Boost.Python Pickle Support

Boost.Python Pickle֧㖼/h1> Pickle is a Python module for object serialization, also known as persistence, marshalling, or flattening.
PickleʇPython攜ԏ㐲P믣賥rialization㩄㿩㬍 Ҳㆎ갥rsistence᢭arshalling㬻⦬atteningᣍ
It is often necessary to save and restore the contents of an object to a file. One approach to this problem is to write a pair of functions that read and write data from a file in a special format. A powerful alternative approach is to use Python's pickle module. Exploiting Python's ability for introspection, the pickle module recursively converts nearly arbitrary Python objects into a stream of bytes that can be written to a file.
㐨Ҫᣴ涔ϳ億ڈݵ펄쾺ͻָ䶔ϳᣍ ��渃Ί̢儷��֮һʇ㬐䒻收P享ʽ㬒Ԍ؊⸱ʽ䓎ļȡꍐ䈫ʽ㍊ mһ֖Ӑ罷芇ʹӃPython儰ickleģ視㍊ ;ӃPython儷䊡Ĝf㬍 pickleģ冀ݹ鵘ת뻼躵ȎҢ儐ython攏㣬 ��䗪뻎꿉Ҕȫ΄쾵ėֽځ硣
The Boost Python Library supports the pickle module through the interface as described in detail in the Python Library Reference for pickle. This interface involves the special methods `getinitargs`, `getstate` and `setstate` as described in the following. Note that Boost.Python is also fully compatible with Python's cPickle module.
Boost Python ͨ齔ڼa href="http://www.python.org/doc/current/lib/module-pickle.html" >Python Library Reference for pickle ֐Ϫϸeʶ儽ӿږ糖pickleģ視㍊能ӿڰ쀨̘ʢ罷荊`getinitargs`ᢍ `getstate`ꍼtt>setstate㬍 eʶȧςᣍ עҢBoost.Python빍ꈫ즈ݐython廣Pickleģ視㍊

The Boost.Python Pickle Interface

Boost.Python Pickle퓿ڼ/h2> At the user level, the Boost.Python pickle interface involves three special methods:
ԚӃ맲ɏ㬂oost.Python pickle퓿ڰ쀨Ƚ趌؊ⷽ穌ꍊ

`getinitargs`
When an instance of a Boost.Python extension class is pickled, the pickler tests if the instance has a `getinitargs` method. This method must return a Python tuple (it is most convenient to use a boost::python::tuple). When the instance is restored by the unpickler, the contents of this tuple are used as the arguments for the class constructor.
層oost.Python)չ 儊啥Ựickleʱ㬍 pickler⢊ԸE啥ӐuӐ`getinitargs`罷衣胸��ᘐ뷵똒븶PythonԪש㨊鈴boost::python::tuple׮罱㣩ᣍ 屆啥Ӊunpickler떸䊱㬸Oꗩ億ڈݓ×瀠儹錟ꯊ��⎊��
If `getinitargs` is not defined, `pickle.load` will call the constructor (`init`) without arguments; i.e., the object must be default-constructible.
ȧ黼tt>getinitargsδ樒壬 `pickle.load`��瓃Ξ⎊��鷫캯ʽ㨼tt>init㩣덊 촸oԏ㱘ኡ鷫졣

`getstate`
When an instance of a Boost.Python extension class is pickled, the pickler tests if the instance has a `getstate` method. This method should return a Python object representing the state of the instance.
層oost.Python)չ 儊啥Ựickleʱ㬍 pickler⢊ԸE啥ӐuӐ`getstate`罷衣胸��Ӧ絻ر튾ʵ=״̬儐ython攏㡣

`setstate`
When an instance of a Boost.Python extension class is restored by the unpickler (`pickle.load`), it is first constructed using the result of `getinitargs` as arguments (see above). Subsequently the unpickler tests if the new instance has a `setstate` method. If so, this method is called with the result of `getstate` (a Python object) as the argument.
層oost.Python)չ 儊啥Ӊunpickler떸䊱㨼tt>pickle.load㩣썊 ˼ʗψҔ`getinitargs`儽ṻΪ⎊��鷫죨컉ϣ顣 ˦공npickler⢊ԐμĊ啥ӐuӐ`setstate`罷衣 ȧӐ㬾͒Լtt>getstate儽ṻ㨐ython攏㣩Ϊ⎊��巔o÷��ᣍ
The three special methods described above may be `.def()`'ed individually by the user. However, Boost.Python provides an easy to use high-level interface via the `boost::python::pickle_suite` class that also enforces consistency: `getstate` and `setstate` must be defined as pairs. Use of this interface is demonstrated by the following examples.
ɏʶȽ趌؊ⷽ稿ɓɓu紎懟tt>.def()ᣍ ⻹��Boost.Python ͨ齼strong>`boost::python::pickle_suite` ̡驁˒븶җӚʹӃ儸߼潓썊ͬʱҲ쓇һւ㺍 `getstate`ꍼtt>setstate ᘐ볉收蒥ᣍ ς=ѝʾK能ӿڵĊ鈴ᣍ

Examples

=ד

There are three files in boost/libs/python/test that show how to provide pickle support.

boost/libs/python/test֐ӐȽ趎ļϔʾKȧꎌṩpickle֧㖡㍊

`pickle1.cpp`

The C++ class in this example can be fully restored by passing the appropriate argument to the constructor. Therefore it is sufficient to define the pickle interface method __getinitargs__. This is done in the following way:

脹햐儃++ ԍ蹽ϲ鷫캯ʽ䫵݊ʵᵄ⎊퍪ȫ떸䡣 Ҳ䋣싼樒尪ckle퓿ڷ��__getinitargs__㹻Kᣍ բʇഒԏ·튽׶迤ģꍊ

1. Definition of the C++ pickle function:

樒惡+ pickleꯊ��

  struct world_pickle_suite : boost::python::pickle_suite
  {
    static
    boost::python::tuple
    getinitargs(world const& w)
    {
        return boost::python::make_tuple(w.get_country());
    }
  };

2. Establishing the Python binding:

��␹thon೶裺

  class_<world>("world", args<const std::string&>())
      // ...
      .def_pickle(world_pickle_suite())
      // ...

`pickle2.cpp`

The C++ class in this example contains member data that cannot be restored by any of the constructors. Therefore it is necessary to provide the __getstate__/__setstate__ pair of pickle interface methods:

ᾀ햐儃++ ꬓЈκι錟꯶쎞稻ָ䵄㉔ኽ㍊ Ҳ䋣쓐ᘒꌡ駝tt>__getstate__/__setstate__ pickle퓿ڷ��攣ꍊ

1. Definition of the C++ pickle functions:

樒惡+ pickleꯊ��

  struct world_pickle_suite : boost::python::pickle_suite
  {
    static
    boost::python::tuple
    getinitargs(const world& w)
    {
      // ...
    }

    static
    boost::python::tuple
    getstate(const world& w)
    {
      // ...
    }

    static
    void
    setstate(world& w, boost::python::tuple state)
    {
      // ...
    }
  };

2. Establishing the Python bindings for the entire suite:

Ϊջ足uite��␹thon೶裺

  class_<world>("world", args<const std::string&>())
      // ...
      .def_pickle(world_pickle_suite())
      // ...

For simplicity, the __dict__ is not included in the result of __getstate__. This is not generally recommended, but a valid approach if it is anticipated that the object's __dict__ will always be empty. Note that the safety guard described below will catch the cases where this assumption is violated.

Ϊ첵冰컣켴t>__getstate__儽ṻ֐uӐༀ輴t>__dict__ᣍ һࣲ덆축６׶㬍嫈繻Ԥƚ攏㵄__dict__��܊ǎ꿕㬍 բȔʇһ趓組ଷ裬 עҢȧ黎巴왉裬ςʶ儰∫瀷洫ʩ��旽彸C鿶ᣍ

`pickle3.cpp`

This example is similar to pickle2.cpp. However, the object's __dict__ is included in the result of __getstate__. This requires a little more code but is unavoidable if the object's __dict__ is not always empty.

ᾀ퀠ˆӚpickle2.cppᣍ ⻹��膠ԏ㵄__dict__Ự쀨Ԛ__getstate__儽ṻ֐ᣍ բҪ栒뵣亂룬嫈繻膠ԏ㵄__dict__⢷Ǘ܊ǿյģ썊 բʇ⻿ɱ܃ⵄᣍ

Pitfall and Safety Guard

ϝڥ찰∫瀷漯h2> The pickle protocol described above has an important pitfall that the end user of a Boost.Python extension module might not be aware of:

ɏʶ儰ickleҩӐ趖ؒ굄ϝڥ㬍 Boost.Python)չģ冀ėӃ맿Ʉܲ떪倣ꍊ

__getstate__ is defined and the instance's __dict__ is not empty.

樒偋__getstate__㬲⇒ʵ=儼tt>__dict__⻎꿕ᣍ

The author of a Boost.Python extension class might provide a __getstate__ method without considering the possibilities that:

Boost.Python)չ 儗畟ܻጡ駝tt>__getstate__罷裬慎듐〞ǒԏ¿Ʉܐԣꍊ

his class is used in Python as a base class. Most likely the __dict__ of instances of the derived class needs to be pickled in order to restore the instances correctly.
˻儀ԚPython֐Ӄ׷빀ࡣ ΪKսȷ떸䅉ɺ 儊啥㬍 ꜓Ʉܐ蒪pickle胊啥儼tt>__dict__ᣍ
the user adds items to the instance's __dict__ directly. Again, the __dict__ of the instance then needs to be pickled.
Ӄ맖ὓϲʵ=儼tt>__dict__Զ̭ϮĿᣍ ͬѹҪpickleʵ=儼tt>__dict__ᣍ

To alert the user to this highly unobvious problem, a safety guard is provided. If __getstate__ is defined and the instance's __dict__ is not empty, Boost.Python tests if the class has an attribute __getstate_manages_dict__. An exception is raised if this attribute is not defined:

Ϊ̡Ӄ맕▖쫲냷ϔ載ʌ⣬Boost.Python̡驁˰∫瀷洫ʩᣍ ȧ黶蒥K__getstate__㬲⇒ʵ=儼tt>__dict__燿գ썊 Boost.Python��⊔脹ӐuӐ__getstate_manages_dict__ʴᣍ ȧ黸E䐔δ樒壬Բҽ崟쳣㺍

    RuntimeError: Incomplete pickle support (__getstate_manages_dict__ not set)

To resolve this problem, it should first be established that the __getstate__ and __setstate__ methods manage the instances's __dict__ correctly. Note that this can be done either at the C++ or the Python level. Finally, the safety guard should intentionally be overridden. E.g. in C++ (from pickle3.cpp):

Ϊ��渃Ί̢㬓渃ʗψȷ樍 __getstate__ꍼtt>__setstate__罷荊 սȷ需튵=儼tt>__dict__ᣍ עҢ㬕⼈Ԕڃ++ҲԔڐython⣴Η浽ᣍ ׮곣찲ȫ瀷洫ʩӦ胓ⵘ��⸇ᣍ =ȧ㬔ڃ++֐㨀䗔 pickle3.cpp㩣ꍊ

  struct world_pickle_suite : boost::python::pickle_suite
  {
    // ...

    static bool getstate_manages_dict() { return true; }
  };

Alternatively in Python:

범ڐython֐㺍

    import your_bpl_module
    class your_class(your_bpl_module.your_class):
      __getstate_manages_dict__ = 1
      def __getstate__(self):
        # your code here
      def __setstate__(self, state):
        # your code here

Practical Advice

ʵӃ��鼯h2>

In Boost.Python extension modules with many extension classes, providing complete pickle support for all classes would be a significant overhead. In general complete pickle support should only be implemented for extension classes that will eventually be pickled.
Ԛ��)չ 儂oost.Python)չģ節썊 Ϊ˹Ӑ ̡驍ꕻ儰ickle֧㖽늇֘䳵Ŀꏺᣍ ͨ㣣썪ջ儰ickle֧㖓渃��ꗮ֕롱방ckle儀久 ʵϖᣍ

Avoid using `getstate` if the instance can also be reconstructed by way of `getinitargs`. This automatically avoids the pitfall described above.
ȧ黊啥Ҳԍ蹽`getinitargs`儷튽֘Ђ鷫죬 ᜃ⊹Ӄ`getstate`ᣍ բ��Զﱜ憎ϊ淨ϝڥᣍ

If `getstate` is required, include the instance's `dict` in the Python object that is returned.
ȧ黑ꇳ`getstate`㬍 Ԛ絻صĐython攏㖐ༀ芵=儼tt>dictᣍ

Light-weight alternative: pickle support implemented in Python

ǡ춌洺罷裺ԚPython֐ʵϖpickle֧㖼/h2>

`pickle4.cpp`

The pickle4.cpp example demonstrates an alternative technique for implementing pickle support. First we direct Boost.Python via the class_::enable_pickling() member function to define only the basic attributes required for pickling:

pickle4.cpp=דѝʾKһ֖ʵϖpickle儌洺켊塣 ʗψ㬎҃Ǎ蹽class_::enable_pickling() ָʾBoost.Pythonֻ樒尪ckleҪǳ儻鱾ʴ㺍

  class_<world>("world", args<const std::string&>())
      // ...
      .enable_pickling()
      // ...

This enables the standard Python pickle interface as described in the Python documentation. By "injecting" a __getinitargs__ method into the definition of the wrapped class we make all instances pickleable:

բ惘䁋Python΄碴Ѓ芶儱ꗼPythonpickle퓿ڡ㍊ ͨ齏ⱻ碗攜蒥֐ᰗ∫ᱼtt>__getinitargs__罷裬 ΒCʹ˹Ӑʵ=漿ɰickle㺍

  # import the wrapped world class
  from pickle4_ext import world

  # definition of __getinitargs__
  def world_getinitargs(self):
    return (self.get_country(),)

  # now inject __getinitargs__ (Python is a dynamic language!)
  world.__getinitargs__ = world_getinitargs

See also the tutorial section on injecting additional methods from Python.

闠ڴӐythonעȫ軾ӷ��㬒⿉컍 tutorial sectionᣍ

Updated: Feb 2004.

筒룺��켯a>

ҫ΄輐￡결08.5.29

Boost.Python Pickle Support

The Boost.Python Pickle Interface

Examples

=ד

pickle1.cpp

pickle2.cpp

pickle3.cpp