Main Content

Turning objects into something you can store - Pickling (Python)

Archive - Originally posted on "The Horse's Mouth" - 2007-04-15 08:06:28 - Graham Ellis

If you're working with objects in an OO language and you want to transfer them to another computer ... or simple save them from the current application for reloading into that application or another one on the same computer later on, you need to serialize the objects.

Data within OO programming languages is stored in the heap and as its name implies, the heap isn't neat - there's pointers and information all over the place, and even if you save an object correctly into a file of database, without extra information you probably couldn't read it back in as the memory locations and addresses used internally probably wouldn't be available in the next program. Serialization is the process that adds the extra information you'll need

Python's cPickle module allows you to serialise an object. The dumps method writes any object to a string, and the loads method reads the object back from a string. The reloading program does need to have imported any modules that were used within the dumped object, of course. If you want to write objects straight to a file, you can use the dump method instead, and reload using the load method.

Source code examples:
Picking
Unpickling
Class of objects to be stored / recovered

Note also the marshal module, which allows you to dump and restore simple objects, and the pickle module which is written in pure Python rather than a mixture of C and Python. Marshal is OK for simple objects (perhaps its a bit more efficient sometimes) but you should only use pickle rather then cPickle if there's a chance of cPickle not being available.

The term "shelving" is often used in association with pickling. Objects picled can be stored in strings, files, databases ... and the shelve module allows you to use an anydbm 'simple' database for the purpose. Personal choice, though - MySQL, but remember not to store objects for the long term as an upgrade / change to the internals of a class may render pickled object unloadable.