Hello Pythonist,

In a few of the previous articles, we've talked about storing data into files.

We saw some techniques likes JSON and CSV for this purpose.

But these methods have their limitations.

For instance,

- if you ever tried to save a tuple object into JSON, you will notice that it got converted into a list!

- sets cannot even be serialized into JSON

Thus, while JSON supports quite a few data types, it cannot support even the builtin Python types, let alone your custom class objects.

Then, how do we reliably store any Python object into a file?

Fortunately, there is a way. But before we get to that, we need to look at new file open modes.

Binary file modes

The files we dealt with are text files. These are files that have readable characters in them. Examples are Plaintext files, Python scripts, JSON files, and CSV files.

There is another type of files, called binary files. These files are a sequence of bytes, not necessarily having readable characters.

The technique to store any Python object employs Binary files rather than Text files.

So, we need to tell Python to read/write the file in binary format.

To do that, we need to tweak the file open mode just slightly.

To open a binary file in read mode, use 'rb':

open('myfile', 'rb') # <--- 'b' stands for binary, 'r' for read

To open a binary file in write mode, use 'wb':

open('myfile', 'wb') # <--- 'b' stands for binary, 'w' for write

With that clear, we are ready to learn how to store any Python object to a file.

Using `pickle` to store data

The pickle module can store arbitrary Python objects to a binary file. We import it like this:

import pickle

Files to store pickled objects are commonly given the extension of .pickle, or .pkl.

To write a Python object into a file, we open the file in Binary Write (`wb`) mode, and use pickle.dump, like so:

import pickle
my_obj = [(1,2,3), {1,2,3}]
with open('data.pkl', 'wb') as file_obj:
    pickle.dump(my_obj, file_obj)

And we're done storing the pickle!

One slight disadvantage of pickle is that it isn't human readable. Which means we can't open the .pkl file and examine the content, like we could do for JSON and CSV.

The pickle has to be deserialized through Python itself.

Let's do that now, opening the file in Binary Read (`rb`) mode, and using pickle.load to read the pickled object:

import pickle

with open('data.pkl', 'rb') as file_obj:
    my_obj = pickle.load(file_obj)

print(my_obj)

This prints out

[(1,2,3), {1,2,3}]

which was exactly the value written into the pickle.

You can now store any Python object into a pickle, and retrieve it exactly.

That's all for today. Happy pickling!

Co3

P.S. If you haven't already, check out my posts about Testing. Learning to test well is a surefire way to level up your coding skills
.

0 comments

Making Copies

What Nobody said: When you can't get the right output, at least...

What Nobody said: When you can't get the right output, at least...

Pickling - Save any Python object to a file

Reading and Writing CSVs as Dictionaries

See all posts from Coding Computing Coach