Import Python PKL file to LabVIEW

[jiangliang]

I come across some dataset that is packed in PKL format, I googled it, and found out:

A PKL file is a file created by pickle, a Python module that enabless objects to be serialized to files on disk and deserialized back into the program at runtime. It contains a byte stream that represents the objects. More Information. The process of serialization is called "pickling," and deserialization is called "unpickling."

I'm not sure if it is possible to import these files to LabVIEW? I'm more comfortable processing data in LabVIEW.

I tried to open it with notepad and found only the file head is readable, I'm not sure if the rest of the file are binary or compressed.

The head of the file is like this:

\80\03cnumpy.core.multiarray\n_reconstruct\nq\00cnumpy\nndarray\nq\01K\00\85q\02C\01bq\03\87q\04Rq\05(K\01M\C7\D5\85q\06cnumpy\ndtype\nq\07X\03\00\00\00V22q\bK\00K\01\87q\tRq\n(K\03X\01\00\00\00|q\0BN(X\01\00\00\00xq\fX\01\00\00\00yq\rX\01\00\00\00zq\0EX\01\00\00\00iq\0FX\01\00\00\00tq\10tq\11}q\12(h\fh\07X\02\00\00\00f4q\13K\00K\01\87q\14Rq\15(K\03X\01\00\00\00<q\16NNNJ\FF\FF\FF\FFJ\FF\FF\FF\FFK\00tq\17bK\00\86q\18h\rh\15K\04\86q\19h\0Eh\15K\b\86q\1Ah\0Fh

[GerdW]

Hi jiangliang,

---

@jiangliang wrote:

A PKL file is a file created by pickle, a Python module that enabless objects to be serialized to files on disk and deserialized back into the program at runtime. 

I'm not sure if it is possible to import these files to LabVIEW? I'm more comfortable processing data in LabVIEW.

---

I guess you need to read the documentation about that Pickle module to learn about the options you have in LabVIEW to "unpickle" your data "objects"…

[Niatross]

I have never delved into how the python pickle module stores data. The documentation can be found here https://docs.python.org/3/library/pickle.html#data-stream-format

I know that the start of the pickled data stores the name of the class (read data type) of the data that has been pickled. This is needed so that the pickle module automatically knows how to 'unpickle' the binary data stored in the file.

It therefore looks like your data is going to a numpy multidimensional array. I would therefore be tempted to convert your pkl file to a csv file using python and then do your data processing in LV. You write this conversion code and then call it in LV if you would like.

If your data isn't just a multidimensional array, or if it has more than two dimensions obviously it won't just be a case of converting to a CSV file. How you translate your data will be up to you.

I am sure with enough reading you could read your pickle file straight into LV if absolutely necessary. The difficulty of this will depend upon how the numpy class actually stores the data and how much pickle obfuscates the data.