Read binary with one of many possible definitions - or - how ot loop over different clusters?

[m-ad]

Hello everyone,

I have a cluster of different data types that I save as binary file and read again at a later time with another VI. I saved the cluster definiton as a typedef so I am certain that I use the same type for reading and writing.

Now I needed to change the cluster in the writing program to include another value. However I still want to be able to use the reading VI to read both old and new datafiles. I also want to have the possibility to change the binary file definition in the future (in the writing VI) and be able to quickly adapt the reading VI to the new definition while maintaining compatibility with older ones.

My plan was to have a typedef of the the cluster for each revision, say v1.ctl, v2.ctl, v3.ctl, ... .

Let's suppose I have a binary input file that has been saved using one of those typedefs but I don't know which one. I would now open that file and try to read its binary content with each of my typedefs as data type and stop once the Read from Binary VI gives no error. Then I would translate that cluster to the current version cluster definition by wiring through existing values and assigning dummy/default values to variables that are not on the binary file.

My idea on how this could work is outlined here with three filetype versions:

For every new filetype version I would need a new case structure although the content of the case structures is almost identical.

Because I save the filetype definition version in my files, I could even simplify and use a separate case structure to update the data and fill in the dummy data:

Now the error case structures are completely identical, only their inputs differ in type. How can I work around that? How can I loop over those different cluster typedefs efficiently?

I use LabView 2012 SP1.

[tst]

Rather than saving as a binary file, you can use something like the OpenG variant config VIs or the MGI read/write anything VIs, which save and load arbitrary clusters and handle version changes correctly, by using the default values for elements which are not present in the file when you load.

[crossrulz · Accepted solution]

Since you have a version of the cluster already, just read that numeric byte itself and then use a case structure to go back to before the version and read the entire cluster based on whichever version of the type def you read.

[daveTW]

Hello,

in my current long lasting project, I had the same problem, an it was clear, that there will be a lot of change to the number and type of parameters, that has to be stored in the file.

My solution was to use the power of the configuration-file- features (INI- Files). They can store different types of information (at the end all converted to strings), they are tolerant to missed values and they don't rely on a certain sequence of storing the information.

The only Disadvantage of the shipped VIs for handling configuration- Strings is the absence of a VI that expects only a string as input, not a path to an existing file. And I wanted to avoid the detour over saving my string to a temp- file and reopening that with the shipped  config-file VIs.

That's why I modified the "Open Config Data" and "Close Config Data" VIs and added the new VIs to the NI_LVConfig.lvlib

See attached snippets below:

If I remember right, I had to make the "Private VIs" public to add my new VIs and after that I made them private again.

By now I have stored more than 170 different parameters and I'm not afraid to add (or remove) some more.

[tst]

---

@daveTW wrote:

That's why I modified the "Open Config Data" and "Close Config Data" VIs and added the new VIs to the NI_LVConfig.lvlib

---

Modifying shipping VIs or libraries is a really bad idea. As soon as you open the code on another machine, your code will no longer work. If that's your solution, you should create a copy of the entire library and use that copy to modify and call in your code. This copy should obviously be saved in place which will migrate properly to new computers.

In any case, that won't help the OP, who doesn't have everything as strings. That's where the libraries I mentioned are useful, because they have the code to the conversion.

[m-ad]

@crossrulz: That's great. I didn't know that you can use the Read from Binary VI to read only part of a binary file, I thought it's all or nothing. Luckily, I have the version number at the beginning of my files, so that should work out in my case. 🙂

@tst: I had a quick look at the MGI VI's download page. That looks very interesting, especially the part with the default data. One thing though: what I like about binaries is for me that I can easily load them into Matlab or Python for further processing (if I know the file definition, fo course). The MGI VI saves everything as a formatted string, so I expect parsing that could be a bit tedious.

[tst]

---

@m-ad wrote:

One thing though: what I like about binaries is for me that I can easily load them into Matlab or Python for further processing (if I know the file definition, fo course). The MGI VI saves everything as a formatted string, so I expect parsing that could be a bit tedious.

---

You could simply save two files - one for you (MGI/OpenG) and another binary one for other programs. That should work if you don't expect to modify the files and then load them back up in LV. If you do, then you could also consider working with a database, which is a more "proper" data sharing and storing method, but that would require more work on either side of the code on matching the correct data types, and certainly won't be as simple as the alternative. Personally, I would prefer either to the unbundling-bundling combo you have to do, but that's a matter of preference.

[Yamaeda]

---

@crossrulz wrote:

Since you have a version of the cluster already, just read that numeric byte itself and then use a case structure to go back to before the version and read the entire cluster based on whichever version of the type def you read.

---

What a beautiful solution, never would have thought of that, it works great!

/Y