Error Reading Binary File generated with LabVIEW in MatLAB

[sez]

I have two different types of binary file that contains a cluster and an array of the same cluster that I can R/W fine with LabVIEW.

Matlab has an issue reading and requires me to input field name, type for each field, number of bytes, array size information...etc

Details on the file content in the attachment.

How can I define each element in my cluster data type so that Matlab can read my file or if there is any other method of making this data transfer possible that you would recommend?

Thanks

[Bob_Schor]

This is yet another example of the value to the Original Poster of attaching code (an executable VI or an executable LabVIEW Snippet) to help us help you.  It would be relatively easy to write a description of the Matlab format had you provided such a code, as we could generate an example for ourselves and test/verify that what we proposed was correct.  Oh, well.

But you largely already know the answer, as you've shown us a picture of your Cluster and also what Matlab is expecting.  The tricky parts is knowing how arrays are saved in Binary Files (any why it is important that we see the code you use to write the file -- are you includiing the Array Size, specifically, in your Write command?), as well as being able to interpret the LabVIEW TimeStamp (saved as a 128-bit "strange fraction" format).

Bob Schor 

[sez]

Yes, I need to tell Matlab what it is reading. LabVIEW reads fine as it already knows its own data type as being the writer of the binary file. But talking to any software outside of the LabVIEW would require a similar understanding of the record structure to read the binary file. Some of the data type conversions in binary format are abstract. (timestamp,array structures...)

I found someting on timestamp datatype information: http://www.ni.com/tutorial/7900/en/

I would like to learn how the arrays(1D,2D), strings are being saved into binary file in LabVIEW to be able to read the file from other platforms. 

I am not sure if NI is forcoming with this information but wondering if there is any NI article on how that binary data conversion been done and saved with any elements that I can possibly have within my LabVIEW cluster data type?

[Bob_Schor]

Again, some of that you control by how you wire the Write Binary Data function (which is why you need to post your code.  A lot of this information is readily available, and we are happy to help you, but we need to know what you tell LabVIEW to do (which we can only do by seeing the code you used) in order to say, for example, if there is array/string size information as part of the file.  We are trying to help you, but you need to help us to help you ...

Bob Schor

[mcduff]

Check out https://sourceforge.net/projects/matio-labview/?source=recommended

This is to make native Matlab files in LabVIEW. Or You can also use of fopen in Matlab and parse the file yourself. Note in your picture you are adding a length to the cluster, make this false, it may make parsing it easier. this is the Boolean input on top of the write file vi. Sorry for the typos on a iPad.

mcduff

[sez]

Bob, Here is my Binary Handler VI I used to for my R/W operations. 2D array sizes within the cluster and 1D array size of the main data cluster array for array type binary file depends on user selection in each operation.

I would like to be able to read the binary files with the given format from Matlab generated by the attached VI.

You can also find sample LabVIEW generated .bin files for both types in the attachment.

[sez]

Anyone has any idea on how to decode the typedef in my previous post in a non Labview environment?

I post my binary file IO vi that contains my typedef into my previous post. LabVIEW prepends array and string size to the binary file by default and represents some of the data types in its own way. I would like to learn how labview binary file byte allocation works when you pass a cluster that contains strings,timestamps.2D arrays and some numeric values as seen in my post above. Any ideas?

[Bob_Schor · Accepted solution]

Sorry, I missed your earlier post.  Let me help you with array.bin, as single.bin will be one step easier.  This will not be a "complete" LabVIEW answer, as there are some types that aren't covered here, but here goes:

• Offset 0.  Because it is a 1-D Array, the first four bytes are an I32 (32-bit integer) with the array size.  The bytes are, in fact, 3, 0, 0, 0 = 0003.
• Offset 4.  Next is a TimeStamp.  This is a 128-bit representation of the time, so it occupies 16 bytes.  The Time representation is a little odd, so I'm going to skip it, for now.
• Offset 20.  TimeStamp String.  Four bytes are I32 String Length, followed by String.  As this is a blank string, the values are 0, 0, 0, 0.
• Offset 24.  Another TimeStamp, 16 more bytes.
• Offset 40.  dt, a Dbl, 8 bytes.
• Offset 48.  Notes, a string.  As this is an empty string, just the 4-byte 0 length.
• Offset 52.  Average SPH, a Dbl, 8 bytes.
• Offset 60.  Average LPH, a Dbl, 8 bytes.
• Offset 68.  IndexNo, a U64, 8 bytes.
• Offset 76.  ImageArray, a 2D Array of U16.  As this is an empty Array, just the two 4-byte 0 lengths.
• Offset 84.  DAQ Data, a 2D Array of Dbl.  As this is an empty Array, just the two 4-byte 0 lengths.
• Offset 92.  Beginning of the second Array element.  Again, a 16-byte TimeStamp, but this one has data, so not all the bytes are 0.
• Offset 108.  TimeStamp String, consisting of "This is array type.", 19 characters, so there are 4 bytes for the length + 19 bytes for the String itself.
• Offset 131.  Another 16-byte non-zero TimeStamp.
• Offset 147.  dt, so 8 bytes representing the Dbl 1.0.
• Offset 155.  String "rwegrweg", so 4 bytes for length (8) + 8 bytes for the string.
• Offset 167.  Average SPH, 8 bytes.
• Offset 175.  Average LPH, 8 bytes.
• Offset 183.  IndexNo, 8 bytes.  This is the value 4, or as a U64, 4, 0, 0, 0, 0, 0, 0, 0.
• Offset 191.  ImageArray.  The two 4-byte size values are 2 and 2, so there are 4 U16 values 1, 2, 2, 17.
• Offset 207.  DAQ Data.  The two 4-byte size values are 2 and 1, so there are 2 Dbls.
• Offset 231.  Beginning of third Array Element.  16-byte TimeStamp.
• Offset 247.  String, four-byte length (14) + value "second element".
• Offset 265.  Timestamp, 16 bytes.
• Offset 281.  dt, 8 bytes Dbl.
• Offset 289.  String, 4-byte length (2) + value "hi".
• Offset 295.  Average SPH, 8 bytes.
• Offset 303.  Average LPH, 8 bytes.
• Offset 311.  IndexNo, 8 bytes.
• Offset 319.  Image Array, 4+4 bytes (2, 1) + 2 U16.
• Offset 331.  DAQ Data, 4+4 bytes (2, 2) + 4 Dbl.
• Total size = 371.  This matches the size of the file.

So if you are really interested in parsing this file, you need to know the "file composition" (provided nicely by the ArrayIn description you provided on your JPG file.  Whenever you see an Array, you know you need to read a 4-byte I32 to get the Array size(s), then read that many Array Elements.  If the Element is a Cluster (as is the case here), you need to read each element of the Cluster, which (in turn) may contain Strings (with a prepended 4-byte length) or Arrays (with size(s) + elements).

Look up the definition of TimeStamp.  You'll see that it really is a 128-bit integer representing the number of ticks of a very fast clock (I forget the clock rate, but it is >1MHz) since 1 Jan 1904 UTC.

Hope this is helpful.

Bob Schor

P.S. -- I'm not (now) 100% certain that I've stated the TimeStamp format correctly.  It is a bit messy ...

[RavensFan]

---

@Bob_Schor wrote:

P.S. -- I'm not (now) 100% certain that I've stated the TimeStamp format correctly.  It is a bit messy ...

---

Here is a document that describes the timestamp better. http://www.ni.com/tutorial/7900/en/

The first 8 bytes represent seconds since 1/1/1904 midnight GMT.  The second 8 bytes is a 2^64 bit fraction of a second.

[sez]

Thx very much for your detailed reply Bob. That is exactly what I needed. One more question:

One thing that I included my cluster in a later version is "path" data type that I could not understand how binary represantation is handled. I ended up converting path to string so that I can decode it like a string to resolve the issue. Do you know how path type being flattened into binary string format?