How to write clusters to binary file without element dimensions.

[Maser]

Hi,
I'm trying to save my testdata to a STDF (Standard Test Data Format) file. STDF files are binary and consists out of several predefined records. These records can be build up out of certain Data Types.

I'm using the 'Write File' VI to write the bytes. This VI gives you via 'byte stream type' input the possiblity to define your own combination of data types.

My problem is that LabView writes the dimensions of the individual elements in front of them.
e.g. if my cluster has an array of numbers it always reserves first a long integer with the amount of array elements. Also a single string has a dimension long integer in front of its bytes.
If I want to read this file with LabView this would of cou
rse be very handy but I want to analyse the written data with other existing software which expect the STDF file format.

e.g.
a variable characterstring => FIRST BYTE = unsigned count of number of bytes to follow (maximum of 256 bytes)
Ohter datatype don't have a byte count.

How can I write to a binary file without these dimension bytes. (or how can I manipulate them)
I use clusters and arrays for 'byte stream type' because the STDF format expect al lot of different kinds of records in one file and every record has different data types within. It would otherwise be very difficult to setup the input for this file format.

e.g.
record 1
HEAD_NUM unsigned integer (4 bytes)
TEST_NUM unsigned integer (4 bytes)
TEST_DES Variable length characterstring (with in thirst BYTE (single byte) the length.
record 2
HEAD_NUM unsigned integer (2 bytes)
TEST_LEN unsigned integer (4 bytes)
OPT_FLG Fixed lenght characterstring
........
......
..

Can anyone point me in the
right direction.

Thanks in advance

Kimball

[JeanPierre]

When writing a cluster, the header(length) is always written for variable length data like strings and arrays. To write a string or an array without header, wire the string/array directly to the Write File data input.

A byte length followed by data is called a Pascal string. You have to write your own VIs to read and write Pascal strings to/from LabVIEW string. It is not difficult, just write the lenght of the string converted to U8 and the write the string to file. To read back, read a U8 and wire this number to the count input of the next read.

Obviouly you can't use clusters as they do not have STDF format on file. Create your own file parsing VIs.

[Maser]

Hello Jean-Pierre,

Thanks for the quick respons.
Wiring the string directly to Write File data input is no option because it would make the VI very large (an awfull lot of records with even more data types) and I want to be able to change certain setting during the startup of the test -> meaning controls on front.

Writing my own VI: How to start and which existing File writing VI's can/should I use. I hope you mean by writing your own VI that this is done within LabView itself and not with external code (e.g. C++).
I haven't used any external codes until now and it maybe handy but for this project not really possible (datelines).

Maser

[Greg_McKaskle2]

> My problem is that LabView writes the dimensions of the individual
> elements in front of them.
> e.g. if my cluster has an array of numbers it always reserves first a
> long integer with the amount of array elements. Also a single string
> has a dimension long integer in front of its bytes.
> If I want to read this file with LabView this would of course be very
> handy but I want to analyse the written data with other existing
> software which expect the STDF file format.
>

If you want to write the elements of the array without the size
prepended, the key is to not write an array, but loop writing element 0,
then 1, etc. For strings where they have a fixed size, convert the LV
string to an array of bytes and do the same, padding it to whatever
length
they expect, or null terminating it by writing a zero after the
last character of the string.

One last thing to keep in mind, for types larger than one byte, you will
need to consider endianess of the bytes. LV writes binary data out in
big-endian format and some programs expect it in little endian format.
In the advanced data manipulation palette there are nodes for splitting
apart and rejoining data.

Greg McKaskle

[JeanPierre]

Greg,

the default behavior when writing an array on file is to write all elements without header, which is equivalent to writing individual elements in a loop. The header is written when the array is bundled in a cluster or when the header input of the write function is wired to TRUE. It is the same for strings.

[Greg_McKaskle2]

OOPs. That is what I get for answering without trying it out.
I know that datalog files do prepend the size, so I assumed
that binary files did too.

Thanks for keeping me straight.
Greg McKaskle

[Maser]

Hello Jean-Pierre,

I'm trying the things you and Greg suggested.
Could be the solutions however I've got as I said before an awfull lot of different records to write that the program would be very difficult to maintain.

Writing my own write-vi => HOW TO START

Maser

[JeanPierre]

Well I thought it was a good use case for LabVIEW Data Tools...

Look at the attached example. The record type is provided to the VI as a cluster. When there are many ways to read the same datatype (PString, LVString, Fixed length string), the format in included in the element label. The cluster may be any combination of datatypes and is accepted on input as a variant.

Using the LabVIEW Data Tools VIs, the cluster is converted to an array of variants (its elements). In a loop, each element is read on file. Binary numbers are read according to their length. Strings are read and converted to LabVIEW strings according to the format provided on the label (dataname). The data is then converted back to a variant cluster. On t
he caller diagram, this variant cluster is converted to a typed cluster record as shown, using "To G Data" function.

The main advantage here is that you have a single VI that can read all the record types. You have to create a cluster for each different record type that you expect to use but the same VI is used to read the records from file.

To use the attached example, you have to download and unzip the lvdata and error packages at http://sourceforge.net/projects/opengtoolkit. Copy the VIs folders in your user.lib directory.

Post again for any question or problem about this code.

[JeanPierre]

OOPs...
You wanted to write a file... My example shows how to read a file... Sorry, I hope it will help anyway... The same technique is applicable to write files.

[JeanPierre]

OK...
the modification was easy. Attached is the VI to write records.