Parse array of variants

[shansen1]

I have an application where we read binary data over Ethernet at a fast rate (~500 Hz). This data consists of many hundreds of measurements and other variables of all different types (floats, ints, strings, etc). I already have the code to parse this data and store it as an array of variants.

Now, I would like to take that array of variants and map each element to a specific variable.

Because the data is stored in a predefined sequence (Element #1 = speed, Element #2 = power, Element #3 = ...), I considered looping over the array of variants and using a case structure with a case for each index of the array (see attachment). However, this is very cumbersome as the case structure has hundreds of cases (one for each array element) and it is impossible to insert a new element in the case structure without manually re-numbering all of the cases.

Can anyone point me to a simpler implementation? If I was using C, I would probably memcpy the binary data directly into a byte-aligned structure. Is there an equivalent in LabVIEW using clusters? 

(Note: using LabVIEW 2014)

[Bob_Schor]

If your data are in a specific format (Speed, Power, ...), why are you saving it as an array of Variants?  Why not save it directly into a cluster whose first element in Speed, second is Power, etc.?  You can certainly mix types in a Cluster.  How many variables are we talking about?  Can you provide more information about the data?

BS

[nathand]

---

@shansen1 wrote:

I already have the code to parse this data and store it as an array of variants.

---

Unfortunately, this is your problem. Store the data into a cluster using the correct datatypes in the first place; don't use variants.

[shansen1]

OK, but by removing the array of variants I'm simply pushing the problem back one level. At some point in time, I still have to convert the binary data into the correct format and store it in a variable. The example I originally posted is one way to do it (just change out the array of variants to a stream of binary data). Are there any other ways, preferrably something a bit easier to maintain?

I was thinking along the lines of setting up a cluster of elements in the correct order, and then copying the binary data directly into the cluster. But, I'm not sure how to implement this in LabVIEW, especially considering it would have to be unsensitive to packing/alignment. Any thoughts?

[nathand]

When you read the binary data and convert it to a variant, aren't you converting to the correct data type at that point, and then converting to a variant?

LabVIEW does have a way to go from a binary string containing heterogeneous data to a cluster - the "Unflatten from String" function. Hard to tell if that's appropriate or will help you here though without more details about what you're doing.

[VItan]

I suppose that what you can do - similar to what you wanted to do in your first post about copying it into a structure in C - is create a type definition for your data structure coming across the network, which would be a cluster of all of your values, and then TypeCast / Unflatten From String into that.  From your image, it looks like you have at least 122 values.  That will be one big cluster, but I suppose that if that is your data structure, then that is your data structure.

By creating a Type Definition, you can gain some maintanability in that if you ever need to reorder, add, or remove data, removing it from your TypeDef should propagate everywhere, and giving every element a name and always using Unbundle by Name ensures that you are always getting the relevent element at each point of use even after changing anything.

Attached is an example of a Cluster TypeDef, and two code snippets follow:

The former uses the Unflatten From String function.  Usually the way to go with unflattening strings.

The latter uses TypeCast.  Functionally identical in this exact case, but probably not advised in this case as:

• You don't have options of specifying byte ordering
• You don't have options of specifying if the string contains the length at the beginning
• You don't have error out to indicate that the conversion was unsuccessful
• For some reason you cannot wire non-fixed-size clusters to the TypeCast function.

I do wonder if anyone has a good solution for the initial problem, though.  Consider what if one's data was actually being sent as an array of variants by some other application - how would one best handle that?

[Bob_Schor]

Well, if the Array of Variants is well-defined (i.e. if you know it will be Torque(dbl), Speed(dbl), Position(dbl), StatusWord(u16), Something Else(Boolean), And More!... (Dbl), then you could feed this into a For loop and with a Case statement driven by the loop index "i", you could build the Cluster element-by-element (using Variant to Data, Dbl for cases 0, 1, 2, and 5, U16 for 3, and Boolean for 4).

There are also VIs that can look at a Variant and tell you the underlying Type, if you don't know it in advance, but it would probably be easier (and faster) to use the knowledge you already have ...

BS

[VItan]

---

@Bob_Schor wrote:

...

you could feed this into a For loop and with a Case statement driven by the loop index "i", you could build the Cluster element-by-element (using Variant to Data, Dbl for cases 0, 1, 2, and 5, U16 for 3, and Boolean for 4).

 ...

---

And then we're back to the initial design and the issues inherent with it.

Perhaps the best goal is going to be getting it into a cluster, even in the case that it is originally variants.  It could be done by flattening the array of variants to a string, then using a subVI to remove the variant headers (should be possibly by checking the first two bytes for type to know how long the data is, remove the 8 byte header, and then offset past the data length to the next variant; iterate), then converting that to a TypeDef'd cluster as already discussed.

[Bob_Schor]

You are right -- here's a better way to do it (and I think it solves the question that was posed).

On the left are whatever data you want to transmit as an Array of Variants.  The goal is to reassemble this after a TCP/IP transmission into a cluster having elements Position, Velocity, Activated, and Name (I just built a Cluster Constant with this format, but it would be better to make it a TypeDef, of course).

So I create my Array of Variants.  To mimic TCP/IP, which converts the data to a string and reassembles it on the other end, I do a Flatten to String on my Variant Array, and then an Unflatten From String, using Array of Variant as the Type.  Nothing (yet) about a Cluster or the data.  But I "know" it is my Cluster, which I now want to reconstitute.  So I take a blank cluster through my For loop, feeding it the Array of Variants and letting it process each element.  I use the VILib function GetTypeInfo (available as part of the Hidden Gems package) to get the name of the original Data cast into a Cluster, and pass that into my Case Statement (based on the name) where i reconstitute the data value (I know the data type associated with each data name, e.g. I know that Position is a Dbl) and then Bundle by Name into my Cluster.  When I've processed all of the Variants, I've filled the Cluster and I'm done.

This is perfectly flexible.  If I add more data on the left, I add corresponding more cases on the right.

Bob Schor

[VItan · Accepted solution]

Very nice.  I figured such a VI probably exited somewhere in vi.lib.

However, I was also concerned about the case where the array of variants is not from another LabVIEW application.  In this case, I wouldn't expect that name and TypeDef information would be in the variants, as these are LabVIEW-specific, so I was thinking of something like this: