LabVIEW

cancel
Showing results for 
Search instead for 
Did you mean: 

Safe-ishly "Upgrade" flattened cluster

Solved!
Go to solution

I want to serialize a cluster of n named elements and then later deserialize it into a cluster with n+1 named elements including all the ones that existed in the original cluster. I also want this to work in arrays/collections of clusters etc.

 

The closest thing I've found to this is various JSON packages on VIPM, and specifically vi.lib\addons\_JKI.lib\Serialization\JSON\Library\Adapt To Type.vi from JKI JSON appears to do exactly what I want:

 

avogadro5_1-1668122367657.png

 

 

but I have found some type safety issues with it, for instance the way it handles empty arrays:

avogadro5_0-1668122153894.png

 

So basically what I want is a "best guess" that has more type compatibility and does a better job of matching the provided datatype. The built-in Variant, XML, and JSON deserializers are too strict and will error rather than providing a guess that preserves what it can of the old data.

Class mutation might solve this but is more developer overhead than it's worth for my application, unless there's a good way to script it around a generic variant.

0 Kudos
Message 1 of 14
(1,398 Views)

It works well with to/from flattened variants, just add some filler 0x00's to the flattened string so there's some data to convert from.

G# - Award winning reference based OOP for LV, for free! - Qestit VIPM GitHub

Qestit Systems
Certified-LabVIEW-Developer
0 Kudos
Message 2 of 14
(1,352 Views)

Have you considered just saving as JSON, rather than flattened clusters?

0 Kudos
Message 3 of 14
(1,336 Views)

@Yamaeda wrote:

It works well with to/from flattened variants, just add some filler 0x00's to the flattened string so there's some data to convert from.


If I'm understanding this correctly is it suggesting that I would have to inspect the 2 datatypes and insert padding string(s) to make the old string compatible with the new one?

0 Kudos
Message 4 of 14
(1,262 Views)

@drjdpowell wrote:

Have you considered just saving as JSON, rather than flattened clusters?


With the built-in JSON unflatten, I have been unable to unflatten into a cluster with different elements than the original cluster that was flattened, even when it should be "obvious" how to adapt such as adding a new uniquely named attribute.

I'm trying to avoid using a serializer addon that includes the code to interpret each potential type as I have learned not to trust that kind of addon very much.

0 Kudos
Message 5 of 14
(1,260 Views)

Another way of explaining what I want is to hook into however LabVIEW determines the value of typedef'd cluster constants that have had the typedef updated to include more elements. Somehow LabVIEW decides which values I want to keep and gives me a constant of the new type, so is there any way to get it to do that programmatically/with variants?

 

Other than using scripting to create the constant/typedef and make the change at runtime...which I am seriously considering but would require a big effort and wouldn't directly work in RTE.

 

I just want to do what I think LabVIEW is already doing which is taking flattened cluster data of the old type (constant on a block diagram), plus the new datatype (new typedef.ctl, which I think does not contain any history), and get a cluster of the new type with as much preserved as possible from the old flattened data (which LabVIEW does for us, either with auto-updating or with the review dialog).

0 Kudos
Message 6 of 14
(1,141 Views)

Another way of explaining I want is that I want to programmatically do what I suspect LabVIEW already does under the hood when it encounters a cluster constant linked to a typedef that does not exactly match. It takes the data stored in the constant and creates a new value with the proper type that preserves what it can from the constant.

So inputs are "old flattened data" + "old type" + "new type" and the output is "best guess that preserves what it can of the old data using the new type". I have the old data and type because I stored the variant which contains both. I could script creating a constant/typedef to get LabVIEW's guess of how to adapt the data, and I am considering doing just that (essentially this is rolling my own review typedef dialog box), but it would require a development environment and seems a bit hacky.

0 Kudos
Message 7 of 14
(1,232 Views)

@avogadro5 wrote:

Another way of explaining I want is that I want to programmatically do what I suspect LabVIEW already does under the hood when it encounters a cluster constant linked to a typedef that does not exactly match. It takes the data stored in the constant and creates a new value with the proper type that preserves what it can from the constant.


Actually you want much more than what LabVIEW itself does under the hood. LabVIEW only adapts (converts) as long as a cluster has the same number of elements AND all the elements are simply convertible to the other. It does definitely bark if the number of elements does not match AND also if any of the elements ends up in the cluster order to try to match to an incompatible element (string to anything but string, boolean to anything but boolean, array to anything but an array). It "only" automatically converts numerics to other numeric types.

And the Review Typedef dialog box is an entirely different beast. It tries to match cluster elements based on element name, element order, and element type and then presents a dialog to confirm things and then will rematch the elements in the way according to the recipe confirmed by the user. It's a notorious function that has received countless updates since its inception and always dissatisfied someone in some way, no matter what it did.

 

In short, what you try to do is inherently difficult to solve in a generic way. The most "easy" way is to store the information in an intermediate form that is agnostic to element order and only uses the element name as distinguishing feature. Such as Python dictionaries or JSON objects. The LabVIEW equivalent would be Variant attributes and since 2019 Maps. And yes this conversion is not something you should do if performance is important, as it is a rather costly conversion.

Rolf Kalbermatter
My Blog
0 Kudos
Message 8 of 14
(1,212 Views)

@rolfk wrote:

@avogadro5 wrote:

Another way of explaining I want is that I want to programmatically do what I suspect LabVIEW already does under the hood when it encounters a cluster constant linked to a typedef that does not exactly match. It takes the data stored in the constant and creates a new value with the proper type that preserves what it can from the constant.


Actually you want much more than what LabVIEW itself does under the hood. LabVIEW only adapts (converts) as long as a cluster has the same number of elements AND all the elements are simply convertible to the other. It does definitely bark if the number of elements does not match AND also if any of the elements ends up in the cluster order to try to match to an incompatible element (string to anything but string, boolean to anything but boolean, array to anything but an array). It "only" automatically converts numerics to other numeric types.

And the Review Typedef dialog box is an entirely different beast. It tries to match cluster elements based on element name, element order, and element type and then presents a dialog to confirm things and then will rematch the elements in the way according to the recipe confirmed by the user. It's a notorious function that has received countless updates since its inception and always dissatisfied someone in some way, no matter what it did.

 

In short, what you try to do is inherently difficult to solve in a generic way. The most "easy" way is to store the information in an intermediate form that is agnostic to element order and only uses the element name as distinguishing feature. Such as Python dictionaries or JSON objects. The LabVIEW equivalent would be Variant attributes and since 2019 Maps. And yes this conversion is not something you should do if performance is important, as it is a rather costly conversion.



I understand it's difficult and I want it to bark at me if it can't convert safely. But I think the adaptation is a bit more flexible than you said, it seems to tolerate added named elements gracefully even when added in the middle of the cluster, and tolerates reordered named elements. And I'm willing to throw up my hands if mismatches can't be reconciled, which I think (for my application) is best handled with something like the review dialog where someone has to approve and has a chance to modify the new value.

I also will definitely not store data in a less performant manner than flattened variant, if I have to I'll convert to an intermediate JSON in memory to help resolve type mismatches when needed.

0 Kudos
Message 9 of 14
(1,169 Views)

@avogadro5 wrote:

@drjdpowell wrote:

Have you considered just saving as JSON, rather than flattened clusters?


With the built-in JSON unflatten, I have been unable to unflatten into a cluster with different elements than the original cluster that was flattened, even when it should be "obvious" how to adapt such as adding a new uniquely named attribute.

I'm trying to avoid using a serializer addon that includes the code to interpret each potential type as I have learned not to trust that kind of addon very much.


Well, the limited use of the inbuilt JSON primitives are why other JSON libraries exist.  I wrote JSONtext myself, and from that experience I can see that making the subVI you want is certainly possible, though you didn't convince me that the JKI one doesn't already work.

0 Kudos
Message 10 of 14
(1,162 Views)