Decoding XML and clustering data

[tux1972]

I'm new to LabView and I'm experimenting with XML and Clusters.

I need to read each XML files into a given folder, decode them, collect attributes of a specific node (Product) and store values into structured data.

It's not complete I still need to create an array of clusters, but the main VI looks like that:

I defined xml schema and created some files to test it.

It looks like is working but it sounds quite complex and error prone (bundling values into cluster and control has been a real pain).. is there a simpler way to achieve the same result?

Attached: zip archive with vi, ctl, xml, xtd files.

[Kyle97330]

Well, LabVIEW does have built-in flatten-to and unflatten-from XML nodes:

https://www.ni.com/docs/en-US/bundle/labview-api-ref/page/functions/flatten-to-xml.html

Word of warning:  I used these in the past and no longer use them because they do not work well with changes.  If you change your cluster contents, you can't encode or decode anything that doesn't match it.

As for using XML the way you are now... it may work, but as another warning, the built-in LabVIEW XML manual encoders and decoders do not scale well.  At least, they didn't a few years back, and I haven't heard of them getting better.  There's a "squared" execution time that comes into play.  In testing, it seems that  if you change from decoding a 1,000 line XML file to decoding a 10,000 line file, instead of taking 10x longer it takes 100x longer.

Does it need to be XML or could it be JSON or another format?  I've had better luck with some of the open source JSON converters out there, and use those almost exclusively now for my data flattening needs.

[tux1972]

> Does it need to be XML or could it be JSON or another format?

I have been asked to use XML .. it's not mandatory but I think they want to use it because of validation.

>There's a "squared" execution time that comes into play....

I don't thinkt this application will ever grow to a much larger xml file. I think about 100 lines are the maximum 

we could expect to load.

I built up the array of cluster... in this way... should it work? There's a better way?

I have some doubt about the inizialization of the array.... 

[Kyle97330]

Well, it's not strictly related to your XML problem, but you almost certainly want to change to use a type definition of a cluster and not re-create it separately in multiple places in your code.

https://www.ni.com/docs/en-US/bundle/labview/page/creating-type-definitions-and-strict-type-definiti...

Also, you probably want to have the array be either on a shift register, or use a conditional output terminal.  The way your code is set up now won't actually store anything in your array.

[tux1972]

> certainly want to change to use a type definition of a cluster and not re-create it separately in multiple places in your code

Yes I wish to avoid that and I created a .ctl file for this but I don't know how to use it.

I tried but for some reason I've not been able to.

I've only been able to connect it to the "Initalize Array" instance:

But I cannot understand how to do it with the "Insert Into Array".

> Also, you probably want to have the array be either on a shift register, or use a conditional output terminal

I don't get it, can you explain? I'm passing the iterator to the "Insert into array" function... this will not work? Of course the VI was not  'returning' the resulting array I'm working on this but I got some unexpected results:

But in the front panel ...

Why they got two indexes? It looks like boards is 2D....

And also error has become an array of errors....????

[Shuggy1 · Accepted solution]

You have created an array with 'initialise array' and for each file you are reading you are trying to insert the cluster into an 1d array. At the end of the for loop this 1d array is indexed into an array of these 1d array forming a 2d array.

This is a mix of a few methods, and you need to choose one. 

You can create an array the correct size for the number of files you are reading and insert the cluster into the array. But the second iteration must use the array as it was left after the first interaction of the loop. To do this you need to use a shift register on the array terminals. 

Alternatively don't initialize the array at the start and let the for loop build the array from the cluster using the indexing output terminal. 

https://www.ni.com/docs/en-US/bundle/labview/page/processing-individual-elements-in-an-array-or-a-co....

 https://www.ni.com/docs/en-US/bundle/labview/page/processing-individual-elements-in-an-array-or-a-co....

To add a shift regester right click the output terminal of the boarder array and select 'add shift regester'

To get the data from the xml is fine but if it is going to change you could consider using this addin by jki. It made my code much cleaner. 

https://www.vipm.io/package

/jki_lib_easyxml/

[tux1972]

> You have created an array with 'initialise array' and for each file...

oh ... yes... I meant to put the initialization 'outside' the loop not inside it...  I'm fixing that:

I'm not sure this's what you meant but now the function does what I meant... (at least it looks like so).

This's the front panel now... much better now!

For some reason also error has been fixed....

I'll give a look at the site you linked...Thnx

[paul_a_cardinale]

---

@Kyle97330 wrote:

Well, LabVIEW does have built-in flatten-to and unflatten-from XML nodes:

https://www.ni.com/docs/en-US/bundle/labview-api-ref/page/functions/flatten-to-xml.html

Word of warning:  I used these in the past and no longer use them because they do not work well with changes.  If you change your cluster contents, you can't encode or decode anything that doesn't match it.

As for using XML the way you are now... it may work, but as another warning, the built-in LabVIEW XML manual encoders and decoders do not scale well.  At least, they didn't a few years back, and I haven't heard of them getting better.  There's a "squared" execution time that comes into play.  In testing, it seems that  if you change from decoding a 1,000 line XML file to decoding a 10,000 line file, instead of taking 10x longer it takes 100x longer.

Does it need to be XML or could it be JSON or another format?  I've had better luck with some of the open source JSON converters out there, and use those almost exclusively now for my data flattening needs.

---

You can use this to generate a cluster and parser for a JSON string.

[michele.santucci]

I finally understood and managed the whole thing trough shift registers: 

And it works fine. The only minor issue is to find a way to apply the board

type definition along with "Insert into array" instead using a generic Bundle

function.