LabVIEW

cancel
Showing results for 
Search instead for 
Did you mean: 

JSON with nested structure hard to parse

I have a JSON file with the following structure:

  1. Device Type (8 types)
  2. Device Voltage (11 unique values, about 3-5 per device type)
  3. Device Series (maybe 100 unique values, about 10-15 per device type)
  4. Device Name (possibly 500-600 unique values, about 5-15 per device series)

This is just to get to a common type.

Now the common "cluster" type is as follows:

  1. about 10 string
  2. a cluster with about 10 strings
  3. another cluster with about 15 strings
  4. ~10 clusters of 2 arrays
  5. 4 arrays

The JSON file is about 170k lines long. It big.

 

The problem I'm having is that it seems like I have to literally spell everything out down to the literal JSON value I want to pull from that string. That's fine, but the problem is that I essentially have to hardcode in the entire cluster structure just to fill the values. There has got to be a better way.

 

I previously used this in Python. Literally just used "json to dict" and boom, I can parse it as expected. Now in LabVIEW it seems like it would be smarter to literally just type all of these values in...

 

I've tried a few different ways to do this. Unflatten seems to practically want me to literally spell out the entire structure (probably will take me 5 hours just to draft it, let alone debug all the name variations). Anything else I've looked at doesn't seem to quite do what I want. I tried the JSON to cluster in the toolkit and it runs and just gives me an empty cluster.

 

I don't really have a preference about keeping this data in JSON forever, or necessarily how the data is extracted - I could store it in TDMS. Really, all I care about is that I need the data out of it for calculations, whether that means I build a VI and do surgery on it once and save as TDMS or I build a VI that interfaces with JSON well, restructure it, whatever.

 

I attached a photo of the JSON with some info redacted. It's just to show what I mean, how my file is structured. There is no type contained in it, it is basically just nested. The thing is, Python's "json to dict" did not have an issue with that. I fail to see how there could not just be a JSON string to cluster in a similar manner. I figured I would ask before I try and build such a VI.

0 Kudos
Message 1 of 9
(3,403 Views)

I think you forgot to say what JSON toolkit you are using.

0 Kudos
Message 2 of 9
(3,390 Views)

Sorry about that. I tried Unflatten from JSON, JSON text to variant, and From JSON text. To be quite honest, I don't know what a lot of this stuff means. I don't really want to write to JSON as much as just get the values from this JSON file that I previously used in Python. The values would then be used in calculations and such.

 

Generally, everything I run either wants information that I don't have and wouldn't have unless I literally hand-typed in the entire massive structure of this file. That's why I came here to ask, because this seems crazy.

 

Alternatively, I'd be open to using another data structure that LabVIEW is more friendly with, because this is just a static data file. So using XML or TDMS or whatever is an option, if I have to convert it or something.

0 Kudos
Message 3 of 9
(3,388 Views)

Do the keys/values you are looking for have unique names? Are there multiple copies of them in the JSON? If the information you are interested in is contained in a small portion, you may be able to define a cluster for that section and use one of the JSON VIs to retrieve the information. If you only need a hand full of items you could use a more brute force method and use regular expressions to find the areas of interest. Without seeing the actual JSON text it wold be hard to give more specific answers. Ultimately, the various JSON VIs generally require a cluster definition which matches the JSON in order to extract the data.



Mark Yedinak
Certified LabVIEW Architect
LabVIEW Champion

"Does anyone know where the love of God goes when the waves turn the minutes to hours?"
Wreck of the Edmund Fitzgerald - Gordon Lightfoot
0 Kudos
Message 4 of 9
(3,368 Views)

Like Mark said/hinted at, I think the key missing information from your post might be what you want to do?

 

"Parse JSON" is a reasonably vague goal - do you need specific keys? Information about a specific device? A lookup dictionary/map/store of information about each of the devices that you have in the entire file?

 

What will you do with this information - does it form some initialisation step for hardware, do you want to display some of it (all of it?), do you need it for debugging?

 

All of the values appear to be strings or null values, so I'd be tempted to guess it should be possible to parse it somewhat generically.

Could you upload a "dummy" file with maybe two or so different entries/blocks/devices and renamed values to remove the bits you don't want to share, along with the (or an analogous) end goal? If you can make them representative of some possible issues (different length array sections, different keys, etc) that might help weed out some of the simpler solutions that won't work (rather than them being suggested and you having to try, only to find they have a problem not represented in the test data set).


GCentral
0 Kudos
Message 5 of 9
(3,363 Views)

A toolkit is like a python library.  There are multiply JSON libraries for LabVIEW.  Many of them will have VIs with names like you gave, so I can't tell what you are using.  Thus we can't help you.  You also attached a PNG of your JSON, rather than the JSON, and didn't show any code, so no one can suggest a better way.

0 Kudos
Message 6 of 9
(3,352 Views)

cbutcher,

 

I added an example file. It shows the general idea, just with names redacted.

 

Let me give a better explanation: I have a massive library of datasheet values. The parts are organized by general application ("Some Class of Part", "Some Special Class..."..) then by voltage class ("600","750",...), then by series ("Some series","Some fancy series"...) then by part name ("some part name", "Some other part name"...). If you can imagine, we sell many different parts for many applications - this is a simulator for those parts. The parts for a given application will also have different voltage classes, and when we come out with an update, it's a new series.

 

 

 

Each general application "class" will have various voltages, various series (which are only applicable within that class), then various part names. Each part name then contains a standard "cluster" of information which contains all of its data. Under the part name, the class remains constant throughout.

 

Something a little bit more accessible: Imagine you had a list of measurements of every phone from Samsung, Apple, and LG. Then you had whether it was a touch screen phone, a flip phone, etc. Then under that you had the model (Galaxy 5, iPhone 7, etc.) and then under that you had a length, width and height. That's approximately what's going on here. I have a straight-forward set of data, but its nested within a complicated tree structure.

 

What I would like is to be able to pull the structure above it into a tree-view or similar selection thing. Then what I would like is for the user to be able to say "I want to simulate this part" by leafing through the classes, voltages, etc. and find their part. Then the part "cluster" would be sent into a VI that has a type defined cluster of module info, I'd just parse that piece of the JSON file for the info that is needed and calculate the simulation.

 

In regards to what all I need from the JSON, say there are approximately 600 datasheets in it, I only care about the structure up until you select the part name, and then the datasheet underneath of that (which is consistent between all of the parts) for the simulation. The rest of the file won't be used.

 

I'm hoping this is more descriptive. If you have more questions, let me know.

Message 7 of 9
(3,251 Views)

If I were approaching your problem the two packages I would use are ones I've developed: JSONtext for JSON parsing in LabVIEW, and "SQLite Library for LabVIEW".  SQLite has a built-in JSON parser and is a full database in a simple file and could be used to allow the user to type in a few letters of the part they want and near instantly get a list of matches to select from.

0 Kudos
Message 8 of 9
(3,229 Views)

I used the package James Powell just mentioned, JSONtext, to create this demo.

Of course you'd need to fully implement the "Part.ctl" typedef, but otherwise I think this does most of what you want.

I slightly modified the first few parts for testing.

The example is saved to 2017 (I think...?) and heavily depends on the "Get All Object Items.vi" from JSONtext. You don't need to necessarily keep (or parse) the Parts if you don't want - you can use the built-in Unflatten from JSON to get a cluster given a correct path, so all you really need is the path.

This example produces an array of "/" separated elements, you could use Path to Array of Strings to get a "Path" for the built-in Unflatten (changing the separator to "\" would probably be necessary, a Delete From Array might be needed to get rid of a leading "C" or similar (at least for my quick checking whilst typing this)).


GCentral
0 Kudos
Message 9 of 9
(3,224 Views)