array of clusters that change

[nitad54448]

Hi

I am reading periodically some data from a server I get a JSON file. Then I parse this file in an array (about 2000 elements) of clusters. I need to perform some simple operations (like average, derivatives as a function of time..) on the data contained in this cluster. No problem until here..... but the data may change in time and mixes up my array. How can I handle this ?

For exemple :

I read a file at t0 and get an array with the elements :

stringA, valueA; stringC, valueC

Sometimes, the server adds an element in the json file, and I get

stringA, valueA; stringB, valueB; stringC, valueC

Now when I perform the average the second element in the array is not C anymore... How do I handle this ? 

Thanks for ideas

[nitad54448]

I am answering myself after some more thought : 

-I compare the label in the new array to the previous one, before doing calculations... It works but it is a bit clumsy. If someone has a more elegant ideea... I'll take it.

Thanks

[Bob_Schor · Accepted solution]

If I understand your question correctly, you have entries in the form <Identifier>,<Value>,[<Identifier>, <Value>] where I mean to say each line consists of at least 1, but maybe more, pairs of <Identifier>, <Value> strings.

You (probably) need to process these data in two passes (you can probably do it in one, but it will be much trickier).  First, parse each line (checking that there are an even number of entries, in the form <Identifier>, <Value> and accumulate a list of Identifiers that you find.  For example, your first entry has Identifiers StringA and StringC, while the second adds StringB.  Once you've parsed the entire list, you'll have a list of all the possible Identifiers.

Now start the second Pass.  Depending on what processing you need to do, you read a line, extract the <Values> according to their <Identifier> and process it however you need.  For example, if simply counting how many of each type of <Value> you have, you can create an Array indexed by <Identifier> and increment entry <Identifier> for every entry having an <Identifier> record.  If you need the "Mean Value", you'll need to also accumulate the sum of all the Values, dividing by the number of <Values> (which you've also been accumulating) when you finish the list.

Bob Schor

[nitad54448]

Hi

Thanks, let me explain more clearly 

I have an array of

identifier_string, value1, value2,..value_n (always the same number of values for a given identifier)

But the list sometimes adds additional entries as identifier_string....

Right now I made an array of original identifiers and compare with the new ones before doing the calculations. It is working but I found out that sometimes some of the original identifiers can also disappear so I need to check this too !

thanks

[billko]

---

@nitad54448 wrote:

Hi

Thanks, let me explain more clearly 

I have an array of

identifier_string, value1, value2,..value_n (always the same number of values for a given identifier)

But the list sometimes adds additional entries as identifier_string....

Right now I made an array of original identifiers and compare with the new ones before doing the calculations. It is working but I found out that sometimes some of the original identifiers can also disappear so I need to check this too !

thanks

---

Can you perform different operations depending on the value of "identifier_string"?

[nitad54448]

Sorry, I do not understand what you mean by "perform". 

Now I check all elements in the array 1 and keep only those that are in the array 2.... and then check all elements in the array 2 against array 1. But I feel this is cumbersome, ... and with about 2000 elements is slow. I wish there was an AND operation of arrays based on on one of the elements of the clusters inside...

[drjdpowell]

You might get more effective help if you provided some example code, and some sample JSON that you are parsing.

[nitad54448]

Hi

I am parsing the Json by searching in the text, maybe it is not optimal, but I couldn't figure out clearly how to do it. I was looking at your JSONtext but had no ideea where to start.

The file looks like this

---

[{"channel":"LINE_1","last":"0.000033","voltage":"0.00003412","peak":"0.0000315","change_percentage":"-4.51","slip":"105952.51004092","ident":"3.653525073864948","high_24h":"0.0000369","low_24h":"0.00003257"},{"channel":"LINE2","last":"1.1407","voltage":"1.1434","peak":"1.1372","change_percentage":"-0.87","slip":"38548.6402","ident":"43960.25730161","high_24h":"1.1579","low_24h":"1.1228"},{"channel":"BODY_1","last":"0.0005183","voltage":"0.0005184","peak":"0.000518","change_percentage":"-1.1","slip":"234458.47376911","ident":"121.287934603344545","high_24h":"0.0005258","low_24h":"0.000511"},{"channel":"BODY_3","last":"0.00005593","voltage":"0.00005753","peak":"0.00005373","change_percentage":"2.94","slip":"37730.4220617","ident":"2.0582897091727734","high_24h":"0.00005634","low_24h":"0.00005228"},...

--

Thanks for the message.

[drjdpowell]

Did you look at the provided examples that install with JSONtext?  Here is a quick start, showing formatting for readability ("pretty print") and pulling out all the "high_24V" elements.  Sadly, whoever produced this file didn't use full JSON, as they have saved numbers as Strings, so getting numbers is a two-step process instead of one.

[nitad54448]

Hi

Thanks for the json example, I could fit it in my program. I did not know what the $,* instructions means. Is there a documentation somewhere ?

thanks again

N