local variable or queue with one value

[Xinghe.W]

Hi, I want to use a very long and growing array (with thousands of values) in another loop. Right now I'm simply use a local variable, but I'm afraid its regraph in the front panel will eat too much CPU as the array grows bigger and bigger. Do you recommend me to use a queue with only one value (the array) stored in the buffer?

[James.M]

This depends how you're using the array. Do you need all of the values from the array in the second (consumer) loop? If you only need a few values or one at a time and you don't care about the previous values, you can use a producer/consumer architecture using a queue.

The Producer/Consumer architecture is based on a producer loop adding data to a queue and the consumer loop dequeueing the data. The process-intensive or time-hogging code goes in the consumer loop to free up the producer loop to run at the speed you need it to run.

More information here.

[Sam_Sharp]

You generally want to prevent an array from becoming too large (unbounded) - not only will it use more CPU in memory allocations, but it will also use increasingly more memory.

Single Element Queues aren't really used/recommended any more - if you want to be really efficient with the data then use a Data Value Reference (DVR) or use a local variable/FGV. To save on memory allocations, you should preallocate the array with the maximum expected size. You can also halve(ish) your memory usage by using SGLs instead of DBLs for your data.

Finally - if you're using a graph it can only physically display so many points (i.e. your horizontal resolution of your monitor) - it's quite common to decimate the data before writing it to the graph (e.g. average every 100 points or something). If you're using a waveform chart - it has it's own history length which you can configure.

You didn't post your code so I can't offer anything other than 'general' advice.

[Xinghe.W]

What's important is all the values in the array. I'm thinking put the whole array as an "value" in the queue and there is only one "array value" in the buffer. As the array grows 100 values every second, I don't really care how it grows as long as I can read it every 50ms (20 times per second).

[James.M]

If that array is growing indefinitely, you're eventually going to have a bad day when you can't meet that 50ms requirement. I would suggest posting your code here so we can better see what you're trying to do. Otherwise, I agree with Sam that you should look in to DVRs.

[Sam_Sharp]

This is how I would do it using a DVR*:

*With the obvious exception that I would probably preallocate/index into array, and do some sort of trimming to stop it from growing too large.

[Xinghe.W]

I managed to cut part of my code out of my project. The XY graph has two conditions: running and freeze. In running condition, I use multi-touch toolkit (which is not here) to adjuct the scale and the array selected is determined by the scale. I have problems "chopping" the data, right now I'm just selecting a part (2000 points) of a long array to show on the graph, and which part being selected depends on the maximum x scale and the time stamp of every 1000 data points.

[James.M]

... Are you talking about the Full array (of DBLs) and the case structure below it that uses a local variable?

Those case structures don't need to be in parallel. Just wire the data directly from the top case structure to the bottom one.

[Xinghe.W]

In freeze condition, I don't need to update the Full array any more so the first case only run once, while the second case will use the full array from the first case and play with it.

[Xinghe.W]

I decimated the data every 10, 100, 1000 points and keep them in three other arrays with maximum size of 10000. I have no problem running the graph in "running mode", but in "frozen mode", I want to check the data 2 hours ago, and here is all the headaches come from. Will DVR solve this problem? I need to find a way to qucikly locate a length of subarray of a long array, or chop the long array in small pieces which I feel will be harder to handle.