Infinite waveform graph

[miyamky]

I have a monitoring tool (using LV v6.5), that outputs several 1D waveform charts.  Data is pumped into them continuously for hours.  Well, after about 14 hrs, all memory of the Windows PC was expended and the system hung.  Is there a way that I can limit how much data LabView keeps in memory for the chart (occasionally, we save the data to a file so I don't mind if I lose some realtime data)?

[johnsold]

Yes, there are ways to do that.  It depends on your program structure to some extent.  Can you post your program or at least the part which acquires, accumulates, displays, and saves your data?

As a general rule wiring more data points to a graph than the number of pixels in the graph's width is useless.  Any more data than that will be reduced in some way by LV or the OS.  You want to select the data points you want to display and send tose to the graph, not all the data you have accumulated.

Lynn

[miyamky]

No, I can't post any code (sorry, my environment prevents that from happening).

But picture just writing in a random number to a waveform chart over and over again.  I just add another point in every iteration of the loop.  It's a realtime view of all the data collected.  You can scroll backwards to see previous data.

I just want to limit how much of that data is saved in memory by LabView.

[johnsold]

A chart has a property to specify the length of the history saved.  Make that length the same as the width of the plot area in pixels.  That will limit how much data is stored in the chart.

Your main data is probably in an array or maybe a waveform? Outside your loop initialize an array to the size of the maximum amount of data you want to keep in memory. This may be much larger than the chart history length. Wire the initialized array to a shift register.  Inside the loop use Replace Array Subset to update the array when new data is acquired.  Use a second shift register to keep track of the index for the next replacement.  Before you start to overwrite the data save to a file.

If the total amount of data is larger than you can keep in memory at one time, then keep a summary for the scrolling display.  The summary could be an average of the data over some period, such as a minute.  That is 840 points in 14 hours or about 43000 points per month.  Easy to keep in memory. When the user scroll over that summary data and asks to see the details, read that part of the data back from the file for temporary display.

Lynn

[miyamky]

The data is only stored in the chart itself.  We just wire in a single entry every time through the loop.  The chart history property is set to 500k (there are about 8 charts in all).  So if I just lower the chart history (I don't want it too small because the amount of data that is sampled is high and it doesn't take long for the data to fall off of the displayed chart - need to scroll back to previous data for as much data as possible).  I believe 500k is the default since I don't remember (although I wasn't the one who originally coded this VI) ever setting that property.

[johnsold]

The default history length for a chart is 1024.  It might have been different in the past, but I doubt it was ever 500 k.  Eight charts is still only 4 million samples so you should not have memory problems unless you are making extra copies of the data.

Search the NI site for Managing Large Data Sets or something similar.  A white paper on handling large amounts of data is available.  The LV help also has useful information on memory management.  

The keys are to avoid re-allocating memory for arrays. This means that arrays (and strings) should not grow within the program.  The chart history is preallocated (I think) so it should not be a problem.

I know you said you cannot post your code, but without seeing the code, it will be difficult to give you specific assistance.

Lynn

[Spruce]

---

@johnsold wrote:

The keys are to avoid re-allocating memory for arrays. This means that arrays (and strings) should not grow within the program.  The chart history is preallocated (I think) so it should not be a problem.

---

The Labview help says that the chart history buffer is statically allocated which is why its size is not settable at runtime. However, I have looked at this in some detail recently as I have a lot of charts each containing a lot of plots in my program and have found that the total data space occupied by a chart does grow considerably as the program runs and more points are fed into it. It gets much bigger than seems reasonable, reaching something like 500 bytes for each data point in each plot. I would really like to get to the bottom of this as I too am running out of memory.

[Jeff-P]

Hi,

As a quick example of how to implement this using a preallocated array and a graph instead of a chart, see the following snippet which implements two different behaviors of displaying the data.

To change the direction of the Continuous Update Graph, change the -1 to a 1 on the rotate array. 

Regarding your statement about the memory usage of the waveform chart: the memory for the waveform chart is allocated at runtime , so it should not be growing in memory usage. If you can attach a sample VI where you see this behavior of the memory usage growing quickly (500 byes per point is very significant) I will look into whether it is a possible CAR. One other thing to consider when using waveform charts for large data sets is the following knowledge base about why you should limit your chart history size.

I hope that is helpful,

Regards,

[miyamky1]

THINK we found our problem, in the same VI where the data is being output to the chart, we open a file (to save the data periodically).  Turns out the file was never closed.  Going to add that to the code I inherited to see if that fixes our memory usage.