Excessive memory usage causing non-sequential processing?

[grahamwebb]

I have a vi collecting data from a TCP port, doing some processing then displaying ensemble curves to a few graphs. At random points after a few minutes of data collection LabVIEW memory usage increases up to the region of 1.5 GB! Then causes problems with data collection and processing.  

I'm using 33-bit LabVIEW 2011, on a standard intel dual core desktop PC running Windows XP, SP3.

Data being handled are int and double numbers collected at 120 Hz and the collection / processing functions are split across a producer-consumer architecture. The code works sequentially under normal conditions, but when the high memory usage occurs data are queued up and the subsequent processing falls out of sync.

I can reduce the problem by updating the UI less frequently, but this doesn't explain why the processing occurs non sequentially under load.

Any thoughts?!  

Many thanks

Graham

[crossrulz]

Are you increasing the size of arrays and/or strings on the UI.  I have found that displaying long strings slows down programs quite a bit and could account for the large memory use.  But it's really hard to say without seeing your code.

[altenbach]

That entire thing really depends on the actual code implementations. Poor coding can easily increase the memory use by a factor of 10 or more.

You only mention 120Hz. We also need to know what kind of processing you are doing and on how much data at a time is processed (simple offset correction? 2D convolutions?). What are "ensemble curves"? How many graphs are "a few"? What consumes the memory? Are there growing data structures, for example? How are you growing them? Are the large data structures accessed via local variables or value property nodes? Are the data structures constantly changing in size? Any charts with the history size set very high? XY graphs with millions of points? Complicated graphs (anutoscaling ON , fancy point and line styles, overlapping graphs, etc.)

Do you have the exact model of your CPU ("dual core" is typically used for low performance chips, while e.g. "core 2 duo" are much better chips.). How much memory do you have?

[grahamwebb]

Ok I take your point! I'll dig a bit deeper and come back with more specific questions.

Thanks

[altenbach]

Most likely you have a synergistic effect between your queue and the processing operations. If the processing is inefficient, the queue will grow, requiring new memory allocations, slowing down the code even more. That entire thing will exponentially snowball into an avalance. 😄