2D array and memory leaks

[Flo-w]

Hello everyone,

I'd rather start a new topic for that very problem.

I have been told that there could be a memory leak in my VI: displayed in this VI is the third loop of my P/C code. This should be the consummer one, where I obviously display my data and record them in a text file. the producer loop above should be timed at 500ms, hence that one too. As I want to record one set of data every minute, I store my data in an array for 120 iterations (~1 minute), then average them, and put them in an another array of strings (my case structure).

Some people told me that stacking them in this string array indefinitly could lead to a memory leak. I would like to tknow if there could exist some better way of doing what I aim to do, and if I can avoid that ?

Sorry if this is confusing, I am just trying to explain as clearly as possible what this loop does 🙂

Thank you !

Flo 

[Bob_Schor]

Flo,

     You really need to find a LabVIEW Guru and apprentice yourself for a week or two.  Failing that, at least purchase Peter Blume's "The LabVIEW Style Book" and read it, cover-to-cover, at least twice (I've read it at least 4 times).

• Never create a cluster of more than 2 elements with Bundle.  Whenever you are going to be doing serious work with Clusters, build one on a random Front Panel, right-click it and choose "Make TypeDef", right-click it again and choose "Open Type Def", clean up your TypeDef cluster (right-click and choose an AutoSizing option), save it in a Types folder in your Project (so you can find all of them -- you will probably have dozens of TypeDefs if you code properly), and then use them with Bundle by Name functions.
• Always code in the context of a LabVIEW Project.  Create sub-Folders for Types, Sub-VIs (you should have hundreds of them in a moderate size Project), Tests (where you put your Test code -- please write Test code), and other Folders to keep your code organized.
• Use some form of Version Control!  Subversion is pretty popular with LabVIEW Users (I use it, too).
• Your Front Panel should fit on a single Screen!  If you use dual monitors, consider running a Modal sub-VI in the other Monitor.  Do this for your own (programming) sanity.
• If the previous is true for the Front Panel, it is much more true for the Block Diagram!  My monitors are 1280 x 1024.  None of my Top Level routines have a Block Diagram that fills the monitor, and less than 1% of my sub-VIs are more than a screen-full (and even then, they are usually only 10% "too wide" or "too tall").
• Learn to use Arrays and For Loops intelligently to avoid having wires all over the place.
• Avoid the Dynamic Wire.  You have a perfectly good Array of Waveforms, but instead of treating it as an Array and either using Index Array or a For Loop, you cast it to a Dynamic Wire and Split Signals.  Booooo.
• To drastically shrink the size of Block Diagrams + make your code transparent to read, use sub-VIs with Icons (which could be text -- "Scale Input Data" or "Save to File").  Use a 4-2-2-4 connector pattern with Error In and Error Out on the lower corners.  Because this pattern is designed for (ideally) 3 inputs, use Clusters and Arrays to bundle data together.
• It really is a Good Idea to Write the Documentation First.  At least think about the data flowing through your VI, what is common, what can be bundled, etc.
• Now for Producer/Consumer and the Memory Leak question.  If there is a serious worry about Memory Leaks, then you aren't doing P/C correctly.  If you want to collect a minute's worth of data and then process it, here's one way to do it:
  • In the Producer, accumulate the data in an Array in a Shift Register (I'm assuming that you have a loop that runs at 2Hz and produces one point each iteration.
  • When the Array size reaches 120, put the array on the P/C Queue and clear the Shift Register in the Producer.  The Consumer now has a full minute to process the single array, which should be plenty of time.  Do not attempt to do any "timing" in the Consumer -- it will be automatically "clocked" by the Producer, as it sits idle until the Producer says "Here is some data" (once a minute), then it works for, say, 10 seconds, then goes idle again.

Bob Schor

[Flo-w]

Hi Bob,

Sorry for the lack of reply, but it was winter break 🙂

So I've quickly read through your advices, and I know I need to drastically clear my code. It will be done on time. While I clear it, let me give you the full version of it. 

I think I haven't been clear enough in my first post. So here is my concern again:

- the second loop produces the data every ~500ms and sends them to the 3rd (consummer) loop.

- the third loop receives those data every ~500ms I guess (cadenced by the 2nd loop). the data are procesed and put into an array where every values are going to be averaged every minute (every 120 iterations). I then release the memory of the array (case True of my case structure in the 3rd loop).

I feel like I am doing what you suggested, expect that I do it in the 3rd loop, not in the 2nd one.

My concern is that I'd like to display those data in an array of string, and few people noted that displaying those data through time may lead to eat all my RAM. 

Any suggestion to avoid that? is there a way to display those data without eating all my memory ?

PS: code hasen't been cleant yet, I will pass through all of your sugestion and do my best to make it more readible!

Cheers,

Flo

[Yamaeda]

---

@Flo-w wrote:

My concern is that I'd like to display those data in an array of string, and few people noted that displaying those data through time may lead to eat all my RAM. 

Any suggestion to avoid that? is there a way to display those data without eating all my memory ?

---

[Flo-w]

Yamaeda ? did you just qoted my post above ? 

[Bob_Schor]

Aha, now I see the problem/question.  Your "Consumer loop" wants to do two things -- save the data as they come in to a file (or "do one thing to each piece of data") and "Further process the data in two-minute segments".  These are separate processes that operate on two different time bases (where have we seen this before?), and can efficiently be handled by (drum roll ...) a Producer/Consumer Loop!

Consider a "Data Averaging" queue.  As the individual data points are dequeued by the Consumer, you first put them on a Data Averaging Queue (thus acting as a Producer) and also write them to a file (thus acting as a Consumer), formatting them for the file as you want.

In your fourth loop, the Consumer side of Data Averaging, you accumulate 2 minutes of data, average it, do whatever you need to do with it, then clear the array.  To prevent the problem of building the Array up one element at a time, which potentially means allocating more and more memory, you can pre-initialize the array to 120 elements.

Does this make sense to you?

Bob Schor

[Flo-w]

Actually, I am not saving the data as they come in that 3rd loop, I am just saving them into a 2D array, then I average those values when the count is at 120.

So I still don't get why I should need a 4th loop ?

Cheers,

Flo

[Yamaeda]

Hmm, seems my post was sent to the void ... I was basically saying that: While true in principle, is it a real problem? You can sent data to a Chart with a limited history length to limit memory used. Even if you keep it all in memory, will it fill up this side of 2049? 🙂

/Y

[Flo-w]

2049/Y ??

[Yamaeda]

From what i gathered you had a low sample rate, and with todays big memories it'd take years to fill it. 🙂