Memory usage of signals

[s.h._tech]

Hello everyone,

I have a problem understanding the memory usage of LV.

I have an event driven producer-consumer machine, where one producer generates 6 signals. The waveform datatype needs double entries in the array. The generated signals are sent via a queue to the main interface where a preview is shown (the signal is not reduced). Here is the problem. How often does LV store the signals when transfered via queues? In the end I store the data in only the main producer ( I excluded every other machine that needs the data).

For example: I have 1000000 Samples x 8byte x 6 Signals = 48MB memory. But LV needs almost 500MB and even when I reduce the samples significantly it does not go down to the initial value again. So if I constantly change the signal and the old memory is not deleted I will run out of memory.

Is there a way to clearly store the values in one place and just point to the address similar to C or Java?

[s.h._tech]

I played around with only the signal generator (Basic_Function_Generator.vi of the Baselib) and there the memory problem is not as severe as in my state machines but still noticible high.

Doing the whole measurement with my c++ code (other card for signal generation) the memory  corresponds to the calculated values. It needs a base amount of memory, but when I change the size of the signal the memory corresponds to the size and back. LV just keeps demanding memory even if the signal is resetting.

I can (have to) live with small increase in memory demand, but 500MB for a 48MB Signal is just greedy

[Yamaeda]

---

@s.h._tech wrote:

For example: I have 1000000 Samples x 8byte x 6 Signals = 48MB memory. But LV needs almost 500MB and even when I reduce the samples significantly it does not go down to the initial value again. So if I constantly change the signal and the old memory is not deleted I will run out of memory.

Is there a way to clearly store the values in one place and just point to the address similar to C or Java?

---

Is this what is happening or are you just worried? Asking the OS for memory is expensive, so LV is reluctant to return memory but keep it for reuse in its internal memory manager. Usually this means that memory will grow for a while until there's enough with some spare to go around. The closes thing you'll get in LV is DVR, Data Value Reference, but as in C, if an array grows it'll need to reallocate memory and it'll leave a 'hole' behind increasing the memory footprint.

Another solution would be to preallocate a 50-100MB block/array and keep track of how much you're actually using yourself.

Usually this is moot though, as things will stabilize. If they don't, you're probably creating data copies which will, in fact, fill the memory, but that's not a fault of the memory manager, is it?

/Y

[mcduff]

Can you post any code?

If you are using queues to pass around data, then you might see this high memory usage. For example if you fill up a queue with 10 48MBy elements, then empty that queue, LabVIEW may not release the memory. (See this thread) I believe you can write an empty array(s) to the queue to force it to empty its memory.

Judging from your comments you may have data copies in your state machine; here use in-place structures to try and eliminate those.

mcduff

[altenbach]

---

@s.h._tech wrote:

The generated signals are sent via a queue to the main interface where a preview is shown (the signal is not reduced).

---

How do you show the preview? That's going to make extra data copies too.

---

@s.h._tech wrote:

Is there a way to clearly store the values in one place and just point to the address similar to C or Java?

---

LabVIEW has DVRs, But you might not be able to outsmart the memory manager.

You might get more detailed feedback on your problem by showing us some code. Me might discover some places where improvements are possible..

[s.h._tech]

Thanks for the replies and sorry for the late answer (company pc).

I tried to 'store' the data only in the 'generating' queue (and then send it to my main queue). If I choose a preview a second set of data will be generated and displayed => 2x48MB. 500MB is still too much.

@McDuff: this might be the solution, or at least the right approach to change my code.

I also suspect that the data is temporarly stored or not released properly. Since the memory usage is almost correct in the simplest way of generating the signal (see post 2) it has to be my state machines/ queue management.

Updates follow

P.S.: My code (machines) are very huge with classes, dependencies and so on. It would be a pain the trim it for a forum

[Yamaeda]

---

@s.h._tech wrote:

Thanks for the replies and sorry for the late answer (company pc).

I tried to 'store' the data only in the 'generating' queue (and then send it to my main queue). If I choose a preview a second set of data will be generated and displayed => 2x48MB. 500MB is still too much.

---

Are you looking at LVs memory usage in Task Manager? If so you'll have to remember that the LVRT is also loaded for a starting memory of some 300ish MB. Similar to how this tab on the forum uses 200 MB right now ...

/Y

[s.h._tech]

I can seperate the memory usage from LV (and the other stuff that is opened). The signal is generated via button so I can identify the portion of the memory used after signal generation.

UPDATE: I slimed my queue transfers and have been more careful with reallocation of unused dataarrays (flashback to C). Still there are some chunks of data that I am still missing. In the linked thread of @mcduff it is statet that LV not always reallocate data, even after flushing the queue and it has the be rewritten or manually emptied.

I will run the simplified version of my project a few days and monitor the memory usage. I added a randomizer to generate, preview the signals and if all goes well the memory is not constantly increasing.

[altenbach]

---

@s.h._tech wrote:

UPDATE: I slimed my queue transfers ...

---

According to Peter Venkman, that's not a pleasant experience..... 😄

[s.h._tech]

@altenbach: I stand by my comment