Why is virtual memory not freed or reused.

[Martin_]

Hello,

In my programs I convert huge textfiles (50mb). With some help of Labview manuals and some helpfull veterans I cleaned up by code. I use now fixed arrays in combination with replace element, I avoid local and global variables and try to use the follow wire programming technique. These tricks were indeed helpfull to speed up the system. But if you check the windows process manager you can see that the swapfile (virtual memory) is growing and growing (hundreds of Mb`s) after each converted textfile.

Of course it is possible to optimise my code further, but my question is: Why is the swapfile growing and only freed after closing the VI (which is resulting in a long wait periode (minutes) to give the memory back to the system. Is there not a way to re-use memoryblocks, or force the used memory block to be released after each file conversion, (and not wait and stagger the memory until you close the program?

to avoid these questions: my computer is containing a pentium 4 chip, 512Mb RAM and 40Gb HD, OS is MS windows Xp.

Martin.

(Do not simple blame the OS, LabView has to tell the OS when it is ready with the memory I think)

[falkpl]

There is a memory deallocation request vi which will flag your subvi to release memory when it stops execution.  It is "Advanced>>DataManipulation>>Request Deallocation", there is also an application note on memory management.

Paul

[falkpl]

Also is it possible to perform your data manipulation routines on smaller chunks of a file at a time?  You could see some speed increases when working with smaller data sets.

[Martin_]

Hello falkpl,

Thank you for your reply.

I still did not invest in a new version of LabView, is that the reason that I don`t have the deallocation vi you sugest? I still use 6.01.

For your other comment: smaller files will indeed speed up the program, but I prefer not to split the data in the files at this stage of debugging (only when I think there is no solution for freeing the memory, because the runtime of the VI it self is fast enough with the (first) large files)

Also when I cut the data in several files I still have the problem of staggering the data in the memory, with smaler files I have more files converted, but still the same amound of data is passed the memory. It can not be that I have to restart LabView each time I have converted a file to free the memory.

Is the sugested subroutine availible for 6.01?

Thanks,

Martin.

[falkpl]

LV 6 will cause a small problem since you cant explicitly force a garbage collection but you can still improve your code by using the best memory management strategies.  Try to preallocate arrays and replace elements, or use autoindexing in and out of loops.  Building arrays will cause huge problems when used unnecessarily, but I don't know if you did this. 

[Kevin_Price]

Here's a little trick I learned under LabVIEW RT 6.1 to force a vi to re-use the EXACT SAME memory space on subsequent runs.  I imagine it'd work similarly under regular LabVIEW 6.0  We were grabbing a memory chunk approx 200 MB in size.  This is the best way I know of to pre-allocate big chunks of memory.

1. We put a single-iteration loop around the entire vi and added an unitialized shift register to hold our big chunk of memory.

2. We used the "first call?" primitive to know when to pass the array through vs when to "allocate."

3. Instead of calling "initialize array" for allocation, we instead called "Reshape Array".  On the very first call of the day after launching LabVIEW, the "Reshape" is called on an input array that's empty (the default value of the unitialized shift register).  In such a case, real memory allocation takes place to create an array of the designated size/shape.  (I forget how we learned this, but I don't think it was in the help for "Reshape".)

Every subsequent run calls reshape with an input array that's already of the designated shape, and reshape returns immediately b/c it has nothing to do.   So we keep re-using the exact same RAM, thus avoiding fragmentation issues that used to arise before using this technique.  Note that part of what helped us was that the total array size was always the same constant value.

-Kevin P.

[carpat]

Hello,

I am currently developping an acquisition vi so as to acquire measurements using high frequencies (at least 200kHz up to 2MHz if possible).
For now, we have difficulties in retrieving all the data (indeed, using the buffer read.vi, we can see that scan backlog isn't 0, meaning the vi has missed a few values).

I am using a circular buffered acquisition with Cont&Acq (async acqu).vi so, normally, data writing inside the buffer (by the card) and data reading (by the cpu) must be more or less synchronised.
In order to make sure that we catch every data, i allocate a very large buffer (e.g. 20 000 000) and set "scans to read" at 400 000 scans.
Thus my vi parameters are:

acquisition frequency = 200 000kHz
buffer size = 20 000 000
scans to read = 400 000

If I understood, the vi will wait for 400 000 scans to be writen inside the buffer before transfering them into the RAM memory. knowing the acq frequency is 200 000, this means we will have to wait 2sec before the 400 000 scans are available.
BUT if the vi waits, this also means that it isn't supposed to miss any value??
however my scan backlog can reach 10 000 values missed or more.

Kevin Price i didn't understand everything in  your explanation about using firs call? and reshape. can you post an example vi so i can see where i am supposed to wire those functions?

I'm joining my vi so you can see how things go for me right now.

for information, my computer is using LV 6.1, pentium 3 and 128Mo of RAM memory. free HD space reaches 5Go, no pb.
(I must try to have a minimum data loss with such a computer since the goal is to be able to run the app on any cpu...)

[Jennifer_R]

Hi carpat,

You may be interested in pages 4-2 through 4-5 of the Traditional NI-DAQ User Manual.  There are sections on double-buffered input and the possibility of missing data. 

Kevin's method seems similar to the VI called One Copy in Memory.vi posted here.  That example uses the loop iteration terminal instead of First Call? to determine whether memory needs to be allocated for the array.