More efficient memory management needed

[Faraclas]

I am trying to get increased memory performance. I understand that property nodes and local variables should be avoided giving preference to wiring. However, this leads to some fundamental problems for me and I would like to find out if there is a memory appropriate way to declare variables (directly in memory?) in LabVIEW.

In my application, I have to read several large data sets (>1-2MB) from file. Also, to save speed, when these data sets are filtered, with various filters, each one needs a copy saved to memory so it can be accessed rather than refiltering. the problem is that when I read the data from file, I have nowhere to store it except in a control (which I hide). This is the same for all of the other variables that I need; I wind up creating controls for all of them and then reading/writing values from/to the property node (which I understand is a no no.) I really need a way of declaring variables without using controls / indicators simply for data containers.

One of the primary reasons that I cannot wire everything throughout my block diagrams is that I am using the Event - Queue architecture. Basically, the top loop collects events from the user in an event structure (and there can be additional event collection loops from ActiveX controls also) and then places the appropriate action on the Queue to be exectuted. I find this much more efficient than using the event structure only because my Queue only requires the name (string) of the task to execute which is passed to a case structure. Otherwise, it is much to time consuming to have to declare and maintain all of the User defined events that could compensate for this. Of coarse, since I am only passing the name of the task too the Queue, ALL of the data must be available in the form of variables (or in my case the horrioble practice of a property node to a hidden control.)

A little long winded, but I hope I am being clear. I am hoping that there is some "variable" concept supported directly in LabVIEW that I can use to maximize my memory useage. The only thing that I can think of now is to write an entire library in C to dynamically manage, read, write variables. This is not so bad a thought, say for one array or so, but to do this for all the data type, custom clusters, etc. is a daunting task.

Thanks,
Elias Faraclas

[Ben]

What you are looking for is a functional global as mentioned by DF Gray in this thread.

http://forums.ni.com/ni/board/message?board.id=170&message.id=116854&jump=true

Functional globals use a shift register to store the data. A shift register is similar to a "static local" (?) in C.

The data persists from call to call but its scope is limited to the sub-VI.

Matering functional globals is required to do any serious coding in LV.

Ben

[altenbach]

Even easier than functional globals would be to use a local shift register in your main loop to hold the data. If you need to read or generate new data, feed it to the right shift terminal from the "read" case. In all other cases (or events), read the array from the left shift register to perform your operations, but also wire the unmodified array back to the shift register. 🙂

In your particular case, the shift register would be in your lower loop.

Message Edited by altenbach on 05-23-2005 09:39 AM

[Faraclas]

From the link that was posted, I think that using GOOP makes the most sense here. The persistant data combined with decoupling of data access / manipulation from the UI thread is exactly where I wanted to go. However, I couldn't find the GOOP Wizard for download for 7.1 (only up to 6.0 is listed in the ftp page).

ftp://ftp.ni.com/support/labview/vis/windows/

Does this matter, is there a new version or similar tool for 7.1, or can I use the old one?

[Ben]

Chase down the post by JanK.

http://forums.ni.com/ni/tracker?user.id=18069

He was involved in the development.

Ben

BTW: Let us know if GOOP and speed can be used in the same sentence. Lat time I looked they did not mix.

Message Edited by Ben on 05-23-2005 12:14 PM

[Faraclas]

Ok, I guess I missed that GOOP would be slow. Why is this or is there a link to some data that says it is?

[Ben]

Large does not always = fast.

It may have changed since the last time I looked.

When I last looked a GOOP close the data in the repositories are copied out of the repository, manipulated, and then returned to the repository.

For large data sets the CPU is doing a lot of work just to copy things around.

I may not fully understand this aspect of GOOP.

What I would really love to do is "attack the core" and add methods that will allow me to operate on the data in-place thereby eliminating the copy out/in work.

But if I understand correctly, attacking the core is not allowed.

The SR approach allows you to utilize LV operators that will just touch the single element you need to touch and the rest of the data is static.

The Dictionary Challenge (see this link)

http://www.ni.com/devzone/lvzone/codechallenge2_results.htm

showed that all of the top entries were variations on the LV2-global, functional-global, action engine approach.

Trying to help,

Ben

[Faraclas]

Thanks a lot for all of the comments! This is deffinately good information. I will research the LV2 approach you are mentioning. If I have enought time, I will try and compare this approach to the GOOP approach as well. Thanks again and I will post when I have the results.

[brian175]

Hello,
I had run into some of the performance issues with the NI GOOP toolkit mentioned in this thread and as a result I ended up developing a new GOOP framework that takes advantage of the performance improvements in the native LabVIEW queue in version 7.0 and later.

The dqGOOP online description is available at this link
http://www.dataact.com/dqGOOP.htm
The latest version is available for FREE download from this link
http://www.dataact.com/downloads.htm

The performance comparison charts show how dqGOOP compares against the other free GOOP toolkits. Since the actual execution times will vary depending on computer processor, operating system, RAM, etc..., the times have been normalized using the time for dqClass.

Legend	Description
dqClass	dqGOOP using default functions
dqClass (Locked)	dqGOOP using functions that allow locked data to be read
niClass	National Instruments GOOP Toolkit
openClass	OpenG GOOP Toolkit

[Faraclas]

Thanks for the link, I will check it out for sure!

I did an experiment yesterday and got some really strange results. The test program consists of
1) Create a 1MB signal (consisting of 3 tones) and write this to a variable
2) Write this to the display
3) run a filter (single Notch) on the data and store it to another variable
4) enter an event structure where the raw data, notch filter data can be toggled and low pass filtered with an adjustable cutoff frequency.

Not very complicated, but it is similar in concept to the way my main application handles the filtering so I thought it would be a good test.

I had three .vi's
1) the "Old School" system I have been using which is indicators on the block diagram to hold the variables and local vaiables placed accordingly to access them
2) GOOP approach using the Ni Class Wizard
3) LV2 (functional globals) with the LPF filtering performed on the raw / notch filterd data as a method

I observed that the GOOP approach was by far the slowest and used the most memory as well.
The most surprising observation was that I did not see any performance gain by using the LV2 style variables. I tested this (build an .exe) on a PIII (1GHz), P4-HT (2.4GHz), P4-HT (3GHz), AMD 4000+ all with 1GB ram and did not see any difference. (Between the styles of code, obviously there were differenced between the copmuters).

Does anyone have any thoughts?