For the program I am writing, I need to be able to aggregate a certain amount of data before writing it to disk (due to the legacy format I am writing it in). I am planning to use an array to do this, but I need to know how much the array can hold before it is full.
On paper, the array seems to have a limit of 2,147,483,647 items (derived from the long-int fed into the 'dimension size' parameter of the "initialize array" function). However, when I tried to create and initialize an array of this size (using "initialize array", nothing happened. There were no errors, and the array was not created. When I tried creating a smaller (but still very large) array, I got the failure "fpsane.cpp," line 269.
So: Is there a maximum array size? If not, is there a way I can tell at runtime what the largest array size possible is?
You will likely be limited by the largest block of free memory currently available on your system rather than by LabVIEW. You will also need to be very careful to prevent LabVIEW to make unncessary copies of the array.
What does actually prevent you to write data on disk as it goes? Or to aggregate an amount of data in a file instead of in an array? Files are random access too e.g you can access any location of a file as you can index an array (well at least 2GB...). For large amount of memory, the OS will manage this with huge disk swapping anyway.
General question: To minimize making extra copies of an array (if an application/VI needed such), would it be better to receive/send the array data via Reference (address/pointer) versus an actual copy of the array variable? I believe wiring in/out an array as well as using a reference for array values is better than making local variable copies. Doug
As far as possible, this is already done transparently when you pass an array to and from a sub-VI.
I'm not sure what happens with locals if the panel is open. If the panel is closed, then I'd assume pass-by-reference is also taking place. The reason for my uncertainty is that panel items maintain their own copies of the data if the panel is open. But certainly wiring direct is always faster than using a local..
Doug (CapeFl) wrote in message news:email@example.com... > General question: To minimize making extra copies of an array (if an > application/VI needed such), would it be better to receive/send the > array data via Reference (address/pointer) versus an actual copy of > the array vari able? > I believe wiring in/out an array as well as using a reference for > array values is better than making local variable copies. > Doug
When using a reference to modify an array, you have to read the "Value" property, an operation which makes a copy of the array the same way as for reading a local or a global. You can't modify a single array element using an array reference without reading the whole array. In LabVIEW, references are references to LabVIEW objects (controls,indicators), not data pointers. They are mainly intended to manipulate cosmetic/appearence properties of the object rather than its data. LabVIEW is optimized for dataflow to pass data from place to place in your diagram. As Craig pointed it out, passing an array to a subVI is transparently done with a data pointer, provided that this data does not need to be displayed e.g. its front panel is closed. LabVI EW makes data copies for display purposes.
"Doug (CapeFl)" a ecrit dans le message news: firstname.lastname@example.org... > General question: To minimize making extra copies of an array (if an > application/VI needed such), would it be better to receive/send the > array data via Reference (address/pointer) versus an actual copy of > the array variable? > I believe wiring in/out an array as well as using a reference for > array values is better than making local variable copies. > Doug
I wanted to avoid writing the data to the disk in real-time for performance issues. I found that when I was writing data it would make the waveform display very choppy, and put me further behind in the buffer. So, I was hoping to just keep the info in memory until the run was finished (around 5 minutes/6 megabytes).
I will add my answer to the ones already posted, the array size is limited by your computer's memory... although I will tell you that this kind of huge size arrays are not a good programming technique, because even though your computer or LabVIEW may be able to handle such array, your performance and system memory will be affected. I've been looking at some application notes and your array size is "acceptable" for how LabVIEW manages to know the size of an array, but I will really recommend you to decide a solution different than that.
The answers posted already are very good, as an added suggestion think about using a database and accessing it from LabVIEW, instead of handling a so large array.
Good luck and thanks for the info from the other person s very precise answers...
Nestor Sanchez Applications Engineer National Instruments
.... > So: Is there a maximum array size? If not, is there a way I can tell > at runtime what the largest array size possible is? >
As was already posted, your array will most likely be limited by the amount of memory that your computer can address. The next generation of OSes will have 64 bit addressing, but for the moment a pointer in a process is only 32 bits, which means that it is quite difficult to have an array with more than 2^32 elements in memory even if they are only a byte apiece.
If your array elements are 256 bytes, then you can expect the OS to start throwing up major difficulties at around 2^24.
As you noted earlier, the size element that is a part of each array and string in LV is a signed int32 and will impose one limit on an array size. In addition, some of the nodes in LV, like the array initialize node will not operate if wired a negative number, and it should stop execution with an out of memory dialog when run with really large numbers. In these situations, LV will ask the OS for the appropriate amount of memory, but if the OS doesn't have RAM or disk swap space, it will fail to allocate the memory for LV, so LV will pass on the bad news. It shouldn't lead to a crash or anything too bad, but it will stop the program as many of the fundamental nodes have no way to return this error to the diagram.
It sounds like for your application, you want to keep memory in memory while being acquired. You don't state how much data that is, but this is quite common. Preallocate an array in a LV two style global. In your acquisition loop you can call the LV2 style global to store the elements one at a time, or more likely a buffer at a time. In the LV2 style global, use replace subset, or if not using LV6, loop and replace element and write the array back into the shift register. After your data acquisition is complete, call the global with a command to write the global out to disk, and it can loop through producing the file or files for you.
Like I said, this is not uncommon, and it will not make lots of copies of your data since only the LV2 style global subVI even has all of the data. The buffer will be preallocated to avoid failing partway through and to avoid introducing timing delays as the OS memory manager looks to move the block around as it is resized. Your major concerns should be the amount to preallocate, which is determined roughly by how long the experement will run.
If for some reason you do not have an upper limit on the length, another approach would be to make more than one array stored by the global. Init the global to have and array or cluster of arrays. The initial size is one array of cluster of 10M lets say. Once the 10M is used up, make another element in the array rather than growing the first one. It may not sound that much different, but it is quite different to the OS as LV keeps the arrays in contiguous memory.