Memory considerations while using the Queue and Functional Global techniques

[Ben]

---

nathand wrote:

...  However, call-by-reference doesn't know if terminals are required or recommended and can't take advantage of this optimization....

---

Hi nathand,

That was thread where nobody believed me when I stated there was over-head in handling the "extra" terminals on an icon connector. Arostos Queues never did get back to us on that Q.

I just did a sanity check on myself and the call-by-reference will break the VI containing it if an input on the defined icon connector is required and nothing is wired to that terminal. Change the terminal to optional, re-link to the icon connector and the Call-by_reference is now happy.

Just trying to help,

Ben

[amaglio]

Great, thanks Ben. I am going to begin digging into that thread right now. Thank you for pointing me to the right place. 

Thanks for your help nathand. You might be the guy to ask this question. If an output of a subVI that is being used as functional global is not used (it is not connected to anything) is there a way in general to know if a buffer will be allocated for this output?

[nathand]

---

Ben wrote:

Hi nathand,

That was thread where nobody believed me when I stated there was over-head in handling the "extra" terminals on an icon connector. Arostos Queues never did get back to us on that Q.

I just did a sanity check on myself and the call-by-reference will break the VI containing it if an input on the defined icon connector is required and nothing is wired to that terminal. Change the terminal to optional, re-link to the icon connector and the Call-by_reference is now happy.

Just trying to help,

Ben

---

Hi Ben - oops, thanks for pointing that out - I should have verified about the required versus recommended terminals first.

However, I did test the second part of my statement and I'm certain that much is correct - a connector pane is not sufficient to indicate if a subVI can operate in-place, and even if a subVI can operate in-place on it will still do buffer allocations when called by reference.

[nathand]

---

maggs145 wrote:

Great, thanks Ben. I am going to begin digging into that thread right now. Thank you for pointing me to the right place. 

Thanks for your help nathand. You might be the guy to ask this question. If an output of a subVI that is being used as functional global is not used (it is not connected to anything) is there a way in general to know if a buffer will be allocated for this output?

---

The best way that I know of is to show buffer allocations on the calling VI.  When you look at buffer allocations for the functional global subVI you'll see that it always allocates buffers for all outputs, but when that same subVI is called as a subVI it may not make those some allocations.  That said, NI employees have occasionally explained that buffer allocations are not as simple as they seem, so you might see an allocation but it may not cost as much memory as you think.  For example, the "Array Subset" function may show a buffer allocation, but it's not actually copying the entire sub-array, it's just creating a new array reference that only refers to that portion of the original array.

Much as it's fun to try to optimize away all buffer allocations, it's rarely necessary unless you're dealing with large amounts of data - and I've never found a case where it was necessary for scalars.

[Ben]

---

nathand wrote:

...

The best way that I know of is to show buffer allocations on the calling VI.  When you look at buffer allocations for the functional global subVI you'll see that it always allocates buffers for all outputs, but when that same subVI is called as a subVI it may not make those some allocations.  That said, NI employees have occasionally explained that buffer allocations are not as simple as they seem, so you might see an allocation but it may not cost as much memory as you think.  For example, the "Array Subset" function may show a buffer allocation, but it's not actually copying the entire sub-array, it's just creating a new array reference that only refers to that portion of the original array.

Much as it's fun to try to optimize away all buffer allocations, it's rarely necessary unless you're dealing with large amounts of data - and I've never found a case where it was necessary for scalars.

---

Agreed!

Ben

[Ben]

---

nathand wrote:

...Hi Ben...

However, I did test the second part of my statement and I'm certain that much is correct - a connector pane is not sufficient to indicate if a subVI can operate in-place, and even if a subVI can operate in-place on it will still do buffer allocations when called by reference.

---

I just ran a benchmark on the "Clear-as-Mud" example code after modifying it to use Call-By-Reference (CBR) and it confirms your statement that buffer can not be re-used with CBR calls.

Now I remeber why I am a sucker for this type of question. I seldom come out of the other end without learning something.

Thank you nathand!

Ben 

[amaglio]

I just got through reading that "Clear a Mud" thread. The info there was very helpful, it is great to have resources like this on this board. I have another question:

Say if I have an array of strings that is 1,000,000 x 10 and I want to access the entire array to do some data processing on it. Is it more efficient to store this array using a the Functional Global technique or the Queue technique. If I am going to store it using the Functional Global technique and I am want to read all the data, is it more efficient (memory wise) to read chunks at a time, or should I just read the whole thing at once?

[nathand]

Functional Globals and Queues can do some of the same things, but they have different uses, and without knowing what your use is, it's impossible to say which will be more efficient.  From my reading of your description I'd suggest that you use a queue, unless the processing will be one of the functions of your functional global - but it really depends on what you're trying to achieve.  Once you've got data into a functional global I'm not sure how you get it out without making a copy (the data needs to remain intact in the shift register between calls), but if you do your work within the functional global then that's not an issue.  Ben is the functional global ("Action Engine") expert and might have a way around this.

What are you trying to do here?  Where are your strings coming from?  If you're reading from a file, and you don't need access to all elements in the array at once, you can be very efficient by reading only one chunk of the file at a time and operating on that, and you might not need a queue or a functional global at all.

[PhillipBrooks]

If you are processing a large log file, I have seen considerable performance improvements by using two loops and a queue. A fixed size queue is created, one loop reads chunks of data from the file, the other loop processes the elements in the queue.

I was processing multi-gigabyte files (performing a running least squares fit, checking for out of limits and recording max-min values). I realized a 5x improvement ( < 1 min. vs. 5+ minutes) using a queue. Performance will depend obviously upon the complexity of the calculations being performed and the amount of memory available for the queue.

[amaglio]

Nathand: My data is coming from a number crunching VI, but all the values are converted to string format becasue some of the variables and things are not numeric.... Looking back I think I would've kept the numeric values numeric because they use much less memory... But it is also a bit easier to program if everything is consistent. So the data is created by LabVIEW, so it is already in memory. I was looking at the differences between Queues and Functional Globals, and notices that when you want to access your data from the Queue you have to make a copy of all the data stored in the Queue (this seems to be a downside to the Queue, it disables you from being able to chunk the data). I am not sure about this bacause I am not too familiar with all of the details about all of the memory management, but using the Show Buffer Allocation tool when you want to get your data out it seems that a buffer is created and I am assumming it is the size of all the data being stored becasue that is what is available (there is pic showing this on page 1 of this thread). But if you use a Functional Global (FG) you could create the FG VI to be able to output subsets of the data at a time (this way the data could be chunked).

So here is what my application entails:

1. Data is first read from files by a Loading VI

2. The data that has been loaded is used by a number crunching VI, and in this step very large amounts of data is generated aside from the data that has been read from files. (all generated data is converted to string format) - At this point my program used to crash but I modified it a bit and got it beyond this step, but I fear that it may barely be passing by and if more data were to be loaded in it crash again. Should I be writing the data to files at this point??

3. The data is then sorted through and organized in a specific format to be able to be plotted. (This is the spot where my program is currently crashing)

I guess I can do a few things here:

1. I could write the data to files as it is being produced by the number crunching VI. (But will this make sense becasue I am going to need all of the data in the next step of my program, so I will just be reading it back again.) Also, the framework of my program does not support this method but could be implemented if it is without a doubt the best way to go.  Then with this mehtod I could access the data in chunks as I sort and organize it.

2. Store all of the data that is generated in the number crucnhing VI in a FG and either:

        i. Access it all at once during the sorting and organization

        ii. Access it at chunks at a time and sort and organize a chunk at a time

3. Store all of the data in a Queue and access all of the data at once to sort and organize because that seems to be the only option with the Queue if I am not mistaken.

Thanks so much for all your help