File Splitting utility, "LabVIEW out of memory" message....

[caunchman]

I put together a quick utility that takes a .txt file and splits it up into n-segments.  The problem I have is that when I feed a larger file into it it gives me a "Out of Memory" error.  What is a more efficient way todo this.  Attached you will find the VI.  It is written in 8.6 don't pick it apart to much, lol, it works great as long as the file is under 35 MB. 

[Ben]

Did you try to open the file as a binary and just read the bytes one at a time?

If not post an image so we can look without using LV.

Ben

[nathand]

Your code needs some work, unfortunately.  Do not use a sequence structure nor local variables (the local variable "array" containing the entire contents of the file is probably part of the problem).  You may also find that you need to convert the numeric data into strings in chunks rather than all at once in order to keep the memory usage reasonable.  Better, however, would be to rewrite your code so that it reads the file in chunks as text and then writes out the new file in increments (say a few lines at a time).  To do this you'll probably need to use file operations instead of express VIs, but you'll avoid the memory and computation involved in converting text to numbers and then back to text.

[caunchman]

Any examples of this I can reference? 

[nathand · Accepted solution]

Here's a 15-minute pass at reworking your VI.  It may solve the out of memory problem and it does demonstrate much better LabVIEW style.  You may be able to find examples of similar VIs by searching this forum.

[caunchman]

Thanks....where do I send the beer.....

[altenbach]

The original code was full of race conditions and unecessary code. For example inside the FOR loop there is no way to guarantee that the "numeric 2" terminal gets written before the local variable of it gets read, but the outcome will critically depend on the order of events.

You have many indicators for the sole purpose of making local variables. Indicators are used to show something to the user! Keep the data on the diagram (wire or shift register). All these locals, especially the 2D array causes huge extra data copies in memory.

Here's my quick take at a cleanup. There are still glaring erros for example the array min/max can give your problems depending on the shape of the array (portrait vs landscape).

Use some of my ideas and nathand's and modify as needed. There are probably better ways to do it from scratch.

[caunchman]

Good information.  I haven't really used LabView for anything  like this.  So please excuse my ignorance.  I always have trouble when it comes to working with issues like this in LV are there any good exercises I can do to help me better understand a better and more efficient way to program this? 

V/r,

Chris