slow 2D string array

[ken@ngc]

I am working with a 92833 by 9 element 2-d array of strings.    The strings in a single column have the same length  but string length varies among different columns.  the maximum string length is 72 characters.  When I build this array as an output of nested for loops performance degrades for each successive run of the vi (the fourth execution takes three times longer than the first execution).  As an alternative I've tried allocating the array first and initializing it to the single longest string, then replace array elements, but this seems to make performance worse.  I wonder if there would be any benefit to intializing each array column individually to its string length.

Labview tends to become unresponsive for several minutes after running this vi several times.   Probably busy with memory management issues.  I have 1 Gbyte of memory.  Windows Task manager shows several 100 Mbytes of available physical memory.

[altenbach]

How are you building the array. Make sure you initialized any shift register or the data from previous runs would accumulate. Could you attach a simple example?

[ken@ngc]

I've attached a slimed down vi that shows the incremental increase in execution time.  This vi reads a large file which I can't easily provide.  The increase in execution time is associated with the "delete from array" vi.  WIthout it, execution time is roughly constant.   I've discovered "index array" (used to provide a subarray as output) performs much better.  Still curious what's wrong here.

[falkpl]

You might be able to see a speed up by using 9 1d arrays of  strings.  The indexing will be simpler.  Since you also have fixed array sizes, you can use byte arrays instead of strings but this can complicate the code.  I would suggest posting your code if possible, it is easiest for someone to fine if you are overlooking something.

[Dynamik]

Should group-selection be ordered highest to lowest?  For instance, with a 10 cell array, you can remove pos 9, then pos 0, but not 0 then 9...

[altenbach]

"Delete from array" should never be used in a loop, because it is very inefficient. For example your second loop contains 3 array buffer allocations!

What are the "header rows"? Just empty lines in the original file?

Could you provide an example data file that has only about 20-50 lines + header?

[altenbach]

OK, I made a few wild guesses about the data file and coded an alternative version (LabVIEW 7.1). It generates the same resuts as your code but seems simpler. Assumptions:

1. unknown number of empty lines.
2. First non-empty line contains headers, items seperated byt tabs.
3. Rest of file contains table, items seperated by tabs.

I don't understand the purpose of the "Group Selection Array, Lowest to highest". Is this something the user is supposed to be able to modify at runtime? You only take 8 elements, skipping element 0 and indexing nonexisting element 9. Remember that LabVIEW indices start with zero, thus I added a "-1". Once this is done, the convoluted FOR loop code simply acts as a "transpose" function. Maybe that's all you need.

Do you really need to transpose?

See if it is any faster with your real data. There are probably a few improvements possible. 🙂

[CoastalMaineBird]

As an alternative I've tried allocating the array first and initializing it to the single longest string, but this seems to make performance worse.

That would work if your array elements were fixed-size (i.e. I32). But strings are variable sized. Each string has to be allocated. When you initialize each element to a large string, you create more work because you alocate all those empty strings, and then de-allocate them and replace them with another allocated string.

[CoastalMaineBird]

Part of the problem is that you're using DELETE FROM ARRAY multiple times, when you only need to do it once.

If you want to delete the top row of an array, then delete the top row in the array, don't delete the elements one by one. Remember that DELETE FROM ARRAY will have to re-arrange memory to accomodate the shrinking array EVERY TIME.

Better yet, just extract rows 1-9999 - don't try to delete row 0.