array limitation

[bmw318]

Hi,

I need to process more than 10E+6 data values. I want to put all these values into an array to process them but it seems to be that I can only put 10E+5 (100000) values into an array.

Where does this limitation come from and how can I solve this problem?

Anybody?

Thank you.

[Dennis_Knutson]

There is NOT a 100000 element limitation unless your pc does not have enough free and continuous memory available. Show your code, please.

[Jarek]

You can find the answer already on the forum (several similar threads). The problem is memory related - if there is a continuous block available (fragmentation) and if copies of the array are made - to avoid running out of memory.

One of the threads on the forum: http://forums.ni.com/t5/LabVIEW/what-is-the-maximum-size-of-an-array/td-p/25263/page/2

[P@Anand]

The limitation really depends on the size of the Physical memory you have in your PC. How do you say that you are not able to handle array 10E6? Are you getting Not enough memory error?

I would suggest even if you have a larger memory don't handle such a big array, which will endup in memory leak. Try to divide the array as chunks and perform the operation what you want.

[bmw318]

Hi,

thank you for your answer. I know that PC memory is directly related to the size of an array but I have 3GB of RAM and only Labview is open.

I need to reed a txt file containing a bit more than 1 million sampling points and put them into an array.

Sometimes  I need a 2D array.

Please see the code.

thank you very much.

[altenbach]

• What is the size of the file? (formatted files often take up much more space than actually needed)
• Why do you have an indicator for the string (data string)? That will cause significant extra memory.
• Only the number of elements count, It does not really matter if it is 1D or 2D. What is the final array size?
• Why don't you use "read from spreadsheet file" directly?
• Wy do you convert to a 2D string array first? You can use "spreadsheet string to array" to create the DBL array directly.
• You are misusing "spreadsheet string to array" anyway. don't use %f if you are converting to a string array. That makes no sense.

So overall, all these stepwise manupulations will create a lot of extra data copies in memory. The code could be streamlined significantly!

[bmw318]

Hi,

thank you for your answer.

you are completely right, I open too much indicators.

The file I have to open is a txt file of about 1MB.

I want to read the text file and put right away into an array of doubles, no string.

[crossrulz]

---

@bmw318 wrote:

Hi,

thank you for your answer.

you are completely right, I open too much indicators.

The file I have to open is a txt file of about 1MB.

I want to read the text file and put right away into an array of doubles, no string.

---

Read the entire file and use the Spreadsheet String To Array, just like you currently have.  But use a 2D array of Doubles instead of string for the data type.

[bmw318]

Hi,

I did so but no change. The max. number is still 10E5 and there are integers. I use %f and even %e as format string for the spreadsheet to string vi.

I read a TXT file.

Weird.

[P@Anand]

How do you say you are reading only 10E5 data? Can you attach the txt file here? What is the text file has only that much of data?