LabVIEW

cancel
Showing results for 
Search instead for 
Did you mean: 

Not enough memory to complete this operation, a 30M data points waveform

Solved!
Go to solution

@yeah wrote:
Sorry, I don't know how to edit the replay and fail to upload the picture. Question 1 : which is the best way to create an array with elements from 0 to 20000000, currently I'm using FOR loop. Question 2:which is the best way to transfer a DBL data array to string array and build 2D array according to the DBL data array quantity. currently I'm using "number to fractional string" to transfer the DBL Array, and then, still using a For loop to buid the 2D array. Is there best method to save memory useage?

2. Why do you need to many transformations? What are you trying to achieve? You'll create many data copies and this can easily be a reason for your issues.

DBL data array to 2D array? Do you just want to reshape from 1D to 2D? Use Reshape array, it is a 0 memory action (you just change the size parameter).

G# - Award winning reference based OOP for LV, for free! ADDQ VIPM Now on GitHub
"Only dead fish swim downstream" - "My life for Kudos!" - "Dumb people repeat old mistakes - smart ones create new ones."
Certified-LabVIEW-Developer
Message 11 of 19
(432 Views)

@yeah wrote:
Question 1 : which is the best way to create an array with elements from 0 to 20000000, currently I'm using FOR loop. 

What's the datatype you want and why do you even need that. Such an array can be described with a few words, no need to waste hundreds of megabytes.

 


@yeah wrote:
Question 2:which is the best way to transfer a DBL data array to string array and build 2D array according to the DBL data array quantity. currently I'm using "number to fractional string" to transfer the DBL Array, and then, still using a For loop to buid the 2D array. Is there best method to save memory useage?

How should the string array depend on the data? Is each element a formatted number (Wasteful and lossy) or just a DBL cast to string? A formatted string is useful when writing to a simple file that needs to be readable by humans and other software. Having that kind of data in memory is pointless, because you cannot really do math on strings. If this is for a table indicator, you can format the visible subset, which is a tiny fraction. What's the purpose and shape of the 2D array and how are you using a FOR loop to create it.

 

We clearly need much better specifications on what you are trying to do. We don't want to know how you are trying to do it, that might be completely misguided!

 

Message 12 of 19
(424 Views)

Hello altenbach,

It is just for the purpose of displaying the data in a table, I know it is useless but they insist that it should be there.

the 2D Array is just a 4 channels data. 1D Array is 1 channel data.

question1 is for the first column, 2D arrays are for column 2-5.

 

So cut the data into pieces seems to be the only way, if I don't do that, I even can't save data with such abnormal requirement.

0 Kudos
Message 13 of 19
(404 Views)
Solution
Accepted by topic author yeah

@yeah wrote:

Hello altenbach,

It is just for the purpose of displaying the data in a table, I know it is useless but they insist that it should be there.

the 2D Array is just a 4 channels data. 1D Array is 1 channel data.

question1 is for the first column, 2D arrays are for column 2-5.

 

So cut the data into pieces seems to be the only way, if I don't do that, I even can't save data with such abnormal requirement.


Are they the same size? If you, you can add the 1D as a 5th column and just have 1 2D array.

As for your table indicator (and assuming it's a hard requirement it's in string format) Array Subset is a good tool before using Format fractional string. Only the shown data should be as string, the rest in a shift register.

G# - Award winning reference based OOP for LV, for free! ADDQ VIPM Now on GitHub
"Only dead fish swim downstream" - "My life for Kudos!" - "Dumb people repeat old mistakes - smart ones create new ones."
Certified-LabVIEW-Developer
Message 14 of 19
(383 Views)

Hello Yamaeda,

After reset the browser, I was able to upload the attachment now. attached please find the VI, thanks.

0 Kudos
Message 15 of 19
(360 Views)

Hi yeah,

 


@yeah wrote:

After reset the browser, I was able to upload the attachment now. attached please find the VI


One problem is to convert an array of 10M floats into an array of 10M strings.

A different problem is InsertIntoArray to prepend a row to a 2D array: why don't you use BuildArray instead?

This might help with memory allocation: it only converts a single float into a string and build the string array at the loop border...

 

Do you really need to calculate this 10M string array for each waveform in your waveform array?

Why is that large FOR loop set to iterate just once?

Why don't you use autoindexing more often in your VI?

Do you really need to convert large waveforms into CSV files???

Best regards,
GerdW


using LV2016/2019/2021 on Win10/11+cRIO
Message 16 of 19
(354 Views)

Hi GerdW,

Thanks for the advice. I have updated the problem you mentioned.

 

Why is that large FOR loop set to iterate just once?

It is auto index before, I just tried to see if the loop 1 or 4 have difference.

 

Why don't you use autoindexing more often in your VI?

I saw somewhere that autoindex have defect on the memory, but I'm not sure about it.

so I try in this diagram.

 

Do you really need to convert large waveforms into CSV files???

I don't want to do this, but my boss insist that it is a customer requirement.

 

A different problem is InsertIntoArray to prepend a row to a 2D array: why don't you use BuildArray instead?

And I not sure about the memory useage difference between InsertIntoArray and BuildArray.  

0 Kudos
Message 17 of 19
(343 Views)

@yeah wrote:

 

I saw somewhere that autoindex have defect on the memory, but I'm not sure about it.

so I try in this diagram.


Vague conspiracies and you don't even give a link. The LabVIEW compiler is extremely powerful and will probably end up with similar code, no matter how convoluted you program it. Autoindexing on a FOR loop is extremely efficient, because the number of iterations can be fully determined and all outputs fully allocated at once.

 


@yeah wrote:

Do you really need to convert large waveforms into CSV files???

I don't want to do this, but my boss insist that it is a customer requirement.  


Then talk to him/her and explain how misguided this is.

 


@yeah wrote:

A different problem is InsertIntoArray to prepend a row to a 2D array: why don't you use BuildArray instead?

And I not sure about the memory useage difference between InsertIntoArray and BuildArray.  


"Build array" is simpler and can be used to prepend (not recommended) or append to an existing array. "Insert into array" is a much more complicated function designed for cases where you need to insert somewhere else (that's why we also have an index input!). Inserting one array into another array at a random location requires significantly more work because arrays are contiguous in memory. This means that all higher elements need to be shifted in memory and often the entire output needs to be copied to a new memory allocation because  you run out of space locally. If you use "insert into array" to append or prepend, you also need to know what happens if you leave the index disconnected, making the code less readable.

 

 

 

Message 18 of 19
(331 Views)

Hi Altenbach,

Thanks for the clarification. I never consider memory useage before, thank you all for your patience.

 

 

0 Kudos
Message 19 of 19
(280 Views)