Not enough memory to complete this operation, a 30M data points waveform

[yeah]

Hello all, is it possible to update the data with 30M data points to the graph? is there a limit for the data points in the graph?

the VIs are from below link, 

Tektronix MSO44 Oscilloscope - Instrument Driver - National Instruments

sometimes I can store the data for 10M data points X 3 Channels(without updating the graph) . but when I try to open it by TDMS read, I have disabled the follwing diagrams.it still shows that memory is full.  

one channel with 10M data points seems OK,  but record length of the equipment( tektronix scope) is 31.25M X4 channels. I'm not sure if it is possible to do so. and we would like to save them in one file because they are the data of just one screen/one time captured.

[Intaris · Accepted solution]

Are you using 32-bit or 64-bit LabVIEW?

Note that 30Mx3 Waveforms of DBL Data will require 30Mx3x8 Bytes of memory (560MB). This will have to be contiguous AFAIK, meaning allocated in a single memory chunk. Depending on your memory available, this can be a problem. It's not that you don't have 560MB available, it just might not be available as a single chunk.

Can't you read it piece by piece? You're surely not going to display 90M datapoints (A Full HD monitor only has 2.3M Pixels). Even 4k UHD has "only" 8.3M Pixels. So if you're going to be further processing the data, it is generally wise to try to do it in chinks if your requirements allow.

[yeah]

Hello Intaris,

Thanks for the reply. I'm using a 32-bit LabVIEW.

The memory occupied by the program goes up to about 2GB when I pull the data from the equipment. 

the VIs for this equipment in above link (first picture) are put in a for loop to generate the waveform array for selected channels. don't know why the memory goes so high.

I saw a program developed by Python could draw the graph with a 30M datapoints. it's slowly but can finish finally. it didn't try 30MX3 too.

But in my LabVIEW program, it seems really hard to draw the graph with 30M datapoints. So I'm not sure if there's something wrong with my program. but what I have done was just transfer the waveform array from the VI to the graph of front pannel in main VI.

And for TDMS read, 10M*3 channels makes the memory goes up to 3GB and the program stops.

10M with 1 channel seems working, but the occupied memory is also about 1.5GB. Not sure if other things I can do for TDMS read. I don't understand why I can save the data but could not read.

And maybe I have to cut the data into pieces. but don't know if I will meet the similar issue since the data pull from the equipment include a DBL DATA Array with 31.25M elements at a time.

For display the graph, it is required to generate the picture and enable to zoom in if can not update in the LabVIEW graph, but I'm still confused that if it is limited by the resolution of screen.

[altenbach]

As has been said, you are sending gigantic amounts of data to the front panel (overtaxing the UI thread!), which is completely unreasonable because your monitor only has a few thousand horizontal pixels.

Arrays need to be contiguous in memory, so you will run out of contiguous space much more quickly than total free memory. You also always have multiple copies of the data in memory, meaning gigabytes! (the wire data, the transfer buffer, the data in the indicator, etc.)

 

We can't really help by looking at some truncated code pictures. You have an array of channels determining the number of iterations of the FOR loop, but you also have a case structure that outputs blank data (via "use default if unwired" tunnel). Since you are autoindexing on channels, the maximum time restrictions means that only some of the channels might even get processed. This design seems completely wrong! What is the size of the autoidenxing arrays? What are the values of the various controls?

Do you know that your "maximum time" calculation is wrong?, the ticker before the FOR loop gets read immediately and in parallel to the sequence structure, so subtracting the elapsed time of the initialization is wrong. Execution order is determined by the data dependency, not position on the diagram. Typical beginner mistakes.

You might also read e.g. this post (and the entire thread).

[yeah]

Hello altenbach,

Thanks for the information, I have been searching for several days but didn't see the thread in your reply. I didn't pay much attendtion on the VI download from NI, I never doubt about it. but it is ture that I am a program beginner.

the for loops depend on the selected channels, for this model, 4 channels at the most. and the time seems compare with the timeout setting, it include many VIs, so I put the link in my first post.  

 let me have a think how to persuade the demander, they also what to combine all the screens together, even there are more than 1000 screens......

But I think it meaningful to save the data for one screen and be able to do further analysis.  is it save data to pieces will be the best solution?

[yeah]

Hello Altenbach and all, I have another 2 question? could you help to give some advice?

[yeah]

missed the picture,try it again.

[yeah]

Sorry, I don't know how to edit the replay and fail to upload the picture. Question 1 : which is the best way to create an array with elements from 0 to 20000000, currently I'm using FOR loop. Question 2:which is the best way to transfer a DBL data array to string array and build 2D array according to the DBL data array quantity. currently I'm using "number to fractional string" to transfer the DBL Array, and then, still using a For loop to buid the 2D array. Is there best method to save memory useage?

[Yamaeda]

---

@yeah wrote:
Sorry, I don't know how to edit the replay and fail to upload the picture. Question 1 : which is the best way to create an array with elements from 0 to 20000000, currently I'm using FOR loop. Question 2:which is the best way to transfer a DBL data array to string array and build 2D array according to the DBL data array quantity. currently I'm using "number to fractional string" to transfer the DBL Array, and then, still using a For loop to buid the 2D array. Is there best method to save memory useage?

---

1. That's good enough. Maybe the Ramp function would be easier, or why do you need it at all? Could it be part of your other question so you only handle 1 at a time?

2. Sorry, i did not understand that description, do you have a VI?

[yeah]

Hi Yamaeda, I tried to attach a VI but also fail, I pulled the attachment to the area such as doing with a new post. but it didn't work. click browse file was the same. It is not suitable to create a new post, how can I pass the VI/picture to you?