LabVIEW

cancel
Showing results for 
Search instead for 
Did you mean: 

Memory problem with loading a csv file and displaying 2 xy graphs

Hi there, i'm having some memory issues with this little program.

What i'm trying to do is reading a .csv file of 215 mb (6 million lines more or less), extracting the x-y values as 1d array and displaying them in 2 xy graphs (vi attacked).

I've noticed that this process eats from 1.6 to 2 gb of ram and the 2 x-y graphs, as soon as they are loaded (2 minutes more or less) are really realy slow to move with the scrollbar.

My question is: Is there a way for use less memory resources and make the graphs move smoother ?

Thanks in advance,

 

Ierman Gert

 

0 Kudos
Message 1 of 5
(2,467 Views)
Only way to use less memory is to use fewer data points.  Since I assume you do not want that, I can't think of anything else that would help in a substantial way.   Maybe getting more memory for the PC?
-Matt Bradley

************ kudos always appreciated, but only when deserved **************************




0 Kudos
Message 2 of 5
(2,466 Views)

Thanks for the reply Matt.

I've read somewhere that splitting the array while loading it, can reduce the memory usage, but i have no idea on how to implement it. 

 

0 Kudos
Message 3 of 5
(2,459 Views)

Hi lerman,

 

how many datapoints do you need to handle? How many do you display on the graphs?

 

Some notes:

- Each graph has its own data buffer. So all data wired to the graph will be buffered again in memory. When wiring a (big) 1d array to the graph a copy will be made in memory. And you mentioned 2 graphs...

- load the array in parts: read a number of lines, parse them to arrays as before (maybe using "spreadsheet string to array"?), finally append the parts to build the big array (may lead to memory problems too).

- avoid datacopies when handling big arrays. You can show buffer creation using menu->tools->advanced->show buffer allocation

- use SGL instead of DBL when possible...

Message Edited by GerdW on 05-12-2009 10:02 PM
Best regards,
GerdW


using LV2016/2019/2021 on Win10/11+cRIO, TestStand2016/2019
0 Kudos
Message 4 of 5
(2,452 Views)

Do you really want to try to show 6 million or so data points on a graph at once?  You can only show so much do to the limited number of on screen pixels.  Initialize an  array to a much more reasonable number of points like a couple thousand.  Read the data in from the file in portions and do a replace subset on once out of very so many lines  (like one in a thousand).  Now you'll have a smaller array you can pass to the graphs.

0 Kudos
Message 5 of 5
(2,446 Views)