From Friday, April 19th (11:00 PM CDT) through Saturday, April 20th (2:00 PM CDT), 2024, ni.com will undergo system upgrades that may result in temporary service interruption.

We appreciate your patience as we improve our online experience.

LabVIEW

cancel
Showing results for 
Search instead for 
Did you mean: 

Memory problem with loading a csv file and displaying 2 xy graphs

Hi there, i'm having some memory issues with this little program.

What i'm trying to do is reading a .csv file of 215 mb (6 million lines more or less), extracting the x-y values as 1d array and displaying them in 2 xy graphs (vi attacked).

I've noticed that this process eats from 1.6 to 2 gb of ram and the 2 x-y graphs, as soon as they are loaded (2 minutes more or less) are really realy slow to move with the scrollbar.

My question is: Is there a way for use less memory resources and make the graphs move smoother ?

Thanks in advance,

 

Ierman Gert

 

0 Kudos
Message 1 of 5
(2,440 Views)
Only way to use less memory is to use fewer data points.  Since I assume you do not want that, I can't think of anything else that would help in a substantial way.   Maybe getting more memory for the PC?
-Matt Bradley

************ kudos always appreciated, but only when deserved **************************




0 Kudos
Message 2 of 5
(2,439 Views)

Thanks for the reply Matt.

I've read somewhere that splitting the array while loading it, can reduce the memory usage, but i have no idea on how to implement it. 

 

0 Kudos
Message 3 of 5
(2,432 Views)

Hi lerman,

 

how many datapoints do you need to handle? How many do you display on the graphs?

 

Some notes:

- Each graph has its own data buffer. So all data wired to the graph will be buffered again in memory. When wiring a (big) 1d array to the graph a copy will be made in memory. And you mentioned 2 graphs...

- load the array in parts: read a number of lines, parse them to arrays as before (maybe using "spreadsheet string to array"?), finally append the parts to build the big array (may lead to memory problems too).

- avoid datacopies when handling big arrays. You can show buffer creation using menu->tools->advanced->show buffer allocation

- use SGL instead of DBL when possible...

Message Edited by GerdW on 05-12-2009 10:02 PM
Best regards,
GerdW


using LV2016/2019/2021 on Win10/11+cRIO, TestStand2016/2019
0 Kudos
Message 4 of 5
(2,425 Views)

Do you really want to try to show 6 million or so data points on a graph at once?  You can only show so much do to the limited number of on screen pixels.  Initialize an  array to a much more reasonable number of points like a couple thousand.  Read the data in from the file in portions and do a replace subset on once out of very so many lines  (like one in a thousand).  Now you'll have a smaller array you can pass to the graphs.

0 Kudos
Message 5 of 5
(2,419 Views)