Problem reading and ploting large lvm files

[jcannon]

Hi everyone,

I have some large .lvm files that I need to process offline; however, the files are quite large and I am regulalry getting "out of memory" messages (7 channels sampled at 4k for 15 minutes or so, maybe longer)... I have managed to reduce the rate this message occurs by first converting the lvm files to tdms, then potting the tdms data, but I still get the "out of memory" error message on somewhat regularly.. .I also down sample the data back to 2k, but that dont help a great deal.

Any suggestions on how I can handle this data? I have read a number of online resources related to managing large data sets (e.g. http://www.ni.com/white-paper/3625/en/) but I am not sure how to implement these suggestions.

Basically, I want to view the content of the ensure file... Then use queues to extract data subsets as selected into another while loop that will handle the analysis/processing (producer/consumer)... I do this regularly for smaller files; so the issue is mainly how to managing the large files... Decimating the data for the intial whole data plot may not work as I have spikes 10ms in width in some channels that I need to see in the main plot.

Any help would be appreciated.

Many thanks,

Jack

[STASHkal]

jcannon,  I did some quick math and I dont think you should be reaching the memory limit of LabVIEW.  However, it is possible that you are running out of contiguous memory on your computer while the program is running.  See this for a quick breif about contiguous memory.  

If I were you I would try to reduce the number of times LabVIEW coppies information in memory.  Use show buffer allocations to find out where in your code you are making coppies of memory.  

best of luck!