Not enough memory - DTbl Uncompress Digital

[engwei]

Hi All,

I m running a quite large of waveform file (5M lines) and a "Not enough memory" error is prompted out. Here I attached the screenshot of the error. 

I m using HSDIO 6552, for digital data generation and acquisition purpose. 

Please advise if any of you have any idea about it. Thanks a lot.

[engwei]

Here's some addition. The error is actually happen when it try to append the waveform, by Append Digital Signals VI.

Any idea to have this VI append bigger waveform ? ..

Thanks in advance. 

[JaredW]

The waveform datatype is a memory hog. Internally, the datatype is a 2d array of U8s, where one dimension represents signals and the other represents samples. In your case for just one device, 5 million samples accross 20 channels would mean a waveform that requires 5M*22 bytes or 110MBs of RAM. Now couple that to the fact that you are acquiring from multiple 6552s and that LabVIEW inherently is going to make additional datacopies, it's easy to see you you could run out of memory.

Now, I think we may be able to remove the error at the Uncompress Digital location, and hopefully this will help you alot. There is a known issue in the Append Digital Signals VI where it always tries to compress your data after it does the append. I don't want to discuss what compressed means, but it is an expensive operation for little benefit. You need to go into the append digital signals VI and physically remove the call to do the compression. Double click on the Append Digital Signals VI in your code and go to the block diagram. You will see a call to another subVI called Append Digital Signals DTbl, go into this subVI. Modify the VI by removing the last VI in the call chain, it is called Compress Digital Wavform. I added a screenshot of the VI and how to modify it.

Removing the call to compress the waveform will make sure that the uncompress code is not called after the first iteration of your for loop. This act alone will reduce the total number of data copies being made and will likely fix your issue. Repost if it doesn't.

Message Edited by JaredW on 12-09-2008 09:16 AM

Message Edited by JaredW on 12-09-2008 09:18 AM

Message Edited by JaredW on 12-09-2008 09:19 AM

[engwei]

Thanks a lot, JaredW. I will tried it once I get back to office. Will update here asap. Thanks again.

[engwei]

Hi, I have already removed it but the error message still prompting out..

Thanks for your time and pls help if you guys have idea about it 

[engwei]

Hi,

Anybody there faced this proboem before ? So far I still cannot append larger waveform, Please help, thanks. 

[JaredW]

My solution was kind of a shot in the dark anyway. I was hopeful that removing the overhead of the unnecessary compress/decompress code that enough data copies would be reduced and your code would start working, but that doesn't appear to be the case.

I think you only have a few options from here:

1) Don't acquire so much data at one time from your device, rather stream it in small chunks to a file. I believe there are examples of how to do this in HSDIO. The fetch Waveform VI in HSDIO lets you read smaller chunks of data from your disk in a for loop to help with memory management.

2) Read your data as binary rather than as Digital Waveforms. This will drastically reduce the memory needs. You may still have to stream to disk if your available memory on your PC is small.

Try one of the above approaches and see if they help.

You may also consider reading the following on NI Dev Zone:

Managing Large Data Sets in LabVIEW

Message Edited by JaredW on 12-15-2008 02:34 PM

[engwei]

Thanks for the advise. Maybe I have to save them to the disk first then retrieve part of them to display.