Sample Compression

[Norick_17]

Hello

I do have a Digitizer with following setup:

   - Sample Rate    = 20000000
   - Record Length = 12000000

This works fine so far and gives me a sampling time of ~600ms. Where the problem occurs is when I like to display the signal in a graph. Here I get the error message "run out of memory". Therefore I used the express vi "Sample compression to reduce the data by a factor 10'000 for example but this still does not work when I set the input of the "niScope Fetch WDT.vi" to the value "-1".

It works when I set this value to 1'000'000 for example but this is no solution because I do not the complete signal up to ~600ms! I also can not reduce the sampling rate of the digitizer because I will loose the accuracy of the measurement.

How can I solve this issue?

Thanks in advance

[GerdW]

Hi Norick,

handling 12M samples each 600ms requires ~100MB RAM per block (samples probably are using DBL datatype). Processing 100MB requires some time and easily fills up available memory...

Ideas:

- Use queues to implement a producer-consumer-scheme. That will help to distribute work on more than one CPU core...

- I suggest to use sampe blocks of ~200kS. That will give you data for each 10ms. Decimate data with a small loop picking every 10000th sample and calc the average of those 20 samples. Display that single point on a chart (with a decent history)...

Working with such high sample rates and those big blocks of data requires more than just some simple programming. There are guides on how to handle "big data", read them!

[Norick_17]

Hello GerdW

thanks for your reply. In fact there is no issue in the acquisition time of the digitizer. My problem is "only" to reduce these data and have it

displayed on a graph. Therefore the compensation express-vi is perfect but as I mentioned I still run out of memory.

I need to gather all data (~600ms) at once.

[CemY]

Hello Norick,

first of all you have to initialize and create a Session for NI-Scope (How-to-Tutorial: LabVIEW -> NI High-Speed Digitizers Help or Getting-Started with NI-SCOPE: http://www.ni.com/white-paper/3382/en/) and close it after measuring. Try to use then a "Property Node" (Block Diagram >> Functions >> Programming >> Application Control) after initializing/creating NI-Scope Session. Choose "Enable Records > Memory" (listed in "Horizontal") as property. After that you have to change the property to write (right click >> "Change All to Write"), then create a Boolsch Constant and set it TRUE.

I'm looking forward to hearing from you. Good Luck.

Cem Yalcin

National Instruments

Applications Engineering

[Norick_17]

Hello Cem

thanks for your reply. The scope is working without problem so far but the property node which you described is not available with my scope-5162. What is the purpose the "Enable Records > Memory" property and where can I find it?

best regards

[CemY]

Hello Norick,

you'll find it as attached. Try to set it before your "Sample Compression.vi".

Best regards

Cem Yalcin

National Instruments

Applications Engineering

[Norick_17]

Hello Cem

oh I have not seen it..!

Nevertheless I still get the "Out of memory" failure. Is there no easy way to fetch the data instead at once, in lets say 10 steps and compress the data for each loop?

[GerdW]

Hi Norick,

---

Is there no easy way to fetch the data instead at once, in lets say 10 steps and compress the data for each loop?

---

You did read my post #2, written several days before?

[Norick_17]

Hello Gerd

I did read your message but how do you partially read out the digitizer memory?

With the fetch.vi I can only get the data at once I suppose, without the possability to do it in a sequence. It would be a good solution if I could read out all data in let's say blocks of 10% of the whole set. Is there a way to do that maybe with an example?

thanks in advance

[DFGray · Accepted solution]

The NI-SCOPE examples contain sample code explaining how to fetch the data from the scope card in chunks (e.g. niScope EX Fetch In Chunks.vi).  As has been previously mentioned, there are tuturials on how to manage large data sets in LabVIEW, including one in the LabVIEW help and one here.  Between these three sources, you have sample code to fetch from your scope card in chunks, decimate for display, and display without running out of memory.  This is the algorithm and method used by the NI-SCOPE Soft Front Panel.