High-Speed Digitizers

cancel
Showing results for 
Search instead for 
Did you mean: 

Increase recording time during continuous acquisition with NI PXIe-5160

Hello everyone,

 

Indeed, we have a NI PXIe-5160 oscilloscope card and with this card we can make the continuous acquisition at a speed of 625 Mech/s (in 8-bit mode) without having the overwritten error.

 

I used the example "Stream to disk" in order to realize the continuous acquisition and to make the recording of acquired data, the problem is that the memory RAM (8 Giga of RAM) is consumed in a few seconds (of 5 to 8 seconds) and then the program hangs .. Is this normal? Do you have any solutions ? knowing that the principle of loop producer/consumer is applied in this example and I put "False" to disable the "waveform graph" and not consume the RAM.

 

thank you in advance

 

Best regards
Ali

Message 1 of 8
(5,437 Views)

Hello Ali,

 

The reason why this could be happening is that the data are not written quick enough to the file, making the RAM filled faster than it is emptied.

I made a test, and while writing a binary file, Windows store datas onto the RAM before writing it in the file.

 

1) Maybe you could check that with the Ressource Monitor ?

 

2) Have you modified the writing process ?

 

3) Also, can you give us more precision about the configuration you are using ? (Chassis/Controller/OS/Module)

Which version of the NI PXIe-5160 are you using ? As it exists many versions of this module, each one having a specific configurations (number of channels, onboard memory size, maximum sampling frequency).

 

4) Another question that could help us : how did you configure the "NI Scope Fetch Binary 8" VI ?

The fetch parameter indicate how many samples you want to retrieve from the module onboard memory each time this function is called.

 

Have a nice day,

 

Regards

 

0 Kudos
Message 2 of 8
(5,414 Views)

Hello Salah,

 

I appreciate your help and thank you for your comments.


Yes, I understand, the speed of transfer from RAM to the disc is very low especially that is an HDD disc.

 

Here are my answers to your questions:

 

1) I do not really know what I can do with the resource monitor. However, I stopped the programs and even internet explorers to exploit the maximum possible from RAM memory...

 

2) Concerning the writing process, I tried once to write directly in decimal (instead of binary) but the result was worse (the RAM was consumed in less than one second)

 

3) Here are the characteristics of the equipment we have:
- Chasis NI PXIe-1071, PXI Express 3U
- NI PXIe-8840 Quad-Core (Intel Core I7), Windows 10 64-bit (Multi-Language) with a bandwidth of 8 GB/s .
- The NI PXIe-5160 Scope has a bandwidth of 500 MHz, sampling rate of 2.5 GS/s, 2 channel, 10 bit input resolution and a 2 GB on-board memory.

 

4) In order to be in 8-bit mode, I added the property node '' Binary Sample Width '' to the program entry in the diagram interface and gave it a value of 8. In the niScope Fetch Binary module, I set the "single waveform" to "1D I8"
I put the program in attachment

 

Have a nice day

Best regards
Ali

0 Kudos
Message 3 of 8
(5,366 Views)

Hello Ali,

 

>Yes, I understand, the speed of transfer from RAM to the disc is very low especially that is an HDD disc.

 

Ok then it is a HDD disk ! I'm even suprised that your application could last 8 sec.

I don't think that there are other solutions than upgrading your system, that is the price to achieve these writing speed.

These modules are dedicated to these situation, where you need to store large files, with a very high throughput

 

The other solution is to pre-process the datas, usually done by a FPGA module (FlexRIO modules) in order to extract only the useful informations, and send them to the controller disk.

 

To answer to your remarks :

 

1) I was simply telling you to check the ressource monitor in order to see how fast was filled the RAM.

 

2) Sure, writing to a text file is way more expensive in terms of data throughput, as each digit that will be written to the file weigh 8 bit (writing 255 will then weigh 24 bits rather than 255). Plus the writing process is more expensive in term of time (binary file can be accessed "randomly", text file can't).

 

3) Even if the controller have a 8GB/s bandwidth, the bottleneck here is still the disk. See above for the workaround.

 

Have a nice week-end,

 

Best regards

Salah

0 Kudos
Message 4 of 8
(5,318 Views)

Hello Salah,

 

Thanks for your time and your explanations.

 

Indeed, I have not yet been able to exploit the 7 or 8 seconds of continuous acquisitions since the system hangs once the RAM is full. And I believe that only a small part of the RAM is transferred to the disk. I might have to save the data in a table before transferring it to the disk in order to exploit the 7 seconds of acquisition...

 

I am indeed very interested in the module you propose: the HDD-8261 with SSD disks and a speed of 2 Go/s. I will discuss it with my colleagues. Thank you !!

 

The output signal of my system (signal to be sampled) has a frequency of 111 MHz and it is recommended to sample it at 4 or 5 times hence the interest of a high transfer speed.
For the moment, I am trying to switch my signal to base-band with an electric mixer, it could be a temporary solution to permit a low speed transfer...

 

Have a nice day 🙂

 

Best regards
Ali

0 Kudos
Message 5 of 8
(5,304 Views)

Hello Salah,

 

Actually, I need again your help Smiley Sad

 

Indeed, after a lot of tests on labview and RAM monitoring, I realized that the RAM was filling up fast because I put a high block size of the function "fetch" in the parameter "max points to fetch ".


I also noticed that simplifying the fetch loop makes it possible to increase speed of data recovery operation from the scope card to the RAM.

I leave below the simplified program. With this program and after setting the sample rate to 600 MS/s (always in 8-bit mode) and the "max points to fetch" to 100 MS, I could launch the program between 2 to 8 minutes !
For 2.2 minutes, for example, the data acquired by scope is 79 GB size (with 600 MS/s of sample rate) but the file recorded in the disk has a size of approximately 11 GB, which corresponds to 18 seconds, which is due to the low transfer speed to the disk (max. 100 MB/s). So my question is this: Is the data recorded correspond to the first 18 seconds of the acquired data? or there is lost data?
Do you have a way to check it? thanks

 

Best regards
Ali

0 Kudos
Message 6 of 8
(5,283 Views)

Hello Ali,

 

Thanks for this return, I did not thanks changing this parameter would allow the program to run longer.

Though, reading some documents on the subject it seems to make sense, as HDD are really terrible at writing small chunks of datas (random access to disk array) due to their mechanical structures

By chosing to write bigger chunks of data, the disk is writing in sequential mode, and don't have to jump area fro area

 

Spoiler

Coming back to the situation you're facing, I pretty sure that the data written are the first 18s, as you are using a FIFO, and datas are written in the disk in the order they are taken out of the FIFO.

 

You could maybe, if you have the hardware for, generate a signal with a known value changing over time, and check the recorded data after by reading the binary file and tracing it to a graph.

Or you can replace the binary write by a TDMS write, and use the TDMS Viewer to check the recorded signal.
TDMS file are binary file, the writing speed should be pretty the same.

 

Don't you have any error while running the code ?

 

Best Regards,

 

 

 

 

 

0 Kudos
Message 7 of 8
(5,252 Views)

Hello Salah,

 

Thanks again for your help and for the HDD link.

 

Indeed, I solved the problem of the code. This was because of the False / True structure in the producer / consumer loop. It is always necessary to enter the structure so that all datas emptied (dequeued) from the queue are sent to the disk. Otherwise, we lose data.

 

I modified the code in such a way as to impose the data recovery time and not block the system (when the RAM is at 100%). Data are saved in queue and slowly dequeued and transferred to the hard disk.  I could actually record from 12 to 14 sec complete without data loss. That would be enough to validate our application.
I checked this by sending a sinusoidal signal by changing its frequency at specific seconds and by reading the data, it was accurate and all is good! I leave the code in attachment

 

You have proposed me in previous comment some recording modules (http://www.ni.com/fr-fr/shop/select/pxi-data-storage-module) with parallel SSD disks to make long continuous acquisition: the HDD-8261 requires 3U in the chassis and we have just 2U free.

I contacted the National Instrument engineers to ask them about the PXIe-8267 module but I was told that this one requires the use of 58W slot cooling capacity to function. Our chassis (PXIe-1071) has unfortunately a capacity of 38 W..
If we use the PXIe-8267 module, we're not going to exploit its higher speed, the chassis of 38 W cooling capacity would be enough? What do you think of it ?

 

Thanks!

 

Best regards

Ali

0 Kudos
Message 8 of 8
(5,238 Views)