High-Speed Digitizers

cancel
Showing results for 
Search instead for 
Did you mean: 

PCI-6110 continuous data

Solved!
Go to solution

I am attempting to take continuous data of 4 channels at the maximum 5Msamples/second and simultaneously writing that data to disk using the TDMS writer.   I am using the daqmx tools within labview to read and write the data.  I do get failry good data rates but it is also not quite continuous. 

 

1.  Is this possible within Labview? 

 

2.  Is it possible with NIScope?  Is NIScope at all related to labview?

 

Thanks,

Ben. 

 

0 Kudos
Message 1 of 10
(7,967 Views)

Starting with your second question, storing 20MS/s to disk is possible with NI-SCOPE, but your device is run using NI-DAQmx, not NI-SCOPE.  The PCI-6110 is in the NI-DAQ series of devices. NI-SCOPE and NI-DAQmx are drivers for particular classes of devices. Both can be called from LabVIEW (and other languages).

 

It is certainly possible to store 20MS/s to disk from LabVIEW, but it will depend upon your hardware and how careful you are with your coding. The 6110 is a 12-bit device, so raw data will be 16 bits or two bytes per sample. Your data rate is 40MBytes/s to disk. This is easily possible with a RAID or solid state disk (SSD) system, but on the edge for a traditional, single, rotating disk system (assuming something reasonable modern). Things get worse if you are saving scaled data (DBL or SGL) to disk. You then are saving eight (DBL) or four (SGL) bytes per sample to disk and your data rate goes up to 160MBytes/s and 80MBytes/s respectively, and most single rotating disks will not be able to keep up with this. RAID and SSD systems may or may not.

 

So, continue to use the DAQmx API with TDMS to stream to disk directly from the device. Verify you are saving integer data, not scaled data. Look at the specs on your hard drive to see if it can keep up. Be aware that operating system overhead can and does degrade disk speed. There may also be another bottleneck in the path to the disk, such as an older controller or bus which is limiting your speed.

 

If you have further questions, please post your code so we don't have to guess what you are doing.

Message 2 of 10
(7,953 Views)

Hi Ben,

 

What do you mean by "not quite continuous"? At such a high sample rate, this might become a LabVIEW VI architecture issue as compared to an acquisition issue. If you set your acquisition to 4 ch, 5MS/ch and don't get any errors, this means the hardware is properly configured and returns the data on the DAQmx Read function. You might lose some continuity if your while loop in LabVIEW takes too long to execute (probably due to the writing to file).

 

If you are taking this data and writing to file, I would recommend a producer/consumer architecture to ensure that the acquisition is done as fast as possible. Here's some information on producer/consumer loops: http://www.ni.com/white-paper/3023/en/

 

NI-Scope s a driver for digitizers/oscilloscopes devices, similar to DAQmx for DAQ devices. Since the 6110 is a multifunction DAQ card, it is only compatible with the DAQmx driver. If you wanted to use NI-Scope driver, you'd need something like the PCI-5105. Hope this helps!

Xavier
Message 3 of 10
(7,952 Views)

Thank you both for such thoughtful answers.  You have given me a lot to think about and work on.  NI-Scope is out as an option and I won't waste anymore time learning about that based on your responses. 

 

DFGray mentioned not making it a scaled value in order to conserve data.  I originally used DAQ assistant as a starting point for my code and I see it uses "Volts" as the current scale.  Also, I can see in the TDMS file that the values stored are not integer values like you might expect from an ADC.   I tried to change the "Volts"  but all that is available is "Custom Scale".   Can you explain how to use integer values in DAQmx?  (Sorry, if this is a newbie question.  I am fairly new to using Labview.) 

 

The "not quite continuous" definition in this case is I ran for 2 minutes with four channels and logging to disk with 1k samples at a time.  At 5Msamples/second I would expect (120 sec)*(5 Msamples/sec)  = 600 MSamples.  I forget the precise value but it was exactly a multiple of 1024 bytes and approximately 64% of the expected number of samples. 

 

On the producer/consumer front that is exactly what I want to do. I am a little sketchy on how I would do that.  In my code I have to enter a value for the number of samples and then I loop indefinitely collecting that number of samples each time.    If I continue doing that there will always be some small delay between loops.  I am not clear on how to make the producer just continue forever putting data into the queue.   Or another way of saying it is, doesn't my DAQmx loop need to be both a producer and consumer to really maintain that 5Msamples/second?  A consumer of the DAQ card and a producer for the TDMS loop. 

 

 

 

 

0 Kudos
Message 4 of 10
(7,942 Views)
Solution
Accepted by benjoromo

I am not a DAQmx expert, but this is a fairly common use case covered by the API. You can set DAQmx up so that when data is collected, it is also efficiently written to disk. You can do this for a continuous data stream. You can find an example of this here (and more examples using the LabVIEW example finder: Help->Find Examples...):

 

<LabVIEW>\examples\DAQmx\Analog Input\Voltage - Continuous Input.vi

 

This VI uses scaled data, so you will want to select unscaled I16 data for the output data type for maximum speed.

 

Alternately, you can use a producer/consumer architecture, acquire the data in one loop, and stream it to TDMS in another. This is essentially what the above example does, but in a much easier manner, and faster, since it involves fewer data copies.

 

Good luck!

0 Kudos
Message 5 of 10
(7,939 Views)

Thank you for not just giving me the example but showing me how to find examples in general.  That little menu option had eluded my attention before.

 

I ran the example a few times with varying inputs and I learned a few things.  First, the hard drive I was using was either extremely slow or it was routinely being accessed by other programs or a combination of both.   It was the C drive so it could have had other constraints on it.  With the example program I had issues going as high as 1MHz data rate.  I would periodically get complaints about the buffer being overwritten before I got to it.  I had purchased a USB drive for this project, so I plugged that in and stored the data on that drive. With that change I was able to take data seemingly indefinitely at the full 5MHz rate and simultaneously store it to disk. 

 

That was only one channel though.  So, I took what I learned from the example and applied it to my VI.  I ripped out the TDMS stuff I had previously put in and recorded using the same method as in the example and I also changed the destination to the new hard drive.  It seems to work perfectly now.  I just ran for an hour(3599.76 seconds).  I got just a little under the 18x10^9 samples I would expect for an hour.  One thing that surprised me.  I expected to have approximately 288x10^9 bytes because each sample I thought was four bytes.  After compensating for the fact that Windows reported KB is 1024 bytes I am actually getting 143.99x10^9 bytes or almost exactly half of what I predicted.  So, it must already be storing the ADC values or something equivalent and also storing how to scale those values.  The TDMS viewer that I am using must be scaling the values before I view them. 

 

I will eventually go to a producer/consumer architecture because I want to have more control over the TDMS file and also I want to do some minor pulse processing on the data as it comes in.  But what I have will do the job immediately if needed. 

 

Thanks again for your help. 

0 Kudos
Message 6 of 10
(7,910 Views)

I believe that under the hood, the DAQmx storage mechanism stores raw, unscaled values to reduce size (two bytes per sample in your case, since your ADC is 12 bit). It also stores the scaling factors so you can read it as scaled data. When you make your producer/consumer architecture, you will want to do the same. TDMS supports this.

 

Good to know you succeeded. Let us know if you run into further issues.

0 Kudos
Message 7 of 10
(7,894 Views)

I have ran into another issue.  I have begun implementing the producer/consumer architecture so I could add in some data reduction on the consumer side.   I believe I have the queue functioning correctly.  In order to implement the queue you need a data type and I took the output of the DAQmx read function and selected "Create -> Constant"  and then I used that constant as the input to the queue type. 

 

I am using the "TDMS Advanced Asynchronous Write" as my TDMS writing example to follow.  When I attempt to connect the constant that worked for the queue data type to the "TDMS Set Channel Information Function" it complains with an error "Polymorphic terminal cannot accept this data type".   Any thoughts on how to address that issue? 

 

P.S. In my original program, that did not work well,  I connected the output of the DAQmx read directly to TDMS write and it worked fine if a bit slow.  So, there should be a way to do this... I hope. 

 

0 Kudos
Message 8 of 10
(7,846 Views)

Hi Ben,

 

The TDMS Set Channel Information only accepts specific data types (integer, floating points, booleans and timestamps). The queue you are using is most likely an array of numerics (doubles, I16, or something else) or a waveform. This is what sets the discrepancy - if you try to wire in a full array of values, you'll get an error (we're just declaring the data type so all you need is one element).

 

I would expect if you use DAQmx Read as an array of DBL (configure the queue with an array of doubles), wire a single element (DBL) to the Data Type input of Set Channel Information, and pass in the full dequeued array of doubles into the TDMS Write function, things should work fine. If not, please put a screenshot with the relevant functions showing (queue creation and the TDMS part). Hope this helps!

Xavier
0 Kudos
Message 9 of 10
(7,827 Views)

As Xavier said, you are probably taking multiple channels of data and this results in a two dimensional array of data from the DAQmx read. This could be an array of waveforms or simply a 2D array of values (we would need your code or a screenshot to know how you set it up). TDMS stores data as 1D arrays (channels). When you do not use the internal DAQmx method, you need to stream each of your channels separately. Use your current DAQmx output as the queue data type. On the storage side, create an array of channel names for each channel of data you take (this can be static or generated at run time). When you get a data packet from the queue, use a FOR loop to index each data channel and its corresponding name and write them to disk one at a time (autoindex is your friend here). So you will be writing multiple times per data acquisition.

 

Good luck!

0 Kudos
Message 10 of 10
(7,772 Views)