Digital I/O

cancel
Showing results for 
Search instead for 
Did you mean: 

Streaming Data to Disk Using NI-6534 DAQ Card

Hello, I'm so happy to have found this discussion group. I'm still a new to Labview and have a long ways to go to feel confident with Labivew. Anyways, I have developed a small application to stream 16-bit data at 20Mhz to disk. I would like some suggestions on my small application. The .vi streams data to disk until the Stop button is pressed. The data is stored in binary format. The .vi starts streaming data as soon as the user enters in the name of the file to stream data to. I am using the NI-6534  DAQ card. If you could look over my .vi and give me any suggestions that would be great, and feel free to modify my app.
 
thank,
joe
0 Kudos
Message 1 of 6
(3,640 Views)
Joe,
 
I'm curious to hear if this program actually works for you.  Looking at the code, I see that you have the DAQmx Read VI configured for U32 data.  At a sampling rate of 20 MHz, this means you are attempting to stream to disk at a rate of 80 MB/s.  If, as you say, you are actually reading U16 data, this corresponds to 40 MB/s.  These rates may be possible, but would be highly system-dependent. 
 
There is actually an excellent tutorial on this topic called Streaming Data to Disk from the NI 6534.  As the title implies, this tutorial discusses benchmarking that was done to determine what throughput rates could be achieved when trying to stream data to disk from a 6534.  You can see in the results section that the highest achievable rates were less than 4 MHz.  The program that was used to perform this benchmarking is attached at the bottom of the page, so feel free to use it to test your system.  Though it was written in Traditional DAQ rather than DAQmx, the results should be very close. 
 
If your application requires higher data transfer rates than you can achieve with this benchmark, you may want to look into some of the high-speed stream-to-disk solutions that are provided by Conduant Corporation, one of our Alliance Members.  This is mentioned in the following knowledge base:
 
 
Hopefully this information is helpful as you further develop your application.
0 Kudos
Message 2 of 6
(3,621 Views)

Jarrod,

Hi, thanks for the information on StreamStor I've contacted the company and requested information on their cards. Do their card integrate with Labview easily? I mean, will have to do lots of reprogramming? I've tested by Labview program with the process priority set to Real-Time and it appears that the data I'm capturing is correct. I have two versions of my software, one version I buffer up the data and then stream it to disk and the other version I stream it directly to disk. I have configured the incoming data to containg a 16-bit count that increments every 20-Mhz, therefore each recorded sample should be one value higher than the previous. I will be setting up my computer for a RAID disk to help stream the data faster. My final application will be to stream 32-bit data at 20-Mhz to disk. The customer may intend to record up to 20-minutes of data. I've only started with the 16-bit version to get a handle on Labview and later to upgrade to 32-bits.

My current computer is a 3.8Ghz Pentium with 4GB of Memory, if that makes a difference in the performance of my application.

One more question, do you see anything wrong with my program? The program starts recording data as soon as the user presses the Acquire button, is that a good way to start the recording processing? I've read another newsgroup post and another person who was developing a recorder mention used a Event Structure. I'm new to Event Structures, should I use one to start my recording?

thanks,

joe

0 Kudos
Message 3 of 6
(3,600 Views)
Jarrod,
 
Hi, I've looked into the cards from Conduant and they start at $8.5K, do you know if Conduant is the only company that streams data directly from an NI card to disk such as Conduant's products do?
 
Thanks,
joe
0 Kudos
Message 4 of 6
(3,592 Views)

Hi Joe,

I am afraid that I am not familiar with a specific company that does the same sort of thing as Conduant, but we have many Alliance members in our directory at ni.com/alliance

To answer the previous post, your program looks good for what you are doing.  If you are happy with the program starting as soon as the user clicks run and then entering in a file, there is no need to use an event structure.  You could use an event structure if you wanted a separate button on your front panel to start the acquisition. 

Let us know if you have any more questions.

Thanks,

Laura

0 Kudos
Message 5 of 6
(3,579 Views)
Hi Joe,
 
Your application looks basically right.  The main bottleneck most people run into is the speed of their hard drive.  Basic off the shelf drives can be as slow as 10MB/sec.  However, with your current application, you are streaming 4 bytes at 20 MHz, which is 80MB/sec.  This is likely to exceed the rate of most off-the-shelf drives.
 
You can test out this theory on your system by eliminating the DAQmx driver altogether and just write binary arrays to the file in a loop.  See how fast you can write data to disk. 

Another possibility is that the DAQmx driver is not keeping up.  You can test it by eliminating the file write from the loop and see how fast DAQmx can go.  Unfortunately, to get maximum rate, you may want to eliminate the AvailSampPerChan property query from your loop.  You will have less visibility about whether you are keeping up, but you will be able to run a little faster.  If you have a recent system, DAQmx by itself should be able to keep up at 80MB/sec.  It looks like your read size is 10000, which should be sufficient.  Play around with different read sizes and look at Task Manager to check the system load.  If you can't keep up this way, you may want to consider a system upgrade, which by itself will be much cheaper than a Conduant solution.

Trying to stream 80MB/sec will basically max out the PCI bus just getting the data from the DAQ device to CPU memory.  (The theoretical maximum over PCI is 133MB/sec, but in practice 100-120 is the maximum sustained rate).  Since PCI is maxed out, you will want to make sure your file writes do NOT go across the PCI bus on your system.  A good solution for this would be PCI Express system with SATA drives.  On most PCI express systems, the SATA bus is separate from the PCI bus, so they don't need to share bandwidth.  If you still cannot keep up with one drive, you can look into a RAID solution with multiple drives and striping. 
 
If you can solve the problem by simply upgrading your computer, it will be much cheaper than the Conduant solution. 
Another simple solution to consider is to try to reduce the data rate.  Do you really need to stream 32 digital lines of data?  Can you get by with just 16 bits or 8 bits.  Or can you compress your data somehow by doing some online analysis before writing it to disk?

The Conduant solution is nice because it only sends the data over the PCI bus once.  The data never enters system memory and can't be slowed down by the DAQmx driver, the file driver, or LabVIEW.  However, keep in mind with the Conduant solution you may need to use Traditional DAQ because DAQmx doesn't currently support the memory mapping required to stream data directly to the Conduant device.  Our experience has been that simply upgrading the computer system to faster off-the-shelf parts and RAID can usually increase the rate sufficiently.

 
0 Kudos
Message 6 of 6
(3,567 Views)