Capture 4-Seconds of Data using PCI-6534

[joe5213]

Hello, I'm trying to develop a Labview application to store 2 to  4-seconds of 32-bit data arriving at the rate of 20MHz. I have set up the DIO Config for burst signal mode and to do 80 million scans. Once I the scans are complete and in the computers memory I can then process the data and then store it to a spreadsheet. The problem I'm having is that  the maximum number of scan I can do is about 1,048,580 scans and I don't know who to get around this problem. Is it not possible to buffer up that may scans? I was reading the NI-6534 users manual and it states that the buffer size can be 1GB in Burst mode. Does anyone know if this buffer is on the card or in the computer's main memory? I thought about streaming the data to the harddisk and I could do that but the data must be processed before saving to a file. I guess could do that processing after I have the data in the file but I don't think that is the problem. Does anyone have experience capturing large amounts of data or if anyone has some suggestions I would greatly appreciate your thoughts.

thanks,

joe

[Ryan_V[DE]]

Joe,

The 1GB memory is in the PC Memory. The 6534 cannot acquire at 20MHz while streaming to PC memory. Here are some performance benchmarks on the 653x series boards. Acquiring at 32 bits at 20MHz corresponds to 1048576 samples (32bits x 1MB), or .0524 seconds. Depending on your configuration, you should be able to acquire the GB buffer around 5MHz.

Hope this helps,

[joe5213]

Ryan,

Hi, thanks so much for answering my post. I now have a better understanding of the bench mark graph on page E-8 of the DAQ 653x User's Manual. Correct me if I'm wrong but the Buffer Size the User's Manual mentions is located on the computer's memory. If you wanted to capture more than 32MB of data, which is the size of the buffer on the card, then you would have to operate in the Pattern-Input Continuous mode which requires that the data be DMA to the computer. In this mode you are capable of capturing 1GB of data, but the number of bits you sample depends on the rate you want to sample. I'm just going over this stuff to make sure I have a clear understanding. So back to capturing 1GB of data, if the rate of the data was coming into the card a 20MS/sec then to capture the data without dropping samples you would have to only sample 8-bits. If you try to capture 32-bits you could only sample the data at 5MS/sec. Am I correct on my thinking? I will look over the manual one more time, but doesn't the card have a fixed sample rate, it's always sampling at 20MS/sec? Ryan, one more questions, in you first reply you said that capturing 1,048, 576 samples would only capture 0.05 seconds of data, is that the max time I can capture, or is that the max sample of the driver?

Thanks for all you help,

joe

[joe5213]

Ryan,

Hi, I've spoken with the design engineers and they have decided to buffer up 4-seconds of data on an external card they are designing and then send the 32-bit data to the NI 6534 card at 5Mhz, how's that sound?

thanks,

joe

[Ryan_V[DE]]

Joe,

I think you are a little off on your understanding. The 6534 has two user-configurable clocks, each up to 20MHz frequency. On each edge of this clock, the 6534 will read/write the entire port width (8, 16, or 32 bits) of data into the on-board 32MB buffer. This data will then be transferred into the PC's memory. The KnowledgeBase article on the benchmarks presents the limitations of transferring data over the PCI bus, from the 6534's buffer to PC memory. You can capture data for any duration, but if you are acquiring or writing too quick, the buffer will overflow or underflow. These benchmarks describe the maximum rates which do not overflow / underflow.

Hope this helps,

[Ryan_V[DE]]

Joe,

5MHz should be fine all the way up to 32-bit ports.

Regards,

[joe5213]

Ryan,

Hi, do you think it is possible to capture 320Million 32-bit samples? The user wanted to capture 4-seconds of data which was coming at 20Mhz, but now wel will buffer up 4-seconds of data on a custom board and them stream the data to the NI card at 5Mhz. I will still have to capture 4 seconds and that would be 4 times as much data as when it was coming at 20Msamples/sec. So that about 320Million samples, I'm a little worried I'll run into problems. Do you have any comments you'd like to share about capture such a large amount of data. I've increased the memory in my computer to 4GB, I hope that will help. I'll appreciate anything you have to offer.

thanks,

joe

[Ryan_V[DE]]

Joe,

What is it that you plan to do with this large amount of data? Are you streaming it to disk? Do you want to keep it in memory and process it? In either case, I would recommend using a continuous task (as opposed to finite) and stopping it immediately after acquiring 320MS in LabVIEW (to prevent a buffer overflow). This will help to avoid having 2 copies of the entire data array in memory, as in a finite task (1 in the 320MS, 32 bit DAQmx PC buffer, and one in LabVIEW memory, for a grand total of approximately 2.6GB). If you plan on streaming to disk, you will only need a "small" DAQmx buffer and "small" LabVIEW array, which you will continuously write to disk. The caveat here is that you may not be able to stream to disk at 20MB/s, depending on your computer. I would set up a test acquiring invalid data before building any hardware. If you plan on keeping the entire ~1.3GB array in LabVIEW memory (for processing, etc.), I would initialize an array of the appropriate size before the acquisition is started. Then, inside your acquisition loop, I would index the most recently acquired data in to the appropriate location in your array. This is far more efficient than continuously building up new arrays as your acquisition runs. In this case, you would have a "small" DAQmx PC buffer and an enormous LabVIEW array. Please keep us informed as to your progress and decisions. I am curious to see how this turns out.

Good luck!

[joe5213]

Ryan,

Hi, the user wants to do an FFT on the data. The incoming data is a 32-bit word  consisting of two words of data, a data word in the upper 16-bits and a data word in the lower 16-bits. I had planned on capturing the samples to computer memory, separating the data, and then saving it to a file. I only need to run this sequence once, but need buffer up 4-seconds of continuous data. The user will then open the file and do whatever analysis he wants to do. I've written my application using the traditional DAQ drivers, also it is basically a sequence structure, first configure the DIO vi, read a large number of scans to computer memory, process the data (separating the data into 16 bit chunks), and then saving the data to a file, and then I'm done until the user hits the start button to do the process again. Sounds simple but the only problem is that I'm capturing a ton of samples.  How's that sound?

thanks,

joe

[DFGray]

If you haven't done so yet, I would recommend you read the tutorial Managing Large Data Sets in LbVIEW. This will help with keeping copies to a minimum. One word of caution, however. You will not be able to store more than about 1.5GBytes of data in memory using Windows XP due to OS limitations. If you store your data in one array, you will not be able to store more than about 1GByte in LV7.1, about 800MBytes in LV8.0, due to memory fragmentation issues. Maximum theoretical memory use on any platform is 2GBytes, since LabVIEW uses signed 32-bit integers for memory pointers.

I would also recommend you use NI-HWS to store your data. It can be found on the driver CD under the computer based instruments (it is the default file exchange mechanism for their soft front panels). This is a fast binary format based on HDF5 that includes compression, should you wish to use it. Most analysis packages (MathCAD, Matlab, Mathematica) will read HDF5 files, so you should not have any problems there.

Good luck! Keep us informed. If you have any more problems, let us know.