Multiple Devices Continuous Data Acquisition

[crickmaster]

 I have two NI PCI-6561 data acquisition boards.   

The two NI high speed boards are synchronized and are collecting data.  We
want to collect data continuously for a long period of time and save it to
a file.  This way the file size would blow up, so we want to be able to
save the data to multiple files, i.e. write x samples to one file, then
write the next x samples acquired file to another file, and so on.  I am
not sure how to go about doing this.

Alternately, I have done the following.  Start one niTClk and niHSDIO
session, save the samples acquired, then start another session, and so on.
 However, I am not able to continuously acquire data in this way.  I lose
data between the end of the previous session and the beginning of the next
session.  And I would like to be able to keep track of how much time is
lost between the start of each session.

Would you have any idea on how i could either of the above two problems...
1. continuously acquire data, and be able to save in the middle of this
data acquisition without interrupting the data acquisition. or if not
possible, 2. be able to keep track of how many samples I am losing between
closing one data acquisition session and opening the next.

I have attached the VI i am working with right now.  I would very much
appreciate it if you could guide me in the right direction.  Thank you

[John_P1]

Hi crickmaster,

At what sample rate are you acquiring data and on how many channels? If acquiring continuously you may begin to run into PCI bus bandwidth limitations at higher rates (the 50 MHz that is defaulted in your code with 32 lines of input is going to result in 200 Million Bytes per second which far exceeds theoretical PCI bus bandwidth).  If you do want to acquire data continuously then the following example should be a good starting point:

NI-HSDIO Stream to Disk Using Win 32 File IO

The above example uses a producer/consumer architecture to optimize streaming in LabVIEW.  You can always change the file reference once the file gets too big, but opening the file reference will slow down the consumer loop so you might want to create your file references before starting the code (depending on how many there are).

As far as measuring the delay between acquisitions after re-starting, you can get an estimate by using the Windows System clock.  If you want more precise value you'll need to use an external counter to measure the time between the End of Record Event and the Ready for Start Event (both Signals are accessible externally by using the niHSDIO Export Signal.vi):

Unfortunately we do not have a current PCIe option for LVDS signals:  PCIe/PXIe would provide higher bandwidth for streaming purposes.  I hope this helps--if you have any questions don't hesitate to let me know!

Best Regards,

[crickmaster]

Hi John,

I am sampling at 25.6 MHz on 16 channels per board, so 32 channels total.  I will look through the example you have suggested and let you know how it goes.  Thank you for looking into this.

Regards,

Nishanth

[crickmaster]

Also, what do you mean by a PCIe and PXIe option?

[crickmaster]

Wouldn't I need the "Start Trigger" instead of the "Ready For Start Event?"  What is the difference between these two?

[John_P1]

PCIe stands for PCI Express, it is a higher throughput bus that is becoming the standard for streaming applications.  PXIe is the extension of the PCIe bus that we use with our PXIe chassis--it includes timing and trigger routes necessary for multi-device synchronization.  More information here:

Introduction to PCI-Express

I was just mentioning that we do not have a current PCIe or PXIe LVDS digital board.

The time between the End of Record Event and the Ready for Start Event is the amount of time that it will take to re-start the device in software.  The Start Trigger is when the new data begins to be acquired--you can export this as an alternative but it should be very close to the Ready for Start Event if you are using immediate triggering.

[crickmaster]

This has definetly gotten me started with my problem.  Thank you very much for the help, John.

Regards,

Nishanth

[crickmaster]

Hi John,

I tried exporting the signals to figure out the delay between acquisisitons, and the delay seems to be vary every time.  I was under the impression that the delay would remain constant.  That is not very useful to me unless it is constant. 

So, I am now working with the NI-HSDIO Stream to Disk Using Win 32 File IO.vi that you suggested.  My immediate goal is to be able to collect about a minutes worth of data.  Eventually I want to be able to collect arbitrary amounts of data.  The VI, the way I have it right now, runs for about a minute and craps out on me.  I get a "data has been overwritten" error.  At first, i thought this is the data in the queue, so I increased the "max queue size."  The VI crashes way before the queue is filled up.  Where else could the data be overwritten in this case? Would it be on the data acq board's memory itself?

- Nishanth

[Keith.Shapiro]

Hi Nishanth,

4 bytes / sample * 25.6MHz = ~98MB/sec.

It seems unlikely to me that you can sustain that amount of traffic on the PCI bus through Windows and onto a single disk, so it is likely that what is happening is that the data cannot be read quickly enough from the 656x's onboard memory.  Eventually, the on-card memory will fill up, resulting in the error you're seeing.

Assuming you don't have any other PCI traffic, your best bet is to use a computer with a large amount of memory (your data set is around 6GB!) and buffer entirely to RAM, then write out to disk when there's not a bandwidth critical situation.

Can you try altering the VI to just throw away the data that it's reading back from the 656x cards?  See if that speeds things up sufficiently to prevent an overflow.  If it does, then a computer with large memory will solve your problem.  If it doesn't, then it's likely you have bus contention that will prevent you from streaming data at that large a rate.

Do your data sets really need to be 6GB?  Would a better trigger and more targeted approach to when to sample be a better solution?

Keith Shapiro

National Instruments R&D 

[crickmaster]

Hi Keith,

No, my data sets do not need to be 6GB.   Is it possible for me to go about the problem in the following manner:

- acquire a large enough dataset where the size isn't going to fill up the on-card memory.

- write this dataset to disk

- clear on-card memory and start over with a new acquisition session

except I would like to know the exact time that I haven't been sampling, i.e., the time between the last sample read in the previous acquisition session and the first sample read in the next acquisition session.  John above mentioned exporting two signals: "start trigger" and "end of record event."  Is there anyway I can get an accurate measure of the time difference without having to export the signals to an external sink?  If so, how accurate will this be?

Also, I tried implementing an NI-TClk session using the producer/consumer architechture, but I am unable to get any sort of data written to disk.  I suspect the problem is somewhere between initiating an acquisition session and fetching the waveform. But I am not able to pin point what exactly the error is.  I have attached the VI i am working on.  I have tried my best to keep it organized and commented.  I used the niHSDIO (U32) Stream to Disk Queues Win32 File IO.vi as reference. I would very much appreciate it if you could take a quick look at it.  Thank you

- Nishanth