Stream to disk- Why cant i use the full 200MS/s acq rate if I only want a few seconds of data

[Mark Mutton]

I am currently trying to stream data to disk using a 5122 and or 5124 and the example vi "niScope EX Save to file - HWS Low level Single channel stream.vi" Everything works fine as long as i keep to the defalt 20MS/s. If i try to run this at any higher rate say 100MS/s I get the error " BFFA4018 Error occured at: NiScope Fetch Binary 8.vi"  I need 3-5 seconds of data at 200MS/s IS this possible or am i exceding some limits.

[psisterhen]

You won't be able to keep up for an extended period of time, but you may be able to keep up for a little while.  How long you can keep up depends on your sample rate, onboard memory size, how you've written your application, and what else is going on in your system.

The error you are getting is that data is being overwritten in hardware.  You can think of the onboard memory as a big circular buffer.  If you continuously fetch data out before it is overwritten, you can acquire indefinitely.  If you can fetch it out before it is overwritten by newer samples, the driver will return that error- you have lost the data.

Remember, the maximum theoretical PCI bus transfer rate is 132 MB/s.  Realistically, let's assume 60 MB/s.  Let's say you have 100 MB onboard memory and assume each sample is 8-bits wide.  Let's also say the sample rate is 100 MS/s.  So you're consuming the on-board storage at a rate of 100 MB/s and "adding" storage (by fetching data out) at a rate of 60 MB/s.   If I'm doing the math right, you will be able to keep up for 2.5 seconds before the not-yet-fetched data starts getting overwritten.

If you increase the on-board memory size, this time limit gets longer.  You can also extend this time by decreasing the sample rate or increasing your fetching throughput.  Since you want to increase the sample rate, you need to increase the fetching throughput (decrease the amount of time it takes to fetch data from the device and store it to disk).

Based on the example you're using, I can offer a couple of tips:

1) Set the Binary Sample Width to 8.  Right now, even though you are fetching 8-bit samples, you are actually storing 16-bit samples on the device and transferring 16-bit samples across the bus.  Software is throwing out the least significant byte before returning the data.  If you set the Binary Sample Width to 8, you will not only effectively increase your on-board memory size (storing one byte for each sample instead of two), but you will increase your fetching throughput because only one byte is transferred over the bus for each sample.

2) Don't fetch as often.  Right now, the example calls fetch as often as it can and fetches whatever is available (passing -1 to the numSamples parameter).  A more efficient way would be to set a minimum fetch size (say, 500 KB to 1 MB).  This will transfer larger chunks of data with one DMA operation, which is going to be more efficient than the current method.  So try different numbers for the numSamples parameter.

Let us know if this improves your performance.

Patrick

[Mark Mutton]

Thank you for the great advice. After a couple hours of trying various paramitors, I have come to the conclution that i cannot run at the full 200MS/s with this paticular vi.  My application is more dependent on the sampling rate than the speed of save, but I want my cake and eat it too. Instead of streaming the waveforms to disk is there any vi's out there that save at a very fast rate? i.e. I will still sample at 200MS/s however if i can save a waveform say every 100us this would be acceptable. A side not to my previous post is i only need about 5uS of data is there any triggering tricks i could preform to stream to disk but only acq the 5uS of interest.  I was thinking about having a 40us long low with a 5us high, would this enable me to aquire at the full 200MS/s and stream to disk if so how would i go about implementing this.

Again thanks for the great post. I have to say ni has one of the best on-line help forms I have yet to come across.

[psisterhen]

If you are only interested in 5 us of data out of every 100 us, you don't want to use a continuous acquisition since you're wasting time transferring data you don't want.  It sounds like you want to configure a multi-record acquisition. 

At 200 MS/s and 5 us of data, you should configure the record size to be 1000.  If you only want to store a waveform every 100 us, that means your fetch throughput only needs to be 10 MB/s.  (1000 samples * 1 byte/sample / 100 us).  This fetching speed should be easily achievable.

Perhaps the bottleneck in this scenario will be the write to disk.  I'm not sure off the top of my head how long this will take.  If you're only getting 1000 samples at a time, you can store a bunch of these records in memory before writing them to disk.

You'll need some sort of periodic trigger to start each 5us record. 

One more thing: only the 5124 has a max real-time sample rate of 200 MS/s.  If you set this sample rate on the 5122, it will require an analog edge trigger and enter RIS mode.  This mode acquires and averages several records for each record fetched, so the throughput calculations above don't apply.  You'll need to run the 5124 at 200 MS/s and the 5122 at 100 MS/s.

Finally, if the insurmountable obstacle does prove to be the write-to-disk speed, you may want to investigate this product:
http://sine.ni.com/apps/we/niaa.ind_pro_view?p_all_id=1225&p_display_all_id=1229

[Mark Mutton]

Thanks for the help. I am using the multi fetch more than avalible memory vi and am curious as to why i havn't hit a limit yet. I am fetching X amount of waveforms every 100us these waveforms are approx 512 points, after all the waveforms are collected I then save them to memory.  How do i know that the cirular buffer has been overwritten or not?  I am not getting any error with the paramitor enable records > memory set to true.  This is a good problem, however, I need to know what my limits are. Does anyone have any idea?

[Tica]

Hi Mark,

The description provided for the VI might be the information you are looking for:

In a multi-record acquisition, each record is one waveform with at least "min record length" points as specified with the Configure Horizontal Timing VI.   Furthermore, each record is triggered, so when the trigger arrives for the first record, the hardware quickly rearms for the next record.  Refer the to Multi Record example for more information.

In this example, the Fetch VI is used with the Fetch Record Number and Fetch Number of Records properties to fetch each record individually.  Using the Fetch VI with a positive timeout only waits for the requested waveform to be done - rather than waiting for all the records.  In this example, the fetch VI only waits for the next record, and than it retrieves the record while it is acquiring other records.

The Allow More Records Than Memory property is a Boolean value that allows you to configure more records than fit in the onboard memory at one time.  When this is enabled, you can specify any number of records.  The records are treated circularly in the onboard memory, so you must fetch them before they are overwritten.  If you attempt to fetch a record that has been overwritten in the digitizer's memory, an error is returned. 

Hope this clarifies some of your questions.

Regards,

Tica T

Applications Engineer

National Instruments

[JStairs]

Psisterhen,

I know it has been a while since these posts went up, but I have a question:

You mentioned that to increase the effectiveness of the onboard memory, you should change the memory from 16 bits to 8 bits so that when you fetch 8 bits it is not actually just throwing away the extra 8 bits and you are recording 1 byte/point instead of 2.

How do you switch the PXI-5124 from recording 16 bits to 8 bits?

Jason

[Sean_C.]

Hi Jason,

To set the sample width to 8-bits, you would set the Binary Sample Width property to 8.  In LabVIEW, you use the NI-SCOPE property node as shown in the attached picture.

In C you would use the following function:

ViStatus niScope_SetAttributeViInt32 (ViSession vi, ViConstString channelList, NISCOPE_ATTR_BINARY_SAMPLE_WIDTH, 8);

I hope this helps.

Regards,
Sean Close

[Mark Mutton]

If you use the fetch vi from ni-scope select the pull down menu under the vi and choose ID I8 option. see attachment

To get around the problem i was having in this post I "focused" in on the section of the waveform i was interested in by inputing the proper trigger delay and only collecting a few hundred points. This way i could collect a large number of sequential waveforms.

[JStairs]

Thank you Sean.  I will give that a try.

eeniu, I think that what you suggested will only change the number of bits that are retrieved from the scope, but not change how many bits the scope actually aquires as is mentioned in the rest of the post above...or am I mistaken?