How to read the entire buffer of a continuous acquire?

[pmackwood]

I have an application using 2 NI6221 cards.

Card1 controls and monitors a machine doing a test.

Card 2 is set up to do a continuous acquire at high speed with a predefined buffer size. 

When the application determines that the test is finished the machine is stopped.

At this stage I want to stop/pause the continuous acquire and read the entire buffer of data - in other words, get the most recently acquired data.

The test is so long that the buffer will have been overwritten.

If I stop the task the data is unavailable so I set up a pause trigger to pause the acquire.

Unfortunately, once a buffer is overwritten, the AvaialbleSamples property returns 0

However I thought I could set the ReadRelativeTo "most recent sample" and RedOffset to -BufferSize

but this casues error -200279 "Attempted to read samples that are no longer availalble......."

If I reduce the Offset slightly I can get most of the data.

How do I dertermine how much data is actually availalble?

I am using the DAQmx C API under VB6

Regards,

Paul Mackwood

[JamesC]

Hi Paul,

Please can you confirm if you are using the DAQmx Read Property node (Overwrite property), and that you are giving the property a value of 10252, "Overwrite Unread Samples" before starting your acquisition, This should then allow you to read your full buffers worth of data. If you have used a Buffer Property node and set the (Input.BufSize property).

Please can you try this and post back.

If these suggestions do not help you please can you post you code so I can look at what is going on.

Regards
JamesC
NIUK and Ireland

[pmackwood]

Hi James

"Overwrite Unread Samples" is the default when a Continuous acquire is set up. I check this with a call to DAqmxGetReadOverWrite - however I have also set this explicitly with no change in behaviour. 

Buffer size: initially I was allowing the driver to set the buffer size - I read the buffer size with DAQmxGetBufInputBufSize and use this to specifiy the read Offset. I have also set the buffer size explicitly to values smaller and larger than the default and the problem persists

As I said in my last post, I can read back most but not all of the buffer. I did some testing today, and it turns out that I can read all of the buffer except for 32 samples. In other words, I can read 968 from a 1000 buffer or 500 from a 532 size buffer etc.

Strange that it is always 32 samples  - is this possibly an addressing/indexing bug in the driver?

For Info: After setting ReadRelativeTo to MostRecentSample and reading a few samples, the CurrentReadPosition = TotalNumberOfSamplesAcquired as expected.

I have also noticed that if the acquisition is paused before the buffer is overwritten everything works OK.

Regards,
Paul

[Kevin_Price]

Sounds like you've tried doing the sensible things already.  So, not to be Captain Obvious or anything, but could you simply configure a bigger buffer so that you'll be sure to be able to retrieve the # of samples you *really* want?

-Kevin P.

[pmackwood]

Hi Kevin,

The application is for a customer, and is to control a machine doing destructive testing - the data required is that up to the point of failure. The sampling rate and amount of data is configurable by the customer. I agree that the pragmatic approach would be to do as you say if I could guarantee that the behaviour would always be the same for whatever buffer size. In my expereince,"if it can go wrong it will go wrong"   and since the tests can take several days I don't want to lose the data. I suppose I could trap the error and re-read the data with a smaller offset until the error doesn't occur.

Unfortunately, in a circular buffer(once overwritten) the AvailableSamples property is only a reflection of the programmed Offset and not how much data is actually available (which should of course be the entire buffer)  

Paul

[Kevin_Price]

Ok, gotcha, and I agree that it's reasonable to expect to be able to read the entire buffer.  In the apps I'd done that only needed most recent data, my buffer was generally times bigger than the amount of data I requested, so I never encountered the quirk you found.

Just a few more fairly long-shot thoughts:

1. Ever toyed with the DAQmx property for "data transfer request condition"?  This property controls how samples move between the board's own built-in hardware FIFO buffer and system RAM over the PCI bus.  I'm wondering if this magic # of 32 missing samples somehow relates to that FIFO?  Perhaps you'd get different behavior by trying different settings there? 

If so, it would seem like an implementation issue (bug?) because any data stuck on the board should be *more* recent than what you can read from system RAM. 

2.  Did you ever try reading 2 half-buffers and then combining them?  Maybe DAQmx is trying to do you a favor you don't need.  Many instances where someone attempts to read an entire buffer in a continuous acquisition would be a bad idea because the task is trying to write data to a memory location that is locked out due to the in-progress Read.  You've paused your acq to try to avoid this problem, but perhaps there's something deep down in DAQmx that leaves 32 samples of fail-safe cushion...

-Kevin P.

[Mcdan]

Paul,

If I understand your requirements correctly, this seems to be an application where using a reference trigger may be a good solution.  If I am not mistaken, you start a test which runs up to some undefined point in the future (but this point is not known ahead of time).  When this test is finished (failure point reached), you are able to send a trigger to the device to pause it.  Once paused, you would like to read the last pre-defined buffer size worth of points (note if I read this correctly, you are not logging all points, but only 1 buffer's worth that occurred right before your failure point).

Rather than pausing a continuous acquisition, you could set up a reference triggered acquisition.  To do this you specify a trigger (which would be the same signal you are using for your pause trigger now), and a number of pre-trigger points to acquire, and a total number of points to acquire.  My idea here is to set up the pre-trigger samples to the user-specified buffer size, and the total number of samples to number of pre-trigger samples +2 (note: I believe 2 is the minimum number of post-trigger samples possible with your hardware... this number is not really important, as you are interested only in your pre-trigger samples, so it should be set as small as possible).

So what would happen is as follows:

You start your task, and the device acquires into a buffer on your computer, overwriting old samples as it goes (this buffer should be sized the same as your total samples to acquire).  Once the device has acquired at least the number of pre-trigger samples, it will then be able to recognize the reference trigger.  Once it receives this trigger, it will acquire the specified number of post trigger samples (total samples - pretrigger samples) and stop acquisition.  At this point, you can read all sample acquired, and discard the ones which occurred after the trigger, or use the read property node and read relative to the trigger to read only the points you are interested in.  Take a look at the shipping example Acq&Graph Voltage-Int Clk-Dig Start&Rev.vi for a simple example of how this works, though you will may want to strip the start trigger out.

Just an idea,

Dan

[pmackwood]

Dan

You understand my requirements exactly. When I first statrted looking for a solution to my problem I was going to use the reference trigger but then discovered the pause trigger which seemed more approprite as I did not need post trigger samples. I will try the ref trigger next week as the exact number of samples is not really that important - what is more important is that the app doesn't crash un-necessarliy and annoy the customer.

It would however be much more satisfactory to have confirmation from NI, is it a bug or not, and if so is the 32 samples always true - then at least I could code a reliable workaround.

Whilst on the subject, is there a way of setting up a software trigger for the pause or reference triggers?

Thanks again everyone for the helpful advice.

Paul

[Mcdan]

Paul,

I did some investigation of the 32 sample issue you were seeing and can explain the 32 sample issue you are seeing.  Essentially the chip on your device responsible for DMA has a FIFO that is 64 bytes deep (or 32 samples for a 16 bit device).  The code which is responsible for monitoring the buffer to determine where the hardware has written can not accurately track where these 64 bytes of data is (it may be in the DMA FIFO, it may be in flight, or it may be in the buffer).  The monitoring code therefore, presumes that it is not in the buffer.  The thing to consider here is that you have configured a continuous acquisition, so there is no notion of being done until you stop the task (hence the buffer monitoring code always has to account for the possibility that there are up to 64 bytes of data in flight).

The advantage of using a reference trigger is that the driver knows when the device has completed its acquisition.  In this case, we can wait for FIFO's to empty and ensure that the data has made it to the host buffer.  Therefore when you read relative to a reference trigger, we don't have to account for in-flight data.  The disadvantage here is that you have to configure some number of post-trigger samples, which is not required by you, but is required by hardware.

One other consideration to keep in mind is that for PCI/PXI M-Series devices, the 64 byte work-around should work.  However, if you want to envision your code being run USB devices or devices in other families, you probably should not rely on this workaround.  This is one other reason I would recommend the reference trigger approach.

As to your last query, I don't believe that we support software pause or reference triggers.  You can imitate this behavior if you have a spare DIO line and PFI line.  Essentially wire the DIO to the PFI line, the perform a write on the DIO line when you want a reference trigger or pause trigger (which may already be how you're pausing your acquisition).

I hope that answers most of your questions.  I'm headed out the door and for a week's vacation, but will check back here if I get a chance.

Hope this helps,

Dan