DAQmx TDMS streaming

[David@Work]

When streaming data to a TDMS file in LabVIEW I have a DAQmx task that uses reference trigger.  Is it possible to read the file during the acquisition?  It appears that the code is not writing to the disk until the acquisition is complete, by then it's too late for the application I have in mind.

I tried to open the file, set the buffer size (smaller), and close the file, but that resulted in the file not even getting created (no data even after the acquisition had completed).

Any suggestions would be greatly appreciated.

David Dupont

[Brad_K]

Hi David,

For reference triggered acquisitions, by default, DAQmx returns data starting at the first pretrigger sample. However, DAQmx can't determine which sample is the first pretrigger sample until the trigger occurs, because doing so would require predicting the future.

(Furthermore, most devices supported by DAQmx need all of the posttrigger samples to be acquired as well, so that they can backtrack from the last sample to determine where the pretrigger and posttrigger samples are. I think X Series and the NI 6535/6/7 are currently the only DAQmx devices that do not have this limitation.)

You can tell DAQmx to return data sooner using the "Relative To" and "Offset" properties in the "DAQmx Read" property node. Setting Relative To = Current Read Position and Offset = 0 would be a good starting point. Note that this will return data from before the first pretrigger sample, so you will have to backtrack from the end of the acquisition to determine where the pretrigger and posttrigger samples are.

I haven't tried simultaneously reading and writing the same TDMS file but there is a community example so I assume it works. Note that another approach would be to set the logging mode to "Log and Read" when you call the DAQmx Configure Logging VI, and to use DAQmx Read during the acquisition.

Brad

[David@Work]

Ok, from your response, I guess I might not have been clear enough in my description of the problem.

[David@Work]

Sorry about that post, for some reason, pressing return sent the post.

I have a DAQmx task that contains 20 channels.  I use the DAQmx Configure Logging (TDMS).vi with Create or Replace, and .......

DOH! Log mode selected.  I would have SWORN I used Log and Read.  Let me try that, and maybe my problem will go away.  If not, I'll be back on Monday (off tomorrow).

Have a good weekend all.

David

[David@Work]

Finally got back in the lab today.  Log and Read function only works if you are reading from the DAQmx task.  what I'm trying to do is simultaenously read from a channel that I'm logging to from a DAQmx task that has reference triggering i.e. triggering from an external source.

What I observed today is that the pre-trigger samples (500 in my case) are all that appears to be getting into the file until the acquisition is complete, then the rest of the file gets written.  Since I'm trying to optimize by reading the number of samples that matches the sector size of the HD (512 in my case), I get nothing back from the read, and the rest of the code doesn't work as a result.

[Brad_K]

Hi David,

Let's simplify the problem by ignoring TDMS for now. When you do a reference triggered acquisition with DAQmx, when does the data become available to be read from the DAQmx task? The answer to that question depends on the type of device.

I wrote a quick test VI (attached) to demonstrate the difference. It performs a reference triggered AI acquisition, acquiring 5 seconds of pretrigger data and 20 seconds of posttrigger data (at 100 S/s, so 500 pretrigger samples and 2000 posttrigger samples). A counter/timer is used to generate the trigger 7 seconds* after the start of the acquisition. During the acquisition, the "available samples per channel" and "total samples per channel acquired" properties are queried at regular intervals, as well as the "counter output pulse done" property indicating where the trigger was generated.

Here is the behavior of an M Series board: none of the samples are available to read until the entire acquisition is complete.

On M Series, until the acquisition is complete, AvailSampPerChan returns 0. This happens because the beginning of the posttrigger samples is determined by counting 2000 samples back from the last sample, and the beginning of the pretrigger samples is determined by counting 500 samples back from the beginning of the posttrigger samples. (As I mentioned previously, a workaround is to use the RelativeTo and Offset properties to make the samples available sooner, but then you have a different problem: finding the trigger.)

X Series devices (as well as SC Express and NI 6535/6/7 devices, as far as I know) display a significant improvement: the pretrigger samples are available as soon as the trigger is detected, and the posttrigger samples are available as soon as they are acquired.

On X Series, AvailSampPerChan jumps from 0 to 500 when the trigger is detected, and then it increments continuously. Note that it is still impossible to read any pretrigger samples before the trigger is detected, because doing so would require predicting when the trigger will occur, before it occurs.

When you enable TDMS logging, DAQmx can't write samples to the TDMS file before they are available for reading, so reference triggering still limits when you can read back the data from the TDMS file.

What type of device are you using? Is it a USB device? USB also affects when data is available for reading, because data is transferred in larger chunks to reduce communication overhead.

Brad

* I didn't use a start trigger to synchronize the counter with the AI task, so the 7 second delay is not exact.

[David@Work]

Brad,

Thanks, that is very informative.  BUT!  You're making it sound like I can't get to where I need to be when using reference triggering.  To re-state my original challenge, I have a task that includes one channel that is feedback for a control loop.  This task turns out to be a very important data collection task during the test event and must include the control feedback channel.  In order to control the attached system I need to know the current value of one channel in that task.

So more simply, I need to collect data from a channel in a task during a test event, while simultaneously keeping up to date on it's value to know when to control the related system.  The timing of the control loop isn't too critical.  Off by as much as a second or two wouldn't be a problem.

Thanks in advance for any assistance you may be able to offer.

David Dupont

[John_P1]

Hi David,

What are you using the reference trigger for? A continuous task with Log and Read would do what you need except you wouldn't be able to stop it deterministically.  It might honestly be easier to acquire everything and post-process the data to pick out the section that you're interested in.  I'm assuming that you want to log a specific segment of data according to the reference trigger, but acquire everything continuously to provide some feedback.

If you really do need that reference trigger...

If you set the DAQmx Read Relative To property to Current Position, you can retrieve your pre-trigger samples while the acquisition is taking place.  Integrated TDMS logging does not support this property however, so you'll have to implement your own file I/O in LabVIEW.  Furthermore, you'll have to handle the trigger appropriately in software.  You could possibly use a Task Is Done? event, check how many samples are in the buffer that haven't been read (DAQmx Read Available Samples Per Channel property), then figure out how many samples to go back and retrieve to make sure you get your pre-trigger and post-trigger samples (adjust the DAQmx Read Offset property accordingly). 

What DAQ hardware are you using?

Best Regards,

[David@Work]

The reference trigger lines up the data with another task that controls other parts of the system.  There's an analog output channel on the same device (M6255 in a PXI chassis) that controls a pressure dome valve.  The "event" is to open the dome valve according to a waveform profile that varies the pressure over a period of time with a finite endpoint.  The waveform to the dome valve is what goes out the analog output channel on the 6255. The start of the waveform must be initialized by part of the system under test, hence the external trigger.

(In case you are unaware, a dome valve is basically a voltage controlled pressure regulator)

You may be right that continuous collection, and post processing might be a viable solution.  I've never used the DAQ in that way.  (For the record this is the first project I do using DAQ, and I don't feel like I've fully mastered this yet.)

Couple more questions (and I reserve the right to ask more after this too):

1. Can I have continuous data collection on the same device while an analog output task is triggered by some external hardware?

If so, I've noticed that the TDMS files don't have a column for time in with the data.  All the information regarding the start time, and the increment value is there on the header sheet (when imported into Excel), but not the individual timestamps on each sample.

2. Am I missing something in the streaming setup?

Thanks again,

David Dupont

[John_P1]

Hey David,

Thanks, that clears it up a bit.  I actually found out that you have a Service Request with Starla T. and it sounds like you have a small number of pre-trigger samples but a large number of post-trigger.  So... it sounds like you really only need the pre- and post- trigger data around the reference trigger, but you'd like to monitor it while the acquisition is still taking place (which might take some time).

I think the attached code may be pretty close to what you need.  It will return "pre-pre trigger" samples as well (since we can't deterministically tell when the reference trigger has occurred), but at the end the resulting data will be a deterministic acquisition around your reference trigger with the specified amount of total samples and pre-trigger samples.  During the acquisition, data will be returned continuously.  You can not use the integrated TDMS logging, but you could just write the end result to a file after the loop has completed.  I'm not sure how much data you are acquiring in total.

If you are on a PCIe or PXIe X Series board, you can start acquiring data from the task using default settings once the reference trigger has occurred (see Brad's graph in the above post).  This would certainly be more straightforward then the attached code if you don't need the pre-trigger samples until after the reference trigger occurs.

For your specific questions:

    1.  Certainly, the tasks can be triggered independently. 

    For clocked tasks, the TDMS file just uses a t0 and takes the dt from the inverse of the sample rate from the driver.  The binary TDMS file actually stores the scaling information in the header as well, but it is applied for you when converted to excel.

    2.  I haven't seen your VI so I don't know if you're specifically missing something.  The attached VI is missing streaming altogether, but you can use the TDMS functions to implement the file write after the acquisition is complete.

One caveat with reference triggering on an M Series is that your buffer in memory needs to be able to hold all pre-trigger and post-trigger samples for your data.  This might be a problem if your number of samples is very large, in which case the only options would be to avoid reference triggering altogether (you would lose out on the pre-trigger samples but can still trigger post-trigger samples with a Start Trigger) or switch to an X Series (you can start reading back data in a more standard manner as soon as the trigger is received).

Best Regards,