DAQmx Read Slow

[coolhandLV7]

All attached is my VI, i've read the forums looking for an answer to why my loop executes so slowly but have not found an answer.  Right now im getting between 70-80ms/iteration.  I need to capture 20S/s, none of the data is being lost however the lag by the end of a 200second capture window is seconds!  I need to be able to capture the data but also monitor it, once the data hits a limit i want to trigger the process equipment.

The data is (2) channels of AI Voltage and (1) channel of TC AI.

Let me know if you have any suggestions.

[GerdW]

Hi coolhand,

which DAQ device do you use?

Some devices need a "huge" amount of time to read a TC value...

[RavensFan]

Why are you using DAQmx Read N channel 1 sample repeatedly in a loop?  Use N sample version.  Letting DAQmx read a quantity of samples for you in one shot is always going to be faster than reading 1 sample repeatedly.  Especially when you are writing to file each iteration.

PS:  With Index Array, you don't need all of those index constants.  They automatically give you index 0, 1, 2, .... if you don't wire in constants.

[coolhandLV7]

GerdW - I'm using an NI-9214, sampling capability of 68S/s, which is fine for this application.  I actually started out with an old school NI-9211 @ 14S/s and nuked the application.  I have disabled the TC DAQmx Read to test this though with no change in performance.

RavensFan - I'm using the Read N channels (1) sample version for (2) reasons:

1. I have an indicator on the front panel for the user to have real time view of what is going on

2. I need to trigger from the measurement data, this state is not added to the code yet, but it's the next step.

I am no expert at this by any means but no rookie either, at these capture rates (20-50S/s) having this problem seems ridiculous, there must be something in my code that is killing the loop speed....  I just don't know what it is.  Yet!

[crossrulz]

Here is the list of things I would consider to speed up your loop:

1. Read more samples at a time.  For your display, you can show an average or peak or even just the last value.  But reading more samples keeps the backlog from filling up.  You are also doing a lot of processing that assumes many samples when you only have 1.  So you can get an overall speed up just by reading more samples.

2. Use a Producer/Consumer for your logging.  The idea here is that you move your logging to another loop so that it does not interfere with the timing of your DAQ loop.  You send the data to the logging loop via a queue.  You will really see the benefit here if you combine with suggestion 1.

3. Combine tasks.  If you analog input and thermocouple are on the same cDAQ chassis, you should be able to put them together into a single task.  This will make your diagram cleaner and possibly avoid the cDAQ trying to maintain multiple sample clocks, which could potentially be causing timing issues.  Not likely causing timing issues, but you might want to anyways to make your life simpler.

[Bob_Schor]

You need to plan your DAQ task.  In your Initialize step, you configure the device to take 20 samples/sec continously for 200 seconds, or 4000 samples at a time (which is fine).  Once you issue a DAQmx Read for those 4000 samples, the Read command will "block" its loop from repeating for those 200 seconds, which gives you a lot of time to do a parallel task.

Instead, you "change your mind" and take only a single sample, which (at 20 samples/sec) should take 50 msec.  However, in the same loop, you do a bunch of TDMS writes and write to a Waveform Chart, all of this taking place after the 50 msec.  Let's say it takes 30 msec to so that.  You have now changed your loop duration from 50 msec (or a rate of 20 Hz) to 80 msec (or a rate of 12.5 Hz).  Do you see the problem?

I realize that you may be using the data to make a "decision" during the sampling period.  How quickly do you need to make this decision?  Do you have to look at every sample, or can you look at a second's worth of data at a time?  You ideally want the "sampling time" to be significantly shorter than the "decision" time to avoid the kind of problem you are having.

When you have a "clocking" function like a Continuous DAQmx Read inside a loop, you need to ensure that you don't "overload" the loop with other tasks that "overrun the clock".  One way to do this is to use the Producer/Consumer Design Pattern, which involves taking the data-to-be-processed and "exporting" it from the loop that "produces" it (typically via a Queue) and sending it to a Consumer loop running in parallel that can display it, save it to disk, compute with it, etc.

Bob Schor

[coolhandLV7]

Producer/Consumer loop was my next "thing to try", i'm specifically using TDMS streaming which from the literature is fast enough to not need it.  I'm not sure so i may try it.

I'm not sure how to actually Read more samples and output the last value, as you mentioned in item (1), although i can picture how it should work.  If i set my DAQmx Read for N samples, i could read a chunk ->output the last value, then read another chunk repeat until the time is up...Hmmm  Not sure how to set that one up may have to play with that idea.

I didn't know you could combine tasks from (2) separate modules, how is that done?  I'll research it too.

Great reply, got me thinking...

[coolhandLV7]

Hey Bob,

I understand that the processing within the DAQmx Read loop may slow down the loop speed.  I don't understand the "change your mind" by only taking one sample.  If the DAQmx Timing.vi blocks the loop from executing then i should get rid of that and just use a timer in the loop?

I need to send a trigger signal, i think for this process a fraction of a second matters so i need the trigger to happen as real time as possible.  As for data display, that can be at a 1s update interval.

[Bob_Schor · Accepted solution]

---

@coolhandLV7 wrote:

Hey Bob,

I understand that the processing within the DAQmx Read loop may slow down the loop speed.  I don't understand the "change your mind" by only taking one sample.  If the DAQmx Timing.vi blocks the loop from executing then i should get rid of that and just use a timer in the loop?

I need to send a trigger signal, i think for this process a fraction of a second matters so i need the trigger to happen as real time as possible.  As for data display, that can be at a 1s update interval.

---

When you configured the DAQ Task using the Timing function, you specified Continuous Sampling, 20 samples/second, and 4000 samples to acquire.  Go open MAX (I hope you know about MAX), and configure your device with those parameters.  Start the task.  After 200 seconds, you will see 4000 points displayed, and after another 200 seconds, the next 4000 points will appear -- DAQmx is doing what you configured it to do, namely sample continuously at 20 Hz for 4000 samples (or for 200 seconds).

What you are actually doing is something else, namely letting your DAQmx hardware run at the speed you programmed, but only reading one sample at a time at a rate dependent on how long it takes to do all of the things inside the loop!  Remember the Principle of Data Flow -- the Loop cannot "loop" until every function inside the Loop has run, functions cannot run until all their inputs have data, and functions do not produce outputs until they are finished.  So loading in a lot of processing that requires data from the DAQ Read slows down the Loop.

I may be wrong about this, but I vaguely recall that Continuous Sampling "wants" a number of samples >1.  Here's what I would suggest that you do to "do what you want" and keep DAQmx happy:

• I understand that your Device has a limited Sampling Rate (68 Samples/sec?), so I'm going to suggest you sample at 50 Samples/sec, and set the number of samples to 5 (so you get 5 samples every 0.1 seconds).
  • Adopt a Producer/Consumer Design.  In the Producer, program the DAQmx Read for N Chan N Samples, and set # Samples = 5.
  • As the samples come out, "export" them (discussed below) to your Consumer Loop.
  • In the Consumer Loop, start by averaging the 5 samples, giving you a Signal-to-Noise improvement.  Do whatever you need to do with the data -- you've got 0.1 seconds of processing time, since the Producer loop is "waiting" for the DAQmx Read to gather the next 5 points and is not taking any CPU time while waiting.
• Here is an NI White Paper on the Producer/Consumer Design Pattern.

Bob Schor

[Kevin_Price · Accepted solution]

A couple minor comments/corrections on yet another really excellent bit of advice (already kudo'ed) from Bob_Schor.  Here's the part I'd like to address:

I may be wrong about this, but I vaguely recall that Continuous Sampling "wants" a number of samples >1.  

My much more long-winded version of this is that what Bob said is almost always true *in practice* when specifying a number of sample to Read in a loop.   However it isn't *technically* necessary in terms of syntax.

The trivial example is that the default value when unwired is -1.  This has the "magic number" meaning of "read all available samples (that haven't been read previously)."  I use this one fairly often when I need something else to govern the timing of my reading loop.  The loop timing can vary, but each iteration lets me catch up to the latest data in the buffer and prevent cumulative lag or overrun.

   But again, that's a trivial example and not what Bob was getting at.  I just wanted to highlight that -1 is both valid and the default input value and there are reasonable use cases for it.

I've also had occasion where I calculate the # samples to read based on, well, who-knows-what.  In some of those cases I set a threshold to prevent handling too-small blocks of data.  So my calculation might result in me requesting 0 samples.  There would be other ways to code around this so it isn't *important* to be able to request 0 samples.  But it *is* legal.

But for the very vast majority of Continuous Sampling tasks, potentially large total quantities of data are being captured.  You'll generally want to specify a # samples to read that's (much) > 1.  A decent starting rule of thumb I follow is to read about 1/10 sec worth of samples at a time.  A sample rate of 1000 Hz means read 100 samples.  Bob's example followed that same guideline, 5 samples per read for 50 Hz sampling.

-Kevin P