Multifunction DAQ

cancel
Showing results for 
Search instead for 
Did you mean: 

Inconsistent DAQmx Startup Behaviour

Solved!
Go to solution
Highlighted

Application with a cDAQ Chassis, 9401 Reading an encoder, 9222 reading 4 AI, 9220 Reading 16 AI & 9213 reading 16 TC.

The Encoder needs to be read in sync with 9222 & 9220. 

 

Code developed from example found: 

https://forums.ni.com/t5/Example-Code/Synchronize-Encoder-Counter-Input-and-Analog-Input-with-DAQmx/...

DAQmx Test.png

 

Encoder & 9222 @ 500Khz, 9220 @ 100Khz

 

When starting this code it will either run exactly as expected, or generate an error (multitude of errors from all tasks).  There appears to be no consistency, it can run first time, or take 10 retries before it will work.  It seems generally worse after a cdaq / laptop restart.

Removing the parallel temperature task appears to make it less prone to error, and reducing it back to the two tasks in the example appears to remove the error completely.

When the system works it works reliably for 12+ hours and the data is good, there appears to be no issue with the hardware capability to run this code, but there appears to be a race condition on the startup. Debugging has included:

Multiple laptops, multiple cdaq chassis (USB & Ethernet), changing buffer sizes, sequencing task setup / clock setup.

Any suggestions appreciated

0 Kudos
Message 1 of 7
(467 Views)

I don't see an obvious "smoking gun", but here are some things to try, consider, or answer:

 

1. When you get errors, what error codes are they?  What clues do they offer?

 

2. Your temperature device 9213 isn't spec'ed for anything close to 1000 Hz sampling.  Try dropping that down to 75 or less as shown in the spec sheet.

 

3. Your 9220 and 9222 aren't really sync'ed.  They're merely *almost* in sync because their task starts reside in the same sequence frame. 

 

4.  I would normally approach true sync on desktop boards using sample clocks alone, but because the same cDAQ chassis drives timing for all your devices, triggering might be a simpler way to sync them.

    I would configure the 9222 task to be triggered by the *internal* start trigger signal of the 9220 task, and then make sure that the 9222 task and the 9401 encoder task are started *before* the 9220 task.  See the embedded snippet as an illustration.

 

5.  You're requesting 1/50 second worth of samples each loop iteration, i.e., you have a nominal intended loop rate of 50 Hz.  Try requesting more samples each iteration so that you iterate and update GUI graphs & suchlike at a lower rate.  10 Hz is a fairly typical rule of thumb, though 3-5 is probably more than fast enough for a user's brain to interpret the complex pattern changes in a graph display.

 

 

-Kevin P

Message 2 of 7
(443 Views)
Highlighted

Hi Kevin,

Thanks for the response, very much appreciated.

 

1. -200279 - Hardware unable to keep up with acquisition is the most frequent.  But others regarding buffer sizes, not all samples could be collected, increase timeout (despite the error occurring before the timeout expiry) and half a dozen others whilst attempting to get this to act consistently. (apologies I have not kept a note of all the codes and only have limited access to the hardware).

 

2. Typo when creating the vi for the snippet, Temperature task is a 1Hz task, have tried software timing this one as well as removing it.  Removing this task completely does not resolve the issue.

 

3. In that case I would suspect this is most likely the root cause, given the same code does not do the same thing each time I suspect depending on if these tasks happen to synchronise on startup or not leads to either running OK forever or falling over immediately.

 

4. Be great to try this approach.  It would appear the embedded snippet you refer to is missing, I have attempted to set up (see snippet) but am hitting -89131 source and destination the same.  I am unsure how I separate the internal start trigger of the 9220 AI task from the 9222 AI task and have been unable to locate an example of this

 

5.  The number of samples requested per loop (and therefore loop speed) is something I have experimented with quite a bit attempting to cure this problem. Based on the error codes received the 200279 suggests reading faster so the number has ended up where it is.  The UI / analysis / logging are all very separate and can handle the data at any rate from this DAQ engine (the loop rate is in no way linked to anything other than trying to get the DAQ to behave)

 

Thanks again

Start Trigger.png

0 Kudos
Message 3 of 7
(401 Views)
Highlighted
Solution
Accepted by johnporter

First the missing snippet.  Oops.

 

share start trigger.png

 

What you tried is very sensible, *almost* the same thing, and might have worked.  There's just one really subtle difference, and it's a result of using cDAQ in particular.

 

In cDAQ chassis there are several "timing engines" available to be used for different tasks.  I don't know all the rules for how they get assigned, but I'm aware that on one chassis I used, the 3 timing engines available for AI tasks were designated with aliases "ai", "te0", and "te1". 

    In your code, you're explicitly asking for a signal from a task using the "ai" timing engine and it appears that the task you want to get it from *happened* to be assigned either "te0" or "te1" instead.  (This is subtle stuff.  Don't go thinking you missed something simple or obvious.)

   I think my snippet should work because the task is queried for its signal without explicit reference to any particular timing engine.  That leaves it up to DAQmx to figure out the correct one, and I do believe it will.

 

WAITAMINUTE!  Looking at your screencap, this kind of thing is probably one of the *main* problems!  I skipped right over it before as well.  But the same kind of thing should be done for your shared sample clock signal.  See next snippet.

 

share sample clock.png

 

This is making more sense -- as to why you sometimes get correct behavior and other times you get failure on launch.  It comes down to the unpredictable sequence of when the tasks start having their timing engine resources assigned, and which one gets assigned to each device.

 

I suspect this'll clear up any of the discussion for items 1-4.  Item 5 remains a bit puzzling.  I would think any timing engine issue would have produced at least 1 task error and terminated the main reading loop immediately.  It seems there must be something else, but that's a pretty lean, tight loop without much going on.

    The *only* thing I can think of is to be suspicious of the "Build Array" primitive where you append a 2D array to a 1D.  Some primitives kinda "take orders" from the top input wire.  If you wire 2 different datatypes to a + primitive, the output datatype will match the upper input for example.

    I've been carrying around a habit where I would make my 2D array the top input and then append the 1D array.  My reasons (if there ever were any) are lost to the fog of memory.  But maybe it's worth a try to append in opposite order?

    Even better, I'd recommend that you redefine your cluster type so you just separately bundle the 1D encoder data, and the 2 sets of 2D AI data.  (You might also consider using 1D arrays of waveforms for the AI data to carry your sample timing info along.)

 

The help for error -200279 does suggest reading faster, but that suggestion is based on a hidden assumption that your code is mistakenly slowing down your reading rate.  DAQmx does a really good job with buffering, and it's generally most efficient with the least overhead when you request large-ish chunks of data less often from DAQmx Read.  The 1/10 second chunk size rule of thumb I mentioned tends to work pretty well.

 

 

-Kevin P

Message 4 of 7
(389 Views)
Highlighted

Thanks for the additional thoughts Kevin.

I have updated as suggested and all *appears* to be working. I am going to leave it with the end users for more extensive testing but is looking really good.

I did hit a -200983 error from the property nodes but reserving the tasks solved that.  For clarity this is the current setup. 

DAQ Working.png

With regards to 5.  My response was unclear, the system will happily work at 1/10 of a second, I had started much closer to your rule of thumb, just by the time i posted the test code it had ended up running a lot faster in an (unsuccessful) attempt to solve the startup issue.  Point 5 was never an *issue* to be resolved.

 

Many thanks, you have resolved an awful lot of head scratching and frustration.

Message 5 of 7
(372 Views)
Highlighted

Kevin is right. CompactDAQ chassis has 3 AI timing engines: ai, te0, and te1. When the AI task is moved to the reserve state, the driver automatically pick an available timing engine. In the first screenshot, "ai/SampleClock" is hard-coded to route to the counter task. But since the 2 AI tasks starts in parallel, it is up to luck whether sample clock of the 9222 running at 500,000 Hz or the 9220 running at 100,000 will route to the counter task.

 

If the application queries the sample clock terminal from the task instead of hard-coding it, that should address the issue. Sample clock terminal can only be queried from a cDAQ AI task when the task has moved to the reserve state. A task can be moved to the reserve state without starting the task by using DAQmx Control Task.vi.

Message 6 of 7
(369 Views)
Highlighted

FWIW, you could get by with a single sequence frame now -- the last frame.  The upper 2 tasks can wire their task refs and errors through frame tunnels after starting them.  Dataflow will then make sure they're started before the 9220 generates its internal start trigger to sync them all.

 

 

-Kevin P

0 Kudos
Message 7 of 7
(353 Views)