Properly Manage and Release Multiple Queues in QMH

[RavensFan]

---

@tsr26 wrote:

Kevin,

Another question about your snippet, would using the timeout feature limit my loop speed to 1000Hz? I assume the smallest timeout I can set is 1ms.

Tom

---

You can set a timeout to be 0.  The loop will run as fast as it can.  If something isn't already in the queue when the execution arrives at the dequeue, it will move on.  It won't wait that millisecond.

[Kevin_Price]

D'oh!  RavensFan is right.  Somehow I didn't even think of timeout=0 possibility, even though I've used it myself in certain situations.

But much the better solution starts with what Gregory already mentioned back in msg #4.  Just one Dequeue function.  And the datatype for the queue would be a cluster containing both "message" and "data payload".   The QMH template demonstrates this using a string for the message and a variant for the data.  

Then you retain only the single "outer" dequeue function.  And your Data Acq data now arrives there as the data payload part of the cluster (contained in a variant), alongside the string message part that could be something like "DAQ data".

All that said, I paid a bit more attention this time to some of the content of the loops rather than just focusing on the queuing operations.  And I think you may have gone a little *too* far with distributing things into separate loops.

In particular, you seem to be wanting to run a PID loop at 1000 Hz, but you have its code spread across 3 loops.  One to acquire, one to process and calculate an output, one to generate the output.

Since your tasks all read or write 1 sample at a time, I assume they're all of the unbuffered on-demand type.  (If buffered, there are other problems waiting for you...)  I'll further assume that you're aware of the timing uncertainty that brings, but you don't have realistic better options.

Here's the thing: you really shouldn't have queues as part of the data path for your control loop.  Queues are great for buffering and asynchronous stuff.   Control loops want to be synchronous.   You don't want to set yourself up to be acting on old stale data, and queues enable that possibility.

Instead of 3 loops (DAQ input, control law, DAQ output), all that stuff should happen in 1 loop.  This control loop could combine the concept of "timeout behavior" from my timeout snippet with RavensFan's idea to iterate as fast as possible by setting the timeout value to 0 once you start acquiring.

But that might not be such a great move either.  I would worry about a control loop whose timing variability is larger than its nominal iteration period.  It might be hard to produce stable, fairly repeatable behavior.  You might be better off slowing the control loop down intentionally so that most of the timing intervals are at least *relatively* consistent.  Then the control gains are more likely to still be fairly appropriate.

-Kevin P

[tsr26]

Kevin,

I agree that the better solution is the combination of message and data. I couldn't see before the real benefit behind combining them. I will work on changing that over as well as making sure I only have one dequeue. 

As for the control loop, I'm actually glad to hear I've gone a little *too* far! I guess you could say I try to be slightly under damped in my approach to getting things right. I will make my control loop into 1 loop - that does make more sense. I haven't used queues enough to appreciate the timing uncertainty they bring.

For the tasks, the hardware I am using is a USB-6003 and USB-6008 (this is just the hardware I had available in the lab). I hate to admit it, but I'm not fully familiar with the unbuffered on-demand type. I set them as single point in/out and I didn't delve too much further. I have been focusing elsewhere but I am doing more of that research now.

Lastly, if I combine my message and data into a cluster and send that to each loop - I would no longer need to mess with the timeout, yes? My thought was perhaps I could have the DAQ Loop send the 'Log Data' message each time it send a data point to the Logging Loop for example?

Tom

[Kevin_Price]

The fact you're using USB devices becomes the *next* problem for closed loop control.

I use the terms "unbuffered", "on-demand", and "software-timed" pretty interchangeably to refer to tasks that do not use any kind of hardware clock or other signal to control sample timing.  Instead, a single immediate act of sampling (thus "unbuffered") will happen "on-demand" whenever the application software happens to issue the Read/Write command ("software-timed").

This task mode is the default for any task that never calls DAQmx Timing to configure a buffer and hardware-based sampling.

Going forward, your software-timed control loop should certainly configure your output task to be unbuffered, software-timed, and on-demand.  Things are less clear for your input task.

With USB devices, you will have more latency than with a desktop device.  I don't have definitive advice here, but would not want to count on control loop rates above 20-50 Hz.  Situations likely exist where faster speeds can be (usually) attained and sustained, but I wouldn't start by assuming it.

However, I probably *would* start by making the task for the feedback signal(s) be buffered and hardware-timed.  Let's just suppose you sample at 1000 Hz.  Then, you can run a control loop rate in that 20-50 Hz range, reading 50-20 samples at a time.  The advantage of doing things this way instead of single samples on-demand is that you can do things like filtering, averaging, other special processing to "smooth out" your feedback signal.

Now a direct answer to your following thought:

---

Lastly, if I combine my message and data into a cluster and send that to each loop - I would no longer need to mess with the timeout, yes? My thought was perhaps I could have the DAQ Loop send the 'Log Data' message each time it send a data point to the Logging Loop for example?

Generally, yes.   I'm suggesting you don't have a dedicated "DAQ" loop though so much as a combined "Control" loop that includes DAQ input and processing and DAQ output.  From there, you could send a "Log Data" message paired with the data itself over to the separate Logging loop.  Then indeed the Logging loop wouldn't need that default timeout behavior.

-Kevin P

[tsr26]

Kevin,

Wish I could kudo that last message twice...great insights thank you.

For the Output Task, I believe I had originally been using the "unbuffered, on-demand, and software-timed" implicitly simply because I was not using the Timing.vi. This is all well and good, but I would love to have things as explicit as possible since I am still learning. So I want to make sure I'm aware of the most important knobs I'm turning. After inspecting the USB-6003 manual closer, I found the section about software vs. hardware timed. I also looked at the Examples for DAQmx of 'Voltage- On Demand Output.vi'

Would you be able to look at the attached .vi and see if I successfully *explicitly* set the Output Task to "unbuffered, etc."? This is a program used for academic research after all, and I'd like to have it teach both myself and my successors...

For the control loop, I combined the AI, Processing, and AO all in one loop. I'll attach the project again in case you are interested in checking it out.

For the control loop speed, would you say that if I benchmarked the loop with the method I use in my PID Loop - and that average loop speed amounted to ~1000Hz - then I can conclude I'm able to get that speed with my setup? I ask because it appears that the loop is operating that fast, but perhaps this is because of some settings I used unknowingly and perhaps it isn't actually processing 1 new data point each time. I'm unsure where the discrepancy comes from between what the manual claims it can do and the seemingly common recommendation of USB being ~50Hz max closed loop.

-Tom