What is Most Efficient Log to Disk With PID Control?

[tsr26]

Hello,

My attached project (LV 2015) is, among other things, continuously acquiring analog data with DAQmx via an NI USB-6003. 

I have a control loop to acquire 1 data point, process it, and write 1 AO value to the servo drives of the motors I am controlling.

In parallel, I have a loop which receives data via a queue from the control loop. It performs a write for every queue element it receives.

My question is two-fold: 1) Is this the most efficient solution to log while controlling? 2) What would be a better way? I considered just creating a large array and then logging when done, but that may be bad as well. My file sizes will get to about 20kb max. I'm sure a middle ground is the best way, but I'm not sure what it is.

Ideally, I'd like to get to ~1000Hz control loop latency. Right now, I can average that, but the variation is ~+- 100Hz. I'd like a variation +-5Hz (or zero hah)

When I wrote this code, I assumed the parallel loops meant they wouldn't interfere, but I'm not computer savvy enough to know for sure that write never causes a delay to the control loop. 

Computer specs: Windows 7 64-bit; Intel i7-6700; 8 CPU @ 3.4 Ghz; 16 GB RAM

Side note: I know that the DAQ hardware I'm using isn't ideal and may be the limiting factor, but I'm trying to get as efficient as possible from a software standpoint before I move to hardware.

Thank you in advance for your time.

Tom

[crossrulz · Accepted solution]

---

@tsr26 wrote:

Ideally, I'd like to get to ~1000Hz control loop latency. Right now, I can average that, but the variation is ~+- 100Hz. I'd like a variation +-5Hz (or zero hah)

---

Not likely going to happen on a Windows system.  You are in the realm where you need to look into a Real Time system, or possibly even hardware (FPGA).  I would look into getting a cRIO system to do your control.

As far as logging, using a Producer/Consumer is the way to go.

[JÞB]

---

@crossrulz wrote:

---

@tsr26 wrote:

Ideally, I'd like to get to ~1000Hz control loop latency. Right now, I can average that, but the variation is ~+- 100Hz. I'd like a variation +-5Hz (or zero hah)

---

Not likely going to happen on a Windows system.  You are in the realm where you need to look into a Real Time system, or possibly even hardware (FPGA).  I would look into getting a cRIO system to do your control.

As far as logging, using a Producer/Consumer is the way to go.

---

Again I reply use TDMS native data-logging

[crossrulz]

---

@JÞB wrote:

---

@crossrulz wrote:

---

@tsr26 wrote:

Ideally, I'd like to get to ~1000Hz control loop latency. Right now, I can average that, but the variation is ~+- 100Hz. I'd like a variation +-5Hz (or zero hah)

---

Not likely going to happen on a Windows system.  You are in the realm where you need to look into a Real Time system, or possibly even hardware (FPGA).  I would look into getting a cRIO system to do your control.

As far as logging, using a Producer/Consumer is the way to go.

---

Again I reply use TDMS native data-logging

---

If taking my advice and going with a cRIO, DAQmx Logging is not really applicable.  While DAQmx is possible on a cRIO, I'm more inclined to use the FPGA to do the control algorithm and passing any data up to the RT system via DMA FIFOs.

[tsr26]

I looked into using TDMS instead of writing strings to text files, but I ran into trouble when I wanted to combine LabVIEW-generated data into the file with the acquired data in the same file. I'm sure there is a solution, but I admit I threw in the towel. 

I also have been given a cRIO to use, but I do not have the FPGA module and don't have the money to buy it. I have the Real-Time module so now I'm going to work to migrate my project over to using the cRIO instead of the USB-6003/6008.

I realize my question should have been more pointed like: Should I be concerned about my log to disk loop slowing down my PID loop? I think the answer is ultimately 'no', but I wanted to confirm that. 

[cbutcher]

---

@tsr26 wrote:

I realize my question should have been more pointed like: Should I be concerned about my log to disk loop slowing down my PID loop? I think the answer is ultimately 'no', but I wanted to confirm that. 

---

This was the point that Crossrulz was addressing above when he mentioned Producer/Consumer - basically you remove the slow logging to disk from the fast PID loop, so that irritating behaviour like your disk having contention with other applications (more prevalent on Windows than cRIO+Linux RT, but anyway) doesn't cause variation in the frequency of your PID loop.

Depending on exactly how you implement it and the required throughput of your logging, it might make a big difference. It's certainly likely to be an improvement (and is also a valid improvement on RT, where you can use something like a Timed Loop for the PID and then a normal While loop for the logging).

[tsr26]

---

@cbutcher wrote:

---

@tsr26 wrote:

I realize my question should have been more pointed like: Should I be concerned about my log to disk loop slowing down my PID loop? I think the answer is ultimately 'no', but I wanted to confirm that. 

---

This was the point that Crossrulz was addressing above when he mentioned Producer/Consumer - basically you remove the slow logging to disk from the fast PID loop, so that irritating behaviour like your disk having contention with other applications (more prevalent on Windows than cRIO+Linux RT, but anyway) doesn't cause variation in the frequency of your PID loop.

---

I actually do implement a Producer/Consumer already in the VI I attached. I have a PID loop which enqueues data, and then the logging loop dequeues and logs. I know that, from the high-level reading I've done that LabVIEW parallelizes those tasks and they shouldn't interfere. But my advisor was wanting me to double-check and I don't know enough about processors and LabVIEW parallelization to be 100% sure they wouldn't ever interfere. 

So, I wanted to first make my logging as least-CPU consuming as possible just in case.

Tom

[cbutcher]

Hi Tom,

Of course, you're quite right - sorry. Clearly this was a poor post to reply to from a phone - I've looked through some of the code now.

My understanding in this case is that if you're not hitting CPU limits on your device, on a crude level you're probably ok.

More specifically, LabVIEW allocates priority using one of several methods or rules - some useful (perhaps) links could be:

• Suggestions for Using Execution Systems and Priorities - very introductory for you probably, but the first paragraph might be of interest. The rest covers avoiding race conditions, not exactly the topic you're looking for...
• How Does Priority Affect the Execution of VIs? - some discussion of the idea LabVIEW uses for subVI prioritization. Of course, this covers subVIs, not loops. You could in some cases make use of this by shifting code into subVIs directly, but that has impacts on readability (your preference/mileage may vary).
• Prioritizing Parallel Tasks - suggests some methods, but the more interesting material is probably lower down and discusses some of the underlying details for the model.
• Understand the Priority-Based Scheduling Model (Real-Time Module) - as the title suggests, more specific to the RT module. Here Timed Loops (actually also on Windows, but less effective there) can have priority set. Copying from somewhere in the middle:

  • Timed Structure Priorities

    LabVIEW uses two separate but related priority schemes: VI priorities and timed structure priorities. Timed structure priorities are numeric, and a higher value represents a higher priority relative to other timed structures executing on the target. However, all timed structure priorities fall between the high and time-critical VI priority levels.

• Setting the Priority of a Timed Structure (Real-Time, Windows) - as the title demonstrates, timed loops can be used on Windows, but really they're more useful on RT.

Is that closer to what you wanted?

Separately, the Array To Spreadsheet String Function might help with the text file writing (if you wanted to continue with that). It's less useful when you have varying format specifiers though (like in the "Logging Parameters Loop").