DAQmx program using almost 100% of RT controller CPU

[SarahW]

Hi,

I have a fairly simple DAQmx VI (attached) which I want to run on an RT system.

When i run th code (RDD2SC2 - Acquire.vi) on a Windows 7 system it is perfectly happy just sitting there using about 8% of the CPU. When i deploy it down to a RT system it maxes out the CPU, and while this is happening the shared variable is not updating properly (presumably because the CPU doesn't have time to deal with the SV?).

Does anyone have any suggestions why this VI could be hammering my CPU?

Many thanks,

Sarah

[SarahW]

I have found some info here:

http://digital.ni.com/public.nsf/allkb/61B0E8B83C799DD586256ECC007CAB24

By adding a Read All Available Samples property node and setting it to true in the setup read VI, then having a start, wait until done, read, then stop task in my read vi (see attached picture) I have managed to reduce the CPU load to about 85%, which is much better, but still seems fairly high to me?

Any suggestions?

Thanks,

Sarah

[Supreeth.K]

Hi Sarah,

How many cores do you have in your processor? If you have fewer cores in your processor, they are shared between the OS pool and the TS pool. This results in overheads due to inter processor communication. Please have a look at this article which throws more light on how threads are assigned to  processor cores in a RT system:

http://www.ni.com/white-paper/6583/en

Thanks and Regards,

[SarahW]

Hi Supreeth.K

Thank you very much for your response.

I have tried two controllers, one has 1 core, and the other has 2.

I have changed the 2D array shared variable to singles (instead of doubles), and the dual core PXI seems to cope with it now, but it made no difference to the single core PXI.

I am using the LabVIEW 2012 RT module.

When I was using doubles i did try reducing the size of the array by a factor of 10 (120*21 data points instead of 1200*21), and this made no difference. So how come using singles instead of doubles works? I assume this just reduces the size of the array by a factor of two?

Thanks,

Sarah

[Texas_Diaz]

I just wanted to chime in with a few things here - your timed loop has a lot of nasties in it that you may want to reconsider.

1. First of all, reading and writing shared variables in a timed loop is very bad, unless you have an RT FIFO enabled on the variable; without the RT FIFO enabled you might as well be doing TCP/IP within the timed loop (actually, you are).  The RT FIFO will reduce the overhead of the call and make your timed loop "not do very bad things" under the hood. The major rule of thumb for a timed loop is to NEVER do these three things within a timed loop:  (1) TCP/IP, and (2) File I/O, (3) Read/Write front panel objects.
2. Why on earth are you waiting 20ms each loop iteration within a timed loop?  With the timed loop, you've said, "hey, stop everything and do my deterministic work as quickly as possible without interruption" but with the 20ms wait you're violating that remark.  Remember that the timed loop is saying, "run this at the highest level possible, NOW!" and then with the wait you're telling it that you were just kidding, it really isn't that important, I want to wait for a while."  Remember, the timed loop has a built-in periodic firing mechanism that a standard while loop does not, so you musn't try to treat them the same.  You've wired in a dt of 20ms (it's ms since you're using a 1KHz clock) which means every 20ms the loop will be allowed to execute again - it does not re-execute immediately after completing an iteration like a while loop does, so it does not need a wait within the loop to prevent it from consuming 100% of the CPU like you would a while loop - with a 20ms dt, that means that if it takes 5ms to execute the loop, once completed the loop sits idle for 15ms to let other things run before starting the next iteration.  In actuality, putting waits in a timed loop drops a bomb on your loop timing that you really don't want/need.  If the code shows really what you intended (which is to take a sample every 20 ms for a set number of iterations), remove the FOR loop within the timed loop and instead wrap the timed loop within a while loop so that each iteration of the timed loop controls the timing of a single point of data collection (you can also sum within the loop), and stop the timed loop after a set number of iterations and then perform the averaging outside the timed loop, and also read/write your variables outside the timed loop.

Don't worry, everyone runs into this at least once (or in some people's case - like me - much more than once) in their quest for RT programming perfection.  🙂

-Danny

[SarahW]

Hi Danny,

Many thanks for the suggestions, they were very useful!

Attached is my modified code based on your suggestions.

I think I tried adding the 20ms wait to the timed loop to see if that helped, it didn't, but I forgot to remove it again. I understand now that was a very bad idea!

I've tried enableing the RT FIFO on the shared variables before, and the 2D array doesn't seem to like it, I get no error, the variable looks good in the distributed system manager, but when i use the read/write variable VIs to read it from somewhere else i just get an empty array??

The attached code seems a bit better, but is still using 80-90% of the CPU, do you think i can make it any better, or is this as good as it will get?

Thanks again,

Sarah

[Texas_Diaz]

Yeah, I have a couple suggestions:

1. DAQmx Start and DAQmx Stop should not be done each loop iteration.  Start the DAQmx session prior to the timed loop, do the DAQmx Read within the timed loop, and then perform the DAQmx Stop after the loop.  Each read operation does not need to be started and stopped before performing another read operation.  Think about it like driving your car; the DAQmx Start is like "Crank up Car", the DAQmx Read is like "Drive to the end of the street and do something", and the DAQmx Stop is like, "Turn off your car."  If you're driving somewhere, do you need to turn off the car and turn it back on every time you turn onto a new road?  No, you crank it up, continue driving until you're done driving, and then turn it off.  Running the DAQmx Start/Stop within a Timed Loop is not recommended.
2. I see that you're using a DAQmx Wait Until Done, but you're using that between the Start and the Read.  That's not how that was meant to be used.  The DAQmx Wait Until Done was really meant for asynchronous operations, like a finite signal generation, to ensure that the asynchronous task has completed before stopping the task.  When using synchronous tasks like the DAQmx Read - which only returns once it has completed capturing its data - the DAQmx Wait Until Done is not necessary.  If you still want to use it (for future-proofing), put it outside the timed loop and immediately preceeding the DAQmx Stop operation.  The only DAQmx operation that you should actually be doing within the timed loop is the DAQmx Read operation. 
3. To be completely honest with you, you shouldn't even be starting/stopping the DAQmx task even within the While Loop.  Prior to the While Loop, you should call DAQmx Start, and then once the While Loop completes you should call the DAQmx Wait Until Done and then DAQmx Stop.  Your DAQ session doesn't change within the loop (you're not trying to reconfigure the channels you're reading, etc...) so there's no need to call any DAQmx calls within either loop except DAQmx Read.
4. If you take my advice in (3), you'll likely want to create a "Stop" button on the front panel of your VI and wire that into the condition terminal within your While loop.  Then, instead of pressing the VI's "Abort" button you'll just click the VI's "Stop" button and the DAQmx task will be cleaned up.  Otherwise, if you press the "Abort" button the DAQmx driver has to figure out that you may have aborted the VI - with a DAQmx session running - and it has to do a bunch of work up front to stop the session and clean up before it can start a new session the next time the VI is run.  This is cleaner and leaves the DAQmx driver in a better state.

Give that a try.

-Danny

[Texas_Diaz]

Oh, and about the RT FIFO on the Shared Variable - only flat data types are supported in the RT FIFO, because all of their memory has to be allocated beforehand.  I don't think you need to have an RT FIFO enabled on your shared variables (because of your use-case), but in general you don't want to read/write shared variables within the timed loop (the RT FIFO is just in case you absolutely have to, which in your case you did not).

-Danny

[SarahW]

Hi Danny,

Thanks for your suggestions, I'm glad you brought up the DAQmx stuff as I was a bit unsure about that!

If you notice in my very first set of VIs I was just doing the DAQmx Read inside the loop. When I added the Read All Available Samples property node and set this to true I lost all the timing structure in my 2D array. I want my array fixed at 21 by 1200 (21 channels and 1200 samples in 12ms, triggered). The array size started varying, I assume because the read just reads whatever is there, regardless of whether the whole acquisition has finished? Adding in the Start, Wait and Stop fixed this. At the moment it is set to finite Samples, should this be changed to continuous?

I didn't add a stop button or any shutdowns because when this is built to an rtexe and deployed down onto the RT system it won't can't ever be stopped. It probably would have been useful for development, but I had assumed it wouldn't take long to develop and I could get away without it!

Thanks for your help!

Sarah

[SarahW]

I've tried using continuous samples and wiring 1200 to the number of samples per channel input to the DAQmx read. This has fixed my array size again. I've moved to a producer/comsumer pattern (see attached picture) to strip back the read loop and avoid the -200279 error. My trigger is 50Hz, so I set the timed loop period to 20ms, this gave the -200279 error pretty quickly, presumably because I'm not reading the data quick enough and am filling up the buffer? This gets better if I reduce the period to 10ms, but presumably this means the loop will sometimes have to wait at the DAQmx Read because there won't be any data to read yet? Will this upset the timed loop?

Thank you!

Sarah