What architecture is best for accurate data logging

[Monse]

Hello,

I'm desiging some LabVIEW code for a standard DAQ application that is required to plot about 100 variables onto the screen on several different graphs and numeric indicators, as well as perform some simple feedback control and log data into a file once a second.

I've done this before, and used a simple state machine architecture, where one state takes care of my logging, and I have a timer vi in there that countsdown 1 second and then writes to file.  However, this method makes me miss a second every once in a while.

I started looking into the producer/consumer architecture as a possible remedy for this.  Because I hear it's good for running two things at different times, so I"ll have my quicker loop handling data acquistion, plots and feedback control, and my slower logging loop, executing once a second.  But I don't see how to implement this

questions:

1. is a simple producer consumer the right topology for my application?

2. when I create my queue do I create it a 100 element array (my data for logging) and then enqueue that in my producer loop from my data acquistion, then pass that to the logging VI.... this seems wrong to me, cause I'm going to be enqueing alot of 100 element arrays... and will be de-queing them slowly at once a second..

3. How do I trigger my consumer loop to execute every second, should I set it up as a timed while loop? or should something from the producer loop tell it to?

I'm sure this is a pretty standard thing to do, I'm jus tnot sure how to implment the correct architecture.

much thanks! 

[jyang72211]

When you said miss 1 second.  Did you mean that your 1s interval log will miss an entry once in a while, did you mean the indicators, or did you mean the DAQ?  

[Monse]

my log, my indicators and plots run fast and execute every iteration of the while loop, but my log will sometimes miss one of the 1second enteries

[Monse]

I found the image below at this link http://decibel.ni.com/content/docs/DOC-2431

And it seems really close to what I'm looking for, but I'm passing alot more data in my queue, about 100 values, so a 100 element array, and if this gets added to the queue every iteration of my producer then it's going to overflow, the consumer I only  need to execute every 1 second to log, but it has to be certain it will log every second, can I change my consumer to a timed loop in this case?  If you have 1 regular while loop and 1 timed loop does the priority input on the timed loop apply over the non-timed while loop or should I have both the producer and consumer as timed loops and change their priorities?   Much thanks!

[Austin_S]

Hi Monse,

In my opinion, the producer/ consumer seems to be the best architecture to do what you're wanting to do.  However, I'm kind of having a hard time understanding exactly what data you're trying to log, though.  Are you only wanting to pass one array of these 100 variables to your consumer loop to be logged once per second?  Or are you trying to have several 100 element arrays (however many are put in the producer loop) dequeued and logged once per second?

[Monse]

Hi Austin,

I would like to log 100 sensor variables every second.  In my program I have these as an array of 100 elements, in my main while loop I pull elements from this array as needed (for plots, feedback control and other things), it's this overhead and the fact that I'm not running on a realt-time OS that I believe makes my code miss a 1 second log every once in a while.  I'm new to producer/consumer architecture for data logging, so I'm not sure how I would use this design pattern to log 100 sensors every second. 

would I be feeding my 100 element array into a queue? or would I be feeding something else into a queue? maybe some kind of flag that indicates when one second is up? how would the 1second timer work?

much thanks!

[Austin_S]

Monse,

Thanks for clearing that up.  Yes, you can feed the array directly into the queue in your producer loop. I've attached a really basic example I put together to just show you how to go about enqueueing an array like that.  Just a note- on the Obtain Queue VI, make sure you set your data type to an Array Constant with a DBL numeric constant inside (I'm assuming you just have a 1-D array of these elements to enqueue. 

As far as timing goes, add a Wait until Next ms Multiple VI directly into your consumer loop and add a constant of 1000 ms to your "millisecond multiple" input.  That should do it.  That way, your consumer loop will only execute once per second.

[Monse]

Hey Austin,

If I enqueue that 100 element array of sensor data in my producer loop on every iteration (I'm reading all my DAQ sensor channels every iteration of my while loop and displaying these on plots) wont my queue be very large, since my consumer loop only needs the 100 element array every one second, it doesn't need anything prior to that 1 second just at the 1 second.... seems like my queue is going to be full of data that the consumer doesn't need....

[Austin_S]

Ok, let's try this.  I've put together an example that should do what you need.  I put notes in the block diagram, but essentially it runs data in a while loop at whatever execution rate you specify, then sends the data to another graph (or in your case, a log) every one second.  Basically, I've used a 100ms execution rate for the while loop, then every 10th time (you can change this if you want), it sends a boolean 'true' to a case structure within the while loop that contains the enqueue element.  The graphs that I included show that it does indeed add a new point to the second graph once a second while the first one is adding a point every 100ms.

The actual wiring of this Vi could be cleaner for sure, but it was a quick and dirty example I put together.  Hopefully this will help you accomplish what you're trying to do.