producer -consumer code with several variables (types)

[Flo-w]

Hi there,

I have developped this VI and, so far, it is running fine. It does the job but as stated in a precedent post, if I set my loop to 1s (for ex. to get the amount of injected fluid by my pump every second) I will lose in accuracy as the loop will be slowed down by the large amount of interaction with my devices, even in the case of a timeout event.

I have been advised to look throught the Producer-Consumer setup, and I get what it does. what I don't get is how to implement that solution in my case: I am controlling two devices, displaying real time data (pressures, etc...) and have to send informations to another VI (let's call it the recorder). my data are strings, doubles, etc...

Could someone help me and eventually guide me in the process of re-designing my VI in more... efficient way?

Cheers everyone!

Flo

[Bob_Schor]

Hello, Flo.

I don't see a Producer/Consumer Design Pattern here, but that is possibly because there is too much "clutter" and not enough "encapsulate the Messy Details into a sub-VI" (incidentally, you don't need to put (SubVI) in the sub-VI's name -- starting it with "Fct", which I assume stands for "Function", is an equivalent Developer Cue).

You actually have several possibilities for P/C (Producer/Consumer, not "Politically Correct") code here, as well as implementing a Queued State Machine.  But before I "start a Tutorial", I want to point you to some reference material that you might already have seen.  If you open LabVIEW and click the File button, choosing "New ..." (the three dots are significant), you can choose New, VI, From Template, Design Patterns, and then choose P/C Design Pattern (Data) and P/C Design Pattern (Events).  You want both of these.

You have three things going on -- responding to User Interaction with the Front Panel (something that rarely happens, if measured as "fraction of time LabVIEW spends handling the Value Change"), "Data Generation" (which includes the processing of the Front Panel stuff to set controls, start, stop, etc. the Who-What-When-Where-and How of acquisition), and "Data Processing", which includes displaying the data and possibly saving it to disk.

Each of these wants to run in its own loop, asynchronously with the others.  The Event Loop is the easiest -- if you put an Event Structure inside a While Loop, it will simply "wait" until one of the Events inside it "fires", then it does "something quick and simple".  In the case of the P/C, it enqueues a State for the Queued State Machine.

[Side note -- there is no Design Template for a Queued State Machine, but there is one for its Big Brother, the Queued Message Handler, which I think of as a "Queued State Machine with Optional Parameters".  If you are really ambitious and want to learn and expand your LabVIEW skills quickly, take a look at the QMH, which I think is a "Project Template", which you can find by opening LabVIEW and clicking Create Project and choosing QMH from the choices presented.]

Do you know about State Machines?  There's a Design Template for a Simple State Machine in Create Project that illustrates the basic idea -- there's something called a State (either an Enum or a String) that feeds into a Case Statement inside a While Loop, with each Case being the code to carry out when in a particular State.  The Simple State Machine uses a Shift Register to hold the State -- the Queued State Machine uses a Queue to hold (potentially a list of) States, dequeuing them inside the While loop and feeding the "next State" to the Case Structure.  So when considering the Event P/C Design, the Event Loop says "You push the Start Button" and puts the State "Start" on the State Queue, the State Machine dequeues it and does whatever code is appropriate for Start.  The interesting step is what happens when your State Machine is actually taking data.  You want to make a State called "Take One Sample" (I hope I don't have to explain what you do in this State).  The trick is that at the end of this State, you decide (based on whether you'd pushed the Stop button and the State Machine, in processing the "Stop Me" State, had put "True" on a "Stop Me" Shift Register) whether to do the "Stop Acquisition" State next or to do another "Take One Sample" State.  There's nothing to prevent the Queued State Machine from enqueuing its own "next State" -- you just have to join up the Enqueue Element function with the State Queue.

So where is the second P/C design, and the third (Consumer) loop?  In Take One Sample, you get data.  What do you do with it?  You put it on a Data Queue and say "Let's let the Consumer take care of this".  You now make a third parallel loop (I arrange these loops vertically to give the mnemonic appearance of "These all run simultaneously") that has a While, a Dequeue Element, and whatever code you need to display the data, save the data, etc.

So why is this good?  Well, the main State Machine, while it is doing Take One Sample, is also mostly idle, as it is waiting for the next Sample to be produced.  Once it gets it, it "gives it away", and goes back to waiting.  While everything else is idle, the Consumer "consumes" the data.  Suppose while the Consumer is consuming, you push a button or the Producer says "I've got the next Sample ..." -- LabVIEW will recognized that another loop wants some CPU time, and will "share" the CPU with whatever loops are asking for it.  So the Producer will get its slice, the Event Loop will also get its slice, but since both of these are designed to be really quick, there will be plenty of cycles available to the Consumer, which does most of the "heavy lifting".

Give it a try.  Do try to have many more VIs so that your top level code fits on a single (small) screen showing lots of VIs, and the overall 3-loop structure is evident.

Good luck.  Post your code and we'll make more suggestions.

Bob Schor

[Flo-w]

Thank you very much for this very long precise reply, I read it twice and I will probably need two or three more try to get the substance out of it!

my problem is not how to implement the P/C scheme, but more how to make it with many variables treated throught time. can I use more than one queue ? (say seven for seven variables ?)

Flo

[Blokk]

---

@Flo-w wrote:

Thank you very much for this very long precise reply, I read it twice and I will probably need two or three more try to get the substance out of it!

my problem is not how to implement the P/C scheme, but more how to make it with many variables treated throught time. can I use more than one queue ? (say seven for seven variables ?)

Flo

You can use a single Queue to send different data types if this is what you ask for?

Besides, be careful with naming in LabVIEW, call it wire or data(type), but not variable! 🙂 Variables are different things in LabVIEW (local variable, (Functional) Global Variable, Network shared variable, etc...), not the same naming used here as in other text languages...

In the linked post below I show an example how to use a two producer/one consumer design with the help of variants. So you can just manage all this with a single Queue, and your string (or enum) inside the cluster will "tell" the consumer loop how to convert the variant data back to the required data type, and what to do with it...

http://forums.ni.com/t5/LabVIEW/TDMS-logging-from-multiple-loops/m-p/3368581/highlight/true#M991756

[crossrulz]

---

@Flo-w wrote:

my problem is not how to implement the P/C scheme, but more how to make it with many variables treated throught time. can I use more than one queue ? (say seven for seven variables ?)

---

Use a cluster for your queue data type.  The cluster can contain an enum (or a string) and a variant.  The enum tells the consumer what the data is and the variant actually holds the data.  The consumer will need to use the Variant To Data to use the data, but that is a small price to pay.

[Bob_Schor]

Time for a "time-out" to write some documentation and think about what you are doing.  You are talking about multiple data streams and how to handle them, and have gotten several suggestions.  But a key question (or questions) is what is the nature of the data and what sort of "handling" does it need?

Here's an example from my own work.  I'm controlling an Experiment, and have a number of channels of analog data that are being sampled at 1KHz.  I also have what I call "Event" data, such as "The Subject pushed a button", or "We turned Laser 3 on" or "It's time to end this phase of the stimulus and start the next phase" -- these are characterized by being "occasional", "infrequent", and "varied" (i.e. sometime On/Off, sometime a single numeric value).

Because the data rates for these two streams are so different, each is assigned its own P/C Queue and Consumer (there is actually a single Producer that most of the time handles analog data, but occasional puts an "Event" on the Event Queue).  The Analog Queue goes to a Consumer that (a) streams the data to disk and (b) plots a decimated Chart (every 50th point, so the chart updates at 20 Hz, showing me the last 30 seconds of data), while the Event Queue's Consumer (a) streams the data to a different file on disk, (b) sets/resets some Front Panel indicators so I can tell what's going on, and (c) can "react" to such Events as "The Subject Pushed the Button".

So are your varied data streams "associated"?  Do you have a single Producer that can bundle them all into a Cluster?  You may deal with different elements differently, but can they all be dealt with "at once" or does the processing need to be separated?  The nature of your data and how you need to interact with it will largely determine the optimal data structure(s) and method(s) of handling them.

If you haven't done so already, time to break out that indispensible LabVIEW Tool, the Document Creator (either Word, LaTeX, or Something Else).  Setting some of this down "on paper" (who uses paper?) will not only clarify your thoughts, but, if shared with us, will allow specific suggestions for you to consider.

Bob Schor

[Flo-w]

Sorry for the very late reply, I think I need to sit down as you guys said, and to think about what I need.

I am really really really lost about the way to code that way. I'll pass through tutorials and come back to you.

Flo

[Flo-w]

Ok, after a bit of reading I slowly start to see that I will have to use 3 loops:

- one that enqueues my produced data (this one would check with my DAQ/pump 1/ pump2, collect the data, and put them in a cluster)

-one for displaying the data (pressure reading, flow rate reading, etc)

- maybe a third loop, but not sure, that would record everything in a text file. maybe that piece of code could beset in the previous loop.

I see three loops because I want the first one to run at 500ms, the second one at 500ms too, and the third one would either dequeue at 500ms and average the collected data, write them every minute in a text file, or would run at 60000ms loop every minute.

Is my thinking okay?

also, here is an example of reading I get after dialing with one of my pump (see attached snippet) You can se that there is 11 data I'd like to read at every iteration. I see that I should use a cluster, but I struggle in understanding how to store those 11 data in the queue, before dequeuing them.

Thank you for your precious help !

Flo

[crossrulz]

Your DAQ loop should not have an Event Structure in it.

Displaying data is so fast, I would just go ahead and put that in your DAQ loop and then have the logging loop be your consumer.

[Blokk]

"I see three loops because I want the first one to run at 500ms, the second one at 500ms too, and the third one would either dequeue at 500ms and average the collected data, write them every minute in a text file, or would run at 60000ms loop every minute.

Is my thinking okay?"

You have your DAQ loop which is the producer. Only this is what you time, so it produces data at equal intervals. Your consumer loop (display plus file logging) does NOT need timing, when a new value is available it simply Dequeues that coming from your Producer loop (so set the Dequeu function to never time out!). Besides, you can produce a time stamp in your producer loop, which you also dequeue in the consumer, and write it into your File alongside with your data.