Multiple producer one consumer

[ANDRES_VANEGAS]

Greetings, I am developing an application where data are obtained at different rates and different processes, but I want to store in one file, store this data as I can if I have multiple producers and one consumer??

[Omar_II]

If the data is the same you can use a single queue with Multiple producers pushing data on to the single queue and a single consumer loop read the queue.

[RavensFan]

Yes, you can.

[JÞB · Accepted solution]

Sounds easy enough- if you use a queue!  

There is a pretty good example called Queue Multiplexer that ships with LabVIEW and does collect data from multiple Producers.

[ANDRES_VANEGAS]

Thanks friend... Grettings of Ecuador... Latin America

[johnsold]

As others have said a single queue works well with multiple sources.  If you need to distinguish which loop was the source of the data, you can make the datatype of the queue a cluster with one element identifying the source and another containing the data or use a separate queue for each source.

Lynn

[VeeJay]

On the same note, I have a question (I digress a little)

What if I need two producers and one consumer loop that receives data from both through 2 queues because data is not synchronized or more like they cannot be synchronized and hence cannot be pushed into the same queue?

Can we use 2 queues and dequeue both in the consumer loop (which is a state machine by the way) and perform actions from what is received?

Please refer to the attached example. thanks!

[Mark_Yedinak]

You can but you will basically turn it into a polling loop since you will need to use small timeouts so both queues can be checked. However I don't know why you can't use a single queue for the producer. Each consumer posts its message to the single queue and you process the data in the order they were queued. If you truly need the data streams to be independent then you may need to have a processing task for each data stream.

As someone mentioned earlier, if you need to identify the source of the data then create a message structure that includes that as part of the data. Your processing task can then take different actions based on the data source.

[johnsold]

Yes,

You can use two queues that way.

Because you have not timeouts on either Dequeue function, the loop will wait until data is present at each queue and will remove one element from each queue on each iteration.  If one producer runs faster than the other, eventually one queue will overflow.

Repeating the VISA read in the consumer (commands 163 and 164) defeats the purpose of the producer consumer architecture.

If "boolean" is false, then nothing ever gets Enqueued into the command queue and the consumer loop will do nothing, including stop.  Well, maybe stop will work because the queue will get destroyed after Producer 1 stops throwing an error in the consumer.

I prefer to dispose of queues after I am sure that all data has been removed and do not like using that error to stop the loops.  It is too difficult to assure a graceful shut down. Similarly the way you close the VISA session in three places has me wondering how many VISA errors you get.

Lynn

[Mark_Yedinak]

I took a look at your code and you might run into problems with the various ACK messages you send. You send them in parallel based on the data you read. However, there is no guarantee which one will get sent first. If you need to make ensure the order of the ACKs then I don't think you want to process the receive data in parallel like that. Also, as previously noted your consumer loop will ONLY operate when there is data at BOTH queues. For a given consumer task I prefer it to use a single queue and define a more generic message structure that will allow multiple producers to post data of differing types. If you have a limited and finite set of data types you can use a typedefed cluster. This will mean that for each message one or more elements of the cluster will have no meaning. You could use a variant as the data type and the message ID will indicate how that data should be interpreted. If you want to use a LVOOP solution you can define a base message class. Part of this again is the ID. The children classes would have the necessary methods for dealing with the specific messages and data types. Then your elements are an instance of the message class.