Queue or Shift Register for Lossless data?

[TooEagerToLearn]

I have multiple queues from various hardware that I need to collect data from and queue them again in the consumer loop to dequeue in the DATA LOGGING loop (I am mostly doing this for timing) and also my consumer loop performs other actions in between reading data.

Is it more efficient to use queues (as many as required) to pass data or would it be easier to use AE's (shift registers) to store data (Write) and retrieve data (Read) to avoid the criss cross of multiple wires?

[RavensFan]

Why do you think you need multiple queues?

If you have one consumer loop, then you only need 1 queue.  Pass the reference to that queue into all other parts of your code that are collecting data.  If you have a second consumer loop, then use a second queue.

[crossrulz]

There is a huge difference between Action Engines and Queues.  Queues are a LOSSLESS form of transport.  This means that every element you add into the queue will be dequeued seperately.  An Action Engine is a LOSSY form of transport.  It only holds the latest data.  This means that you could be getting duplicate data and/or miss data points.  Niether one is good for a logger.

So the short answer is you want to use a queue.

Longer answer: compbine the two.  Put the queue inside of an Action Engine.  Have your logger loop initialize the queue and initialize the Action Engine.  Then everybody else can use that AE to send the commands.  Make sure the Dequeue is NOT inside of the Action Engine or you will have a deadlock situation.

[TooEagerToLearn]

Is there an example of using the same queue and passing data from different producers? I am not sure how it would impact my data logging in terms of timing. Some loops run as fast as possible and some run at predetermined rates.

Now that I think about what my application needs, Crossrulez just answered my question. I do need the latest data and not necessarily queued data (I thought I did). Because I am using a queue, I was having data misalignment on a 2D map - so I was flushing the queue in all other states EXCEPT that state I actually needed the data from.

Queue inside an AE seems very intriguing and I would like to explore that just for learning and future implementations.

[crossrulz]

---

@TooEagerToLearn wrote:

I do need the latest data and not necessarily queued data (I thought I did)

---

This is what we call Tag Data.  Tag data is when you only care about the latest data point.  Best done with a single writer (for a single piece of data).  If going this route, I would recommend just using a Global Variable.  They are extremely fast (MUCH faster than an Action Engine/Functional Global Variable).  Again, make sure you only have 1 place that the variable is being updated at.

[Gregory]

Here is a library I've made which I copy and paste into a project folder and then rename (and edit the icon). I've also included an example of what your module might look like (the part that does the dequeuing).

Edit: It should be easy in module to create new cases for "Temp Data", "Power Data", etc., which is how you can have multiple loops sending data and the logger knows what to do with each one. The data is a variant, so you could have it include a cluster of timestamp and numeric, for example.

And I would be very interested in others' suggestions for how I could improve it.