When to use lossy Enqueue Element vs Regular Enqueue (non-lossy) Element?

[cayenne]

Hello all,

I'm starting to do my first run with queues...running between two loops. I've seen examples using Lossy versus non-Lossy Enqueue functions, and was wondering programmatically and philosophically, when is the best time to use each one? What are the "gotchas" for going one way or the other?

Thank you in advance,

cayenne

[crossrulz]

Personally, I think that with queues I want every single piece of data that hits that FIFO.  So I don't use the lossy enqueue.

But now that I'm sitting here thinking about it, the lossy enque would be perfect for a circular buffer.  I wouldn't be using it as a messaging construct, but as a data holder.  Ok, time to go play around with some ideas...

[P@Anand]

I had one instance where I have to use lossy Enqueue element. We run the test and mean while we have to hold a 20 second of data all the time so we keep on writing the data to the queue and once the data is overflown it will automatically remove the older data and enqueues the new data so we always have 20 second of data in the buffer and when a failure occurs we flush the 20 second data to a file so that is easy for us to analyse the error cause.

For the other type of normal operations like passing the data from loop to loop you can use normal enqueue elements.

I am not sure in persormance wise which one leads but they both have significant use.

[crossrulz]

---

@P Anand wrote:

I had one instance where I have to use lossy Enqueue element. We run the test and mean while we have to hold a 20 second of data all the time so we keep on writing the data to the queue and once the data is overflown it will automatically remove the older data and enqueues the new data so we always have 20 second of data in the buffer and when a failure occurs we flush the 20 second data to a file so that is easy for us to analyse the error cause.

---

Exactly what I meant by circular buffer.  And exactly what I was trying to code up real quick.  Glad to hear that it works well.

[P@Anand]

---

@crossrulz wrote:

---

@P Anand wrote:

I had one instance where I have to use lossy Enqueue element. We run the test and mean while we have to hold a 20 second of data all the time so we keep on writing the data to the queue and once the data is overflown it will automatically remove the older data and enqueues the new data so we always have 20 second of data in the buffer and when a failure occurs we flush the 20 second data to a file so that is easy for us to analyse the error cause.

---

Exactly what I meant by circular buffer.  And exactly what I was trying to code up real quick.  Glad to hear that it works well.

---

Great. Do you have any idea of doing benchmark between these two. It would be very good to know what is the performance difference (I haven't tried before)

[crossrulz]

Due to their completely different needs, I'm not sure a benchmark would really be of any worth.  Off the top of my head, I would think that the lossy enqueue would be faster since it doesn't have to ask for more memory when its full (assuming the "regular" queue has an unlimited size while the lossy has a fixed size).

[P@Anand]

---

@crossrulz wrote:

Due to their completely different needs, I'm not sure a benchmark would really be of any worth.  Off the top of my head, I would think that the lossy enqueue would be faster since it doesn't have to ask for more memory when its full (assuming the "regular" queue has an unlimited size while the lossy has a fixed size).

---

No the size of the queue will be defined when we create the queue so its not a variation whether we use the lossy enqueue or the normal one. I don't think there will be a big difference in their performance. Anyway good to know all these. Thanks.

[crossrulz]

---

@P Anand wrote:

No the size of the queue will be defined when we create the queue so its not a variation whether we use the lossy enqueue or the normal one. I don't think there will be a big difference in their performance. Anyway good to know all these. Thanks.

---

Well, if we define the size, the enqueing should perform the same when not full.  The enqueuing for the lossy will be much faster when full because it can just act.  The regular will have to sit there until there's a spot to put the data.

[P@Anand]

---

@crossrulz wrote:

---

@P Anand wrote:

No the size of the queue will be defined when we create the queue so its not a variation whether we use the lossy enqueue or the normal one. I don't think there will be a big difference in their performance. Anyway good to know all these. Thanks.

---

Well, if we define the size, the enqueing should perform the same when not full.  The enqueuing for the lossy will be much faster when full because it can just act.  The regular will have to sit there until there's a spot to put the data.

---

Yeah there comes the timeout into play for the regular Enqueue. So we have to check by setting the timeout to zero and while generating the error the lossy enqueue may outperform the regular enqueue (Just a guess am not sure about the actual)

[johnsold]

Not just timeout. With a finite size queue the regular Enqueue Element will block until timeout or until an element is dequeued elsewhere.  If the Enqueue uses the default -1 timeout, it will block forever.

If the blocking enqueue is in the same structure node as the dequeue, the program will lock up.  This configuration is seen in some Queued Message Handlers, so the finite queue should not be employed in such situations unless the programmer can guarantee by design that the queue will never get full.

Lynn