07-19-2021 03:03 AM
Hi!
Using more and more queues in my projects, I realized that I still don't really know how they actually work when running on a computer, and more importantly what considerations toward perfomances should I look out for?
As an easy way to understand them, I always liked to see a Q reference as "a sort of pointer to a sort of array in the memory" (I know ... but it does help to understand in my opinion) That way, when you "Obtain a queue", you're actually allocating memory for your data type (with a fixed size or not) queuing is adding an element to the array, dequeueing is removing the first element in this array and then releasing clears your memory.
But what is it exactly?
How much memory does it take really?
What is the maximum recommended number of queues to obtain and work with at the same time (depending on your machine I guess)?
What memory is it actually using, RAM or ROM?
How big the data transmitted through a queue can be?
What happens exactly if you name a queue and how does LV finds an existing named queue?
I couldn't find any detailed explanations, and I'm curious.
If you have answers and weblinks, I would gladly read them 🙂
Best,
Vinny.
Solved! Go to Solution.
07-19-2021 03:28 AM - edited 07-19-2021 03:29 AM
In short, queues are very efficient.
You should use them when called for. If you need a queue, don't look further and use a queue.
Some neat tricks can be done with single element queues...
@VinnyAstro wrote:But what is it exactly?
Probably a pointer to an array of pointers, except maybe for small data types.
@VinnyAstro wrote:How much memory does it take really?
As much as the data, plus a little overhead.
@VinnyAstro wrote:What is the maximum recommended number of queues to obtain and work with at the same time (depending on your machine I guess)?
IIRC, there's a limit of 2^12 queue references per data type.
With normal designes, I won't worry about it. With special designs, test it.
@VinnyAstro wrote:What memory is it actually using, RAM or ROM?
RAM.
Any other memory would be terribly slow. Of course if you don't have enough memory, RAM will be swapped to disk and things will get very slow indeed.
ROM is traditionally read-only, and can't be written to at all.
@VinnyAstro wrote:How big the data transmitted through a queue can be?
42.
But seriously, queues are efficient. The data is usually not copied into the queue and copied out of the queue. In stead, pointers are used, to avoid copies. When possible of course.
@VinnyAstro wrote:What happens exactly if you name a queue
When you name queues, all hell breaks loose 😂.
Naming queues means creating global resources.
Global resources encourage coupled code.
Global resources means you can (and eventually will) get naming collisions. These can be very, very hard to find.
@VinnyAstro wrote:
and how does LV finds an existing named queue?
LabVIEW probably creates a simple LUT (hopefully tree based for fast searches). This is still log(o) at best (I think) so when you get more names, the look up will get slower.
07-19-2021 03:29 AM
Hi Vinny,
@VinnyAstro wrote:
How much memory does it take really?
What is the maximum recommended number of queues to obtain and work with at the same time (depending on your machine I guess)?
What memory is it actually using, RAM or ROM?
How big the data transmitted through a queue can be?
What happens exactly if you name a queue and how does LV finds an existing named queue?
Memory requirements depend on the amount of data you put into the queue. (With a small overhead for managing the queue elements.)
You can use as many queues as you like. (I guess there might be a maximum, somewhere at a power of two like 2^15m 2^16 or 2^20…)
Why do you even think about using ROM (read only memory) to store random data?
The queue element can take as many memory as is available (to LabVIEW)…
When you access named queues then LabVIEW will access the same queue throughout the whole project/executable…
07-19-2021 03:55 AM
Thanks for the quick answers 🙂
wiebe@CARYA a écrit :
In short, queues are very efficient.
You should use them when called for. If you need a queue, don't look further and use a queue.
I am. I've started working extensively with queues about 6 months ago (I'm still rather unexperienced with labview while I've be trained and also followed some tutorials), and in my "learning by doing" I've started creating queues everywhere for all of my "modules" (General UI, Application Control, Data Logging, Acquisition, Devices control ... you name it.) The most I created in one application was 10.
That's when I started to wonder if that was too much for a laptop (or another computer) to handle or not.
While neither the RAM nor the processor usage were going crazy, the "Power usage" in the Task manager would go to "Very high" (whatever that means) so I got curious.
I guess that 2^12 queues is a sufficient limit then ^^
wiebe@CARYA a écrit :
When you name queues, all hell breaks loose 😂.
Naming queues means creating global resources.
Global resources encourage coupled code.
Global resources means you can (and eventually will) get naming collisions. These can be very, very hard to find.
Good to know!
I've seen it being used in a few designs here and there to easier the look of their code (I guess some could call that lazyness^^) for instance to update a status from a random sub-vi in their code without to have to pass the Q reference wire all the way there.
I might have done it the same way from time to time as a curiosity, to test .... and also out of lazyness 😛
Best,
Vinny
07-19-2021 04:25 AM
@VinnyAstro wrote:
Thanks for the quick answers 🙂
wiebe@CARYA a écrit :
In short, queues are very efficient.
You should use them when called for. If you need a queue, don't look further and use a queue.
I am. I've started working extensively with queues about 6 months ago (I'm still rather unexperienced with labview while I've be trained and also followed some tutorials), and in my "learning by doing" I've started creating queues everywhere for all of my "modules" (General UI, Application Control, Data Logging, Acquisition, Devices control ... you name it.) The most I created in one application was 10.
That's when I started to wonder if that was too much for a laptop (or another computer) to handle or not.
While neither the RAM nor the processor usage were going crazy, the "Power usage" in the Task manager would go to "Very high" (whatever that means) so I got curious.
I guess that 2^12 queues is a sufficient limit then ^^
wiebe@CARYA a écrit :
When you name queues, all hell breaks loose 😂.
Naming queues means creating global resources.
Global resources encourage coupled code.
Global resources means you can (and eventually will) get naming collisions. These can be very, very hard to find.
Good to know!
I've seen it being used in a few designs here and there to easier the look of their code (I guess some could call that lazyness^^) for instance to update a status from a random sub-vi in their code without to have to pass the Q reference wire all the way there.
I might have done it the same way from time to time as a curiosity, to test .... and also out of lazyness 😛
Best,
Vinny
Well, some say programmers should be lazy and stupid 🙄. I have.
Lazy, as you should try to do as little as possible. Not less. But also not too much more.
And stupid, because you always have to think that there are smarter ways to do what you're doing. (A lazier way, probably).
In fact, both are really smart things to do of course.
The thing is, what is lazy is circumstantial. A names queue can be OK if it is helping the situation. It's not for me, because it doesn't help my situation. Some decisions are almost always incorrect, but there are very little decisions that are never correct. A stacked sequence structure is almost never called for, but I won't say it's never useful.
I choose not to use named queues anymore, as I've been burned by them. But a lot of programmers might never get to the point where they'll get burned. For me, not naming queues is the lazy way.
I'm pretty confident to say though that it's pointless to name queues if the intent of the name isn't to make the queue global. Because that is what it does.
For documentation, give the data type a label, and the wires will get that label. CTRL+H will show the queue wire label. That works even better than naming the queue.
07-19-2021 11:32 AM
You can look at the following thread,
https://forums.ni.com/t5/LabVIEW/Queue-Memory-Allocation-Weirdness/td-p/1988609?profile.language=en
Not sure if things have changed, memory has become super cheap now. Queues are more like a circular buffer if you go through that thread.
mcduff