Obtain Queue memory leak?

[cmteuffel]

I think there might be a memory leak in "Obtain Queue.vi". It does NOT occur in a simple setup like this:

Instead it seems to be connected to multithread setups, where several open references might exist. This is escpecially cumbersome since "Obtain semaphore Reference.vi" makes also use of "Obtain Queue.vi". The effect is insignificant with respect to memory consumption. But it affects cpu usage if a lot of queue operations occur in a long running application. I have an application that calls "Obtain Queue" at least 30 times per second and runs for several days. After 24h it likely holds more than 2,5 Mio unused memory references. The memory management makes cpu usage go up from 15% to 100%. A good hint that this behavior really is from memory management is the fact that that closing the application also takes longer the longer it ran before. The leaks can be reproduced using the following test procedure.

According to the attached trace from "Desktop Execution Trace Toolkit" this test produces lots of memory allocations that are never freed up.

[Ben · Accepted solution]

That behaviour is normal and expected.

You can get similar effects by opening a VISA ref.

Refs in LV have resource associated with each and those resource are allocated everytime you open the ref. Closing the refs mark them as in valid but does not dealoocate the resources.

The accepted practice for avoiding the issue is to cache the ref once opened and to use the same ref repeatedly. The ref can be tested to see if it is valid and only when it is not valid, open the ref.

Ben

"Back in the day..." of LV 5 or there-abouts... using a ref that had been closed could result in a blue screen of death. I like it much better now.

[cmteuffel]

Thanks for this hint Ben,

this could be a partial workaround. But since "Obtain Semaphore Reference" makes also use of "Obtain Queue" and is not using your trick, the issue persists for almost all multithreading applications.

[cmteuffel]

No Wait .., you are right. Of course I can cache that semaphore reference as well 🙂

[AristosQueue (NI)]

Every call to Obtain using a name will either create a new queue/semaphore or return a reference to the existing q/s. Regardless of which one happens, a small amount of memory is allocated for the reference itself, so if you open multiple references to the same named q/s, each one of those is a small allocation.The q/s will be destroyed when all the references to that q/s are released. *Every* call to Obtain needs a matching Release (unless you rely on the LV behavior that all will be released when the VI goes idle -- not a recommended practice but a safeguard we include just in case). You can have the Obtain and Release both inside the same loop and it works fine assuming you have at least one q/s obtain and release pair outside the loop to keep the queue in existence... otherwise, every call to the Obtain is creating a new queue and every call to Release is destroying it.

This is documented in multiple places in the online help, ni.com and LAVA forums. Search around if you want more details...

[cmteuffel]

---

@Aristos Queue wrote:

.. *Every* call to Obtain needs a matching Release ...

---

Your statement is not true in every case. "Release Queue" has a parameter "force destroy" that I connected with TRUE in the example code given above: "If TRUE, the function destroys the queue and you do not have to call the function multiple times or stop all VIs using the queue reference. Destroying the queue invalidates all references to the queue." The trace attached to the first post gives hints that this behavior is not working as I expected. It seems there are memory allocations coming from "Obtain Queue" that are left open. Ben stated correctly that the manual reads: "destroying the queue invalidates all references". It does not deallocate them. One therefore has to be careful with frequent calls to "Obtain Queue" even if there is a call to "Release Queue" inside the same loop.

[Ben]

It is not just queues.

It is all refs that allocate resources.

It can be prooved by writing a VI that repeatedly opens and closes a VISA ref. Let it run as fast as possible, mark the memory useage. Let it run over the week-end and Monday morning will show increased memory usage. It wll be released when the app is stopped and closed.

Ben

[cmteuffel]

Just a little update: Removing all calls to "Obtain Semaphore Reference" and to "Obtain Queue" by using several single cached references instead solved my cpu usage problem. But in my opinion this is merely a dirty workaround which is not obvious from the manual.

[AristosQueue (NI)]

---

@cmteuffel wrote:

---

@Aristos Queue wrote:

.. *Every* call to Obtain needs a matching Release ...

---

Your statement is not true in every case. "Release Queue" has a parameter "force destroy" that I connected with TRUE in the example code given above: "If TRUE, the function destroys the queue and you do not have to call the function multiple times or stop all VIs using the queue reference. Destroying the queue invalidates all references to the queue." The trace attached to the first post gives hints that this behavior is not working as I expected. It seems there are memory allocations coming from "Obtain Queue" that are left open. Ben stated correctly that the manual reads: "destroying the queue invalidates all references". It does not deallocate them. One therefore has to be careful with frequent calls to "Obtain Queue" even if there is a call to "Release Queue" inside the same loop.

---

Force Destroy... a technicality that I had forgotten. Force Destroy does both invalidate and deallocate the references and the queue itself. The refnum table is still large enough to hold as many refnums as has ever been needed. If you have an Obtain and Release both in the same loop then the number of refnums never gets large, so no problem there.

[AristosQueue (NI)]

---

@cmteuffel wrote:

Just a little update: Removing all calls to "Obtain Semaphore Reference" and to "Obtain Queue" by using several single cached references instead solved my cpu usage problem. But in my opinion this is merely a dirty workaround which is not obvious from the manual.

---

*blink* We're talking about memory allocations and you turn it into a CPU usage problem? Huh? If the problem was CPU usage then *of course* you're using CPU cycles every time you create a new refnum.