DataSocket memory leak problem (2VO0SF00) -- more info?

[PMCR]

When upgrading to LabVIEW 8.5 recently, I noticed the following known issue in the readme file:

"ID: 2VO0SF00

DataSocket/OPC Leaks Memory using ActiveX VIs to perform open-write-close repeatedly
If you call the DataSocket Open, DataSocket Write, and DataSocket Close functions in succession repeatedly, LabVIEW leaks memory. Workaround — To correct this problem, call the DataSocket Open function once, use the DataSocket Write function to write multiple times, and then use the DataSocket Close function."

Looking back, I think this problem may have been present in previous LabVIEW releases as well, and might be giving rise to a problem that's been dogging me for quite some time (see my thread, "Error 66 with DataSockets", http://forums.ni.com/ni/board/message?board.id=170&thread.id=187206), in addition to general slow/glitchy behaviour when my VI's have been running continuously for a long time. But in order to determine whether or not this issue affects me, and how I should go about fixing it in the context of my own programs, I need a bit more information about the nature of the issue itself and the inner workings of the DataSocket VI's. Any help or insight the community can provide into this would be greatly appreciated!

Here are my questions:

1. It is my understanding from the "known issue" description above that the memory leak happens when you have a DS Open wired to a DS Write wired to a DS Close, all inside a loop (example 1), and that the suggested workaround would be to move the DS Open and DS Close functions out of the loop on opposite sides, wired to the DS Write which remains inside the loop (example 2). Is this correct?
2. Does this leak also happen when performing DS open-read-close's repeatedly (example 3)?
3. What happens when a DS Write (or DS Read) is called without a corresponding DS Open and DS Close (examples 4a and 4b)? Does it implicitly do a DS open before doing the write operation and a DS close afterwards? What I'm getting at is this: would having an isolated DS Write (or DS Read) inside a loop, not connected to any DS Open or DS Close functions at all, cause this same memory leak?
4. If one computer is running the DS server and a second computer is running the VI with the repeated open-write-close's, on which computer does the memory leak occur?
5. In my question #1 workaround (example 2), the DS Open and DS Close outside the loop are routed through a shift register and in to and out of the DS Write inside the loop. If the DS connection id goes into the DS Write "connection in" and then splits and goes around the DS Write and out to the DS Close, without coming out of the DS Write "connection out" (example 5), will the memory leak still be avoided? I.e. if the DS Write function doesn't have anything connected to its "connection out", will it try to do an implicit DS Close?
6. If the VI causing the memory leak is stopped, but LabVIEW stays running, will the leaked memory be reclaimed? What if the VI is closed? What if all of LabVIEW is closed?

FYI, in the examples above "x1a" is a statically-defined DataSocket on the DS server running on the computer Max, to which the computer running the example VI's has read/write access. My actual application has numerous VI's and hundreds of DataSocket items, many of which are written to / read from every 50-100 ms in the style of examples 4a and 4b.

Does anyone have any idea about this stuff?

Thanks in advance,

Patrick

[MeghanK]

Hi Patrick,

I will do my best to answer your questions.

1.) Yes, you are correct.  The workaround is to place the DS Open and the DS Close outside the while loop.

2.) It does not state that the DS Read VI follows the same rules as the DS Write VI, but it is good programming to always have VIs that are opening and closing references outside of a loop.  LabVIEW will slow down if it has to re-open and close the reference many times throughout a program. 

3.) When you place a DS Read or Write VI on your block diagram with no DS Open or DS Close VI, it is going to open and close that reference everytime that Read or Write VI is called.  Therefore, it is basically the same as putting the DS Open and DS Close VI inside the loop, which will cause memory leaks.  This should be avoided.

4.) I am very confident that the memory leak will occur on the computer running the VI, but this can be avoided all together by placing your DS Open and DS Close VIs outside of your loop. 

5.) Your code in example 5 where the DSC Write VI is split from the original path, will not affect whether or not there is a memory leak.  The only step you can take to verify that there will be no leaks, is to place your DS Open and Close VIs outside of the loop.

6.)  The leaked memory will only be reclaimed if LabVIEW is closed.  Stopping LabVIEW will not reclaim the leaked memory.

It sounds like you have a fairly large application to work with, and adding the DS Open and Close VIs will take some time.  However, this is the best route for you to go if you want to minimize memory leaks and maximize performance.  If you have anymore questions, please let me know.  Thanks Patrick!

[PMCR]

Hi Meghan,

Thank you for your informative reply! I have begun the process of revising my VI's; it sounds like it will be worth the time invested.

Just to clarify, in your answer #5, when you say that splitting the DS Write VI from the original path will not affect whether or not there is a memory leak, does this mean that the "example 5" VI I attached above will not have a memory leak (because its DS Open and Close reside outside the loop)? That is, the fact that the DS Write inside the loop has its "connection ID out" connector not wired to anything won't make the DS Write try to close the reference everytime it runs?

The reason I care about such a split-path example is that in some of my more complicated VI's, too many different DataSockets are accessed inside a given loop for it to be feasible to make a shift register or tunnel for each one, so outside the loop I am combining the connection ID's from the DS Opens into an array, then passing this array into the loop and splitting off the individual elements there, before the array exits the loop and sends elements to various DS Closes. It's a bit hard to describe, but I've attached an "example 6" to this post in which this is implemented. I just want to confirm that this implementation avoids the memory leak.

Thanks,

Patrick

[MeghanK]

Hi Patrick,

You are correct that having the DS Write VI inside the while loop, without the reference exiting the VI and being passed directly to a DS Close VI, will cause the DS Write VI to close the reference inside the loop.  I apologize for missing this earlier.  Therefore, you are right, your example 5 will most likely cause a memory leak.

I took a look at your example 6.  Can I ask you why you are passing your references into an array and then indexing the array?  This would make sense if you have hundreds of these connections, but if you only have 3 as shown in the example, I would suggest passing the reference from the DS Open VI directly to the DS Write, and then pass that reference directly to the DS Close.  If this will not work for your application, than I would recommend taking the reference outputs of the DS Write VIs, and building an array similar to the first part of your VI.  I included two pictures of what I would recommend for you to try so that you can avoid memory leaks.  Let me know what you think, thanks Patrick!

[PMCR]

Hi Meghan,

Yes, some of the larger VIs in my application do write to / read from several hundred DataSockets, so it's not feasible to use shift registers for each one individually, and hence why I'm passing the references into an array, etc.

Your Alternate Solution 2 is more along the lines of something that would work for me. However, my actual code has a lot of nested loops, sequences and DataSocket items which are not all written to in the same frame, so this solution would still be difficult to implement: it would be cumbersome to unpack the entire 500-element reference id array and build a new one (maintaining the positions and values of the unaffected elements) every time I write to some small subset of the DataSockets.

I think I have a solution which solves the problem and is also scalable to the size of my application -- I've attached it as Example 7. Do you think this will avoid the memory leak? It's the same as your Alternate Solution 2, except that instead of building a new array out of the DS Write reference outs, each reference out replaces the appropriate element of the original array.

If I understand you correctly, in order to avoid implicit reference opens and closes, a DS Write needs to have both it's reference in and reference out wired to something. Thus, even though my Example 7 replaces an element of the array with an identical value, and therefore doesn't actually change the array (which would be a silly thing to do normally), the DS Writes have their reference outs wired to something, and eventually in a convoluted way to a DS Close, so it should avoid the memory leak.

Just out of curiosity (I don't think anything like this would apply to my application or any fixes I implement), when would the implicit reference close happen in the attached Example 8? The DS Write has its reference in and reference out both connected to temporally "adjacent" DS Writes via the shift register, so perhaps it wouldn't try to close the reference on each loop iteration? Or would it look into the future and see that there is no DS Close and decide to implicitly do that itself? Or maybe only the DS Write on the last loop iteration does this?

Thanks for bearing with me through this,

Patrick

[RavensFan]

What if you wire up the output reference to something even if it is just a "black hole"?  For example, pass the reference into an indicator even though you never do anything with the indicator.  You could even hide it on the front panel.  Since it is 100's of possible references, pass them into local variables of the same indicator.  (Of course that would make a 100 memory copies of the indicator.)  Don't worry about rebuilding arrays, or replacing array subset that you pass through the loop.  Just pass through the a branch of the original reference array.  (Of course that would make another memory copy of the array).  But I don't think these memory copies should grow in time.

It's also possible I have no idea what I'm talking about, but I'm just trying to add some other possible ways to do what you are trying to do.

[PMCR]

Hi Ravens Fan,

I like your "black hole" solution better than my "replacing array subset" solution (posted as Example 7 above) -- I think your method would more smoothly scale to the size and complexity of my application. The question now is whether this would still prevent the DS Write from implicitly closing the reference, and therefore prevent the memory leak (I hope so!). I guess this is similar to what I was getting at with my Example 8: what happens if the reference comes out of the DS Write and goes somewhere, but not to a DS Close?

For discussion purposes, I've implemented the "black hole" solution here as Example 9.

Patrick

[RavensFan]

---

PMCR wrote:

Hi Ravens Fan,

I like your "black hole" solution better than my "replacing array subset" solution (posted as Example 7 above) -- I think your method would more smoothly scale to the size and complexity of my application. The question now is whether this would still prevent the DS Write from implicitly closing the reference, and therefore prevent the memory leak (I hope so!).

I hope to hear how the test works out.

I guess this is similar to what I was getting at with my Example 8: what happens if the reference comes out of the DS Write and goes somewhere, but not to a DS Close?

I would guess they would work the same.  Perhaps better.  The compiler could be smart enough to realize nothing ever happens to the shift register, thus the reference really never goes anywhere.  (That seems like a lot of advanced thinking and looking ahead for it though.)  But with an indicator, I figure since you are telling Labview you want to look out what is coming out (even though it doesn't really show anything) it would have to hold onto the value. 

Also, even if it makes no difference, I figured this was a way to send the data somewhere without a lot of extra wiring due to the shift registers.  Here scaling up would mean more copies of the local variable.  Whereas before, it would mean more clutter from shift registers or replace array subsets causing diagram spaghetti.

For discussion purposes, I've implemented the "black hole" solution here as Example 9.

Patrick

---

Message Edited by Ravens Fan on 11-30-2007 04:02 PM

[PMCR]

Meghan, do you think that Example 9 (ie. Ravens Fan's "black hole" solution) will avoid the memory leak?

[MeghanK]

Hey Patrick,

I'm not exactly sure whether or not that will prevent a memory leak.  What I can do, is create a VI with this setup, and run it overnight here to see what happens.  If there is a memory leak, LabVIEW or the entire system will crash.  I'll update you as soon as I get results, thanks!