Out of Memory Error - why?

[CoastalMaineBird]

Win 10, LV2018SP1 (32 bit, stop laughing) 

I'm stuck on 32bit for the time being, and am running into the NOT ENOUGH MEMORY TO COMPLETE THIS OPERATION error.

The TASK MANAGER's memory display shows around 3225 Mbytes when that happens.

I know that that is the expected limit, inherited from days of old, and understand that.

In poking around, I ran into one of the larger offenders, and I'm having trouble figuring out why.  

This video <https://www.screencast.com/t/B8XEKrdlCR5> shows a breakpoint in one of my classes.

As we start off the memory stands at 998.2 MB.

The PRINTOUT START is an abstract VI, and it is not overridden in this case.

But AS SOON AS I STEP INTO IT (0:10), RAM goes up to 1054.2 MB, an increase of 56.2 MB ! ! !

You can see the code does nothing except pass inputs to outputs.

AS SOON AS I STEP OUT OF IT (0:17) RAM goes up to 1111.0 Mbytes - ANOTHER increase of 54.8 MB ! ! !

When I step into the NEXT method (0:27), RAM goes up to 1225.1 an increase of 114 MB.

Stepping OUT jacks it up to 1394.5

I don't understand how an abstract (no code) method could command over 50 Mbytes.

The CLASS terminals are dynamic, it could conceivably be overridden, but in this case it's not.

You can see that the abstract method is in the SAME PPL as the caller.

That class takes up 1610 KB on disk, and is a child of another class in a 762KB PPL, which is a child of another class in a 54KB PPL.

Altogether that's 2400 K bytes, so how does that get inflated to 56 MBytes?

There are several calls to this class, it looks like each one has an exorbitant cost, both on entry and on exit.

I've tried the DEALLOCATE NOW thingy, to no effect.

Any ideas how to debug this?

[Kyle97330]

FYI the link for Screencast you posted includes an unneeded ">" at the end so I had to remove it to watch.  You might want to edit your post to remove it from there.

Can you add code in there to get the approximate size of the data in the object you are using there?  The "MoSAIC Printout" one?  Something like this where it flattens it to a string, then gets the string length:

That won't be the exact size to the byte, but it will get the general order of magnitude and check and see if there's a large amount of data floating around in that object, and if there is it could just be that it's making a lot of copies of that data which end up being expensive in RAM.

[tst]

My guess would be the same as Kyle's: the object has that data. When you step into the VI, you open the FP, so you will need copies of the data for the input and output controls. I would suggest trying breakpoints before and after without stepping into the VI to see if still behaves the same.

The profile window or the DETT might also help with the analysis.

Also, it's not "deallocate now", but "request deallocation", which has a different meaning (assuming it even does the deallocation).

[CoastalMaineBird]

Corrected link to video: < https://www.screencast.com/t/B8XEKrdlCR5 > 

[CoastalMaineBird]

Corrected link to video:  < https://www.screencast.com/t/B8XEKrdlCR5 >

[CoastalMaineBird]

I would suggest trying breakpoints before and after without stepping into the VI to see if still behaves the same.

Hmmm.  I did that and the RAM usage before and after that abstract VI call is exactly the same.

But if I step INTO it, the usage goes up by a lot, and stepping OUT of it makes it go up more, and that does not come back.

I don't understand why - the only controls/indicators on that VI are the class in/out and error in/out.

The class might have a lot of data, so, apparently it has to make a copy when the panel opens (which makes sense), and another copy when it closes (which doesn't)?

And why isn't that ram recoverable?  

I still run out of RAM later on, so this is not a solution, just a point of info.

[Kyle97330]

Without seeing a lot more code it's hard to have a guess where the actual problem could be.  It seems like you believe that it's due to interactions with this particular class, so knowing how big it is could be helpful in pinning it down.  If you have a class with a memory footprint of 50 megs per instance, then all it takes is one set of badly optimized buffer allocations of it getting slightly out of hand and your problem manifests.  If it's got a memory footprint of 5k instead, then there's a lot more wiggle room that can happen.

It also sounds like you might not have full access to this class since it is inherited from PPLs, so there's also a chance that the parent class could be engaging in some sort of data duplication behind the scenes that you don't even know about.  Did you make the PPLs or have source access to them, or are they libraries you can't see the source for?

[CoastalMaineBird]

The PPLS are all mine, and showing the code is impractical, as there is a ton of it.

I measured the class as 52 Mbytes, which roughly corresponds to the increase I see when I call the abstract class method, and when it closes.

So, it is related to copies, I think.

I found a place where I am unnecessarily duplicating a timestamp array, the class contains 36 Mbytes when it could get by with only 200k, so that's a big deal that I can resolve.

But I don't understand why it makes data copies when calling a simple VI.  Obviously having the panel open up is one part of it, but none of the other panels open up, and I still have trouble.

I'm working on eliminating my own duplication; I hope that doesn't just kick the can down the road.  I'd rather have a better understanding.

I've read the McCaskle: Clear as Mud post from years back, maybe that pertains.

[Kyle97330]

Have you replicated the behavior in a compiled EXE or is it only occurring in the dev environment?

If you turn off debugging on your VIs or your EXE does it stop or at least slow down?

Without seeing a lot of code it's going to be not terribly easy for us to provide much more insight. 

Just as a general reminder... LabVIEW tries to be "smart" but also tends to try to err on the side of "safe", meaning that if it is ever not sure that something in memory might be used again, it always keeps it in memory since that's safer.  So you might need to get a little creative and look for places in your code where you might think that LabVIEW should know that it's done with something, but in reality it can't be totally sure.  

Things like reentrant VIs that might launch and each get a copy of the data... arrays that build over time that might be getting stored in a buffer such as a shift register or feedback node... creation of DVRs... anything like that.

[rolfk · Accepted solution]

That the VI makes a copy on exiting when the front panel is open is not strange. In LabVIEW each control

AND indicator has an internal data  buffer.

With such huge data elements it may be very prudent to start thinking about using DVRs for this data. DVRs are not the solution for everything but in this case it would seem to make sense.