CNV library memory leak?

[ElectroLund]

So over here, I posted that I had patched all of my leaks.  However, for a branch of my project, I'm still seeing some leakage.

The significant differences in this branch are pulling data from CNVs.  I've followed the example projects as closely as possible, but I'm beginning to wonder if the data disposal they use is flawed.  For instance, here's how outline looks:

while (!KeyHit())
{
	// Get next data available in client buffer
	CNVGetDataFromBuffer(subscriber, &data, &status);	

	// Check for new data; ignore empty buffer (CNVNoData) and stale data (CNVStaleData)
	if (status == CNVNewData || status == CNVDataWasLost)
	{
		int value;
		
		// If data is the expected integer type, get the value and print it
		if (CNVGetScalarDataValue(data, CNVInt32, &value) >= 0)
			fprintf(stdout, "%d\n", value);
	}
	
	// Dispose data
	if (data)
	{
		CNVDisposeData(data);
		data = 0;
	}
}

// Cleanup
if (data)
	CNVDisposeData(data);
if (subscriber)
	CNVDispose(subscriber);
// Cleanup Network Variable Library
CNVFinish();

I've implemented this same framework in my branch.  And yet it's leaking like a sieve.  So I'm beginning to wonder if maybe the check on data before disposing is not working.  Perhaps dispose of data and subscriber regardless of their existence?

[ShamWOW!!]

Hello,

When you implement this code over in your branch, is it exactly the same as this code, or are there any major changes? I ask because you are right in that this code is extremely similar to an example project for CVI. 

The only main difference that I can see is that this code does not have any error checking system. If you run this and run into errors, you won't be able to best diagnose them.

What does the resource tracker show when the code is leaking?

Best,

Shamik C.

Applications Engineer

National Instruments

[ElectroLund]

Thanks for your reply, Shamik.  Unfortunately, putting my exact code here would be difficult, as it is quite large.  So I attempted to put in a "skeleton" of it.

In my code, I am using extensive error checking, like is done in the TDMS example projects.

Over the weekend, I had both this branch project (the buffered subscriber application) and another CVI application which functions as the buffered data writer running in debug mode so that I could monitor resource tracking.

Here is the subscriber app (my branch software), which today was up to a total of about 20MB.  The non-CNV trunk version consumes about 12MB in runtime.

You can see that it's all pretty typical stuff.  The UI portion is all the panels.  The difference between 20 (CNV branch)  and 12 (non-CNV trunk) is fairly negligible and I'm willing to chalk that up to the CNV library needing what it needs to run.

But the real offender is my CNV writer application.  It has a true leak I believe.  Since Friday, it had creeped up to 60MB.  And yet, I'm not seeing alarming resources...

It uses two libraries to collect data: TCP connection to my temperature chamber, and CNV to write the data to the network.

[ShamWOW!!]

If the memory leak is coming from the writer application, then what is roughly happening in there? The first message showed disposing of data in the reader, but what about the writer?

Do you have some kind of skeleton of code that you could show from the writer function?

In addition, what version of CVI are you currently using? Prior to 2013, CNVGetDataFromBuffer would have a bug that would allocate more resources than intended. This had a workaround of getting rid of the condition if (data) in the reader, and disposing of data regardless. There is relatively no harm in disposing an empty data handle.

[ElectroLund]

I am running CVI  2015.

Here is an outline of my writer application.

One-time init

	// first the data
	if (nvData)
		CNVDisposeData(nvData);

	// then the data handle
	if (nvHandle)
		CNVDispose(nvHandle);

	// finally the library
	CNVFinish();


	// register this machine
	cnvChk(CNVRegisterMachine(systemConfig.networkDirectory));
	
	// Create a buffered network variable writer connection
	cnvChk(CNVCreateBufferedWriter(systemConfig.networkDirectory, 
					 0, 
					 0, 
					 0, 
					 systemConfig.monitorBuffer, 	// buffer depth is N records
					 500, 				// wait Nms for this call to complete
					 0, 					
					 &nvHandle));			// handle name

Then in a timer, I have the temperature collection and publishing:

	// update status
	sprintf(stemp,"Temperature = %3.2f",temperature);
	MessageTableUpdate(statusPanel,STATUS_TEXT,stemp,MESSAGE_PASS);
	SetCtrlVal(mainPanel,MAIN_TEMPNOW,temperature);
	SetCtrlVal(mainPanel,MAIN_ERROR,OFF);
	
	//-----------------
	// publish temperature to network variable
	
	// create the variable data
	cnvChk(CNVCreateScalarDataValue(&nvData, CNVDouble, 0.0));
	
	// set the scalar data and store it
	cnvChk(CNVSetScalarDataValue(nvData, CNVDouble, temperature));
	cnvChk(CNVPutDataInBuffer(nvHandle, nvData, BUFFER_WAIT));
	
	// clean up network variables
	if (nvData)
		CNVDisposeData(nvData);

I've not included the Error: error handling code for simplicity.

Your comment on disregarding the CNV data condition before disposing is interesting.  I had considered doing just that, as the help didn't imply there was much consequence for doing so.  I'll give that a shot.

[ElectroLund]

Another update today:

Unfortunately, making the CNVDispose change (regardless of presence of nvData) didn't help my memory leak.

The leak is not as severe now, but still present.  My applications (reader and writer) have been running for about 28 hours now and grown from 12MB / 10MB (respectively) up to 25MB / 50MB.  

As you can see, the writer app consumes a lot more memory over time.  At this rate, it's not obvious to me where else to try to "trim the fat."    

[ShamWOW!!]

For the timer loop, you seem to be creating and disposing a scalar data value every time in the loop. If the conditional affect that is affecting the if (data) conditional statement on the reader still exists, then I would recommend two steps. Place the CNVCreateScalarDataValue function before the while loop. Then remove the 'if (data)' conditional and dispose of the data after the while loop. That might be another way to 'trim the fat.' 

In addition, we tend to put some sleep functions into our CNV write functions in order to prevent starvation from occuring. Code like that usually exists in the loop just at the end like below.

#if _NI_mswin_
		Sleep(500);
#elif _NI_linux_
		usleep(500000);
#endif

[ElectroLund]

Hmmm, somehow I missed that call to Sleep(500) in the sample projects.  So I've included it now in my writer application.  I also took your advice and stopped disposing of the data variable while the writer app is open.

It's been running since this morning (about 7 hours now).  Sadly, I'm seeing the consumption grow from ~8.2MB to ~19MB.

I'm so perplexed!

[ShamWOW!!]

I recommend saving a screenshot of the memory allocation at the beginning of the call, as well as after a while to see exactly what is creating more instances. Try to pinpoint the difference and be certain about it.

I know that you have a large application, but if this is a bug that is showing up, then would you be able to create a small piece of code that reproduces the memory leak? We could then use that to possible create a patch for these functions in the near future.

Best,

Shamik C

Applications Engineer 

National Instruments 

http://www.ni.com/support 

[ElectroLund]

Shammy, thanks for your help!

I took a starting screenshot and then one again this morning...

2016-07-20 14:43:00

 

2016-07-21 06:32:05

This is roughly 1MB per hour consumption.

I am running my application in a debug fashion which doesn't even need any physical hardware attached (polling a Watlow controller for current temperature).  Instead, it's just picking random temps and publishing that to a CNV variable.

The application itself isn't large.  Would you like a private copy to inspect?