LabVIEW

cancel
Showing results for 
Search instead for 
Did you mean: 

TDMS Write Memory Leak

Solved!
Go to solution

I am using TDMS to store test data which appears (in a very convincing way) to be causing a memory leak in my program. Shown below is a graph of three versions of my program. Series 1 uses TDMS data storage with the file opened, data appended, and closed every 5 seconds. Series 2 uses TDMS data storage with a referense kept open during operation and closed at the end. Series 3 writes data to a CSV file. The graphs show LabVIEW memory usage every .5 second over the last ~4 hours of a 10 hour test.

 

LabVIEW 8.6.1

Windows XP SP3

 

LabVIEW Memory Usage

Message 1 of 15
(5,039 Views)
Highlighted

Further Information:

 

TDMS Write

  • Group Name - String
  • Channel Name(s) - 1D Array of Strings
  • Data - 2D Array of Strings
0 Kudos
Message 2 of 15
(5,034 Views)
Highlighted
You say you leave the TDMS reference open. What happens when you close it? Does the memory footprint go back to normal or does it stay unchanged?
 
We tried to reproduce this on our end, but couldn't. Any chance you can post a simple VI that demonstrates this behavior?
 
Thanks,
Herbert
0 Kudos
Message 3 of 15
(5,010 Views)
Highlighted
Solution
Accepted by topic author Gleichman

Further explanation ... when a TDMS file is open, LabVIEW will create an index structure in memory that is used for random access to the file. The built-in LabVIEW TDM Streaming API will always create this index, even if you're just writing. The size of the index depends not on the size of the file, but on how fragmented the file is. If you store a relatively small number of large blocks of data, the index will be very small. If you keep adding small amounts of data to your file, it might get pretty large. A remedy for that is the NI_MinimumBufferSize property. You can set this property on a channel in order to make LabVIEW buffer a given number of values before storing them. That way, data from multiple calls to "TDMS Write" counts as one chunk of data in the file. Typical values for this are 1000 or 10000. File fragmentation and memory footprint are reduced accordingly, however LabVIEW will use some memory for the buffers.

 

If all this fails, and you don't need reading access to the file while you're writing, you can also use the VI based API for TDMS. This will not build the index structure I mentioned above, so there should be no memory increase at all. In this case, I recommend to defragment the files before post-processing.

 

Hope that helps,

Herbert

Message 4 of 15
(5,005 Views)
Highlighted

Herbert Engels wrote:
You say you leave the TDMS reference open. What happens when you close it? Does the memory footprint go back to normal or does it stay unchanged?
 

 

This is the blue line in my graph.  When I used this method, the base memory used remain constant, but the maximum memory used continued to grow.

 

 

0 Kudos
Message 5 of 15
(4,974 Views)
Highlighted

Herbert Engels wrote:

 

If all this fails, and you don't need reading access to the file while you're writing, you can also use the VI based API for TDMS. This will not build the index structure I mentioned above, so there should be no memory increase at all. In this case, I recommend to defragment the files before post-processing.

 


I have changed my code to use these VIs and I will report my results on Monday.  Thank you for your responses! Smiley Happy

0 Kudos
Message 6 of 15
(4,972 Views)
Highlighted

I ran my program for over 40 hours and the memory useage was stable using the VI based API for TDMS. Thanks for the quick response Herbert! Maybe in future versions of the TDMS VIs there could be an option for not indexing?

0 Kudos
Message 7 of 15
(4,933 Views)
Highlighted

One tip is just to buffer up the data in your own application, and only doing a write to the TDMS file say every minute or something like that. Queues work brilliantly for this kind of thing.

 

This keeps the index file small as well.

 

Using this technique it is possible to log for weeks (perhaps longer...) without running out of memory.

 

_________________________
All your base are belong to us.
0 Kudos
Message 8 of 15
(4,928 Views)
Highlighted
curious what did you use to monitor your memory usage and create the graph??
0 Kudos
Message 9 of 15
(4,192 Views)
Highlighted

Hello Chandler,

 

In LabVIEW's Example Finder, there is an example called Memory Monitor that will acquire this data for you. The example finder is located under Help>> Find Examples. From here, search for memory and it is the seventh result. I hope this information helps you and if you need any more help please feel free to post.

 

JimS


Jim St
National Instruments
RF Product Support Engineer
0 Kudos
Message 10 of 15
(4,145 Views)