LabVIEW

cancel
Showing results for 
Search instead for 
Did you mean: 

TDMS Write Memory Leak

Solved!
Go to solution

I am using TDMS to store test data which appears (in a very convincing way) to be causing a memory leak in my program. Shown below is a graph of three versions of my program. Series 1 uses TDMS data storage with the file opened, data appended, and closed every 5 seconds. Series 2 uses TDMS data storage with a referense kept open during operation and closed at the end. Series 3 writes data to a CSV file. The graphs show LabVIEW memory usage every .5 second over the last ~4 hours of a 10 hour test.

 

LabVIEW 8.6.1

Windows XP SP3

 

LabVIEW Memory Usage

Message 1 of 15
(7,594 Views)

Further Information:

 

TDMS Write

  • Group Name - String
  • Channel Name(s) - 1D Array of Strings
  • Data - 2D Array of Strings
0 Kudos
Message 2 of 15
(7,589 Views)
You say you leave the TDMS reference open. What happens when you close it? Does the memory footprint go back to normal or does it stay unchanged?
 
We tried to reproduce this on our end, but couldn't. Any chance you can post a simple VI that demonstrates this behavior?
 
Thanks,
Herbert
0 Kudos
Message 3 of 15
(7,565 Views)
Solution
Accepted by topic author Gleichman

Further explanation ... when a TDMS file is open, LabVIEW will create an index structure in memory that is used for random access to the file. The built-in LabVIEW TDM Streaming API will always create this index, even if you're just writing. The size of the index depends not on the size of the file, but on how fragmented the file is. If you store a relatively small number of large blocks of data, the index will be very small. If you keep adding small amounts of data to your file, it might get pretty large. A remedy for that is the NI_MinimumBufferSize property. You can set this property on a channel in order to make LabVIEW buffer a given number of values before storing them. That way, data from multiple calls to "TDMS Write" counts as one chunk of data in the file. Typical values for this are 1000 or 10000. File fragmentation and memory footprint are reduced accordingly, however LabVIEW will use some memory for the buffers.

 

If all this fails, and you don't need reading access to the file while you're writing, you can also use the VI based API for TDMS. This will not build the index structure I mentioned above, so there should be no memory increase at all. In this case, I recommend to defragment the files before post-processing.

 

Hope that helps,

Herbert

Message 4 of 15
(7,560 Views)

Herbert Engels wrote:
You say you leave the TDMS reference open. What happens when you close it? Does the memory footprint go back to normal or does it stay unchanged?
 

 

This is the blue line in my graph.  When I used this method, the base memory used remain constant, but the maximum memory used continued to grow.

 

 

0 Kudos
Message 5 of 15
(7,529 Views)

Herbert Engels wrote:

 

If all this fails, and you don't need reading access to the file while you're writing, you can also use the VI based API for TDMS. This will not build the index structure I mentioned above, so there should be no memory increase at all. In this case, I recommend to defragment the files before post-processing.

 


I have changed my code to use these VIs and I will report my results on Monday.  Thank you for your responses! Smiley Happy

0 Kudos
Message 6 of 15
(7,527 Views)

I ran my program for over 40 hours and the memory useage was stable using the VI based API for TDMS. Thanks for the quick response Herbert! Maybe in future versions of the TDMS VIs there could be an option for not indexing?

0 Kudos
Message 7 of 15
(7,488 Views)

One tip is just to buffer up the data in your own application, and only doing a write to the TDMS file say every minute or something like that. Queues work brilliantly for this kind of thing.

 

This keeps the index file small as well.

 

Using this technique it is possible to log for weeks (perhaps longer...) without running out of memory.

 

0 Kudos
Message 8 of 15
(7,483 Views)
curious what did you use to monitor your memory usage and create the graph??
0 Kudos
Message 9 of 15
(6,747 Views)

Hello Chandler,

 

In LabVIEW's Example Finder, there is an example called Memory Monitor that will acquire this data for you. The example finder is located under Help>> Find Examples. From here, search for memory and it is the seventh result. I hope this information helps you and if you need any more help please feel free to post.

 

JimS


Jim St
National Instruments
RF Product Support Engineer
0 Kudos
Message 10 of 15
(6,700 Views)