Avoiding TDMS Fragmentation

[James_McN]

Overview
This example VI can help you have the TDMS API buffer the data to reduce the effect of fragmentation.

Description
TDMS is a file format designed for very efficient streaming of data to file (and it is very good at this). The way it achieves this though is by expecting the same data to be written each time you write to file. If you change this data then the file becomes fragmented meaning larger files and slower reading performance. To avoid this you should try and always write the same data chunks every time but this is not always possible. To mitigate this you can have the TDMS API buffer the data to reduce the effect of fragmentation.

The buffers don't prevent fragmentation but it does reduce the effect.  By increasing the buffer size you increase the size of data in each chunk written to file and so reduce the overhead associated with each. Because the data is not immediately written to file though any crash will mean that data is lost.

This code compares three scenarios for writing timestamped data to files.

1）Write a timestamp channel then a data channel: This is the 'natural' way to do this but this causes fragmented files.  The problem increases as the data chunks get smaller.

2）Add an internal buffer as discussed in http://digital.ni.com/public.nsf/allkb/63DA22F92660C7308625744700781A8D.

3）Convert the timestamp data to a double type so it can be written with the data in a single write.  This could be changed to then just contain relative time.

Requirements

• LabVIEW 2012 (or compatible)

Steps to Implement or Execute Code

1. Download the attached folder to your computer
2. Open the project "TDMS Best Practices 2012 NIVerified.lvproj"
3. Open the VI "Test VI.vi"
4. Run the program

Additional Information or References
VI Block Diagram of "Test VI.vi"

 

**This document has been updated to meet the current required format for the NI Code Exchange.**

[Technico]

Hi, can you upload a LV 2010 version as well?