TDMS file will not properly import into Matlab

[Mike8977]

I am trying to load the contents of TDMS files into MATLAB for post-processing. I am not too picky on what approach I use to accomplish this, but I want it to be relatively streamlined for batch processing and I want it to work (that's the key)!. I have tried the NI supplied DLL with .m example files (see "Reading TDM/TDMS Files with The MathWorks, Inc. MATLAB® Software", http://zone.ni.com/devzone/cda/epd/p/id/5957). I can get it to work just fine with the supplied TDM file, but it does not correctly process my own TDMS file. I have also tried to process the binary TDMS file using an M-script from MATLAB Central (see "ConvertTDMS", http://www.mathworks.com/matlabcentral/fileexchange/28771). This file was written to process TDMS files based on the file format description from NI (see "TDMS File Format Internal Structure", http://zone.ni.com/devzone/cda/tut/p/id/5696). The M-script is my preferred solution since it doesn't require a bunch of other files like the DLL approach does, but I am willing to take the approach that works best.

All I want to do is be able to read all the properties (names and values) and each group of channels (names and values) from a TDMS file into MATLAB. Is that too much to ask? Any help is greatly appreciated.

See attached document which has several screenshots to explain the problems I encounter. I have also attached my modified versions of the scripts mentioned above and my example TDMS file that doesn't get processed correctly.

Thanks,

Mike

[Fred_V]

Hi Mike,

Do you know if your files are TDMS version 1.x or 2.0? It looks like the m-script solution was written to work with TDMS 1.1 & 1.2, where by default LabVIEW 2009 and later create TDMS 2.0 files. If you can update your LabVIEW program (if that's what you're using) to export TDMS 1.x files, they may work with the m-sript solution.

With the DLL solution, comments seemed to indicate that it may only able to return double values. If you're TDMS files are less homogeneous, that may be the reason.

Hope this helps,

[Mike8977]

Fred,

The version numbering with TDMS files is a bit unclear to me. I have seen things numbering versions as 1.1, 1.2, and 2.0 as you have in your post, but I have also seen version numbers 4712 and 4713 as described on http://zone.ni.com/devzone/cda/tut/p/id/5696. I know that my TDMS files are version 4713 (this is pulled from the TDMS file itself by the script), but I do not know if that translates to 1.1, 1.2, or 2.0. The .m script was written based on the TDMS internal structure described on http://zone.ni.com/devzone/cda/tut/p/id/5696. Is there anything missing or incorrect on that page that might be causing the problem?

Alternatively, is the source code for the DLL (rather than just header files) available from NI so that I can try to enable it to handle more than just doubles? Or will NI update the DLL to make it more robust?

Thanks,
Mike

[Fred_V]

Hi Mike,

You're right that the version numbering is unclear; it turns out that 1.1 (4712) & 1.2 (4713). So, the files you have do appear to be compatible with the m-script. 

What problems did you run into using that m-script with your TDMS files? The comments on that posting seem to indicate that it works well for some. Have you tried using previous versions of the script? 

Also, try posting the specific errors you're getting on MATLAB® Central. The script has been actively developed, and your error may reveal an area for further improvement.

[Mike8977]

Fred,

See the Word document in the zip file attached to my first post. It contains screenshots that show one problem with the m-script. There are "chunks" of data that are missing after the m-script processes some channels. I have been in contact with the author of the m-script, but he's out of ideas. It seems that the NI page "TDMS File Format Internal Structure", http://zone.ni.com/devzone/cda/tut/p/id/5696 is incomplete or inaccurate. Based on the postings at MATLAB Central, it seems that the problem may be specifically related to multi-segment optimization.

There is also a comment on the TDMS file structure page that says, "This article does not describe how to decode DAQmx data. If you need to read a TDMS file with software that implements native support for TDMS (without using any components provided by National Instruments), you will not be able to interpret this data." I hope that this does not mean data coming from DAQmx devices cannot be logged to TDMS files in a way consistent with that structure description.

I have also posted my same initial comment to MATLAB Central's forums (as opposed to directly on the ConvertTDMS page since I have already talked with the file's author) but have not had any replies.

Thanks,

Mike

[Fred_V]

Hi Mike,

Thanks for the update. I'm in the process of talking with R&D about the state of our current documentation, and the capabilities of the provided C DLL.

I hope to have a better answer for you soon.

All the best,

[Mike8977]

Fred,

I have a solution now using the NI DLL to import TDMS files into MATLAB. I basically threw out the ReadFile.m that was provided by NI and wrote my own function from scratch. I have come up with a function that can process one or more files at a time and creates a MATLAB structure that closely resembles the hierarchical structure seen in the TDMS Viewer. It preserves both the data and properties for all channels and channel groups as well as at the top level. I have not tested it with a very diverse set of TDMS files/contents, but it correctly processes my files. I suspect that if there are bugs in there (which might be identified by trying out different TDMS files), they would be minor to fix.

If anyone is interested in what I came up with, I can try to get approval to publicly release the .m file (in which case I would post it here).

As a side note, the native M-script approach on MATLAB Central had more activity and a new version was released. I didn't spend much time on it, but it still didn't work for my files. That's why I wrote my own.

Thanks,
Mike

[Fred_V]

Hi Mike,

Thanks for the update. The feedback I got from R&D was exactly what you discovered; the example script only extracts a subset of the available information, but the DLL is capable of handling the spectrum of TDMS files.

In fact the same DLL is used in all NI ADE's, so is under constant development. 

All the best with the rest of your project,

[MartinMcD]

Hi Mike,

If you are able to share you .m file I would love to see it - I am having the same problem here.   I find it so annoying that National Instruments don't provide a solution themselves.

Thanks,

Martin

[Mike8977]

Martin,

I am in the process of getting my script cleared for Public Release. I don't see any issues with it being approved, so as soon as I have that approval (probably another 2 weeks or so) I will post it here. Maybe NI will go ahead and steal my work and package it with their DLL for even wider distribution since their m-script is so poor.

Thanks,

Mike