TDMS issues

[paljacob]

I want to store some data every 30sec on my cRIO-9075 over a few months (I used to do this into a plain txt file, but wanted to convert to the tdms for easy excel importing an LV support features).

What is the best way to do this?

I created a VI that logs a dataset into a existing or created tdms file. Then I will start on a new file after a while (e.g.. Week) and compress the old one.

The problems:

1. I see that there is automatically created a .tdms_index file close to same file size as the log file. Can this not be disabled? This takes up space and I need to clean it up afterwards..
2. It uses a lot of resources. I see that the processor power increases when the file increases (see a plot of the first 100 iterations). Compared to writing a plain txt file this is very bad performance. What am I doing wrong?

I don't want to have the file open between the iterations.

I have attached the test VI I used.

I use cRIO-9075, LV2011, Win XP.

[YongqingYe]

1. About disabling .tdms_index file. On TDMS menu palette, we have a sub menu palette, called "TDMS Advanced API", it doesn't generate .tdms_index file automatically. For the traditional Standard TDMS API, currently we don't supply any way to disable it.

2. About the performance. Different from writing a txt file, when logging data to TDMS file, it cossumes more memory and CPU usuage, sice TDMS will accumulate more information in memory and do more calculation, please try using "NI_MinimumBufferSize" property, you can find the help information here:

http://zone.ni.com/reference/en-XX/help/371361G-01/lvhowto/setting_tdms_buffersize/
Maybe it cannot solve the problem completely, but it should be an optimazation.

[paljacob]

Thanks, for your input.

I feel that there should be a better TDMS-feature made for the cRIO-platform for logging data. I do not find the "TDMS API" or the "TDMS Advanced API" to easily full my needs.

I now just use this  http://zone.ni.com/devzone/cda/tut/p/id/6471 (gtdms_8.x.zip) to write the raw log, which do not generate a index file. For the other case I do not want to risk loosing data on powerloss by using the buffer feature, so I guess I just do a plain text file as I did before. Nothing seems to beat the performance, but I need to create my own viewer....

[Wayne.C · Accepted solution]

How long did you run your test?   I have a .tdms file that is 2M, the index is only 10K. 

[paljacob]

I just did two new test to answer you.

I now run it on my laptop. At 500 datasets (17doubles), it used over 600ms PER set (opening, writing and closing the TDMS). The file size is 340kB/273kB (tdms/index).

The txt verison uses 4ms on dataset nr 500, and file size dependending on precision (precision(2) gave 46kB and precision(6) gave 124kB).

[Wayne.C]

Please post your code in version LV2010 or earlier so I can take a look at it.