TDMS & Config Files Comparison

[DragonFangEM]

Hello,

I'm not really new to the community since I had many queries that have already been asked before and I found an answer here, until now. I've been studying up on TDMS (File>Group>Channel) and its conceptual heirarchy seems to be very similar to Config Files (File>Section>Key). Now, I think it's possible to use the TDMS template for config files as well aside from measurement results but I thought I should ask...

What are the advantages and disadvantages of TDMS vs Config (*.ini) Files?

I haven't found any form of comparison between the two so far. I would personally excempt the Notepad accessibility of the Config File as an advantage since it would be better to limit the access to the system settings from non-authorized users.

[Stuart.Parkinson]

An interesting idea... however, here's my take on it.

TDMS is primarily designed as a streaming format. Therefore the data written by the TDMS Write vi is an array of values, and generally I would imagine that for config info you only want to write one value per setting. It could be done but adds an extra layer of complexity and is not what they are really designed for.

A distinct advantage of config files is that they are easily readable/editable by humans, and for development and maintenance this can be incredibly helpful. In my personal opinion, if you have a problem with users fiddling with files they are not supposed to then you have bigger problems! There is the whole of the Windows operating system they could fiddle with, so you have to trust your users to some extent!

If you definitely do not want the files easily editable by users then another alternative is to hold your config info in a cluster and write this as a binary file. A halfway house is the xml file format which, although editable, is certainly less friendly to the average Joe. You can also serialize to xml and pass it through a simple encription routine before writing as text or binary if security is your thing.

Just my twopenneth... 

Stu.

[crossrulz]

---

DragonFangEM wrote:

What are the advantages and disadvantages of TDMS vs Config (*.ini) Files?

---

TMDS was designed specifically for streaming waveforms to disk.  It actually sacrifices some on the reading side specifically to make the writing process faster.

Config files were designed specifically for holding configurations for an application.  They are easy to edit.

Trying to use TDMS for configuration data is like trying to use TCP where a simple local variable would work.  Yes, it can work.  But there will be a lot more overhead.

[Sam_Sharp · Accepted solution]

Yeah, I wouldn't even consider using TDMS for storing key/value data as it's designed for streaming time-series data to disk.

I think if you want human readable config files then INI files are the way to go - especially with some of the functions in the MGI toolkits for reading/writing anything (e.g. clusters as sections) to an INI file.

If you want to protect the config file then I think I'd probably recommend some sort of binary file format - probably something like an SQLite database as that allows you to easily look up values. If you wanted to go further than that - placing the file in a hidden location (e.g. %ProgramData%) and doing some sort of reversible encryption of the file contents would help to protect the contents from unauthorised viewing/modifying.

From experience, using the MGI Read/Write anything INI file functions have been the best solution we've come up with until we decide to think about using an SQLite database. Teh fact that if you're configuration cluster changes it still manages to read the fields that are present and uses default values you specify if they're not makes it really handy when you need to change the cluster in a future version. Certainly the LabVIEW Flatten to XML functions don't do this - if the type definition changes then I think it returns an error.

[DragonFangEM]

Thanks for the responses. It just goes to show how little I know of the underworks on both options, but I suppose it wouldn't be a problem to use both. ZIP files with passwords is a workable solution for now for system configuration.

An add-on question though, if the measurement data storage medium is TDMS and I get an error mid-test (e.g. fatal error, user closed windows, etc.) what happens to the data? Currently, I've read that the information are being kept in a buffer until the references are closed although I noticed there's a Flush function to force the transfer based on its help description.

[Sam_Sharp]

For your addon question - this is why it's important to have proper error handling and a proper shutdown routine in your application.

If you have any LabVIEW errors in your application you should handle these properly and if it's deemed to be an unrecoverable error then you tell your application to exit.

You should handle your application being closed correctly to make sure that whenever possible you shut down gracefully and close all of your references (e.g. hardware, file references etc.) before your application exits. There's some good information about that on a recent thread here: http://forums.ni.com/t5/LabVIEW/Best-Practices-for-Gracefull-Application-exit-close/m-p/3094973#M885...

Of course, there is always the possibility that you might get a fatal application crash or you might have a power cut or something - in that case you could periodically flush the TDMS buffer but there is also OS and disk-level caching to consider. 

If you look at the Advanced TDMS VIs - you can disable buffering and according to the help "buffering does not occur at any layer".