How to sort data in a TDMS file

[MrQuestion]

Thanks,

In the pictures that I showed I did a 2D array sort in the TDMS File Viewer. That is how I was able to take the picture of the data sorted.

I really would like to stay away from post processing of data, because it doesn't always scale well. 

If there isn't a TDMS primitive function that allows sorting, that answers my question of whether or not I should use this type of datastore.

[Dave_Brandt]

What is your final goal to view or use the data?  

Your Time Stamp is not a valid time, but some other number representing seconds (?) A number to allow the sequence of the sample when it was  recorded?  Why are is the data out of order to begin with?  If the data and time stamp is valid, the easiest way to sort is in excel , and save back as a new sorted csv.  Or before you convert from csv to tdms sort by timestamp then save as tdms.   Or you can post sort from the tdms file. 

A tdms file (.tdms)  is the only file you need to keep. The tdms_index is a file labview creates for more efficient use of the data in the tdms file.  If the data saved to the tdms file is not saved as a waveform or an array (more is better)  there will be no significant size reduction of the tdms_index file . Both files will almost be the same size.

[Dave_Brandt]

Sort csv data before

[RavensFan]

What does it mean to "sort" a TDMS file?  TDMS primitives are primarily, open, close, read, write, and extract attributes out of them.  Now there are some functions like TDMS Defrag that may go in and rewrite files in a more organized fashion.  Whether it sorts data in the manner you are thinking, I don't know.  At the end of the day, any sorting operations are essentially done in memory.  There really is no TDMS sorting because TDMS is just a special type of file protocol.

At the end of all of your processing, what are you really looking for?   A file stored on the hard drive where all the data is laid out in a sorted order?  Or just the abillity to be able to retrieve snapshots of data out of that file that you can then view in LabVIEW sorted (which is what modifying the TDMS viewer is essentially doing.

If you want a special sort on the actual layout of data in a file, TDMS may or may not be the right choice.  What format do you want that file to be?  A text based CSV?  TDMS?  Something else?  Maybe you can read the CSV files, store them in a database.  That way you can retrieve data in any sort order you want.  If you want to rewrite everything out into CSV files again, let the TDMS or the database be the temporary file storage so you can retrieve sorted data, then write the newly sorted out data into the other file types.

[Daikataro]

So long story short yeah, I don't think there's a TDMS primitive that automatically sorts the data, so if you want that, TDMS might not be your cup of tea if you don't want to do any post-processing. However, have you actually tried to do it with some post-processing? How long does it take? Why do you want to stay away from post-processing so much? It might only take a second or two to sort the data and you only have to do it once for each file.