Multi-Record Digital Waveform to TDMS

[bmann2000]

I'm acquiring Multiple Records from six 6561 cards.  Why can't I write to TDMS file, should I change the format to something the TDMS file will accept?  what format would that be?

Has anyone seen an example of Multi-Record Continuous Acquisiton?  Is it possible?  I'd like to fetch many more records than the onboard memory can accomodate.

[Daniel_Thomas_NIUK]

Hi bmann2000

I have had a look at your code and it doesn't look like you are going far wrong with what you have already produced.  I would first suggest that you may like to try the TDMS 2 file format. This option can easily be changed by adjusting the enum constant highlighted in the attached image.

Kind regards,

[bmann2000]

okay, what's the next step ?

[Rhys - AE]

Hi Bmann.

It is indeed possible, but what is holding you back at the moment is that you are trying to wire an array of data into the data terminal of the TDMS file write function block.

TDMS files are based on Binary Files and thus inherently have fast read/write and random access. What separates a TDMS file from a normal Binary file are the headers for entering meta data, but also the higherarcicle structure they employ. In a TDMS file we write data to "Channels", and bring Channels together into "Groups". The "TDMS file viewer" function in LabVIEW is a fantastic tool for visualising this higherarcy, I recommend playing about with it.

In your application what we need to do is index out each channel of data from that array which we are passing through the queue, and write them to their own channel in the TDMS file using a write function for each channel with the appropriate channel name wired in. I gave this a go in your code and now I've hit a stumbling stone: When we wire the digital waveform datatype into the "data" terminal on the TDMS write function block the channel name terminal changes from accepting strings to accepting string-arrays. This is expected behavior and is documented in the help file for that function.

This has stumped me as the same behavior is not present when we use an analogue waveform data type, with analog we are free to use the normal string data type rather than an array of strings. This suggests to me that TDMS files store the digital waveform data type differently to analogue waveforms, but to my eyes the two data types are just clusters of a few different data types themselves and I didn't expect there to be a difference between storing the two.

I shall look into this, but if anyone else can offer there opinion that would be fab.

If you are wondering where to start with the TDMS file functions in LabVIEW then I hugely recommend navigating to "help" then "Find Examples...". Running a search in the example finder for "TDMS" yields lots of results with explanation of the code. Alternatively you ma want to flick through some searches on our website. Let me spam you a few links I found that I thought may be helpful.

The NI TDMS File Format

http://zone.ni.com/devzone/cda/tut/p/id/3727

TDMS File Format Internal Structure

http://zone.ni.com/devzone/cda/tut/p/id/5696

Writing TDM and TDMS Files in LabVIEW

http://zone.ni.com/devzone/cda/tut/p/id/9334

TDMS 2.0 FAQ

http://zone.ni.com/devzone/cda/tut/p/id/9995

I hope that prooves helpful! I shall get back in touch if I find anything useful in regards to how we write/name the channel of digital waveform data in TDMS.

[bmann2000]

I think a For loop around the TDMS write will convert for Multi-Record Data type, I'll try it out tomorrow.

[Rhys - AE]

Indeed, a for-loop will do the job of indexing out the array, but you will need to also feed the for-loop an array of your channel names too, to make sure you don't just append to the same channel.

In light of the fact that we are now tackling a channel name input node that requires a string-array name, we would need to feed the for-loop a 2D array of our channel names.

I am still looking into the reason why the channel name node becomes a string-array when we wire in digital waveform data, and it appears one use is that we need to add a preliminary channel for the compression data channel if our digital waveform data is compressed. Now how we find that out I don't know... back to work I guess! 🙂

Let us know how you get along.

[bmann2000]

I don't usually bother with the channel names.  I'm acquiring Multiple Records at 10kHz, I'm only interested in time-of-arrival, so the record length is 1.  When I read back the TDMS file, each record has the timestamp I'm looking for.  I don't need names because each record represents the time-of-arrival and state of the 16 LVDS channels I've sampled.  The problem for me is more the time overhead of the for loop inside my consumer loop.  If I fetch say 5000 records, the consumer loop trying to peform 5000 individual write operations, where I'd like to perform 1 write and get all 5000 records on to the hard disk in one go.

[Rhys - AE · Accepted solution]

Right, I'm back!

So, the reason the channel name string input becomes an array input when we wire in Digtal Waveform Data is because the Digital Data table in the cluster that makes up it's waveform creates a new channel in the TDMS file for every new line of data in the table. If the Digital Data has been compressed, you still get 1 channel per line of data plus an extra channel (the first channel) for the compression order.

If any of that slipped you by, and I wouldn't blame you as I had to read it over a few times myself, then I recommend just playing around with the TDMS file vewier to get a feeling for the way the channels are created.

I hope this has all proved helpful. All the best with your LabVIEW work!