LabVIEW

cancel
Showing results for 
Search instead for 
Did you mean: 

Read/Write Variant to TDMS

I may make this an idea exchange item later.  But for now I'm just curious, are there people who have come up with a solution for reading and writing variant data to a TDMS channel?

 

My original idea was to turn a variant into an array of bytes, or a string, and then write that data to a channel.  Turns out the string write is meant for ASCII string and stops at the first null so that doesn't work.

 

The solution I came up with is pretty crappy, but uses the MGI Read/Write anything code which turns a variant into an INI string.  I then write this INI string to a channel and can read it back later.  This usually makes the size on disk much larger than it needs to be.

 

Just before posting I did a search and came up with a post here, which turns a variant into the type and data, then writes these values as ASCII values which again makes the data on disk take up more space than it does in memory by quite a bit.

 

So has anyone came up with any other solutions for writing binary data to a TDMS channel?  Is there any technical reason that the TDMS read/write doesn't support a raw mode?  Is there a work around for reading and writing a binary string to a TDMS channel?

 

Edit: I just did a quick test and write a scalar I32 with a value of 8.  This takes up 4 bytes of memory (obviously) but if I use the code that I linked to to turn a variant into the two ASCII string it takes up 14 bytes on disk.

0 Kudos
Message 1 of 10
(5,091 Views)

First, an admission -- I've never tried writing TDMS files.  However, I think about it as the "NI Way" to write data, including sampled (binary) data, so was surprised by the notion that you could (only) write strings.

 

I haven't done the Experiment, but reading the Help file, which pointed to this White Paper, suggested that it was possible to write "raw" files in binary format (there's an example of an I32 file with values 1, 2, 3 written as four-byte binaries).  I know your topic says "Variant", but did you really want to write "data of variable data type, unknown until run time, and varying from write-to-write" to the TDMS file?  If the data for this particular element will be fixed for the duration of the file, you could pass in the Variant and transform it back to the corresponding Binary, writing "raw" data, couldn't you?

 

Bob Schor

0 Kudos
Message 2 of 10
(5,067 Views)

@Bob_Schor wrote:

I know your topic says "Variant", but did you really want to write "data of variable data type, unknown until run time, and varying from write-to-write" to the TDMS file?


Yes, that is my exact desire.  Probably not what NI thought of when desiging the requirements for TDMS functionality.

 

Reading through that white paper does help a bit, but I think the real limitation comes from the list of data types that can be written at the end of section 2.  The only data type listed that has any variable length is string (maybe DAQmxRawData but I have no idea how that works).  And as far as I can tell the string read write, won't write the data as a raw string, but instead writes it as a string that is intended to be human readable, and will stop at the first 0x00, among other restricting rules.  I've also tried editing the data in a TDMS file using a hex editor and even if you were able to write a string as some kind of raw data, the read also will do things like stop at the first null, even through the string length is defined.

 

So if your data type doesn't change from write to write, you can at least put the Type information into the property of the channel, and then the write you can really just write the data.  In my situation however my variant will actually change from write to write so I need to write the type.  Anyway attached is my attempt to write and read an arbitrary data type modeled after the code mentioned earlier.  Still if anyone from NI in the TDMS group wants to chime in I'd be glad to hear other suggestions, I tried some DVR writing stuff but I think that has the same data type limitations.

 

Write Variant.png

 

Read Variant_BD.png


 

0 Kudos
Message 3 of 10
(5,045 Views)

Where are XNodes when you really need them?Smiley Wink  [Excellent talk, by the way]

 

BS

 

0 Kudos
Message 4 of 10
(5,040 Views)

I am sorry Brian but why di dyou not want to save the data as a data channel?

 

I pushed .jpgs and .pngs into a data chanel in the past. The TDMS viewer di dnot like it but the application worked.

 

Curious,

 

Ben

Retired Senior Automation Systems Architect with Data Science Automation LabVIEW Champion Knight of NI and Prepper LinkedIn Profile YouTube Channel
0 Kudos
Message 5 of 10
(5,022 Views)

Imagine I have a sequence, and each sample of a channel is a step in that sequence.  Each step may have different data to be configured with, since each step could be of a different type.

 

So idealy I have something like two channels, one called "Step Type" which is a string stating what step to run, and then a channel called "Step Data" which is a variant which is specific to that step type.  I understand I can write an array of U8 data to a channel, but that will write each element of the array as a sample, which isn't what I want.  In this example I would want to actually write the array of U8 as a single sample, of a single channel.

 

BTW the TDMS viewer won't be used, it will of course be custom since the viewer needs to be able to display changing data types, either in a single control that can represent any variant as a tree, or as the data type that the specific step uses.

0 Kudos
Message 6 of 10
(5,016 Views)

Hello,

 

I just ran into this post, because I needed exactly this feature recently. As I saw this never got into an Idea, so I did it:

https://forums.ni.com/t5/LabVIEW-Idea-Exchange/TDMS-Set-Properties-allow-saving-Cluster-Variant-data...

0 Kudos
Message 7 of 10
(3,814 Views)

After VIMs were official I did end up making a Read/Write Cluster in TDMS example posted here.  For completeness I'll be replying to the idea, but I did kudo it too cause something native is always preferred.

Message 8 of 10
(3,809 Views)

@Hooovahh wrote:

After VIMs were official I did end up making a Read/Write Cluster in TDMS example posted here.  For completeness I'll be replying to the idea, but I did kudo it too cause something native is always preferred.


Super! Thanks, I was not aware of this document! 🙂

0 Kudos
Message 9 of 10
(3,805 Views)

I copied the two VIM into my project. Works like a charm!

Thanks again! 🙂

0 Kudos
Message 10 of 10
(3,793 Views)