LabVIEW Idea Exchange

About LabVIEW Idea Exchange

Have a LabVIEW Idea?

  1. Browse by label or search in the LabVIEW Idea Exchange to see if your idea has previously been submitted. If your idea exists be sure to vote for the idea by giving it kudos to indicate your approval!
  2. If your idea has not been submitted click Post New Idea to submit a product idea to the LabVIEW Idea Exchange. Be sure to submit a separate post for each idea.
  3. Watch as the community gives your idea kudos and adds their input.
  4. As NI R&D considers the idea, they will change the idea status.
  5. Give kudos to other ideas that you would like to see in a future version of LabVIEW!
cancel
Showing results for 
Search instead for 
Did you mean: 
GregFreeman

Allow non-contiguous waveform data to be handled by TDMS

Status: New

I have noticed a big limitation in TDMS files. When acquiring DAQmx data, often times it is easiest to acquire a 1D array of waveforms for multiple channels, then log this array of waveforms. The problem is that when you have an application that may log, then stop logging, then resume logging again to the same group and channel (such as an application that logs only when an alarm condition is met), the waveforms are automatically appended, rather than showing the gap in time when nothing was being logged. I would prefer that there be some way data logging is managed in scenarios like this, because right now it seems my only options are to 1) convert waveforms to an xy format to be logged, or create a new group every time a logging flag changes to on (which could get quite confusing to a user). I have to assume switching logging on and off is a common scenario, so I'm surprised this hasn't been addressed.

6 Comments
YongqingYe
NI Employee (retired)

Thank you for the post. And it's right that TDMS doens't show the "gap" when appending waveforms. Each wavefor has t0 (the start time) and dt (the interval time), when appending waveforms to the existing channel, TDMS directly ignores the new waveform's t0 and dt, only keeps the first waveform's. If TDMS needs to show the "gap" then I'm afraid TDMS also needs to know whether the dt is different from the orignial waveform, that  means TDMS needs to bookkeep some information in both memory and the physical file on disk, that probably would affect the performance.

SeanDonner
Active Participant

I just ran into this exact issue and came to the very same conclusion that the OP did.  I ended up writing to both a TDMS file to for visualization as well as a CSV file that writes the data and timestamps using the "Get waveform Time Array".    Although you're correct that TMDS will need to do some book keeping, that bookkeeping is extremely minimal during writes.  All it needs to do is see if the current t0 of the input waveform matches the last t0 and if not then create a new (t0,dt) pair in its metadata section.  I can't see this having any significant impact in performance.


GregFreeman
Trusted Enthusiast

All it needs to do is see if the current t0 of the input waveform matches the last t0 and if not then create a new (t0,dt) pair in its metadata section.  I can't see this having any significant impact in performance.

True, even some of the TDMS VIs in the SVTK query the TDMS file before writing (such as FFTs to see if an FFT channel with the same name exists already because you can't really append FFTs) and it doesn't really seem to affect performance. I think the bigger issue is the plotting/reading of the data. A noncontiguous waveform doesn't really make sense, as a waveform assumes equal spacing between all points in the waveform. So having breaks in time within the waveform is a bit counterintuitive, and i think that will keep something like this from being a built in feature and instead will continue to need to be implemented by the programmer. In my mind I have somewhat determined it is a "would-be-nice-to-have" feature but realistically will not greatly inhance the end product of LabVIEW TDMS and I'm ok with it falling by the wayside. 

CoastalMaineBird
Trusted Enthusiast

I have need of something similar to this.

Whether it's a Waveform data type or not, I don't care, but I need to be able to record a TIME channel alongside several (250+ in my case) other channels.

If the user stops recording Time,A,B,C, and adds a channel D, then the new channel D's first sample SHOULD BE associated with the Time channel that I pass in with it.

 

IOW, If I have recorded:
T0   A0   B0   C0
T1   A1   B1   C1
T2   A2   B2   C2
T3   A3   B3   C3
T4   A4   B4   C4
T5   A5   B5   C5
T6   A6   B6   C6
T7   A7   B7   C7

and then I change configuration and append a new line:
T8   A8   B8   C8   D8

 

then I would expect to retrieve D8 along with T8 as a pair meaning that they occurred at the same time.

 

As it is, instead D8 in effect becomes D0, associated with T0.

Steve Bird
Culverson Software - Elegant software that is a pleasure to use.
Culverson.com


Blog for (mostly LabVIEW) programmers: Tips And Tricks

Sam_Sharp
Trusted Enthusiast

To be honest, I had already assumed this was possible (we're currently looking at moving to TDMS from CSV files).

 

I can think of a few situations where this would be useful:

1) Changing the logging rate on the fly (e.g. to capture data at high speed during an event of interest)

2) Start/Pause/Stop logging by the user

3) Device gets disconnected during operation causing a gap in the data

 

If this isn't possible, it requires a separate timestamp channel which surely isn't as efficient?


LabVIEW Champion, CLA, CLED, CTD
(blog)
AristosQueue
Proven Zealot

> Although you're correct that TMDS will need to do some book keeping, that

> bookkeeping is extremely minimal during writes. 

 

But it is also bookkeeping that I believe it would be bad for TDMS to do for exactly the same reasons that I posted here about why it would be a bad idea for a waveform chart to compute non-uniform spacing b.... You would be blending business logic layers with file write layers. I admit that in the TDMS case the problem is less severe than in the indicator case, but the problem still exists.