DIAdem shows data incorrectly

[CoastalMaineBird]

LV2013, Win7, DIADem 2012 (2015 shows same problems).

OK, I have to actually SOLVE this problem this time.

I'm an old hand at LabVIEW, but new to DIAdem.

My system has 300+ channels of info, at various sampling rates.

I need to have a record that shows channels coming and going (active and inactive).

I have it recorded correctly, but I can't get DIAdem to show it correctly.

Here is an example - generated by a little dummy data generator.  When the data is done, it pops up the TDMS VIEWER in LabVIEW, which shows EXACTLY what I expect:

At 5 seconds in, a new channel (A4) becomes active.

At 9 seconds in, the channel (A3) becomes inactive.

The other channels don't miss a beat, so I am doing what I want to do.

If I asked for the value of A4 at 2 seconds in, or the value of A3 at 10 seconds in, I would hope to get a NaN or an error (no such data point) or something like that.

HOWEVER.

If I load that file into DIAdem, and plot the group, it tells me a lie:

It insists on showing the A4 channel as starting at the same time as the others.

I am recording (in LabVIEW) using waveforms - each one has a START TIME and a deltaT.

DIAdem shows me that the START TIME ("Waveform x-start-time") for channel A4 is indeed 5 seconds after the others, exactly as it shoud be.

DIAdem ignores this, preferring to decide for itself how my data should look.

I can type in a value of 5.0 for "waveform x-offset" and the plot shifts to where it should be.

But what is that offset relative to?  It's certainly not relative to the start time that I specified.

SO, no problem - I'll just write that offset value into the file when I record it, right?

WRONG.

It ignores me doing that, because the property is.....  READ ONLY.

So, it ignores the data that's in the file, won't let me write a fudge factor, and expects me to TYPE IN a value at VIEW time.

Yeah, that makes sense.

My previous question on this brought a declaration that my position (wanting to see what was actually in the file) was a minority opinion, and that to make it tell the truth, I have to create a DateTime Channel and somehow connect the data to that X channel.

However, I can't figure out how to do that.  If I tell channel A4 to use the X-value from A4, it just goes wonky.

If I generate a TimeChannel by manually entering the start/stop times, and then use that, the data just disappears.

Can anyone provide step-by-step instructions for getting an honest plot?

Each channel has a START TIME and a deltaT.  I have to be able to extract the plot without typing in numbers that are already in the file.

If the TDMS VIEWER thingy can do it, then surely DIAdem can.

[CoastalMaineBird]

OK, it gets worse.

I can generate a TIME channel, by using the WAVEFORM CHANNELS <---> NUMERIC CHANNELS function and selecting absolute time.

If I then plot PAIRS, the 4th channel now correctly appears 5 seconds later than the others.

But now there's another problem - at 7 seconds, the 3rd (blue channel) goes away, same as before.

At 11 seconds in, it comes back (a new feature I added to my dummy data generator).  I would expect a 4-sec gap in the signal.

But the display, DESPITE there being another waveform structure with a new START time, insists on ignoring it and appending the new data to the old, without a gap.  

What's the point of using WAVEFORMs, if DIAdem just ignores them?

[PedroMunoz]

Hi Steve;

Can you please upload your example data (TDMS file)? I would like to use it to be able to give more precise instructions.

Thanks.

Pedro Muñoz.

[CoastalMaineBird]

Thanks for your attention, Pedro.

Here is what I EXPECT to see, in DIAdem, AND in the TDMS Viewer.

Chan 3 comes ACTIVE at 5 sec in.

Chan 2 goes INACTIVE at 7 sec in, and goes back ACTIVE at 10.5 sec in.

I achieved this by actually "backfilling" the data - when the channel goes ACTIVE again, I "backfill" the channel with NaN numbers, to create the gap.

Seems like an ugly hack to me.

Without the backfilling, here's what I see in the TDMS VIEWER  (notice no gap in Chan 2):

And here's what I see with the exact same file in DIAdem (notice chan 3 is incorrectly shown as starting at 0):

Attached is the TDMS FILE for test 5.  All chans are recorded as waveforms, i.e., each block has a START TIME in it.

DIAdem just ignores them.

[Brad_Turpin]

Hi Steve,

There are two issues you've raised in this Discussion Forum thread:

1)  DIAdem natively plots waveforms differently than LabVIEW does, by ignoring the start time property and showing all waveforms as starting from the waveform offset value

2)  Waveforms in LabVIEW and DIAdem have no way to store inactive ranges other than explicitly filling those ranges with NaN values

You seem to have discovered the ANALYSIS function in DIAdem that creates the needed DateTime channels to display the data in VIEW or REPORT as it appears in LabVIEW, so I'm going to assume issue 1) is resolved, albeit not elegantly.  Issue 2), however, is unresolvable, and it's not a DIAdem problem, it's a waveform containor limitation that both DIAdem and LabVIEW have to live with.  There is just no component to the waveform containor you can use to indicate an inactive period.  What led you to believe that this would work?  How did you try to indicate in your LabVIEW code that the waveform was inactive?  The waveform has one start time and one sampling rate-- how can inactive regions be encoded with this information?

National Instruments waveforms can only contain contiguous signals with a constant sampling rate.  If you want intermittant signals, you must use create a Time channel to form XY pairs for each acquired data point.  Even then you'd need at least one NaN between active regions in order to see the gap you desire in the graph.

I do understand your desire for an elegant logging mechanism that would provide for an "inactive" and "active" flag for specific regions, but to my knowledge no such elegant mechanism exists in LabVIEW.  The waveform is too simple a storage mechanism for that task.

Brad Turpin

DIAdem Product Support Engineer

National Instruments

[CoastalMaineBird]

 issue 1) is resolved, albeit not elegantly.  

Well, I haven't figured out how to do it without manually going through the steps:

1... Choose the ANALYSIS tab.

2... Choose the BASIC MATH - CHANNEL FUNCTIONS - NUMERIC CHANNELS <--> WAVEFORM CHANNELS function.

3... Selecting all the channels in the CHANNELS list.

4... Choosing WAVEFORM CHANNELS to NUMERIC CHANNELS  (it never remembers my selection).

5... Choosing GENERATE TIME CHANNEL (ABSOLUTE)  (it never remembers my selection).

6... Clicking CALCULATE.

7... Clicking OK.

8... Pairing one Y channel with one X channel, and dragging the pair to the view (I have to carfully match them up - sometimes two Y channels produce one X channel - I have to guess when).

9... Repeat 8 for each Y channel.

If there's a better way to do that, I sure need it.  I haven't figured out scripting yet.

There is just no component to the waveform containor you can use to indicate an inactive period.  What led you to believe that this would work?  

It's right there in the data - that is what is so maddening..

Each waveform includes a START TIME value.

If I say that channel X starts at 12:34:00.000 with a delta T of 0.1 and I write 100 samples, then you would expect the last sample to be at 12:34:09.900, right?

That's the start time + 0... start time + 9.9 sec. It has no trouble plotting this.

If I then write another block that starts at 12:34:19.000 with a delta T of 0.1, what the heck else can there be in that time except a gap ? ? ?

I've plainly told it that this block starts some 10 sec AFTER the end of the last block, but it ignores me.

How did you try to indicate in your LabVIEW code that the waveform was inactive?

I have a flag for each channel and if it's FALSE, I simply don't write that channel. it's NAME is not in the NAME LIST, and it's Waveform is not in the WAVEFORMS list, when I call TDMS WRITE.

[CoastalMaineBird]

National Instruments waveforms can only contain contiguous signals with a constant sampling rate.

I understand that a single waveform has ONE start time, and ONE sampling rate.

But I am writing a WAVEFORM every 100 mSec (in my example).  It looks to me like the TDMS VIEWER pays attention to the first START time of a given channel, and disregards any further ones.

DIAdem is completely disregarding the START TIME of all of them.

[CoastalMaineBird]

Neither the TDMS VIEWER nor DIAdem handles a change in sample rate correctly.

Here, at about 12 seconds in, I change the sample rate, not the SIGNAL frequency.

Both viewers misinterpret it.

It looks like DIAdem simply assumes that the first WAVEFORM header applies to the channel forever after, and disregards further WAVEFORM info.

[CoastalMaineBird]

I do understand your desire for an elegant logging mechanism that would provide for an "inactive" and "active" flag for specific regions, but to my knowledge no such elegant mechanism exists in LabVIEW.  The waveform is too simple a storage mechanism for that task.

But I'm already handling that.  I don't expect a single WAVEFORM to cover those bases.

But I DO expect it to pay attention when things change.  I'm writing a WAVEFORM every 100 mSec, with a new START TIME, and (maybe) a new deltaT.  It ignores me.

Are my expectations really that difficult?

[Hooovahh]

One thing that might clear this up.  Is a single waveform in TDMS is going to write data an array of doubles to that channel, but the start time, and dt between samples are written as properties of that channel.  If you look at the properties of the channel you'll see this.  So if you write 100 samples at one rate, and then another 100 at another rate, the channel will only keep one start time, and one dt for each channel.  If you really want to store the data to keep this information you'll probably want multipe channels like Ch0 100hz and Ch0 200hz.  These can then have separate start times, number of samples, and dt.

Beyond this you could do something custom to the file, but I doubt Diadem would like it.  Something like write new properties on the channel with every write.  You'll probably also need to keep track of the number of samples in each waveform write too.