FFT setup in Diadem -- joint time frequency

[jtw131]

Hello,

I am very new to Diadem.  I have data from an embedded system that is measuring acceleration data.  The embedded systems take a 256 sample snapshot of acceleration data at a 3200HZ sample rate.   Each snapshot is timestamped, and written to disk at a rate of about one snapshot per second.  The data is stored in a CSV file such as the one attached.  The format is like this:

date1, time1, sample1, sample2, sample3, sample4, ... sample256

date2, time2, sample1, sample2, sample3, sample4, ... sample256

...

dateN, timeN, sample1, sample2, sample3, sample4, ... sample256

My first problem is how to tell diadem how to describe this data.  I guess I need to have each snapshot in a waveform channel but I am unsure of the optimal way.  I can't even get diadem to load the data by row instead of by column. 

Then what I need to do is calculate the 256 point FFT for each snapshot, then put them all in a 3D plot arranged by timestamp of each snapshot.  This could be a 3 axis plot or a colorized spectrum display. 

Finally, for icing on the cake, I have correlated GPS data in an NMEA file.  I would like link the GPS time and location to the time in the FFT/time plot.  Is this all too much to ask?  Diadem seems like a very powerful tool, I think I have just picked a complex problem to start with.  Can anyone help me get started?  Or, if the task is not best suited for this software, I need to know if now so I can pursue other methods.

Thanks!

[Brad_Turpin]

Hi jtw131,

I created a DataPlugin (JTFA_XLS) for your submitted *.xls ASCII data file-- the DataPlugin loads the acceleration snapshots to the Data Portal in the ideal form for 3D plotting in DIAdem.  Then I created a VBScript and REPORT and VIEW layout files that calculate the averaged FFT and JTFA and display these in VIEW and REPORT.  You'll need to detach and double-click the JTFA_XLS DataPlugin to register it, then load one of your *.xls ASCII data files into the Data Portal (make sure it was empty first).  Then run the "JTFA.vbs" file in DIAdem-SCRIPT to run the analysis and load the VIEW and REPORT layouts.  Note that VIEW is interactive (zooming, scrolling) whereas REPORT is designed for static publication-quality reports.  Check out the attached files and let me know if you have further questions.

I included some *.png images of the 4 REPORT sheets in the "JTFA Images.zip" attachment,

Brad Turpin

DIAdem Product Support Engineer
National Instruments

[AndreasH_Ha]

Hi jtw131,

just a little additional information to what Brad already described: reading the NMEA file should be straightforward. If you go to www.ni.com/dataplugins you will find a plugin you can download. Once you have it installed on your machine, you can read the NMEA files directly into DIAdem.

Andreas

[Brad_Turpin]

Hi jtw131,

Yeah, I forgot about the NMEA bit-- we can cover that too.  Andreas pointed you to the DataPlugin to load those files.  If you happen to have DIAdem 2010 (11.2), then there is a new feature in VIEW to natively synchronize GPS data with the display of any other VIEW information-- in your case JTFA.  If you go with the approach I sent you yesterday of the Time information on the Y axis of the JTFA graph, then we'll need to tweak the cursor synchronation of the VIEW layout so that the GPS display synchronizes with the Y axis of the JTFA graph instead of its X axis (default behavior).  But one step at a time.  First download the DataPlugin and load the NMEA file.

Brad Turpin

DIAdem Product Support Engineer
National Instruments

[jtw131]

THANKS to both of you!  I will download the plugin and see if I can get it going.  I am attaching the NMEA file that corresponds with the X accelerometer data -- the timestamps should line up. 

[jtw131]

Brad,

I was able to load your dataplugin and run the script to get all the FFTs associated with the correct timestamp, it worked beautifully!  I also was able to use the NMEA plugin to import the GPS file.  I now would like to link the map and coutour plot in VIEW.  However, the contour JFTA intensity plot does not seem to be setup correctly.  I see solid blue for the entire plot, when I know from the other plots in the REPORT section that there should be some higher intensity colors on the low end of the frequency range.  Any ideas what is wrong with the setup?  See attached screenshot  -- note that amplitude396 that is highlight has a max value of 153, so we should see some green in the plot. 

Thanks again for your first class support!

Jeff

[Brad_Turpin]

Hi Jeff,

Here is a new version of the JTFA.TDV that includes the cursor mapping between the Y axis (datetime) of the intensity area and the X axis (datetime) of a new GPS graph area above it.  You'll need to load the corresponding NMEA data file AFTER running the JTFA script for this TDV file to work correctly.  The reason you're seeing a difference between VIEW and REPORT is that I configured the REPORT graph to interpolate the time values to a mesh of "only" 400 different time values, whereas the VIEW contour plot uses all 1900 some time snapshots from your data file.  This makes the FFT lines in VIEW extrodinarily thin and hard to see (you don't have 1900 pixels up and down on you monitor).  So there's nothing wrong-- that's just your data.  REPORT is kindly averaging together adjacent times into "only" 400 values up and down so that the FFT lines are thick enough to be visible.  But the cool thing about the VIEW plot is that you can zoom into the area of interest, and when you're zoomed in that extra 1900 vertical division resolution is a benefit.  The image you sent last shows features in the upper left hand corner-- try zooming into just that corner of your data, and you'll see the colors appear.

Brad Turpin

DIAdem Product Support Engineer
National Instruments

[jtw131]

Brad,

Thanks again for your prompt response.  I was able to import the NMEA file, but the times do not quite match up with the acceleration times.  I think you must have assumed that the second field in the time stamp is seconds since midnight, when in in fact it is this format:

HHMMSS.FFF

There may not always be fractional seconds.  When there is not fractional seconds the data will not have the decimal point, like this:

HHMMSS

Is there a way to modify the data plugin to correct the time and handle both cases?

Thanks!

Jeff

[Brad_Turpin]

Hey Jeff,

I'm perplexed by your question.  You posted two data files "AccelX001.xls" and "NMEA001.xls".  I created the JTFA_XLS DataPlugin to read the first data file, and the existing NMEA_GPS DataPlugin that ships with DIAdem 2010 loads the second data file.  The first channel loaded in both cases is a date/time channel, which loads whatever resolution is in the data file.  This is what I see when I load those two files in their first channel, called "Timestamp" in both cases:

 AccelX001.xls    09/22/2010 04:24:11.0000 ... 09/22/2010 12:51:44.0000

 NMEA001.xls    09/21/2010 12:55:03.4460 ... 09/21/2010 13:27:02.0000

As you can see above, you have posted data sets from two different days, so synchronizing this JTFA data with this GPS data won't work, because the time stamps don't overlap.  When I artificially make one of the Timestamp channels line up with the other one, then the synchronizing works great on my computer.

Please clarify your qustion,

Brad Turpin

DIadem Product Support Engineer

National Instruments

[jtw131]

Brad,

Sorry, looking back I can see my question was unclear.  What I am getting at is that these measurements were indeed taken simultaneously.  The NMEA file is showing the correct time, so there is something wrong with the JTFA_XLS data plugin created that improrts acceleration file.  If you look at the time stamp in the acceleration CSV manually (before it is imported), you will see this entries like this

210910, 125452.646

This corresponds to 09/21/2010, 12:54:52.344  

So the second field, time,  is in HHMMSS.FFF format.   Sometimes the fractional seconds will not be there, and it will just be HHMMSS. 

Does this make sense?

Again, thanks for your continued help.

Jeff