DIAdem slow when processing raw TDMS files?

[MichaelBalzer]

Hi all,

I've just started using DIAdem to do some post-processing of some TDMS files generated using DAQmx, and it is painfully slow. Simply zooming in and out of the data in a graph has very visible redraw, and the signal analysis routines are almost unusable due to the preview window constantly updating (and failing).

The files themselves are ~500MB (12ch, I16, 1MS/s), and were logged as raw + scale using DAQmx's "DAQmx Configure Logging" VI. If I take the same file, apply the scaling, and save them to a new TDMS file (a ~1GB file of SGLs), then feed that into DIAdem, there's almost no noticeable delays with the graphs, and the processing times are very quick.

Is this my only option for speeding up processing when using raw + scale TDMS files? Also can I globally disable data previews?

[AndreasK · Accepted solution]

When DIAdem opens the TDMS, by default data is not prepared but read on acess. This causes the waits while working on the scaled values. If the file is resaved the data is rearranged for better reading performance.

It is possible to change the loading behavior of DIAdem.

Settings->DIAdem Settings->Navigator->Loading Behavior

  "Always Load Bulk data"

  x "Automatically calculate characteristic values"

This will increase your loading time but work will potentially be equal to resave.

instead of changing global setting you can also use a script like the following one to change behavior just for a single file.

Option Explicit

if "IDOk" = FileNameGet("ANY", "FileRead", DataReadPath, "TDMS data (*.tdms),*.tdms", "", True, "Select TDMS file to load completely") then
  dim imAuto : set imAuto = new CImportToPortal
  Call DataFileLoad(FileDlgFileName, "TDMS")
end if

Class CImportToPortal

  Private Sub Class_Initialize()
    ' remember old values
    BulkDataLoadingMode_  = BulkDataLoadingMode
    ChnCharacteristicsUpdate_ = ChnCharacteristicsUpdate
    ' load and calculte min/max, ....
    BulkDataLoadingMode = eBulkDataLoadingImmediately
    ChnCharacteristicsUpdate= "Automatic"
  End Sub
  
  Private Sub Class_Terminate()
    ' set back
    BulkDataLoadingMode  = BulkDataLoadingMode_
    ChnCharacteristicsUpdate = ChnCharacteristicsUpdate_
  End Sub

  private BulkDataLoadingMode_
  private ChnCharacteristicsUpdate_
end class

Hope this helps.

Andreas

[MichaelBalzer]

Thanks for your response Andreas, much appreciated. The bulk load data option has certainly improved performance, especially in the analysis previews. If storage and pre-processing time isn't an issue for this application, I'll probably stick with the resaved TDMS files, but it's good to know the raw + scale TDMS can be used without too much overhead (and on a per file basis too).