LabVIEW

cancel
Showing results for 
Search instead for 
Did you mean: 

Manage and open large TDMS data files

Hi NI Community,

 

I'm looking for advice on how to manage data collected from PXI chassis.

 

Currently, we are taking multi-channel data at a very high sampling rate (up to 1MS per second) and each run can last anywhere from 1 to 5 minutes. The output TDMS file can end up being 10-20GBs which is quite large! I've had no problems opening a 20GB TDMS file with DIAdem or FlexLogger to visually inspect the traces. However, I'm looking to see if there are ways I can avoid using DIAdem or FlexLogger to view the data. I found Scout (free version) online but it is not that great to open a large 20GB data file as it fills up your RAM quickly. Can you suggest to me how to deal with a large amount of data? And how to open and view large TDMS data files?

 

Thank you!

0 Kudos
Message 1 of 4
(451 Views)

Hi Victor,

 


@victor55 wrote:

The output TDMS file can end up being 10-20GBs which is quite large! I've had no problems opening a 20GB TDMS file with DIAdem or FlexLogger to visually inspect the traces. However, I'm looking to see if there are ways I can avoid using DIAdem or FlexLogger to view the data. I found Scout (free version) online but it is not that great to open a large 20GB data file as it fills up your RAM quickly. Can you suggest to me how to deal with a large amount of data?

Use Diadem/FlexLogger as they are optimized for such large data files…

 


@victor55 wrote:

And how to open and view large TDMS data files?


There are TDMS functions in the file palette: you can write your own software to open/read TDMS files.

Think about a good decimation algorithm when starting with those 5min*60s/min*1MS/s=300MSamples long plots…

Best regards,
GerdW


using LV2020 on Win8.1+cRIO
Message 2 of 4
(429 Views)

Hi Gerald,

 

Thank you for your reply as always. Do you have an example of a decimation algorithm or know where I can find some of that online?

 

0 Kudos
Message 3 of 4
(401 Views)

Hi Victor,

 


@victor55 wrote:

Hi Gerald,


Who is "Gerald"?

 


@victor55 wrote:

Do you have an example of a decimation algorithm or know where I can find some of that online?


LabVIEW has a DecimateArray function for a very simple decimation of an 1D array.

There are decimation functions in the signal processing palette. Read their help…

Use GoogleFoo to look up for other decimation algorithms like "Min/Max decimation"…

Best regards,
GerdW


using LV2020 on Win8.1+cRIO
Message 4 of 4
(355 Views)