Reading TDMS file data slows my Application

[Dskp]

Created a  Data Reader Application in LabVIEW   for Analysis the Logged Data.

My logging application have 20 Channels and log the data in TDMS file,once data are logged ,I use my reader application application to read the data from tdms file.

But once the Reader Application read the data ,it slows my application and CPU usage almost reaches 80 to 100% .(Justed tried by opening the tdms file in reader application which is having 50000 smaples from all channels and size of it is around 27MB,then its slows my application )

Also I my appliaction gets slow and displays the data very slowly.

Please suggest ways of using TDMS file to make application works in normal way without affecting its performance and memory.

[mikeporter]

First question: Why bother writing a TDMS reader when LV ships with a really good one?

Second, are you trying to read and write the same file simultaneously? That is not a good idea.

What exactly are you trying ot accomplish?

Mike...

[Dskp]

Hi Mike,

For first Question
We have developed our custom data reader application and due to some constraints we are not now planning to use data analysis application like diadem etc..but in future we may use these..

For second question
We are not writing and reading simultaneously ..once we are done with our tests we do analysis by open tdms file in our reader applications

Any suggestions for our tdms slow performance ..?

[JÞB]

SCOUT by Signal X is a good option

[deppSu]

I think it will be helpful to answer your question if you have your source code uploaded, so that people can take a look and provide the solution/comments.

[crossrulz]

I second the usage of Scout.  It is FREE and very well put together.  Those guys did a really good job with that application.

Now to your specific problem, I would need to see code.  I have done plenty of TDMS writers and readers and never had an issue.

[DFGray]

Many of your problems probably stem from poor handling of the large data sets involved. I would encourage you to read the LabVIEW help on the subject and this article (much of which is mirrored in the LabVIEW help). Handling huge sets of data is not free, but you can get orders of magnitude better performance if you know what you are doing.

If you wish further help, I will join the chorus of those asking you to post your code. We can't help much more unless we know what you are currently doing.

[Dskp]

Hi Gray,
I too read the article before days and find it usefull.

Also about my application , it just read the tdms channel data (20 channels )with time stamp and display all the data in XY graph whatever the samples count may be (I have restricted only to read less than 1 million data from all channels all the time because of memory and performance issue).

Added to that, I zoom the graph , I Play , forward and reverse the graph data display just by simply changing time stamp.

I too have implemented the decimation way of display the data in graph based on graph width and buffer.

But only when I am reading all the channels data I am facing this slow performance.

Please provide us any suggestions you feel.( Source code is something confidential for us)

Thanks
Selvan.

[Dskp]

More than the above mentioned functions my application doesnt do anything. .It just an history viewer of acquired data..
Hope I cleared..

Anymore clarifications pls revert..

[DFGray]

Individual channels in a TDMS file are physically separate on disk. You can usually get best performance for a single channel read if you defragment your file. However, this may or may not improve performance for multichannel reads, since TDMS stores different channels at different location in the file. You may be able to improve performance by interleaving your data into a single channel. You would be responsible for deinterleaving on read. I would try defragmenting your files (if you have no already) and compare before and after. I would then try to interleave data in various schemes to see what works best for your application (e.g. interleave all channels a point at a time or interleave channels by blocks of 65,000 data points each) to see if you can improve things.