TDMS File starts recording data at 20Hz, but after a short time drops to <10Hz as VI slows, why?

[superomario]

How can i find where this is happening? basically if I open my tdms file after about an hour, the first few thousand samples are 20/sec, but then it slowly drops down to 19/sec 16/sec, etc. I am reading and displaying data from two serial-usb ports over VISA. The vi graphs and counters themselves also appear to be slowing. I have preallocated my graphs with 1000000 sample arrays, and am using queues to shuttle everything around what is it I am missing?

I cannot post the code it is proprietary unfortunately.

[Sam_Sharp]

---

@superomario wrote:

I cannot post the code it is proprietary unfortunately.

---

Then it's very unlikely that we'll be able to help you. It sounds like a memory allocations problem - but without seeing the code to look for growing arrays/queues, it's virtually impossible to 'guess' at what the problem might be. Is your memory growing? Have you used the desktop execution trace toolkit and turned on the 'display allocations' option?

[superomario]

How Can i query the queues/arrays to find out if they are growing? Is there an analyzer function like in Matlab?

[Sam_Sharp]

---

@superomario wrote:

How Can i query the queues/arrays to find out if they are growing? Is there an analyzer function like in Matlab?

---

I edited my post - you can use the Desktop Execution Trace Toolkit to view memory allocations.

[superomario]

---

@Sam_Sharp wrote:

---

@superomario wrote:

How Can i query the queues/arrays to find out if they are growing? Is there an analyzer function like in Matlab?

---

I edited my post - you can use the Desktop Execution Trace Toolkit to view memory allocations.

---

I attempted to scrub as much identifying info as I could.....thank you for the help

[Intaris]

TDMS files get continually slower when adding small amounts of data/groups/channels due to limitations of the implementation.   I had a similar problem (albeit a different magnitude).  Try writing fewer chunks of data. Avoid point by point writing at all costs.

Shane

[superomario]

What do you mean write fewer chunks of data?

[Intaris]

Every time you call a TDMS write or a TDMS Advanced write, data is written as a "chunk".  Next time you want to write, the driver does a linear searcht through all chunks to find the next write position.  As you can imagine, if you have many many chunks, write time deteriorates continually as the search takes longer and longer.

You can read about my own problems HERE where NI eventually (it only took a few weeks) admitted it was an issue with the file format and that my data was not "typical".

[superomario]

So I don't care when the VI writes to file, but I do need it to display at 20Hz on all graphs. Is there a way to store in a buffer the "chunks" and then write them to a tdms file with a lag such that it does not slow the whole VI down as time passes? As long as when I hit the stop button and open the tdms file in excel, there is data in all columns at 20 samples/second I don't care when it wrote down that data (real time or delayed).

[crossrulz]

If you are using a Producer/Consumer, it is very easy.  One way is to recieve multiple elements in your queue and then write all of them in a single TDMS write.