TDMS Write data (1000 channels , 10 samples per second) ,

[an123]

Hey,

So I'm reading in 1000 values @10hz each and then writing them to a TDMS file and flushing. Now each value is from a unique source and in the TDMS must be it's own channel. I am building an array (2d) of these single precision data points in a feedback node type pseudo buffer and then every 1 second ( so 1000channels x 10 points go to the true case which writes to TDMS)

For some reason the execution time is very inconsistent when I do this. anywhere from 10ms to 150ms spikes occasionally. What could be causing this. 

[Wolleee]

Always helpful to provide some code if you can

[an123]

The orange wire coming from the left to the write is a 1D array of N values (aka the 100 channels), and the pink/purple wire is the N number of channels (their names)

[PalanivelThiruvenkadam]

Colours Define the datatype, So Its always good  to represent in terms of Programmer.

Try to share the code or Vi Snippet, Not the Image.

[crossrulz]

Personally, I would just use a Producer/Consumer setup where your current loop sends the data to a parallel loop (the consumer) with a queue.  The consumer can then write the data to the TDMS file as it comes in.  No need to "buffer" the data with an inefficient feedback node and build array.

[perhult]

Do you really need to buffer? I write 25 values to a TDMS at 10 hz without a buffer (I just write every 0.1 s) without any problems or spikes, I did not think the number of elements would matter that much. Are you sure it is the TDMS functions that is causing your spikes?

That aside, I think this is the actual problem with your code: Do not resize the array each iteration! Use a fixed size array, you know maximum number of elements. The constant array allocations may cause your problems.

Also, IF you need to buffer, use the native TDMS functionality instead.

Use NI_MinimumBufferSize and TDMS Flush, or simply let TDMS Write do the buffering automatically.

[Hooovahh]

There are other recent new functions that might help out too, like the TDMS in memory which allows you to read and write to a TDMS reference, but that reference isn't to a file, it is to a set of memory.  Have you also thought about using the Asynchronous Write functions which will command the data to be written, and then return immediatly?  It is more work but is an alternative to the producer consumer, which is also a fine solution in this case.