timed loop tdms write

[DennisBengs]

Hello.

I have a simple question about whether it is an acceptable practice to write to a TDMS file inside a timed loop rather than in a while loop using Labview RT. This comment made me unsure if this was a good idea.

"Always perform file IO in non-deterministic threads.

    A rule of thumb is that the complete application should consume less than 80% of the CPU resources to avoid starvation."

http://www.ni.com/white-paper/10435/en/

I have made an FPGA program which writes data coming from a NI 9234 module to a FIFO buffer.

The RT program uses a single 1 Khz timed loop to check if the FIFO size has reach a certain chunk size (25600 values) and if so the data is written to a TDMS file. In this scenario, is there a reason to prefer using a while loop or is it okay to use a timed loop?

- Dennis Bengs

[arevh]

Hello Dennis,

Of course to have a Timed Loop to read data from the DMA is good idea, as you will have determinism and if correctly configured, you will not lose data. 

At the same time it is not preferred to implement file IO operations at first in the same loop as DMA reading, and second, in a Timed loop.

So the best solution is to read DMA data in Timed loop, write it to the queue and read data from the queue in a while loop running in parallell, and write to the TDMS file.

Thanks,

Arev 

CTO | RAFA Solutions

[DennisBengs]

Thank you for your reply.

I understand that this is the prefered way of doing it using the consumer/producer pattern. It does add a lot of extra complexity so I'm still curious about a few things.

When you say that it is possible to lose data, how does that happen? Take my scenario for example. In my timed loop I use FIFO.read twice. Once to check how many elements there are in the FIFO without reading any, and if there are enough elements a second FIFO.read reads 25600 elements and writes them immidietely after to a TDMS file. I am not sure where there is a possibility for data loss at this point. The FPGA program checks for overflow/underflow errors which should prevent losing data that way. The accuracy if the timed loop is not that important as long as it empties the FIFO buffer faster than it's growing.

- Dennis Bengs

[crossrulz]

What else do you have happening in the looop?

If all you are doing is a read from the DMA and then logging to disk, I would not use the timed loop.  Instead, I would read 0 samples from the DMA just to see how many samples are in it.  When the number of elements reaches the number you want to log at a time, you read that many samples and save it to your file.

Now if you are doing a control algorithm with the timed loop (rarely would I say you actually need the timed loop otherwise), then just put the samples into a queue and write the data in another loop.

[arevh]

For example if your loop has been slowed enough to make the FIFO full, you may lose data. Also if you are handling overflow/underflow in the same loop, for example by reading Timed Out from the FPGA DMA, and emptying the FIFO, it again will not function when the loop is busy with File IO operations.

How are you handling overflow/underflow in the FPGA?

Thanks,

Arev 

CTO | RAFA Solutions

[DennisBengs]

I have included an image of the subsection of the vi with the timed loop. It's rather crowded but hopefully it should be clear what it is doing. Also it's probably rather buggy still.

It has been suggested to me that using a timed loop would increase the priority of certain parts of Labview code relative to other programs like the FTP server running in the background on the CRIO 9074. I do not know if this is true though.

[DennisBengs]

For overflow I'm checking the Timed out? variable in the FIFO write vi with a timeout of 0. For underflow I'm checking for a specific error when reading from the NI 9234 module.

[arevh]

Are you creating a new TDMS file for each batch read from the FIFO every 1 ms?

Thanks,

Arev 

CTO | RAFA Solutions

[DennisBengs]

No. The timed loop is one 1 Khz but most of the time there isn't enough elements in the FIFO yet. The TDMS file is written to once every second (when the number of elements in the FIFO >= 25600). A new TDMS file is created every fifth minute when the number of samples in the current file is 25600 * 300.

[crossrulz]

---

DennisBengs wrote:

It has been suggested to me that using a timed loop would increase the priority of certain parts of Labview code relative to other programs like the FTP server running in the background on the CRIO 9074. I do not know if this is true though.

---

FTP server is already of a lower priority than anything in your code.  I would not worry about that.  In fact, I do not see anything in your code that warrents a high priority.  Just use a normal While loop with a little longer of a wait.