Logging TDMS Data on USB3.0 on cRIO 9045

[ReKat]

Hi LabVIEW Programmers,

I have been dealing with issues trying to log (backup) data on USB3.0 hanging off of cRIO 9045. I have been pulling my hair out for the last few weeks trying to resolve this issue. Here is a description of the problems.

Objective:

1. Log the TDMS data (~200 channels) on USB3.0 on cRIO 9045. The same data is streamed via network stream to a host application and is written to file on the host PC (this is working). Writing the data to the USB is backup in case there is network connectivity issue, e.g. network cable is cut/disconnected….the idea being that if there is connectivity issue, the cRIO will continue to run headless (rtexe is set to run on startup) and is still saving data on the USB drive.
2. Every N^th iteration or so (say equivalent to 1 hr), I would like to create another file. This is so that if somehow a file becomes corrupt, I am not losing the whole data set so I won’t have to repeat the whole, say, 12-hour long test.

Problem:

Writing the 200 channels of data continuously at 100Hz to the USB on the Crio, I encountered a loss of data in random location. My guess is that writing every data point continuously like this is too much on resource-strapped RT environment. So I decided to buffer the data into a queue and dump it every so often (i.e. write less often). I use a simple state machine type structure for this (see attached code). I am still losing 4 or 5 data points in between file creation. I must be missing something!

I am attaching my latest attempt. So my questions are:

1. Is writing data at 100Hz continuously for 200 Channel too much for the cRIO even though the USB3.0 is supposed to be better?
2. Is the attached buffered implementation with state machine a right way to do this?

Any input is appreciated. Many thanks in advance!

PS: I am using LabVIEW 2017 (32-bit).

[PiDi]

If I understood this logic correctly (it's a bit late and I'm not thinking straight):

After opening the new file you're waiting until the queue is full - so the Enqueue Element in the top loop will block until your state machine rolls into Write TDMS state. It's not a problem, but your code doesn't show where the Data comes from, other than the #Note which doesn't explain the transfer mechanism form "another loop". And depending on this mechanism, you migh be loosing data somewhere upstream due to the Enqueue Element block.

Also this one:

It's probably not the case, but what if a new element is enqueued between Get Queue Status and Flush Queue? Why not using "remaining elements" output of Flush Queue?

[ReKat]

Hi PiDi,

Thanks for the quick response. I have been out of the office so sorry for the delayed response on my end. I will look at your comments this week and get back to you.

Thanks again.

Renato