circular buffer USB

[Dindex82]

Hello,

I'm wondering if it is possible streaming of data to a memory connected to cRIO 9063 USB connector.

My application currently streams data to a circular buffer implemented in Pc Host, using Network Streams. I need data of 24 channels sampled at 250 Hz for half an hour. This because sometimes the trigger condition is given from a web service which is updated every 20 minutes giving the timestamp of the event. So I use the timestamp to search data in the buffer and extract it from there. 

This cannot be carried out by 9063. I would remove Pc Host from my application and use USB memory. Is it possibile? I'm looking for an example but it seems there are no examples on the web.

KR

Davide

[James.M]

This is very do-able and is quite easily. Here's a very simple example.

If you're trying to automatically find the usb drive, you can create a polling loop to check for attached drives. When I did this before, I checked for attached drives, checked what data was already on the drive to find the most recent data, and then "sync'd" whatever data from that point to current.

Make sure you write data in chunks, not just every sample, because the File I/O will be relatively slow.

[Dindex82]

Hi James, thank you!

I had already seen this solution. The task is different, I'll try to explain it in a clearer way.

I want at each time a buffer containing acceleration measurements sampled at 1 kHz and the indication of the timestamp of the first value. For example now I have an array of 24(channels) columns and 30(minutes)x60(seconds per minute)x1000 (samples per second), and a timestamp. They will obviously change in the next loop interation.

The problem is the trigger. The trigger is from Twitter. It gives the timestamp of the event, I can find the measurements of interest having the data array and knowing the timestamp of the first event in the array and the sampling rate. This is trivial using a shift register. But how can be done with a txt file?

[Bob_Schor]

As I understand it, you are only interested in retrieving (from a Web inquiry) "recent" (within the last 30 minutes) samples.  Just for the sake of argument, let's assume that you sample once a second, sampling 24 channels at 1KHz, and each sample takes 2 bytes (say, an I16).  By my calculation, sampling for half an hour would take 86 MBytes, big, but not impossible.  You also save the time of the first sample, so when asked for a Sample at time T, you can compute which sample in your buffer to return.

This works fine as long as you are still filling the buffer.  But once the buffer is full, what then?  I'd suggest "Double-buffering" -- have an array of 2 buffers.  When Buffer 1 has 30 minutes of data, start filling Buffer 2 (recording its Start Time).

Now a Request comes in.  If it is after the Start Time of the current buffer, you already know how find the Sample.  If it is before the Start Time, it must come from the previous Buffer, so extract it from there.

Note that you need to access these Double Buffers in something like an Action Engine, as you don't want to be "adding data" (and potentially switching buffers) while you are processing a "Read" request.  I assume that you are using something like a Producer/Consumer Design Pattern to fill the Buffer, so if the Consumer is temporarily "blocked" because a Read is keeping the Action Engine busy returning data, the Producer/Consumer Queue will ensure points aren't missed.

I'm assuming that your cRIO has enough memory for two of these buffers.  Keeping everything "in memory" will certainly simplify your code.

Bob Schor

[Dindex82]

Thank you Bob,

actually after the request I need for at least 30 seconds of data, not just a sample, and my samples are Doubles not I16, but this is not the point.

I already tried to use an RT FIFO with a 30 minutes buffer and there is not enough memory. This is because I asked for an USB solution, using the memory of the USB instead of RAM Memory of the cRIO.

But your solution is interesting: I could log a first file stopping logging when it contains 30 minutes, then log into a second file, when also this is full delete the first one and log there. What do you think?

[Bob_Schor · Accepted solution]

You don't need to delete anything.  Say you name the files "File 1" and "File 2" (very imaginative, I know ...).  I'd write a sub-VI called, maybe, "Open Double Buffer" that:

1. Opens the specified file (File 1 or File 2 -- you need to keep track of which), using "Open or Create" so it doesn't matter if the file exists or not.
2. Set the File Position to 0.  This effectively "deletes" any data that is already in the chosen file, combining the Delete Old and Open New steps.
3. Start saving until it is time to Swap Buffers again.  Close the File and exit.  Caller then switches buffers and calls us again (or you could do this with a reentrant call ...

Bob Schor