CRIO Save to Memory

[Wmarchewka]

Hi. I am new to Labview and CRIO. I have an application were i want to record 24 channels of data at 50K samples/sec for 2 to 3 seconds. I am able to follow this example.  http://zone.ni.com/devzone/cda/epd/p/id/5919

However, i do not need to stream the data the the PC host, but rather collect it on the CRIO disk.  I have read that it seems the limitation for the disk is around 1.5 - 2MB per sec.  In my example, it would be approx 5MB/sec (50K * 24channels * 4 bytes).   So i created a VI that instead of streaming, just writes into a huge array.  Does that seem plausible?  When i collect enough samples, i will then write to disk. But, it seems that even an array at 500K seems to cause 100% processor usage.  I would use the streaming, however, i will be increasing to 48 channels shortly.  It says the cRIO has 128MB DRAM, i am assuming this is where the array would be created?  Any help is appreciated.  I can attach my code if necessary, but would just like to have some guidance.  Thanks.

CRIO 9014

Labview 8.6

6 - 9233 analog modules for data acquisition, one 9215 analog module (for trigger in)

Message Edited by Wmarchewka on 11-26-2008 10:22 AM

Message Edited by Wmarchewka on 11-26-2008 10:25 AM

[Christian_L]

Instead of storing the data in a large array in LabVIEW I suggest that you use the host side DMA memory to store the data coming from the FPGA before writing it to the disk.This memory is allocated from the DRAM on the controller.

You can define the size of the host (RT) side DMA memory using the FPGA host interface method node DMA Configure function. Here are the basic steps:

1. Before running the FPGA acquisition, in the host VI configure the DMA buffer for the desired size (3600000 elements for 3 seconds)
2. Start the DMA on the host side
3. Start the acquisitionn on the FPGA which will stream data to the DMA buffer on the host
4. While the acquisition is running or after it has completed, on the host read the data in smaller chunks from the DMA buffer and write it to disk doing any processing on the data if necessary.

Message Edited by Christian L on 12-01-2008 10:38 AM

[Wmarchewka]

OK i tried that, and it still can't keep up with writing to the disk.  So abondoning disk writing for now, one thing i notice is that i can increase the size of the RT FIFO, but only to a certain amount. If i go to above 1,000,000 i get an error 2 in RT FIFO CREATE, which i cannot find info on. Even when i go to 1,000,000 on the RT size, i keep getting a RT timeout occuring on the RT FIFO WRITE when i hit about 500,000 elements, which i cant understand. I guess my thoughts were that i could just create a large RT FIFO of say 4,000,000 elements and just write into it.  When i hit 4,000,000 just dump to disk.  But according to what i read, the timeout on the RT WRITE states the timeout "specifies the amount of time to wait for an empty slot to write the data in the RT FIFO. The default is 0, which returns immediately." 

1. Why would it be timing out if the size is 1,000,000 but timing out at 500,000?

2.  Why cant i create a RT FIFO size such as 4,000,000?

3. What is error 2 in RT FIFO CREATE?

[StephenB]

Hi Wmarchewka,

I think there is some confusion as to the role of the DMA FIFO. There is a great document about it here: http://zone.ni.com/devzone/cda/tut/p/id/4534

Basically it is used as a transfer mechanism between the FPGA and the RT controller. It is a 'fast path' between the two. You can stream data from the FPGA through a DMA FIFO to the RT Host (or vice versa), but it is not designed to be a temporary storage element. So if you configure a DMA FIFO to go from the FPGA to the RT controller, you need to write to the FIFO on the FPGA and read from the FIFO on the RT.

The size of the FIFO is flexible, but it is flexible because it is intended to be set by the programmer based upon their knowledge of the device's timing. For example:

• FPGA to RT Controller DMA FIFO is created. 
• FPGA grabs one sample at a time and writes data to the FIFO at a rate of 10 times a second.
• RT Controller reads data off the FIFO at a rate of once per second.

The desired size of the FIFO in this case would be 10. This would ensure that the size of the buffer is adequate to cover the gap in time between read and writes. The RT controller can read 10 elements at a time. 

As you can see, the DMA FIFO is just a transfer mechanism between the RT and the FPGA. The issues you are running into are related to using the DMA FIFO as permanent storage mechanism, which it was not intended for.

It looks like your issue is trying to find a place to store the data. What you can do is transfer the data from the FPGA to the RT in a DMA FIFO and then either store it into memory by just writing it into an array on the RT VI, writing it to disk, or writing it to some network device.

[JMota]

Hi,

 >OK i tried that, and it still can't keep up with writing to the disk.

Can you provide more details on why the solution Christian suggested didn't work for you? I thought it should have worked. Were you using an FPGA DMA FIFO?

Message Edited by JMota on 12-03-2008 09:57 AM

[Wmarchewka]

While i cant say that didnt help, increasing the size of my RT FIFO seemed to be the key.  One of the issues that i ran into was on my RT FIFO CREATE, i lacked understanding as to the difference between SIZE and ELEMENTS IN ARRAY.  I simply wanted to create a large memory area, thus my reasoning behind wanting to create the RT FIFO to 4,000,000. Now that i understand the difference between the two it makes more sense.  Consequently, the error i was getting was out of memory error. 

[Christian_L]

For the method I described above you do not need to allocate any RT FIFO to buffer or store the data. The DMA buffer on the RT controller will store the data before writing it to disk in smaller chunks. The size of the DMA buffer is set using the FPGA Invoke node with the DMA Configure function selected.

To store 3 seconds worth of data completely in the DMA buffer you need to allocate 3600000 elements (50000 samples/sec * 3 sec * 24 channels) for the DMA buffer. I tested this on a 9012 controller and it should work.

I put together a basic example (see attached ZIP file) of streaming data from FPGA to RT using DMA and then writing it to disk directy from the DMA buffer in RT. As I don't have the 9233 modules available I simulated the data on the FPGA.

In the example I start writing data to disk from the DMA buffer as soon as the acquisition starts. When testing this code it acquires and writes data to disk for three seconds. After the acquisition is complete, the RT VI continues to write data to disk for another ~2.5 seconds reading it form the DMA buffer.

Message Edited by Christian L on 12-03-2008 12:43 PM

[Wmarchewka · Accepted solution]

Christian,

Wow!!!! I must say this is the simpliest, most well organized example i have come across.  The project worked perfectly.  I just need to add my I/O.  The program flows, it is lean, and everything has a purpose.  I think that me trying to make that streaming example into something it wasn't designed to do was causing me alot of grief.  I seemed to have all the pieces of the puzzle, yet couldn't get them together.  This project is a fine example of terrific programming.  Thank you tremendously. 

Message Edited by Wmarchewka on 12-03-2008 02:03 PM