cRIO Consumer Producer FPGA RT 100 kHz

[amazzeo]

Hello,

     NI technical support has been great up to this point, but my problems are not completely resolved yet.  I am essentially trying to bring 5 channels (FXP <+/-,20,5>) off of a cRIO 9073 FPA at 100 kHz and save them to a binary file on the cRIO itself.  If I understand things correctly, that would come out to 20 bits*100kHz*5 channels=1.2 Megabytes/second (MB/s).  Given that you can buy the NI 9802, which can read/write to flash at 2 MB/s, my task should not be a problem.  To keep things simple at this point, I am writing the data to a file as a 1-D array without splitting it into 5 separate columns.  I have a separate vi to convert the binary file to a txt file, which I can then process in MATLAB.

      You can see the attached images, which depict the acquring process on the FPGA, the consumer-producer structure on the real-time side, the file saving within the consumer portion of the loop, and the separate vi for converting the binary file to a text file.  I can run the producer loop (loop with DMA FIFO read) at 5 ms, the consumer loop at 1 ms, the queue with a variable size (-1 for queue size), and  the FPGA count at 10 microseconds (100 kHz) without getting timeouts from the FPGA DMA FIFO (I usually have to wait a few seconds for things to get going and reset the latch once).  I can also run the producer loop at 10 ms and the consumer loop at 5 ms.  Now, once I enable the "write", the number of elements in the queue begins to grow and things halt soon thereafter.  When I process the saved binary file, I do get some data but there is "jumping" in the data (http://forums.ni.com/ni/board/message?board.id=170&thread.id=366635&view=by_date_ascending&page=1).  Even when I reduced converted the FXP data into 8 bit integer data before saving and kept the number of elements in the queue from growing, I still got "jumping."

     So, I am requesting any suggestions the community might have for eliminating this "jumping" behavior and keeping the queue from growing too quickly.  Thank you for your time.

Aaron

[RavensFan]

Can you give an example of what you mean by "jumping"?

One problem I do see is the way you are handling a timeout in the consumer loop.  You have a 1 msec timeout on your dequeue.  You say you are running the consumer loop at 1 ms and the producer loop at 5 ms.  So you are only putting in an item every 5 msec.  But your dequeue operation is going to stop after 1 msec, and 4 times out of 5 it will have no data.  If your Write boolean is on, you will be writing 4 elements of default data (coming out of the timed out dequeue) to 1 element of real data (coming out of the dequeue when data has actually been enqueued in the producer loop.)

Why do you have a timeout on your dequeue function?

[amazzeo]

Hello,

     Thank you for the response.  I have attached a text file, which illustrates the jumping I'm talking about.  Some portions are fine, but others are not, and still worse, there are some added 0's. 

     I don't really have a good reason for having the 1 ms timeout on the dequeue.  I think I've tried it without and still had problems.  That said, I will take it out and report.  Thanks again.

Aaron

[amazzeo]

Hello again,

     I removed the timeout on the dequeue as shown in the attached image.  When the producer was set to a period of 5 ms and the consumer a period of 1 ms, everything seemed fine until I triggered the write.  Then, the queue began to fill up quickly, and I disabled the write before things halted.  I was able to manually stop the vi after the number of elements in the queue stabilized or was reduced.  I have also attached the text file results.  Again, the "jumping" is present.  I also tried running the producer and the consumer both with a period of 5 ms, but the queue filled up.  Other ideas or comments?  In case, I have done something incorrectly, I have also attached my MATLAB script for resorting the data.  Thank you for your time and help.

Aaron

[RavensFan]

I believe your queue is filling up quickly because you are putting in new pieces of data 200 times a second.  But file operations are taking longer than.  There is now way you can expect to write a piece of data to the file in less than 5 milliseconds.

By the way, you are not writing to a binary file.  You are actually writing to a text file.  If it was true "binary" file, you wouldn't be able to read it.  It would look like garbage characters.  Someway or another, with the fixed point datatype you are using, the ASCII characters are being transferred in a that is human readable.

What you need to do, is to read from the queue in larger chunks.  Read all available elements in the queue, combine them together, and write the data out to the file at once.

[amazzeo]

Hello,

     Yes, the queue is filling up too quickly.  How do we read from the queue in larger chunks?  I tried using the flush queue command at some point, but that didn't seem to do the trick.  Maybe, I implemented it incorrectly, and I would appreciate some further insight into this point.

     As for the data output, it initially goes to a binary file.  I use the vi shown in the ReadProcessDataFile.jpg included in the original post to create a text file, then I process that text file in MATLAB to get the correct number of columns.  If I am introducing the "jumping" in the conversion process, please let me know.  However, I think the issue occurs during the data acquisition and writing to a file.  Thanks.

Aaron

[RavensFan]

I understand now that you are converting the binary file to a text file.  I don't work with the fixed point datatype, so I'm not sure in what ways that conversion might get messed up.

Flush queue should work for you.  It outputs an array of all the elements in the queue.  So you just have to process an array of elements instead of one element at a time.

[amazzeo]

Hello,

     Thank you.    If enqueue only saves one element at a time (1-D array in this case), shouldn't the dequeue be fine at removing one element (1-D array)?  That said, I guess flushing the queue would ensure that everything possible is removed from the queue.  I'm going to spend a little time trying to flush the queue.  Do you have any tips for trying to flatten or catenate separate elements of 1-D arrays that will come out with the flush?  Thanks.

Aaron

[amazzeo]

Hello,

     I was able to flush the queue as shown in the attached vi without writing enabled, but I do not know how to transfer the flushed elements to a waveform chart.  When I tried to write to file with the consumer loop running at 5 ms, the program halted after the number of elements in the queue began to increase.  It looks like I have lost the connection with my cRIO and will need to reboot it manually when I get a chance (probably on Monday).  I am looking forward to any insights applicable to resolve these issues with the queues.  The ultimate goal is to save data from 5 channels at 100 kHz.  Thank you.

Aaron

[RavensFan]

---

amazzeo wrote:

Hello,

     Thank you.    If enqueue only saves one element at a time (1-D array in this case), shouldn't the dequeue be fine at removing one element (1-D array)?  That said, I guess flushing the queue would ensure that everything possible is removed from the queue.  I'm going to spend a little time trying to flush the queue.  Do you have any tips for trying to flatten or catenate separate elements of 1-D arrays that will come out with the flush?  Thanks.

Aaron

---

Yes, dequeue would be fine at removing 1 element at a time.  The problem is the file write operation takes a long time which causes the consumer loop to run slower.  It can't keep up with the producer loop.

Since you are passing an array of fixed point, and the output of the flush queue is a an array of elements, there is a problem.  You can't have an array of arrays in LabVIEW.  So LabVIEW appears to put the array into a cluster, then outputs a 1-D array of those clusters.  So in order to process them, you'll need to cycle through the elements of the 1-D array, unbundle from the cluster, and then concatenate the resulting arrays together.

This actually tells me something about the problem you are having with your binary file.  You are writing an array of fixed points to the binary file.  Is this really what you want?  What do these fixed points mean in your application?  With your binary write, you do not have the input for Prepend Array or String Size (T) wired, so it defaults to true.  Everytime you write out your array, an additional 4 bytes are placed at the beginning so LabVIEW knows how big the array is.  That wouldn't be bad except you are not accounting for this in your Binary to Text VI.  There you tell the Binary Read VI that you are reading a fixed point datatype as opposed to an array of fixed point data types.  So what happens is those 4 bytes that tell the size get interpreted at a fixed point value and are thus converted to a number that makes no sense to you.  Since the array is small (probably only 1 element in it) 3 of those 4 bytes are zero since the size of the array only requires 1 of those 4 bytes (255 or less elements)

---

amazzeo wrote:

Hello,

     I was able to flush the queue as shown in the attached vi without writing enabled, but I do not know how to transfer the flushed elements to a waveform chart.  When I tried to write to file with the consumer loop running at 5 ms, the program halted after the number of elements in the queue began to increase.  It looks like I have lost the connection with my cRIO and will need to reboot it manually when I get a chance (probably on Monday).  I am looking forward to any insights applicable to resolve these issues with the queues.  The ultimate goal is to save data from 5 channels at 100 kHz.  Thank you.

Aaron

---

Why are you talking about waveform charts now?  Up until now you haven't talked about it, and your VI doesn't have one.  A waveform chart can take either 1 data point at a time or an array of datapoints.  But your compactRIO application shouldn't have any front panel user interface elements like a chart on it anyway.  I wouldn't worry about waveform charts until you get your datatypes (scalar vs. array) and your queue mechanism working right along with your Write to Binary file.