Data Transfer methods between FPGA and RT

[Brinoceros]

It does sound like you are running out of memory for your queue.  Maybe you need to decrease your visuals so your loops can execute a bit faster.  You could also try to send larger chunks of data in the queue so the visuals will update less often.

[guilio_2008]

I switched from the que to the RT FIFO to use in a producer consumer loop and I still get the same thing.  It runs fine but when I put the GTDMS saving in there, that is where the problems come in.  Attached is an excel file that I am saving on the RT controller that shows what I mean. 

Right now, I'm to the point where I'm using a TCP to stream the data to the PC, but when I tyr to unflatten it, it has errors, memory, or corrupt data.  Even when I try this method, it slows the RT program down alot. 

Thanks,

guilio

[nathand]

This is pretty basic so feel free to ignore it, but your earlier comment that taking out the visual aids fixes the problem makes me wonder - your visual aid isn't competing for data with your other loop, is it?  For example, you don't have a single queue that feeds data to both the visual aid and whatever else you're doing?

[guilio_2008]

If I am thinking what you mean, then no.  One loop ques and displays while the other loop deques and saves.

Here is what I did so far.  I cut 2 more inputs out now leaving me with 8.  I then flatten that and then send it through the TCP/IP write.  On the host side, I take the TCP/IP read and get the string data.  However, when I try to unflatten it, I get a error 116.  Anyhow, know why?

Othe rmethods.  I have now 2 time loops that I'm using for producer/consumer.  The good news is, I'm saving data and I don't see any jumps.  The bad news, the data doesn't flow nicely.  When ever the elements left on the DMA FIFO reaches 0, it acts like it waits till the FIFO fills up again, then send the data.  Could this be a time loops setting?  IE the dt is too fast or the source clock isn't right?  How do I find out what to set it as? trail and error?

Thanks,

guilio

[nathand]

Would it be possible for you to simplify your code to the point that it demonstrates the errors you're seeing but could be safely posted here?  I suspect there's a misunderstanding about the way that some part of this works, since I've used DMA FIFOs to transfer large amounts of data (values for a cRIO chassis full of IO modules) and then sent that data over TCP/IP without encountering any of these problems.

Also, are you perhaps confusing Typecast with Flatten/Unflatten from String?  You can use either approach for converting your data, but both the sending and receiving ends need to use the same one.  If you were to typecast your data to a string when you send it, and then attempt to Unflatten from String on the receiving side, you'd probably see this error.  Another possibility would be that the data type you're feeding to Unflatten From String doesn't match the data actually being received.

[guilio_2008]

Would it be possible for you to simplify your code to the point that it demonstrates the errors you're seeing but could be safely posted here? I can and will try that today. I suspect there's a misunderstanding about the way that some part of this works, since I've used DMA FIFOs to transfer large amounts of data (values for a cRIO chassis full of IO modules) and then sent that data over TCP/IP without encountering any of these problems.  What kind of rate where you sending the data?  I am sending 100kS/s.  Do you have code that you can post to show me how much data you are sending?

Also, are you perhaps confusing Typecast with Flatten/Unflatten from String?  No, I am using the Flatten to String in the RT program, and Unflatten from String in the PC Host program.  You can use either approach for converting your data, but both the sending and receiving ends need to use the same one.  If you were to typecast your data to a string when you send it, and then attempt to Unflatten from String on the receiving side, you'd probably see this error.  Another possibility would be that the data type you're feeding to Unflatten From String doesn't match the data actually being received.  This is a possibility.  Anyone have an idea on how to do it?

Currently, I am now using timed loops as producer/consumer.  The question I have now, how do I optimize my timing of the loops?  Meaning, how do I know how to set the dt of the loops for best performance?  Trail and error?  Also, how can I be sure that the RT FIFO is getting all the data?  My concern is I can get the loops running very good if I set the elements of the RT FIFO low compared to the number of elements going into the RT FIFO. (if I have an array of 8000, I can set my RT FIFO elements at 4000 and it runs smooth)  But if I set this number to be equal, then that where it doesn't run smooth.  I don't mind setting the saving loop with a lower elements, but want to make sure it's not taking 4000 of the 8000, then dumping the rest.

Thanks,

guilio

[nathand]

I don't know the exact rate at which I was sending data, but not nearly that fast.  I no longer have access to that code, it was at a previous job.

You can do a quick check of the datatype of your wires by turning on context help and putting the cursor over the wire - maybe on your RT end you're sending data as I16 and your receiving end is expecting I32, so that your diagram looks the same but the data isn't consistent?

The best indicator of whether you're losing data is the Timed Out output on the FIFO write.  If that ever goes true, you're not reading from the FIFO fast enough and it filled up.  You could connect that to an indicator on your FPGA (you'll probably want to use a shift register and an OR to "latch" it, and provide a way to clear it).  Read that indicator in your RT code, and if it's ever true then you lost data and need to resynchronize.  There will be some trial and error in your loop timing.

[guilio_2008]

 The FIFO's are timing out and since I done the time loops I have managed to keep them from doing so and give me data with no breaks.  Before I try posting the code, I am going to try another route and want to get some input and information

The Read/Write control is what I initally wanted to use rather than the DMA FIFO.  Simple and I though it would handle the values that are set up as indicaters on the FPGA.  Although my full wave data is reading 100kS/s and I am successfully getting that data through using a DMA FIFO, the 8 values I want to save are only reading once per cycle.  (ie.  1000Hz signal, lets say I want only the +peak of the V, that is the value I am sending out)  Shouldn't the read/write control fuction be able to read that single data point that quick?  Is the bus between the FPGA to the RT that slow? 

Thanks,

guilio

[nathand]

DMA FIFOs are in general a much better way to transfer data from the FPGA to the RT code, unless you only need to transfer a single point of data.  To transfer an array of data between the FPGA and RT using front panel controls, you actually need two controls - a numeric control, and a boolean to act as a semaphore.  When the FPGA updates the front panel value it sets the boolean to true, and then waits for it to become false before writing the next value.  The RT code waits for the boolean to be true, reads the numeric value, then sets the boolean back to false.  If you don't do this, you cannot guarantee that the RT side will read all the values written by the FPGA.

Why can't you combine these two approaches, and only send over the DMA FIFO the 8 values that you need to read?

[guilio_2008]

That is what I am doing.  I actually have 2 DMA FIFO's (cause there is a loop for each sensor (2 sensors), 4 values in each loop, 8 values total)  that I am sending the data through to the RT controller.  It will get sent through the DMA FIFO, but when I start saving the data on RT, or adding stuff in the loops, that is where problems occurs.  It slows the loop down, then the que/deque fill up, shuts the program down.  So far I got it set up not to do that at least, but figure when I start adding in more stuff, It's going to get worst.

Now, when you say trainsfer a single data point, how often?  Does is send that data point each loop interation on the FPGA?  If so, then I should get that cause the indicator is a single data point that updates with a single new value once each interation on the FPGA.  I have no array's in the FPGA program until it gets to the DMA FIFO which is the end of the loop.  Initially, I was using the read/write to transfer a cluster of 10 values and it didn't seem to update fast enough.