How to measure PCI bandwidth in Labview & Labview FPGA

[TomC_NGC]

I have a PXI backplane with a Rio 7811R (FPGA board). I would like to use this FPGA to measure the achievable bandwidth of the PCI bus.

So far I have a DMA FIFO (FPGA Target to Host) that continuously writes data, and is continuosly read. See attachments. I'm not sure how to leverage that to measure in MB/sec.

[Captain_Kirby]

Hi,

Using this setup to measure the achievable bandwith of the PCI bus will be extremely difficult if not impossible. The bus is used by almost all devices communication in the chassis, and to find the maximum rate, you would need to ensure that no other devices use the bus (which is not possible.) Also keep in mind that other factors will slow down your measurements such as the operating system on your host machine (which will be significant if it is windows, and at least noticable if it is a real-time system.) Also note that there is quite a bit of overhead when you initiate any communication with your FPGA. There is data verification, and device routing data that is passed with each request which you cannot control or factor into your measurement.

With all that said, I would probably start by defining a set block of data in the FPGA, and then, when a certain front panel control is triggered (a boolean control) I would have the FPGA place the entire block into the FIFO, and at the same time, have the real-time VI start a timer. Then, using basically the same loop with a varying element input to the fifo that you have given in your screenshot, I would read data untill the number of elements in the block is reached. Then stop the real time timer, and divide the number of bytes placed into the FIFO by the timer output to get a bytes/sec. value. Be sure to place 32 bit elements into the FIFO, as this is the most efficient size for the FIFO.

[TomC_NGC]

After some changes, I modified it to get a whopping 200 Bytes/Second (see attachment). There are a couple other cards in the PXI chassis, and they might be running background tasks. However, I assume most of my overhead is in Windows, like you said.

Might I get better results in real-time mode instead of in windows?

[Captain_Kirby]

Hi,

You will probably get much better results from a real-time VI then a windows VI. But I just want to stress again, that it will be almost impossible to get accurate readings due to the many limiting factors. Reducing these factors will get you closer (like removing windows from the equation,) but its still fundamentally difficult to measure the raw transfer rate of a PCI bus. And this comes from some personal experience.

Good Luck! 

[reddog]

From your screenshots, it looks like you are executing your FPGA VI on your development computer using simulated I/O.  This will not give you an accurate measurement of PCI bandwidth as the PCI bus isn't used at all in this configuration.  If your goal is to get an idea of attainable transfer rates between the FPGA and the host computer, I can tell you that we typically see maximum transfer rates in the range of 80 - 110 MB/s when transferring from the FPGA to the host computer with your hardware device.  This of course is system dependent and assumes there is very little PCI traffic generated by other devices in the system.  I wouldn't expect this number to change much when run as a Windows VI vs. an RT VI unless you have very tight latency requirements for your processing loop. 

If you would still like to write a benchmark for your system, I would recommend changing your test methodology slightly.  First, make sure you're executing the VI on the FPGA target and not the development computer.  I would then use the FPGA VI to send data to the host at a fixed rate and use a VI on the host to simply read and discard the data in a while loop.  On the FPGA VI, you should track whether the write to the FIFO was successful or late by looking at the Timed Out? output and/or the Late? output inside your sequence structure.  You'll probably want to rewrite your code so that these output are "sticky" until read or reset by some other control.  You can then check these outputs within the while loop of your host VI.  If at any point you detect a late condition, it means your system wasn't able to maintain the data rate without losing data or temporarily slowing down the transfer rate. 

When benchmarking, I usually like to start at a relatively low transfer rate and then gradually increase the rate until failure.  You may also want to play around with the FIFO size on the FPGA and DMA buffer size on the host as these may also impact the transfer rates.  My experience has shown that a FIFO size around 2048 on the FPGA and a DMA buffer size around 1 MS on the host are usually large enough to sustain the fastest rates.  I've also seen cases where the number of points read each loop iteration from the DMA FIFO has an impact on performance so you might want to experiment with this setting as well.  If you want to go faster than ~20 MB/s, you should also rewrite your FPGA VI to use the single cycle timed loop instead of a while loop.  With these changes, you should have a baseline for the fastest raw PCI rates achievable on your system.  If you plan to perform inline processing on your host system, you may also want to add some processing into the while loop of your host VI to give a more realistic benchmark.  This will give you an idea of whether the processing power of your CPUs or the PCI bus will be the ultimate bottleneck for your application.