ETH0 and ETH1 on SOM Dev Board (sbRIO-9651)

[User5551212]

I am getting about 33M bytes per second over ETH0 (dev board label) and 36M bytes per second ETH1 (dev board label) using TCP/IP UDP data  size of 65,500 bytes which gives the best performance.

Is this the performance limit of the SOM or is there a more preferred method to get something like 80M bytes/sec (800M bit/second) which is what I would like to get. 

The one core is running at 70% utilization and the other like 20%.

I have jumbo frame enabled at 9K.

What can I try?

Thanks.

[sassyaspy]

Hello User5551212,

Can you tell me a little about your test, specifically the method you are using to create and send packets? Although I don't believe the physical limitations of the board would not restrict you to 30 MB/s, the speed of software execution and the real-time processor might be a factor here. Ethernet communication is single-threaded, and therefore would not be able to take advantage of your dual-core setup!

[User5551212]

Hi Siana,

I am getting about the same numbers with TCP data transfers but sending 2Mbyles of data per transfer seem to get best performance.

If you can recommend another way of transferring data from the RTOS sbRIO to a PC host with less demanding CPU utiliaztion that would be very helpful.

I might try sending UDP/TCP on two different ports and see if I can get to 60Mbytes/sec since you indicated that they are not multicore functions.

[sassyaspy]

Hello User5551212,

After doing some additional research, it seems that our benchmarking tests for this device put the typical Ethernet communication at around the same rate that you have been getting. There is a limitation associated with the architecture behind the communication that probably will not allow you to get 80 MB/s speeds. The only way around this would be to create your own Ethernet controller in the FPGA, which would take up a lot of the FPGA fabric but could greatly increase the ceiling on communication speeds.

From what you told me, it seems like you have already taken care of most optimizations that we suggest in software (ie, using jumbo frames, increasing packet size, utilizing the Nagle algorithm). The only other suggestion I could make would be using parallel TCP streams. However, I found a couple of resources that might be helpful for the underlying configuration of TCP or UDP. Please keep in mind that changes to the Linux RT kernel will likely not be supported!

- http://www.embedded.com/design/operating-systems/4403058/Accelerating-network-packet-processing-in-L...

- http://www.techrepublic.com/article/take-advantage-of-tcp-ip-options-to-optimize-data-transmission/

- http://datatag.web.cern.ch/datatag/howto/tcp.html

[User5551212]

Thanks for the attached links and ideas on improving throughput over the SOM Gigabit Ethernet. 

The FPGA eth1 connection was an idea I had thought about.  I was thinking that if I had two bit files (one initial configuration and another data transfer) I could switch between and maybe I could reduce the amount of FPGA fabric needed to implement.  The idea being to let the RTOS configure all the basic parameters (since NI has already done the drivers for this) of the Micrel KSZ9031MNX gigabit Ethernet controller (which might consume a lot of memory blocks in the FPGA) then resend another bit file to reconfigure for a FPGA "data only transfer" to the eth1 controller.

I would guess from the FPGA I could get full bandwidth.  I need to check and see if the eth1 Micrel part gets a reset signal when the bit file is configured - which would have to be disabled if indeed it was.  Is there a set of schematics available for the development board?

I will check out the links you provided.

Thanks.

[tannerite]

Hi,

I am not sure I fully understand what you are meaning about swapping bitfiles after configuration of the PHY. You use the sbRIO CLIP Generator to enable the eth1 port (GMII lines are routed through the fabric [EMIO]). That bakes in the necessary behind-the-scenes VHDL into your 'blank' bitfile. Regardless of if it is routed throug the fabric to an external PHY or using eth1, the bottleneck is still going to be the Zynq MACs that limit the bandwidth. As stated above, the throughput you are seeing is similar to iperf benchmarks that have been done.

If you try to load a bitfile that doesn't have the eth1 VHDL baked in, that interface will no longer be accessible.

There is a company that I believe instantiated a MAC on the fabric and used other resources on Zynq to achieve very high throughput; however, they dedicated one of the processing cores for Ethernet communication. Here is a link to that I remember reading a while back. I have not fully read into this stack, just sharing for reference.

Since the SOM has a socketed CLIP, it is possible for you to instantiate a MAC on the FPGA (PL), however, that is something that must be done on your own and is not supported by National Instruments.

As an aside, are you needing sustained 80 MB/s throughput for Ethernet or are you taking burst measurements that can be streamed ex post facto?

[tannerite]

Sorry, I missed your second part of the question.

We have the schematic of the Reference Carrier Board host from the model page, under the Resources tab.

There is also a hardware development community solely for sbRIO, including the SOM that may be helpful to you in the future.

[User5551212]

Hi Tannerite,

Thanks for the schematics link.  I missed the

som_reference_design_files.zip on the right side the last time I was on the page.

Implementing it on the FPGA will be a challenge.  I will give some of the RTOS tweaks a try.

[tannerite]

Hi,

I agree and would not recommend it. I am just being as transparent as possible in saying that it may be possible, but it would be operating outside of the bounds that it is meant for.

[User5551212]

So, I tried using two UDP VIs going to two different UDP ports on one Ethernet adapter (tried both eth0 and eth1) and data transfer was maximum 44M bytes/sec.  Also, sending data to eth0 and to eth1 (dual Ethernet adapters) also yielded the same results of combined transfer rate.

Each timed structure was on a different CPU core and both cores where at 55% utilization.  Also, I tried setting both timed structures to CPU automatic and had same results.

No tweaks to the RTOS NI Linux Real-Time ARMv7-A 3.2.35-rt52-2.10.0f0