Upgraded from an Intel X520-DA2 to Chelsio T520-CR and iperf dropped from 9.9Gbps to 7Gbps



  • My pfSense box consists of the following:

    pfSense 2.4.4-RELEASE-p3
    Supermicro X10SRL-F motherboard with BIOS 2.0
    Intel Xeon E5 2683 v3 CPU 14 cores
    32GB of DDR4 2133 RAM
    Supermicro 32GB SATA DOM

    I ran an Intel X520 in it initially and all was well, but I upgraded over the weekend to a T520-CR and I noticed on the initial boot, that pfSense appeared to update the firmware on it.

    The web GUI seems a little choppy at times compared to the Intel NIC. But today I did some iperf testing and found performance to be only 70% of what it should be.

    Are there known issues with Chelsio NICs in Supermicro motherboards? Do I need to upgrade the BIOS? (current version is 3.2 and I'm on 2.0)

    Do I need updated drivers or firmware for the T520-CR or was that taken care of on the first boot and I noted above?

    Thanks!


  • Netgate Administrator

    The firmware will have been updated by the driver if it was older on the card.
    You should be able to see it from: sysctl dev.t5nex.0.firmware_version

    I believe the version in pfSense 2.4.4 is still 1.19.1.0 but there are newer firmware versions available.
    You can update it using cxgbetool from the cli. However I would nit expect it to gave made that much difference.

    Try testing with top -aSH running to see how the load is spread and what it using CPU cycles.

    Steve



  • Yep, I'm on firmware 1.19.1.0.

    Turns out that my main issue was that the client I was running iperf on was the bottleneck. I got much better results when another client as seen here:

    alt text

    Ran the test again this morning with top as well using the arguments you suggested. For some reason I was down by about 1 Gbps compared to my previous test. Not sure why.

    alt text

    Maybe I'll put the Intel X520 NIC back in there alongside the T520-CR to do some back to back tests.


  • Netgate Administrator

    Looks like you're running iperf on pfSense which is not really a good test. That could be the limit.

    Much better to test between LAN and LAN2 with separate iperf server and client if you can.

    Steve


Log in to reply