XG-2758 10 GbE performance

tzidore

Hi

I have setup a full production environment on a netgate xg-2758 and are now testing my 10 GbE interfaces and are getting these disappointing results.

[ 6] local 172.16.2.110 port 42103 connected with 172.16.2.1 port 5001
[ ID] Interval Transfer Bandwidth
[ 6] 0.0-10.0 sec 425 MBytes 357 Mbits/sec

Any ideas? Why this low on a 10 GbE interface?

heper

what are you testing ?

do not : client<–>router
do: client<->router-lan | router-wan <-> server

Guest

I have setup a full production environment on a netgate xg-2758

Which pfSense version is installed on that device? The same appliance from the pfSense store came with a
pre-installed and pre-tuned pfSense version, but the one from netgate could also be sorted with that, by
creating an account and downloading the ADI image from there for the installation process.

and are now testing my 10 GbE interfaces and are getting these disappointing results.

How do you testing that interfaces? Would you please provide some more information about that?
Are VLANs heavy QoS and many rules in the game? Are these ports WAN or LAN or WAN and LAN ports?

[ 6] local 172.16.2.110 port 42103 connected with 172.16.2.1 port 5001
[ ID] Interval Transfer Bandwidth
[ 6] 0.0-10.0 sec 425 MBytes 357 Mbits/sec

As said before you should do any kind of testing procedure through the pfSense and not by using it as one of the
partners like client or server! Its more important as you could imagine it!

Any ideas? Why this low on a 10 GbE interface?

With how many RAM is that unit sorted?
Did you do any tunings pointed to the 10 GBE NICs?
What kind of SFP transceivers you are using? DAC cables or SFP transceivers and fibre cables?

Create an account at ADI and register your unit
Download the ADI image from their
do a fresh and full install (no NanoBSD)
pending on the inserted RAM, high up the mbuf size to 1000000 (if 8 GB RAM are available)
activate TRIM support on that unit if a SSD or mSATA is used for the installation
Enable PowerD (high adaptive) if needed
enable TSO/LRO settings

please determinate which NICs are in usage at the 10 GbE Ports and what driver is loaded for that NICs and then find out
something about Tuning and Troubleshooting Network Cards this might be the best start to find out more about that issue.

Harvy66

@heper:

what are you testing ?

do not : client<–>router
do: client<->router-lan | router-wan <-> server

What helper said, don't test against PFSense, test through PFSense. PFSense is not optimized to be a high throughput server, just a high throughput firewall and router. They are actually quite different problems.

messerchmidt

freebsd has issues above 4gbs or so atm with the current kernel

whosmatt

@tzidore:

Hi

I have setup a full production environment on a netgate xg-2758 and are now testing my 10 GbE interfaces and are getting these disappointing results.

[ 6] local 172.16.2.110 port 42103 connected with 172.16.2.1 port 5001
[ ID] Interval Transfer Bandwidth
[ 6] 0.0-10.0 sec 425 MBytes 357 Mbits/sec

Any ideas? Why this low on a 10 GbE interface?

In my experience iperf on pfSense is a poor test of performance. I always get low numbers if pfSense is either the iperf server or client. If you have the hardware, test between 2 LAN segments (for routing performance) or LAN/WAN (for NAT performance) with machines that also have 10GbE interfaces.