82576-2T-X1 Speed issues
-
Ok, so you see about the same result in speedtest on pfSense as you do from a LAN side client.
So the first thing I would do is swap the NIC assignments and see if the low speed moves to the LAN.
Steve
-
Nearly the same results just flipped. The only thing I haven’t done yet is swap out cables. I’m going to try that at lunch.
-
Still slow WAN side?
Ok, so something in the interface/port config there. Check the output of
ifconfig vvvm igb1. Assuming WAN is now igb1.Steve
-
ipconfig vvvm igb1 returned an error. (ifconfig: interface vvvm does not exist)
I assume you meant ipconfig -vvvm igb1?igb1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 description: WAN options=e120bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,WOL_MAGIC,VLAN_HWFILTER,RXCSUM_IPV6,TXCSUM_IPV6> capabilities=f53fbb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,NETMAP,RXCSUM_IPV6,TXCSUM_IPV6> ether 80:61:5f:08:2d:75 inet6 ffff %igb1 prefixlen 64 scopeid 0x2 inet 1.2.3.4 netmask 0xfffff000 broadcast 255.255.255.255 media: Ethernet autoselect (1000baseT <full-duplex>) status: active supported media: media autoselect media 1000baseT media 1000baseT mediaopt full-duplex media 100baseTX mediaopt full-duplex media 100baseTX media 10baseT/UTP mediaopt full-duplex media 10baseT/UTP nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> -
Indeed I omitted the
-.Nothing significant there.
The next thing I would check for is flow control:
[2.6.0-RELEASE][admin@t70.stevew.lan]/root: sysctl dev.igb.1.fc dev.igb.1.fc: 3It's probably enabled like it is there. I would try disabling it:
https://docs.netgate.com/pfsense/en/latest/hardware/tune.html#flow-controlSteve
-
I changed it to zero. And ran another speed test and it increased it about 50Mbits/s to 330Mbit/s
-
Hmm, well any effect there is interesting.
Try running
top -HaSPat the cli whilst you're testing. Make sure you're not hitting a limit on any CPU core. -
So I tested several times last night and only one core would spike to 15%. Generally it was below 10% usage.
-
And it was the expected interface loading?
Ideally I would hope to see at least two cores with some loading. If you tested with speedtest that uses multiple connections so it should use multiple queues. But if nothing was at 100% there should be headroom for more.
The next test I would try to do is iperf through the firewall. So that could be to a public iperf server of by using your own iperf server on the WAN side.
Steve
-
So I did one on the firewall and one from my Mac
Firewall
iperf3 -c iperf3.velocityonline.net -p 5210 Connecting to host iperf3.velocityonline.net, port 5210 [ 5] local 1.2.3.4 port 15533 connected to 216.162.135.203 port 5210 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 5.16 MBytes 43.3 Mbits/sec 250 188 KBytes [ 5] 1.00-2.00 sec 4.32 MBytes 36.3 Mbits/sec 1 130 KBytes [ 5] 2.00-3.00 sec 4.69 MBytes 39.3 Mbits/sec 0 175 KBytes [ 5] 3.00-4.00 sec 4.21 MBytes 35.3 Mbits/sec 1 118 KBytes [ 5] 4.00-5.00 sec 4.53 MBytes 38.0 Mbits/sec 0 165 KBytes [ 5] 5.00-6.00 sec 4.13 MBytes 34.6 Mbits/sec 1 112 KBytes [ 5] 6.00-7.00 sec 4.45 MBytes 37.3 Mbits/sec 0 160 KBytes [ 5] 7.00-8.00 sec 4.68 MBytes 39.3 Mbits/sec 0 197 KBytes [ 5] 8.00-9.00 sec 4.31 MBytes 36.2 Mbits/sec 1 133 KBytes [ 5] 9.00-10.00 sec 4.72 MBytes 39.6 Mbits/sec 0 177 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 45.2 MBytes 37.9 Mbits/sec 254 sender [ 5] 0.00-10.11 sec 44.4 MBytes 36.9 Mbits/sec receiver iperf Done.Mac
iperf3 -c iperf3.velocityonline.net -p 5201 Connecting to host iperf3.velocityonline.net, port 5201 [ 7] local 1.2.3.4 port 59478 connected to 216.162.135.203 port 5201 [ ID] Interval Transfer Bitrate [ 7] 0.00-1.00 sec 4.27 MBytes 35.8 Mbits/sec [ 7] 1.00-2.00 sec 4.48 MBytes 37.6 Mbits/sec [ 7] 2.00-3.00 sec 4.71 MBytes 39.5 Mbits/sec [ 7] 3.00-4.00 sec 4.67 MBytes 39.1 Mbits/sec [ 7] 4.00-5.00 sec 4.61 MBytes 38.7 Mbits/sec [ 7] 5.00-6.00 sec 4.57 MBytes 38.4 Mbits/sec [ 7] 6.00-7.00 sec 4.45 MBytes 37.3 Mbits/sec [ 7] 7.00-8.00 sec 4.74 MBytes 39.7 Mbits/sec [ 7] 8.00-9.00 sec 4.43 MBytes 37.1 Mbits/sec [ 7] 9.00-10.00 sec 4.74 MBytes 39.8 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate [ 7] 0.00-10.00 sec 45.7 MBytes 38.3 Mbits/sec sender [ 7] 0.00-10.02 sec 45.4 MBytes 38.0 Mbits/sec receiver -
Ah, so not a great test server I guess, both well below the limit you were seeing.
Are you able to put something on the WAN side to test against? Another pfSense instance is what I would probably use there.
-
I want to be clear about my tests. The Firewall one was from the pfsense box to a public iperf server. The Mac went though my pfsense box to the same server. I want to make sure I did it the way you wanted me to do it.
-
Yes that should be fine but neither reached the speeds you were seeing to speedtest so it doesn't really give us any more information.
If you can connect the pfSense WAN to something running an iperf server directly and test to that it removes any sort of restriction reaching it so any limitation left much be in one of the pfSense interfaces.
As I said I do that sort of test all the time using another pfSense instance on the WAN. That way it hands out DHCP so the test device just works.Steve
