Slow through put after bandwidth upgrade.

[jjudd]

Hey guys,
I've been using pfsense for a couple of years now without issue. I have Frontier Fios fiber to the house. Previously I had 150/150 mb/s and I typically got ~120/150 mb/s actual. I recently upgraded to 200/200 mb/s but now I'm only getting ~95/100 mb/s. I've been on the phone with frontier and they had me try my old verizon router that only has a 100 mb/s wan interface, it got 100 / 120 mb/s, all on wired connections.

My hardware is overpowered so that shouldn't be an issue:

CPU Type Intel(R) Core(TM) i3-6100 CPU @ 3.70GHz
4 CPUs: 1 package(s) x 2 core(s) x 2 hardware threads
AES-NI CPU Crypto: Yes (inactive)

4GB Ram

I have a Gigabyte itx MB with a NICs:
igb0: <Intel(R) PRO/1000 Network Connection, Version - 2.5.3-k>
em0: <Intel(R) PRO/1000 Network Connection 7.6.1-k>

I've tried all the steps from here to tune my cards https://docs.netgate.com/pfsense/en/latest/hardware/tuning-and-troubleshooting-network-cards.html

The fastest configuration I've found so far is this /boot/loader.conf.local configuration.

kern.cam.boot_delay=10000
kern.ipc.nmbclusters=1000000
net.inet.tcp.tso=0
hw.pci.enable_msix=0
hw.pci.enable_msi=0
hw.em.fc_setting=0
hw.igb.fc_setting=0

That's where I get the ~95/100 mb/s.

With all this, my cpu never gets over 5% utilization.
I've tried setting up traffic shaping but that didn't help so it's been removed again. And I don't have any additional packages installed.

I do have a new router coming in from Frontier in a few days so that I can at least confirm whether there is an ISP issue or not.

Anyone else know anything I can try?

Thanks!

[stephenw10]

Check Status > Interfaces for errors on either NIC.

You shouldn't have to set any of those tunables to pass 200Mbps on that hardware. In fact you might try just defaulting the config to be sure it's not something that has been introduced.

What connection type is that? DHCP? PPPoE?

Which NIC is WAN and what is it actually connected to?

How are you testing the speed?

Steve

[jjudd]

WAN Interface (wan, igb0)
Status
up
DHCP
up
...
MTU
1500
Media
1000baseT <full-duplex>
In/out packets
15845533/7079126 (19.68 GiB/882.37 MiB)
In/out packets (pass)
15845533/7079126 (19.68 GiB/882.37 MiB)
In/out packets (block)
18178/0 (796 KiB/0 B)
In/out errors
0/0
Collisions
0

LAN Interface (lan, em0)
Status
up
...
MTU
1500
Media
1000baseT <full-duplex>
In/out packets
6813575/15477365 (890.10 MiB/19.65 GiB)
In/out packets (pass)
6813575/15477365 (890.10 MiB/19.65 GiB)
In/out packets (block)
2427/0 (251 KiB/0 B)
In/out errors
0/0
Collisions
0

igb0 is the WAN interface using DHCP connected through cat 5 to the fiber termination point. It's not a ONT, its more like a desktop box for fiber termination to RJ45.
All connections are cat5 1Gb, and I'm testing with speedtest.net from a browser or the app installed on my machine.

Going to try doing the config reset now.

Jim

[jjudd]

Well that was unexpected, I reset the configuration and when I test with speedtest.net I'm now getting 31Mbps down and 48 Mbps up.

[stephenw10]

Hmm, you might try running a test from pfSense itself at the command line to see if it's the WAN or LAN at fault.
There is a speedtest package you can install and run like so:

[2.4.4-RELEASE][admin@7100.stevew.lan]/root: pkg install py27-speedtest-cli
Updating pfSense-core repository catalogue...
pfSense-core repository is up to date.
Updating pfSense repository catalogue...
pfSense repository is up to date.
All repositories are up to date.
The following 1 package(s) will be affected (of 0 checked):

New packages to be INSTALLED:
	py27-speedtest-cli: 1.0.7 [pfSense]

Number of packages to be installed: 1

30 KiB to be downloaded.

Proceed with this action? [y/N]: y
[1/1] Fetching py27-speedtest-cli-1.0.7.txz: 100%   30 KiB  30.6kB/s    00:01    
Checking integrity... done (0 conflicting)
[1/1] Installing py27-speedtest-cli-1.0.7...
Extracting py27-speedtest-cli-1.0.7: 100%
[2.4.4-RELEASE][admin@7100.stevew.lan]/root: rehash
[2.4.4-RELEASE][admin@7100.stevew.lan]/root: speedtest-cli
Retrieving speedtest.net configuration...
Testing from Plusnet (224.85.18.79)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by Freethought Internet (London) [8.99 km]: 11.883 ms
Testing download speed................................................................................
Download: 67.88 Mbit/s
Testing upload speed................................................................................................
Upload: 18.75 Mbit/s

Steve

[jjudd]

Ok got their router and tried it and it wouldn't connect, so they did something on their end to get it to connect. After that and confirming the speed looked good there, switched back to pfsense box and speeds are where they are supposed to be. So looks like it was actually an ISP issue...

Thanks for all the suggestions!