Performance Tuning for 1.5gbit Internet and 10Gbit LAN
-
a default install on that hardware should be able to handle up to 2 gbit without any further configuration/tweaking/adjustments.
is this PPoE ? if yes: https://docs.netgate.com/pfsense/en/latest/hardware/tuning-and-troubleshooting-network-cards.html#pppoe-with-multi-queue-nics
if not you could check other possible solutions with regards to broadcom nics:
https://docs.netgate.com/pfsense/en/latest/hardware/tuning-and-troubleshooting-network-cards.html#broadcom-bge-4-cardseither way check the entire document to see if anything works
are there any interface errors? (status-->interfaces)
could you provide a 'top' at the time of a speedtest?
it would be ideal to perform a real throughput test with an iperf-server on the wan side , and an iperf-client on the lan side. -
@heper Great! I'll dig into these docs today and see what I find.
As for iperf3 on the WAN side, where would I find a server out on the internet to use as an iperf3 source/destination? Spin up an instance with a cloud provider? Or is there somewhere that offers that as a service?
-
there are services available online but that would defeat the purpose.
any "old" laptop or pc could be an iperf server -
Any old laptop doesn't have a 10gbit fiber NIC (keep in mind I have no copper ports on my pfSense box at all) and I'd have to remove the ISP's transceiver and change my WAN settings to get rid of the VLAN and PPPoE so I don't know how accurate of a test that would be.
I can set up one of my servers on the WAN side and one on the LAN side each with a 10gbit fiber module, clear out my WAN settings and see what iPerf does. It would at least confirm that the hardware can handle the throughput.
-
Alright, so after looking through the documentation my /boot/loader.conf.local looks like this:
hw.bxe.interrupt_mode="1" net.inet.tcp.tso="0" if_bxe_load="YES" net.isr.dispatch="deferred" kern.ipc.nmbclusters="131072" hw.bxe.tso_enable="0" hw.pci.enable_msix="0"
and I've managed to break 1000, just barely. Although the speedtest seems finicky. see attached history of runs all very close to each other.
I also took a TOP while speed-testing but I'm not sure how to interpret the results.
Edit: I checked Status - Interfaces, no errors
-
That looks like plenty in hand in performance terms. No cpu core is anywhere near 100%. The bxe processes are not at 100%. I would have to guess the limit is somewhere else.
You might try running tests from the pfSense box itself. It's not a good way to show absolute values but you have CPU cycles to spare and it will allow you to test the WAN and LAN separately.
So you could run iperf on pfSense and test to it from the client to be sure you're getting speeds on the LAN that are above 1Gbps. You won't see 10Gbps but if you see, say, 4Gbps you know that's not limiting.
You can run the CLI speedtest client on pfSense to test only the WAN. That might show almost anything! My experience is that it usually shows low speeds on high bandwidth WANs but if it shows closer to 1200Mbps that would prove the WAN is good.
Steve