Poor WAN Performance Between Reboots
-
@patient0 thanks for your reply :)
(some edits because I missed one of your questions)
I am letting speedtest.net auto-select my server each time when I test on my desktop PC. When I speedtest in pfsense one of the available hosts is VERY close to me and I left that server manually selected.
Under normal circumstances my traffic would exit my network through NordVPN which I have configured as a wireguard gateway in pfSense. Some of my devices are exempted from using Nord via firewall rules, I am only speed testing on devices that use my normal ISP gateway. When I'm getting full WAN speeds the devices that use Nord are only ~20% slower and I get aprox 1800 up and down. When I'm getting the slower WAN speeds Nord gives me 700-800 mbs.
I'm not sure if I'm clever enough to do a diagram but I can list out my network equipment and how they are all connected.
-
Bell Canada ISP 'gigahub' in bridge mode, it has multiple ports but only one of them is labeled as 10GB, I believe the others are all 1GB. The 10GB port is connected to the 2.5G WAN on my beelink. All of my cabling is either cat6 or SFP+. PfSense does not report the speed of the WAN interface, I think because it is PPPOE.
-
Beelink LAN is connected to a 2.5G switch.
-
There are 3x 2.5G client devices on that same 2.5G switch : 2x Win11 desktops that have intel NIC and 1x ubiquiti wireless AP that has a 2.5G port. I am using a POE injector to power the ubiquiti AP.
-
The 2.5G switch has a single 10GB SFP+ and that is connected to a HP Aruba managed switch. The HP Aruba has 1GB ethernet ports and 10GB SFP+
-
HP Aruba switch has two HP servers each connected via single 10GB SFP+, the other ports on this HP switch are 1GB, assorted slower client devices are all connected to the 1GB ports on the HP aruba (printer, TV tuner, audio receiver, smart TVs, etc)
-
The entire home is wired with cat6 and they all converge into a single patch panel and I patch either to the 1GB switch or the 2.5GB switch depending on the speed of the client device.
-
A small 1GB POE switch is patched from a 1GB port on the HP Aruba to another floor of the house. There is 1x IP camera and 1x ubiquiti wireless AP connected to that POE switch.
The only vlan that I am tagging is vlan 35 which was needed to get the PPPOE session with my ISP working. I do not have any LACP setup anywhere and I do not have any network loops.
-
-
@tjs4ever said in Poor WAN Performance Between Reboots:
I am letting speedtest.net auto-select my server each time when I test on my desktop PC. When I speedtest in pfsense one of the available hosts is VERY close to me and I left that server manually selected.
Do you see the same results if you test from one of the 10GBit connected servers?
I do use speedtest-go and always select the same server, otherwise you are introducing another variable (not sure you can select a server with the standard speedtest client). My ISP provides 2 and an ISP customer connected with 25Gbit provides one, which are excellent options for me. And I never test from pfSense since the test puts stress on pfSense which may lower the speedtest results.
You could:
- check with
top -HaSPhow hard your router is working while the speedtest is running. Any noticeable difference between a right-after-reboot and later speedtest? - can you check that the link speed between your modem and pfSense (
ifconfig ...from the console/SSH) is still at 2500? - is it possible that the ISP has not switched your profile and after reboot (with a new IP?) the ISP bandwidth limitation are not yet in place? But get applied after some traffic goes through?
- Have you done some test with the ISP all-in-one device in router mode?
- check with
-
@patient0 when I woke up this morning my WAN was completely down and would not come back online until I rebooted pfSense. The timing of this issue seems very coincidental as my new internet package was setup over the weekend. I've already spent many hours troubleshooting what I thought was only a speed issue with my ISP. The ISP only offers very basic support since I am using my own equipment - basically if they get the green light from their call centre to the modem that is the end of their troubleshooting. At this point I am at a total loss on what to do next.
-
@tjs4ever Have you checked the WAN interface for errors (Status->Interfaces)?
-
@WN1X no errors at this moment but my last reboot was around 40 minutes ago. I will check the status the next time it cuts out.
-
Check the actual link state of each NIC after some time in Status > Interfaces.
Check the CPU temperature and the current CPU clock speed. Those N100 platforms are known to have strangely behaving power management with anything but Windows. There are a few threads here details users efforts to make them behave rationally.
-
@stephenw10 I appreciate all of these helpful replies.
When I tax the beelink by doing a speedtest-cli within pfSense the CPU reports 2923mhz current and 806mhz max. CPU temp is currently 59C which is around 25 degrees above ambient. Memory usage is very low at 6%. The power mgmt in the beelink BIOS is set to whatever was decided at the factory, let me know if you think I should enable high performance mode or increase the TDP within the BIOS. I have reached out to beelink to ask about a BIOS update and am still waiting on a response.
I called my ISP again this morning, I suspected there was something 'off' about their modem's bridge mode. The tech unbridged the modem, factory reset it, he disabled the wifi, dhcp and all the other services. From his end he left the modem in an unactivated state : if you look at the modem now it is asking me to go online to activate but in this un-activated status my PPPOE in pfSense has been rock solid for the past 10 hours! The tech was calling this 'unofficial bridge mode'. I'm crossing my fingers that this was an issue with the modem all along but I'm going to wait a week or so before I consider this case-closed.
I don't believe I have any double-nat issues with the modem in this state but I honestly don't know how I can prove that.
As of this writing fast.com is saying that I am getting 2.7gbs down and 2.3gbs up, speedtest reports that I am getting 2316 and 2292 - I am perfectly happy with these speeds.
Thank you all for your quick and helpful replies, as one final question - would I be able to squeeze any additional performance out of this beelink by purchasing a pfplus license?
-
@tjs4ever said in Poor WAN Performance Between Reboots:
I don't believe I have any double-nat issues with the modem in this state but I honestly don't know how I can prove that.
Take a look at your assigned WAN address. Is it an RFC1918 address or is it public? RFC1918 would indicate you are double-nat'ed. A public address and you are good to go.
-
@WN1X The WAN interface IPv4 IP is the same IP address that I get when I visit https://whatismyipaddress.com/ using any of my (Nord exempt) client devices, so I take this to be correct then.
There's also the Gateway IPv4 address but it is a different value and I cannot ping it. I think this is yet another quirk with my ISP. A while back I had to change my gateway monitoring to use quad-8 for the monitoring IP - it just didn't ping one day.
-
@tjs4ever No double-nat there. You should be good now.
-
This post is deleted! -
You might be able to get more performance by tweaking the power/thermal management settings in the BIOS. Check the threads for N100 devices.
Since you're using PPPoE you should also get better performance by using the new if_pppoe driver in 2.8-beta.
-
So my first order of business this morning was to run some new speedtests and I am back to gigabit speeds on the WAN.
I checked ifconfig and the interface is connected as 1000baseT
It looks like I'm back to square 1. I'm going to swap out the cable that connects the modem to my pfSense to rule out a bad cable.
-
What NIC type are you using to connect the WAN?
You can probably set to negotiate the link at 2.5G only.
-
@stephenw10 the beelink is using dual Intel I225-V
Where can I hard-code the link speed for the WAN interface? I see that option under LAN but not under WAN settings.
-
@tjs4ever Interfaces->PORT1WAN Speed & Duplex.
-
@WN1X no such setting available under WAN, is it because it's PPPOE?
LAN has it.
The other interfaces : WAN and the two interfaces setup for my Nord have no such option.
-
Assign the parent NIC as a new interface and set it there. You can leave it as IP types none or set it to the modem/ONT subnet if you want to access that.
-
@tjs4ever Someone using PPPOE will need to assist you.
-
@stephenw10 thanks, I think I figured it out. I assigned a new interface, kept the default name of OPT3. I had to enable it though, when disabling I lost WAN immediately. I left the IP type as none and hard-coded the speed to 2500base-T. I guess I need to just wait another day or two and see what happens.