WAN Speed issues.
-
Greetings!
I have been running pfsense on custom hardware now for a couple of years. I started off using an old Core2Duo with 100Mbps connection. We switched to a 300/20 package (through WoW) and it worked ok at that time as well.
Now we are on a 600/40 package, which the Core2Duo could not keep up with as far as I could tell. Had an old i5-2500k and 4GB ramsitting around and decided to use it (would have liked to virtualized it at some point).
Issue is, I can not seem to figure out why the connection is so inconsistent. We will get 120Mbps most times, but can sometimes get around 300Mbps. This is tested with a couple of different systems that are hardwired to the gigabit switch and even with speedtest-cli on the pfsense box itself.
Powerd is on maximum and AES-Ni is active for OpenVPN. I am not running any other packages at this time. CPU does not seem to go above about 20% when doing speedtests. Using an Intel gb dual nic for both the WAN and LAN.
Set the tunable kern.ipc.nmbclusters="1000000" as suggested in https://doc.pfsense.org/index.php/Tuning_and_Troubleshooting_Network_Cards#mbuf_.2F_nmbclusters
Looking for advise on this issue.
-
@seanharsh Was the connection fairly consistent until you switched to the i5? Were you close to the 300/20 with the Core2Duo?
Also, is the 120Mbps during work hours (peak traffic) and 300 after hours? Or is the 300 at random times throughout the day?
We upgraded our router & switched to fiber last month, i3-7100U with 8GB ram. Packages include squid, snort, openvpn & our speed is very consistent. When we had cable our speed varied a lot, was a 300/20 package but often had 150 or less during the day & was a lot higher in the early mornings & evenings.
Since you don't have any other packages installed & your cpu isn't running high, I'm wondering if it's an issue with your internet provider.
-
Speeds vary and don't seem to be affected by time of day. WoW is only 1.5 years old in this small neighborhood and I think most folks are still on Spectrum.
For an example, I just now ran a test with traffic going through the pfsense box. Got 149 down. I then unplugged the pfsense box and connected my MacBook directly to the modem and got 601 down. I can run multiple tests in a row with the MacBook and get 525-608. Go back to the pfsense as the router, back down between 150-300. Everything seems to point to the pfsense box.
I was not always getting consistent 300 prior, but it was not bad enough to be worrisome. I never get higher then low 300 through the pfsense box.
-
@seanharsh Yes the MacBook test confirms it's an issue with the pfSense settings. This might help, see Visseroth's reply:
https://forum.netgate.com/topic/116185/pfsense-has-slowed-down-my-internet-connection-significantly
In the Network Interface section I only have the TCP segmentation offload & large receive offload checked, I can't remember if I changed them or not.
-
Yea, I should have mentioned that I tried multiple options under Networking Interface and nothing seems to change it. I believe that both of your options are default, at least the TCP seg. was from what I recall.
-
When you set up the i5, did you restore a backup from the old Core2Duo? I tried to use a backup when upgrading my router but ran into several issues... when I set it up from scratch everything worked fine. I used the same settings, don't know what was different but it works now!
-
@lburr Nope, all scratch. I have not do a full reset since setup some months ago, so I may try that as the next step. Just spent time re-checking all the advanced network interface settings with no change for any of them. Getting a consistent 240-260 from a hardwired workstation.
-
It sounds like you've tried running speed tests in multiple ways. From pfSense itself, from behind pfSense with a PC and without pfSense in the picture at all. One thing I would suggest is not using the speedtest cli on pfSense itself. That's based on my own experience with it as well as others on these forums with much more experience stating it should not be used. When I tried the speedtest cli, it consistently showed me much lower speeds than a PC behind pfSense going to the same speedtest.net server. That right there told me something wasn't quite right with it and I immediately uninstalled it. There is no sense in adding more confusion to troubleshooting with potentially bogus information.
I notice you have an i5-2500k. That's an LGA1155 CPU which supports PCIe 2.0. In theory if the NIC card is 2.0, even a 1x PCIe dual port Intel Gigabit card should be able to handle full duplex gigabit speed on both interfaces. Is your card 1x or 4x?
-
@raffi_ I stay away from speedtest-cli as well. The variance is crazy with it. Normal speedtest.net site from a workstation/laptop does seem to be more consistent. Seeing the same speed issues with our test sites as well.
As to the card, it's the Intel E1G42ETBLK which was listed as pcie 4x.
-
Great, yea speedtest.net should be pretty consistent.
That NIC looks like it's more than capable. Did you verify if it's a legitimate Intel card? Knockoffs are problem these days. If it has a Yotta mark, that could be checked. Otherwise, try getting in touch with Intel. I had to go through that process with my two intel NICs.
