gigabit wan download speed much lower than upload
-
Hi all, I'm having no luck figuring out why my Verizon fios symmetrical Gigabit WAN is getting such low download speeds on my pfsense box.
Speedtest.net is showing consistently around 5ms ping, 50-70Mbps download speeds vs. 800-900Mbps upload speeds.I have a new pfsense setup 2.4.4-release-p3
Running on a HP T620 Plus 16GB M.2 SSD, 4GB of RAM, with a HP NC364T PCI Express Quad Port Gigabit Server Adapter. (Intel chipset, em0 wan, em1 lan interfaces)I'm only using two of the four ports on the nic for WAN/LAN.
Brand new CAT6 cables on port WAN/LAN.
A 24 port Netgear JGS524 switch for LAN devices.I have disabled:
Hardware Checksum Offloading
Hardware TCP Segmentation Offloading
Hardware Large Receive OffloadingAnd after some googling decided to add these change to /boot/loader.conf.local
kern.ipc.nmbclusters="1000000"
hw.em.num_queues=0
hw.em.fc_setting=0
hw.em.rxd=4096
hw.em.txd=4096I have 3 openvpn connections open to iVPN in a gateway group that by default are allowed to NAT outbound to the internet. And I've added an alias "nonvpnd" for a group of LAN ip's that passes these matched ip's to the WAN gateway and does not pass over the ivpn gateway group. In other words all internet bound traffic from my LAN 192.168.137.X goes over the iVPN gateway openvpn connections, unless the source IP matches one in the "nonvpnd" alias, then it gets NAT'd through the WAN/fios connection IP to the internet.
I also have installed pfblocker-ng dev version. And have tried the speed tests with it on and disabled in every place. And no change in download speed tests.
CPU usage during a speed test never goes past 20%, RAM usage at 13%
https://www.verizon.com/speedtest/ shows 15Mpbs download 935Mpbs upload to device. 935 Download to router and fails on upload. Which I've read is because they try to test upload from the Wi-Fi/Router box I took out of the mix and going straight from the ONT with CAT6 to the HP T620 pfsense box.I"m baffled at this point what is causing such a huge drop in the download speeds.
That speed disparity is HUGE! 50Mbps vs. 900Mbps leads me to believe that nothing is wrong with my fios service. And that something is wrong in pfsense's configuration that is crippling the download speeds from the internet.Any help anyone could give would be so greatly appreciated! I"m throwing my money away on this speed that I'm not getting.
-
What CPU is in that?
Try running
top -aSHat the console while you're testing to see how the load is spread across the CPU cores.That's a DHCP WAN connection I assume?
Have you tested with a client connected directly, without pfSense, and can see the full download speed?
Steve
-
Agree with @stephenw10 - if you connect directly to the ONT, do you see full up and down speeds? That would be the first thing to check to make sure it isn't a provisioning or other issue on Verizon's end.
-
@stephenw10 said in gigabit wan download speed much lower than upload:
What CPU is in that?
It's most likely an AMD GX-217GA or GX-415GA or GX-420CA processor in that box:
https://www8.hp.com/us/en/thin-clients/t620-zero.html
https://www.servethehome.com/hp-t620-plus-thin-client-and-firewall-vpn-appliance/Jeff
-
The HP T620 Plus is a thin client PC I've been told was great for a small, low cost, low power, silent pfsense box.
CPU Type AMD GX-420CA SOC with Radeon(tm) HD Graphics
4 CPUs: 1 package(s) x 4 core(s)
AES-NI CPU Crypto: Yes (active)2Ghz
I tried a speed test with 2 different laptops gigabit Ethernet directly to the ONT and unexpectedly still got the same speeds, worse on subsequent tests.
Now I'm questioning my fios service quality.
Before I used the T620 I'd been using a virtualized pfsense install on esxi but wasn't satisfied with speeds.
And I'd been using the WAN nic with MAC clone of the Verizon provided Wi-Fi box I don't use anymore.Do you think that MAC could impact speeds at all? With provisioning?
Thanks for the input everyone.
-
Some providers will massively limit the speeds you can get if you are not connected "properly". Such as having to use the vlan or priority tags on outgoing packets. Though I didn't think FiOS did that.
I would have them reset the ONT or at least check the connection, there's clearly something wrong there if a laptop sees the same speeds.
Steve
-
I assume you also tried a different cable just in case? Since you can't get full speed connected directly to the ONT, I think you should call Verizon as something doesn't seem be configured properly on their end, or maybe there is an issue with your fiber signal level/quality, etc.
I have been using Verizon FIOS gigabit service together w/pfSense for over two years and have always gotten full speeds - there is really nothing special you have to do to set it up (e.g. no VLAN's, priority tags, gateway bridging, MAC cloning, etc.) besides maybe a bit of network hardware tuning for high speeds.
Hope this helps.
-
Worth checking if your WAN connection linked at 1Gb full duplex. Check the Status > Interfaces page for errors.
Steve
-
@stephenw10 said in gigabit wan download speed much lower than upload:
Worth checking if your WAN connection linked at 1Gb full duplex.
Given he's seeing 800-800 Mb in one direction, the interface is definitely running at 1 Gb. Also, Gb is always full duplex.
-
Yup it should always be full duplex and it should always be autonegotiate. But we have seen equipment that doesn't stick to that. Though I agree it would have to be something more like a hardware issue to get such asymmetric speeds.
Steve
-
Well I've taken another look at speeds directly to the ONT
Started with a power reset of the ONT.
Then with a new cable and laptop, verified I got 1 Gbps Full Duplex link up.
Ran speed test through speedtest.net
Got back PING ms
7
DOWNLOAD Mbps
141.84
UPLOAD Mbps
242.88Then on verizon.com/speedtest
DOWNLOAD
405
MbpsUPLOAD
476
Mbpsboth of these are terrible, and indicate something still ins't right hardware wise or provisioning?
I guess I need to call Verizon now.
-
Um, yeah that looks.... sub-optimal!
Seems like an upstream issue.
Steve
-
@lasermole said in gigabit wan download speed much lower than upload:
Then with a new cable and laptop, verified I got 1 Gbps Full Duplex link up.
Was that with the computer by itself? Or through pfSense? Try the computer connected directly, to see if the problem is with pfSense or the ISP. It's best to keep things simple while testing.
-
Yes the tests were with the laptop directly connected to the ONT with new cable and I got those results...
-
@lasermole said in gigabit wan download speed much lower than upload:
Yes the tests were with the laptop directly connected to the ONT with new cable and I got those results...
Then it's not a pfSense issue. Call your ISP.
-
Yup, good times with ol Verizon....
-
Just curious what you guys think, but do you think Verizon has reason to dismiss a speed test done directly to their equipment? Dismissing your results, pointing blame at your device. Hence why I'm sure verizon's own speed test is the only one they officially recognize, as it separates "Device" and "Router" in its test. FYI, I don't expect to get 1000/1000, even 800/800 would be fine by me to device.
-
@lasermole - do you still have Verizon router/gateway? If yes, it might be worth plugging it in temporarily and run the Verizon speed test to the router. Given what you have described so far, I doubt that even that test will actually see full speed. If it doesn't, I expect them to help your further since the issue is then more clearly somewhere with their equipment.
-
Yes I do still have the router/gateway device. I could hook it up again and try through that to see what speeds I get. I'll let you know what comes of that test.
-
Tried the speed tests again with the Verizon router in place between the laptop and the ONT.
Laptop > Verizon router > ONT
Speedtest.net
PING ms
7
DOWNLOAD Mbps
256.93
UPLOAD Mbps
207.28Verizon speed test
"Device"
DOWNLOAD
568
MbpsUPLOAD
382
Mbps"Router"
DOWNLOAD
949
MbpsUPLOAD
912
MbpsI really feel that "router" test is bull. I mean I don't pay Verizon each month to read how fast their devices are, to their network, on a screen...
That is not end user usable bandwidth. And I'm sure they will use that result to silence my complaints and shift the blame to me any way they see fit.
