Slow throughput on SG-4860 (~600mbs on 1gbs line)
-
Hey folks - I recently upgraded to a SG-4860 to take advantage of a speed bump in my FiOS connection to their 1 gig service.
After the upgrade I am maxing out at about 500-700 mbs …although I rarely get about 600.
I was so sure it was the line or Verizon gateway that I scheduled a service visit before checking anything else. When we connected directly to the Verizon gateway (which is in bridge mode, BTW) it pulled a sold 950mbs. I say we, because I was able to replicate with a laptop directly connected.
I've run speed tests using curl and iperf from the PF box with nothing else connected - so I'm reasonably sure it's not other devices sipping data while I'm testing.
I have 3 site-to-site VPNs running too (the test doesn't go through any of them, FWIW). But I assume they create some processor overhead, even if there isn't much/any traffic flowing. My LAN has 4 vLANS which may also create some overhead...?
On a related note - I have hardware cypto enabled on both ends, and the tunnels max out well below the line speed of either endpoint.
I've also read it might be a single vs multi thread issue. I've tried running speed tests on multiple machines at the same time and the result is a sum total of about 600 mbps.
It could be my cabling - I've swapped the WAN cable already. And I don't think it's the switches on the LAN side since I've connected directly to the SG-4860's LAN port and run the tests from the PF box itself.
Anyone have any more tips for troubleshooting?
-
Are these results from speedtest trough the tunnel? Sorry, it's not that clear to me.
-
Are these results from speedtest trough the tunnel? Sorry, it's not that clear to me.
Totally on me! I wasn't clear!
~600mbs is from the LAN side of the firewall and LAN clients straight to 'the internet'…
I did the ole curl -o spedtest , Fast.com and several public iperf servers(through OVPN tunnel I max out at 50 mbs...again, another thread I need to start once I resolve this :) )
-
Check MTU's on your ports, modem and switch. 4860 can do 1Gbps without any problems so it's MTU or cables perhaps.
-
Check MTU's on your ports, modem and switch. 4860 can do 1Gbps without any problems so it's MTU or cables perhaps.
played around with different MTUs tonight: 1450, 1400, 1350, 1300 and for good measure 9000 … no appreciable change in speed.
I also swapped cables with some from work which were in a setup pulling 1gpbs symmetrically. So it's not the cables.
I watched the processors during the tests and at least one pegs each time. I'm thinking perhaps the 4860 can do 1gpbs total, but not in a single thread?
-
Check MTU's on your ports, modem and switch. 4860 can do 1Gbps without any problems so it's MTU or cables perhaps.
played around with different MTUs tonight: 1450, 1400, 1350, 1300 and for good measure 9000 … no appreciable change in speed.
I also swapped cables with some from work which were in a setup pulling 1gpbs symmetrically. So it's not the cables.
I watched the processors during the tests and at least one pegs each time. I'm thinking perhaps the 4860 can do 1gpbs total, but not in a single thread?
When was the unit purchased? If it's recent, please submit a ticket to https://customercare.netgate.com/ and reference this thread. We'll figure out what's going on a lot faster.
Thanks!
-
If you upgraded your config from an older machine to a newer one you might have to enable powerd to get your speeds.
System -> Advanced -> Miscellaneous, enable PowerD, HiAdaptive
-
Are you having the SG-4860 terminate a PPPoE connection? If so, that's your problem. I get about the same performance on mine because of a FreeBSD bug. There's more info at https://redmine.pfsense.org/issues/4821.
-
If you upgraded your config from an older machine to a newer one you might have to enable powerd to get your speeds.
System -> Advanced -> Miscellaneous, enable PowerD, HiAdaptive
Thanks - I've tried both ways and unfortunately, PowerD seems to –if anything --slow things down further.
-
Are you using Snort? if so turn it off and try again as Snort is single threaded. Your performance with one snort instance is actually above average considering you have an Atom cpu in there.
-
Are you using Snort? if so turn it off and try again as Snort is single threaded. Your performance with one snort instance is actually above average considering you have an Atom cpu in there.
no snort here :)
-
What happens when you plug the SG-4860 WAN interface directly into the ONT - do you still experience throughput issues? From what I can gather form your first post, the pfSense box is actually downstream of the Verizon gateway - is there any particular reason why you decided to set it up that way?
Hope this helps.
-
What happens when you plug the SG-4860 WAN interface directly into the ONT - do you still experience throughput issues? From what I can gather form your first post, the pfSense box is actually downstream of the Verizon gateway - is there any particular reason why you decided to set it up that way?
Hope this helps.
You might be on to something. Admittedly it’s years since I could claim even a working understanding of how VZ was implementing FiOS. Back then I’d get a huge ONT on the side of my house.
In this install they ran a really super slim single strand fiber from the pole straight through my exterior wall and into a Verizon router. That router has an AD/DC supply zip tied to it. Otherwise there’s no ONT that I can see.
Should I get a fiber to ether exchanger for my PF box? I have a jar full of gbics I haven’t used in years ;)
-
I think there might be some confusion as to what we are calling the Verizon Gateway and ONT. The ONT includes the device (looks like a modem) that takes fiber as input and provides ethernet/coax outputs. The Verizon Gateway (at least to me) is the router/firewall Verizon provides - with gigabit I assume they would have given you the black G1100 (Fios Quantum Gateway) unless you decided to keep the (red colored) Actiontec. Do you have your pfSense box currently plugged into the ONT directly or into the Verizon Gateway (router/firewall)?
Hope this helps.
-
Are these results from speedtest trough the tunnel? Sorry, it's not that clear to me.
Totally on me! I wasn't clear!
~600mbs is from the LAN side of the firewall and LAN clients straight to 'the internet'…
I did the ole curl -o spedtest , Fast.com and several public iperf servers(through OVPN tunnel I max out at 50 mbs...again, another thread I need to start once I resolve this :) )
Keep in mind FIOS is rated UP TO @940 megabits even though they advertise gigabit. It may not be anything with your gear you might jsut be hitting a bandwidth limitation in your aarea. I ahve a freind that subs to VZ Gb and he rarely sees above 800 himself.
-
940 Mbps happens to be about what most people will see with a single 1 Gbps interface, after taking into account protocol overhead, so they're still pushing 1 Gbps through the line.
Of course, the caveat is that they're providing "Up to 940 Mbps"… if you're in-depth on the technology behind FiOS, it is actually shared connectivity between you and possibly up to 31 other neighbors, so if a couple of your neighbors also have gigabit service and have major downloads going on, that could actually prevent you from getting full speed.
-
The poster did mention that he was able to achieve a higher speed when connected directly to the Verizon gateway (back in the first post), so I'm not convinced it's an issue with the connection.
I'm still curious though as to how exactly this is setup: Is the pfSense box hooked into the gateway and gateway into the ONT, or is the pfSense box hooked directly into the ONT? I would recommend connecting the pfSense box into the ONT and then using the Verizon gateway downstream of the pfSense box (if it's required to be in use, e.g. to provide internet connectivity to STB's).
Hope this helps.
-
Having the exact same issue. When my computer is plugged straight into the ONT, I get 940 Mbps each way every time. When downstream of the SG-4860 running a 2.4-RC snapshot (Oct 4), download tops out at 400-500 Mbps. Upload can usually hit 700 Mbps, strangely.
This is a fresh install, so no old config slowing it down.
Let me know how I can help triage this. Thanks.
-
Fixed it- was necessary to enable PowerD, and set to Hiadaptive or Maximum (slightly faster). Hey, maybe this thing should be enabled by default, huh?
-
I'm still curious though as to how exactly this is setup: Is the pfSense box hooked into the gateway and gateway into the ONT, or is the pfSense box hooked directly into the ONT? I would recommend connecting the pfSense box into the ONT and then using the Verizon gateway downstream of the pfSense box (if it's required to be in use, e.g. to provide internet connectivity to STB's).
Hope this helps.
VZ box (where fiber terminates) –cat6-->PF Wan port
I know it sounds confusing because I've seen different FiOS set ups, but in this install, fiber runs from the pole, through a hole in my wall all the way to the Vz box (let's just call it the VZ Box). The VZ box has a locked port where the fiber terminates. And it has a switch built in. One switch port is connected to my PF box.
Wifi and routing are disabled on the VZ box (it's in bridge mode).
We don't have phone or TV through Verizon - internet only. No STBs in the mix.
