pfSense throughput performance disparity
-
I have a Netgate 2100 and have just upgraded my broadband to fibre to the premises, and am connecting with if_pppoe turned on in pfSense 25.07 RC. The fibre delivers 910Mbps up and down when tested with the ISP-provided router (a Fritzbox). With my 2100 it can achieve the 910Mbps up, but maxes out at about 600Mbps down.
I do not have Suricata or Snort or other heavy services running, but do have NAT and the firewall.
I appreciate that inbound traffic has a little more checking to be applied to it, and if this was the behaviour I was seeing from a mix of many different connections generating the bandwidth, I'd say "fair enough". But this is the performance using a speedtest app. Presumably each successive packet has the same characteristics as the previous one and requires the same treatment. So if pfSense cached recent packet rule outcomes, surely it should be nearly as fast to process as the outbound case. However it ends up CPU-bound at a much lower throughput.
Without deferred ISR handling it maxes one core at 100%. With deferred it loads both cores, but still maxes out at the lower bandwidth.
Given that the hardware can handle 910Mbps outbound, is there anything I can do to get closer to this inbound?
-
@OracPrime The 910 is surprising tbh; the 2100 is generally considered to max out around 600-700 or so.
-
@SteveITS well now we know it can do more, we just have to make it do more both ways :)
-
@SteveITS just a shame the price step to the 4200 is so much (£550 in the UK) given that non-Netgate hardware is considerably cheaper. I just do like the familiarity of running it on Netgate though.
-
@OracPrime Except Netgate doesn't think it can do that. Measuring error?
-
-
You can get asymmetry like that due to NAT for reasons I've never dug deep enough to discover. But, yes, I would not expect to see 900Mbps through a 2100 unfortunately.
-
@stephenw10 (thumbs up) - apparently I need more reputation to actually use the emoji (?)
-
Hmm, you may not be able to upvote but I didn't think there was a restriction on emojis!
-
@stephenw10 you are of course correct. I meant click on the icon which would generate an emoji-like response.
-
@OracPrime said in pfSense throughput performance disparity:
and am connecting with if_pppoe turned on in pfSense 25.07 RC
Your 2100 might be doing even more if it had not to do the extensive pppoe handling.
True, with the current pfSense version a new pppo driver was introduced that was completely rewritten (== faster) as for some reason pppoe doesn't want to roll over and die.
PPPOE is ok as the big CPU overhead was fine back in the old days, where a typical DSL could be anything from 1 to 16 Mbit /sec.
Doing close to a Gbit/sec using pppoe is ... not sure ... madness ? but for some reason some ISPs still use pppoe these days.What if the Fritzbox did the pppoe for you, if it is capable of doing so ?
In that case you set the 2100 WAN interface to the 'simple' default DHCP.
This means you have to NAT twice - if needed, as your pfSEnse WAN would be using a RFC1918 (like 192.168.20.2) -
@Gertjan Did wonder whether that might help and had a quick try a couple of days ago, but for some reason nothing worked. I'd also have to work out how to make the FritzBox direct all the incoming traffic to pfSense for HAProxy routing. I'll dig further.
-
If you use "pppoe" as a WAN connection method, the upstream device, the Fritsbox, is just a modem type device ...
One of the advantages (probably the only one) of using pppoe on the pfSense WAN : the pfSense WAN interface uses the real outside world IPv4.
No need to 'dmz' or 'NAT' or 'redirect' anything to pfSense. Everything will reach the pfSense WAN interface.
