System Tunables on Netgate Hardware - Not Active?

RobbieTT

@stephenw10 said in System Tunables on Netgate Hardware - Not Active?:

Some of those are read-only in the sysctls and have to be set as loader values in /boot/loader.conf.local.

I don't have that file, only a /boot/loader.conf file, or is the implication there to create a new one rather than vi the existing loader.conf?

️

jimp

@RobbieTT said in System Tunables on Netgate Hardware - Not Active?:

@stephenw10 said in System Tunables on Netgate Hardware - Not Active?:

Some of those are read-only in the sysctls and have to be set as loader values in /boot/loader.conf.local.

I don't have that file, only a /boot/loader.conf file, or is the implication there to create a new one rather than vi the existing loader.conf?

️

https://docs.netgate.com/pfsense/en/latest/config/advanced-tunables.html#managing-loader-tunables

RobbieTT

@jimp said in System Tunables on Netgate Hardware - Not Active?:

The man page suggests that should be configurable but it may require matching some bits of the original mask exactly and not just taking a number from another system.

If it can't be changed at all, that's something that would need to be addressed upstream in FreeBSD.

It might be seen upstream as a contentious change as the 351 setting does trade some security for performance.

It stood-out to me because the default pfSense System Tunables includes that option and shows the 351 as the default setting. It probably requires a low-priority tweak to pfSense to remove the option from the menu, if it is non-functional.

️

RobbieTT

@jimp said in System Tunables on Netgate Hardware - Not Active?:

https://docs.netgate.com/pfsense/en/latest/config/advanced-tunables.html#managing-loader-tunables

Thanks - I've been working hard to avoid an RTFM moment but off I go and run straight into one!

️

RobbieTT

@stephenw10
I've looked as best as I can but I still don't understand why a process limit is set on a firewall/router:

hw.igc.rx_process_limit: 100

Clearly this would be set as-is for a general compute use of FreeBSD but it looks like an artificial limitation for a router application and is probably a bottleneck for pushing packets.

️

stephenw10

As far as I know we don't set that so it's the driver default value.

Try increasing it as see. We've not had to do that to get the expected throughput, but.....

I do know we have looked at increasing hw.igc.max_interrupt_rate. Setting that to 20000 provided gains in some scenarios.

Steve

RobbieTT

@stephenw10

Thanks Steve,

It makes sense as a default for FreeBSD but I will try disabling it on pfSense to see if it helps performance for PPPoE.

For non-PPPoE users it has been tested 'disabled' previously and shown to improve performance on Intel NICs. The first white paper I read on the subject happened to included a pfSense router as part of the bench-testing. I think Orange produced the original paper for a FreeBSD conference. I'll see if I can find a link.

I'll have a look at increasing the hw.igc.max_interrupt_rate as I am unfamiliar with it. I already have net.isr.dispatch set to deferred but any improvement I could detect was probably within the margin of error. It certainly didn't make anything worse though.

As a fellow PPPoE sufferer is there anything else I should consider beyond the pfSense guide?

Rob

️

stephenw10

I'm stuck at g.fast(!) speeds and I'm at the long end of the wire for that so ~140Mbps. Hence PPPoE isnt really a problem for me. Unfortunately.

Setting net.isr.dispatch to deferred usually makes a significant difference if you are hitting that as a limit. So you may not be.

RobbieTT

@stephenw10

The testing is somewhat synthetic for me on FTTH (930/115) as I am not running anything demanding other than FQ_Codel and pfBlocker, so I am currently within the 6100 CPU limits. But I will be switching to the 1.8 Gbit package when launched by Openreach later this year and this is probably beyond the 6100 with default settings.

Ideally I would run ntopng and Suricata too but that looks out of reach for high-bandwidth PPPoE users. In fact, high-bandwidth PPPoE is becoming a significant bottleneck for those hanging off an Openreach network.

I don't miss G.fast at all, so hope you get upgraded to FTTH in the near future!

Rob

️

stephenw10

@RobbieTT said in System Tunables on Netgate Hardware - Not Active?:

hope you get upgraded to FTTH in the near future!

Me too but it doesn't look like it will happen any time soon.