PC Engines APU2C4 Download bandwidth half of upload

themaninspain

I've just installed an APU2C4 with pfSense 2.4.3 (replacing an Alix) , my WAN connection is now 600/600 fiber using PPoE and a VLAN.

With my telco supplied router I get 610/610 on Speedtest with the APU2C4 I get a max of 300/610.

The only services I have configured are some NAT rules for a server

After googling and reading multiple articles, I changed the following options:

unchecked DIsable Hardware TCP segmentation offload
unchecked Disable Hardware large receive offload

PowerD is set to HiAdaptive

There was a slight improvement in results to 310/610

Why the difference between download/upload and is there anything I can do to fix it?

Update:
I reread the hardware tuning guide - https://www.netgate.com/docs/pfsense/hardware/tuning-and-troubleshooting-network-cards.html

Added the following to /boot/loader.conf.local

kern.ipc.nmbclusters="1000000"
net.inet.tcp.tso=0

Now I get 345/610

Also tried changing:

w.igb.num_queues=1

But there was no measurable improvement.

VAMike

Someone had a suggestion in https://forum.netgate.com/topic/96129/gigabit-pppoe-and-intel-drivers/12 but nobody seems to have tried it yet.

stephenw10

Yes, if you're hitting that it would be good to try that sysctl.

Unfortunately I can only dream of a WAN connection that fast.

Steve

superweasel

Sounds average for a high speed PPPoE connection and the current release of pfSense.

I have been waiting 4 years for this to get fixed https://redmine.pfsense.org/issues/4821 with no resolution in sight.

themaninspain

@vamike

Just tried it, download has improved to 450 mbits/s. I'll do some more testing over the next 24 hours. Speed tests are notoriously fickle and I'd like to get some average numbers

VAMike

@themaninspain for grins, you might try to compare with a linux based distro, like openwrt

stephenw10

Mmm, that's quite a significant improvement. More than I expected TBH.

Steve

VAMike

@stephenw10 It's about what I expected; the freebsd defaults are trying to do too much in the interrupt context for these low-power devices. If he runs both tests with firewalling disabled, I'd expect a dramatic improvement in the non-deferred case and hopefully much less so in the deferred case (but that's hard to test in a NAT environment). Also, @themaninspain what are your net.isr.maxthreads and net.isr.numthreads set to?

themaninspain

@vamike

Both are set to 4

VAMike

@themaninspain cool, that should be optimal

themaninspain

Bandwidth measurements are consistent. Max download is averaging out to 450.
I know I won't get the full 600/600 but I don't really need it.

Thanks for all of the help.