PC Engines APU4c4 - Slow LAN

Stella_161

Hi ,

Im having issues with slow LAN interface compared to the WAN and need some help finding the cause.

My Setup:

2.4.5-RELEASE-p1 (amd64) - no vm
APU.4C4 system board 4GB /w SSD M-Sata 30GB MLC Phison

Ive recently switch over to a 100MBit PPPoe HFC connection from a 50Mbit connection. When running a speed test from the apu4c4 or a laptop directly plugged in to the HFC box, i get 98Mbit~ but if i run it from my the same laptop that's behind the apu4c4 its only at 71Mbit no matter what i do.

Ive tried:

Swapping Ethernet cables
Plugging directly in to LAN Port 2 and 3
https://teklager.se/en/knowledge-base/apu2-1-gigabit-throughput-pfsense/
disabling ICMP Redirects to enable the faster tryforward routing path (under System / Advanced / System Tunables set net.inet.ip.redirect & net.inet6.ip6.redirect to 0)
Upgraded to 2.4.5-RELEASE-p1 (amd64) from 2.4.4
Upgraded the BIOS to: Version: v4.12.0.2
Factory reset with no packages enabled

After the BIOS upgrade i did a factory reset, no packages have only setup PPPoe and default LAN with ad blocker enabled on PC. Still only 71Mbit~.

CPU usage is minimal when running from PC: Speedtest.net and Fast.com with the same result

PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU COMMAND
11 root 155 ki31 0K 64K RUN 3 42:59 100.00% [idle{idle: cpu3}]
11 root 155 ki31 0K 64K CPU1 1 42:57 100.00% [idle{idle: cpu1}]
11 root 155 ki31 0K 64K CPU2 2 42:56 98.49% [idle{idle: cpu2}]
11 root 155 ki31 0K 64K CPU0 0 42:42 88.96% [idle{idle: cpu0}]
12 root -92 - 0K 496K CPU0 0 0:31 14.70% [intr{irq256: igb0:que 0}]
12 root -92 - 0K 496K WAIT 2 0:08 5.66% [intr{irq259: igb1:que 0}]
12 root -92 - 0K 496K WAIT 3 0:07 1.66% [intr{irq260: igb1:que 1}]

**

CPU when running Speed Test CLI on pfsense:

PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU COMMAND
11 root 155 ki31 0K 64K CPU1 1 47:13 86.77% [idle{idle: cpu1}]
11 root 155 ki31 0K 64K RUN 3 47:12 82.57% [idle{idle: cpu3}]
11 root 155 ki31 0K 64K CPU2 2 47:09 79.88% [idle{idle: cpu2}]
11 root 155 ki31 0K 64K RUN 0 46:46 71.29% [idle{idle: cpu0}]
12 root -92 - 0K 496K WAIT 0 0:50 33.25% [intr{irq256: igb0:que 0}]
52611 root 74 0 135M 71416K CPU3 3 0:02 7.96% /usr/local/bin/python3.7 /usr/local/bin/speedtest{python3.7}

Any help or advice on this will be appreciated

Let me know if you would like anymore information.

stephenw10

What CPU usage do you see when testing from a LAN side client through pfSense?

Hard to imagine that's limiting it to <100Mbps though. Even with PPPoE.

No traffic shaping set here at all?

Try an iperf test from the client to pfSense LAN side. The absolute value will not be meaningful but it should be far higher that 100Mbps.

Steve

Stella_161

@stephenw10

Correct, no traffic shaping or anything. All tests were form a laptop plugged directly into the LAN of the box:

here the iperf test results:

CPU Usage during speed test:

Speed Test Results:

stephenw10

Hmm, not CPU limited then.

You should try setting net.isr.dispatch to deferred if you haven't already since you are using PPPoE:
https://docs.netgate.com/pfsense/en/latest/hardware/tuning-and-troubleshooting-network-cards.html#pppoe-with-multi-queue-nics

Steve

Stella_161

@stephenw10

Thanks Stephenw, Great News! That change got it up to 84Mbit.

Any other tips to get the last little Mbit out of the box? i know its not much of a speed increase but id like to use all that i'm paying for.

Thanks

stephenw10

You could try the other values shown at that link.

It's odd that you need them though. Even given you're running with PPPoE I would have expected that CPU to pass 100Mbps easily.
Check for errors in Status > Interfaces.

You are seeing the full line rate if you test from pfSense using speedtest-cli so it looks to be a LAN side issue. But you're also seeing >300Mbps to the LAN directly which implies some routing problem maybe.

It looks like you've tried a while bunch of different things here. It's possible you have something left over or a conflicting setting. I would probably reinstall clean and see what you get with a default config there. 100Mbps just shouldn't be a problem. A lot of those tweaks were aimed at getting 1Gbps through that box.

Steve