New if_pppoe backend

TampertK

I'm trying the new if_pppoe implementation on a 1Gbps symmetrical fiber line. My provider uses PPPoE over a VLAN, my device has an Intel J5005 quadcore with an Intel X520 NIC. TSO and LRO are disabled.

I'm noticing a big imbalance between download and upload. Down maxes out around 490Mbps, up goes to full 920Mbps. Looking at System Activity, this imbalance is also seen in distribution across cores.

Download:

    0 root        -60    -     0B  1600K CPU0     0   3:09  84.28% [kernel{if_io_tqg_0}]
    0 root        -60    -     0B  1600K -        3   0:54  14.26% [kernel{if_io_tqg_3}]
    0 root        -60    -     0B  1600K -        2   0:40  10.25% [kernel{if_io_tqg_2}]
    0 root        -60    -     0B  1600K -        1   1:04   9.28% [kernel{if_io_tqg_1}]

Upload:

    0 root        -60    -     0B  1600K CPU1     1   0:30  43.16% [kernel{if_io_tqg_1}]
    0 root        -60    -     0B  1600K -        3   0:34  31.59% [kernel{if_io_tqg_3}]
    0 root        -60    -     0B  1600K -        0   1:41  29.69% [kernel{if_io_tqg_0}]
    0 root        -60    -     0B  1600K -        2   0:21  27.78% [kernel{if_io_tqg_2}]

On 2.7.2 I used the net.isr.dispatch=deferred tunable, and easily reached full download speeds (I've removed this tunable on 2.8). Can it be that the VLAN is causing issues? Or perhaps we need to wait until the RELEASE version?

netblues

@TampertK Have you tried the tunable on 2.8? Worst case, it is ignored.
And no, it can't be the vlan.

There is nothing to wait, the new pppoe code is the same in pfsense plus 24.03 beta and on 2.8 ce

TampertK

@netblues said in New if_pppoe backend:

@TampertK Have you tried the tunable on 2.8? Worst case, it is ignored.

I have not, as the documentation recommends against doing this.

@netblues said in New if_pppoe backend:

There is nothing to wait, the new pppoe code is the same in pfsense plus 24.03 beta and on 2.8 ce

This contradicts a statement by the owner of Netgate:

It will actually get faster in 2.8-RELEASE because there was a rework that occurred too late to make the beta cut-off.

netblues

@TampertK I was refering to plus 24.03 beta, not ce beta.

kprovost

@TampertK That enhancement is in neither the 25.03 plus nor the CE 2.8.0 beta.
It's expected to be in both releases, but it's not there yet.

TampertK

@kprovost This confusion is distracting from the issue. The new if_pppoe implementation is in both the 2.8 CE and 25.03 Plus betas. According to gonzopancho, the implementation will "get faster" in the release versions.

However, the current if_pppoe implementation in 2.8 CE beta is causing issues for me, with a big imbalance between download and upload, which is also visible in the distribution across the CPU cores.

kprovost

@TampertK Yes, that's what's expected to be addressed by the updates that are not yet publicly available.

TampertK

@kprovost Do you have any evidence or reference that the current implementation of if_pppoe in 2.8 CE beta is causing this imbalance between download and upload?

marcosm

@TampertK Yes, internal testing shows improved performance.

TampertK

@marcosm I'm now on version 2.8.0.b.20250410.0059. The overall download speed has improved to around 800Mbps, but the 'imbalance' across cores is still there. See:

    0 root        -60    -     0B  1616K -        0   1:39  79.69% [kernel{if_io_tqg_0}]
   11 root        187 ki31     0B    64K RUN      1   8:34  63.09% [idle{idle: cpu1}]
   11 root        187 ki31     0B    64K RUN      3   8:33  60.89% [idle{idle: cpu3}]
   11 root        187 ki31     0B    64K RUN      2   8:19  60.50% [idle{idle: cpu2}]
   12 root        -60    -     0B   224K WAIT     2   1:02  46.19% [intr{swi1: netisr 1}]
   11 root        187 ki31     0B    64K RUN      0   8:53  23.10% [idle{idle: cpu0}]
   12 root        -60    -     0B   224K WAIT     2   0:30  20.90% [intr{swi1: netisr 2}]
   12 root        -60    -     0B   224K WAIT     3   0:23  10.79% [intr{swi1: netisr 3}]
   12 root        -60    -     0B   224K WAIT     3   0:15   7.86% [intr{swi1: netisr 0}]
    0 root        -60    -     0B  1616K -        1   0:10   7.86% [kernel{if_io_tqg_1}]
    0 root        -60    -     0B  1616K -        3   0:07   3.08% [kernel{if_io_tqg_3}]
    0 root        -60    -     0B  1616K -        2   0:06   3.08% [kernel{if_io_tqg_2}]

This version of the beta should have the improved if_pppoe, correct?

kprovost

@TampertK RSS pins flows to specific CPU cores, so if you're testing with a single flow this is what you'd expect to see.
Repeat your test with more flows and you'll see more of your CPU cores doing work.