New PPPoE backend, some feedback
-
@dsl-ottawa said in New PPPoE backend, some feedback:
a) I could try updating the firmware, never a bad thing
b) unsure about the
dev.ixl.0.eee.enable: 0
dev.ixl.0.iflib.allocated_msix_vectors: 9
dev.ixl.0.iflib.override_qs_enable: 1
and
c)Your queue outputs show that on the RX side only the one queue was active on yours too.I don't think those parameters are related to the problem — they seem like some OEM firmware/CPU settings, maybe.
And about the queue... That’s a good find! I didn’t pay attention to it before. Hmm, interesting...
I have a gigabit PPPoE connection, and the new backend gives me slightly better speeds compared to MPD5 with the deferred option. It almost saturates my ISP’s Ethernet link — over 900 Mbps — which I think is a great result for 20-year-old Cat5 cabling. -
@w0w I would agree nothing there in the settings..
BUT I found some more info.. and I should have kept to my gut feeling and done the further tests before pinpointing the PPPOE.
I disabled my pppoe and ran straight IP, to my provider modem, so double nat'd.
- All the queues were used. as expected.
- my desktop speed tests DIDNT CHANGE, still slow downloads
- I found a copy of the cli speedtest package for pfsense and installed it.
- download test from the pfsense box came back at full 3gig
- upload was slower than normal but I'm less worried about that since my desktop tests were fine - put PPPOE back, and ran the same tests.
- download 3gig from pfsense using speedtest-cli
- and normal 1.7-1.9 on my desktop
This to me proves that it's not the PPPOE or even the wan side of pfsense but something on my bleepin LAN, or at a minimum the LAN side of pfsense.
On the up side it does seem to prove that the single queue on the download isn't an issue :)
-
Ah that's interesting!
What did the per-core CPU usage look like when downloading at 3G over PPPoE? Though the speedtest itself will be using a lot.
-
@stephenw10 it's a pretty quick test so I'm not sure it's a good capture
CPU 0: 11.8% user, 0.0% nice, 28.6% system, 0.0% interrupt, 59.6% idle
CPU 1: 3.9% user, 0.0% nice, 6.3% system, 10.2% interrupt, 79.6% idle
CPU 2: 12.2% user, 0.0% nice, 7.8% system, 7.1% interrupt, 72.9% idle
CPU 3: 5.9% user, 0.0% nice, 6.3% system, 11.0% interrupt, 76.9% idle
CPU 4: 10.6% user, 0.0% nice, 7.8% system, 8.2% interrupt, 73.3% idle
CPU 5: 4.7% user, 0.0% nice, 8.6% system, 12.9% interrupt, 73.7% idle
CPU 6: 5.1% user, 0.0% nice, 6.7% system, 17.3% interrupt, 71.0% idle
CPU 7: 4.3% user, 0.0% nice, 7.1% system, 11.0% interrupt, 77.6% idlebut it certainly looks clean
-
@dsl-ottawa said in New PPPoE backend, some feedback:
put PPPOE back, and ran the same tests.
- download 3gig from pfsense using speedtest-cli
- and normal 1.7-1.9 on my desktop
Is there any chance that some old limiters, rules, traffic shaping, or other configurations are still active?
If this is a clean install, I would suspect a bug in pfSense -
I make a script to monitor the queues of the parent interface IX2 and all data goes through the queue 0 on the RX side.
BTW my ISP use a VLAN (IX2 --> VLAN40 --> PPPoE) maybe it doesn't help to manage the queues correctly...
In 24 Hours
-
@mr_nets
There may be only one queue on the network card, but packets can be distributed across CPU cores. Previously, due to netgraph, everything was handled by a single core. The deferred option slightly improved this, but it was a limited improvement—overall, performance was bottlenecked by a single CPU core. At least that limitation no longer exists now.In theory, if the network queue never fills up, there shouldn't be any issues. I don't know what the theoretical maximum throughput of a PPPoE session is—maybe Netgate specialists have already tested this? I would expect something like 2.5–3.5 Gbps with multithreaded download.
-
It still depends on the hardware but we were able to pass >8Gbps through a 6100 in raw testing: https://www.netgate.com/blog/optimizing-pppoe-performance-in-pfsense-software
-
@stephenw10 said in New PPPoE backend, some feedback:
It still depends on the hardware but we were able to pass >8Gbps through a 6100 in raw testing: https://www.netgate.com/blog/optimizing-pppoe-performance-in-pfsense-software
Thanks, great read. Too bad I missed it, but I'm glad you brought it to me today.
-
I am testing pfSense 2.8 on my pppoe WAN setup that has been working for many years.
I noticed that the weekly periodic reset I set on the pppoe WAN is not automatically performed on the weekend.
If I try to change the setting manually the pppoe connection resets and reconnects but then it will no longer do so automatically -
You're setting that in the PPP settings on the interface?
Interesting. That's an advanced setting that only applies to mpd5. It's hidden on the ppp interfaces page but looks like it's not hidden from the assigned interface and probably should be.
-
I use this setting on a pppoe connection with variable ip.
I need it to avoid the connection being reset by the ISP since I do not have a fixed IP.
On pfSense from 2.5 to 2.7.2 it worked on all versions.
-
I assume you have enabled if_pppoe?
-
Yes, I have enabled system/advanced /networking marked Use if_pppoe kernel module for PPPoE client.It seems like a bug in the new version 2.8
-
Yes, there is a bug there. However currently the bug is that the mpd5 specific options are not hidden correctly. And that the cron-job is still set even thought the script it calls is not present when if_pppoe is enabled.
If you need this for now you should add your own cronjob. This should work:
/etc/rc.interfaces_wan_configure opt2
Where opt2 is the pppoe interface internal name.This should probably also be a feature request. Let me see here...
-
The WAN interface is pppoe1 which is on igb1.
There is already a cron job with the script /var/etc/pppoe_restart_pppoe1.
If you can tell me if this is ok or if this script needs to be modified?
-
That script is created to bring up mpd5 so it's not valid when using if_pppoe.
Here's a bug the covers the invalid options: https://redmine.pfsense.org/issues/16155
I'll open a feature request to add back the periodic reset for if_pppoe.
-
Thanks for reporting.
I hope this feature will be included in pfSense 2.8. -
It probably won't be in 2.8 but it's a pretty simple feature. I'd expect it could be added via system patches after release.
Or you can still just add the cronjob manually.
-
The thing is I don't know how to manually make the script to put in cron.
If you can kindly give me a working example that would be great and would solve the problem permanently.
Thanks
