New PPPoE backend, some feedback

stephenw10

It probably won't be in 2.8 but it's a pretty simple feature. I'd expect it could be added via system patches after release.

Or you can still just add the cronjob manually.

https://redmine.pfsense.org/issues/16159

claudio69

@stephenw10

The thing is I don't know how to manually make the script to put in cron.
If you can kindly give me a working example that would be great and would solve the problem permanently.
Thanks

stephenw10

Edit the cronjob and change the command to: /etc/rc.interfaces_wan_configure opt2

Where opt2 is the internal interface name for your pppoe interface. If that's actually WAN then use wan.

w0w

@stephenw10 said in New PPPoE backend, some feedback:

/etc/rc.interfaces_wan_configure

I'll ask something else while we're at it...

Regarding manually starting and stopping the WAN interface, which is PPPoE — what's the proper way to do it? Especially with the new backend...

/etc/rc.linkup stop wan
/etc/rc.linkup start wan

Or is there something newer now?

claudio69

@stephenw10

It worked fine.
Thanks for your help.

stephenw10

That should work. There's nothing newer I'm aware of. Because it's now a regular interface (mostly) you should be able to do regular interface things with it. If you want/need to.

RobbieTT

@stephenw10

Is there a definitive list of PPPoE performance tweaks that should be removed before using if_pppoe?

If so, it may be worth adding it to Jim's article too.

Apologies if it exists somewhere but I did look... and clearly I have forgotten all the tweaks I did apply to try and get the best PPPoE performance on the current backend.

️

w0w

Does anyone have packet loss on a new backend? The connection is not interrupted just loss of packets, about 5%, once a day or so.

MONITOR: WAN_PPPOE has packet loss

This could be something on isp or on my side also, because I haven't seen this before. IDK...

stephenw10

Not seen any significant loss here. Been solid since I switched both my edge PPPoE links to it:

Screenshot from 2025-04-24 19-20-59.png

And I'd been running it for months before that on one WAN in 25.03.

stephenw10

@RobbieTT said in New PPPoE backend, some feedback:

Is there a definitive list of PPPoE performance tweaks that should be removed before using if_pppoe?

There isn't. The testing we did though was without any of the previous pppoe tweaks AFAIK. So the default sysctls only.

RobbieTT

@stephenw10 said in New PPPoE backend, some feedback:

@RobbieTT said in New PPPoE backend, some feedback:

Is there a definitive list of PPPoE performance tweaks that should be removed before using if_pppoe?

There isn't. The testing we did though was without any of the previous pppoe tweaks AFAIK. So the default sysctls only.

I know, I just need a nudge on what I now need to remove (eg deleting the top entry for a start):

2025-04-24 at 20.24.17.png

️

stephenw10

I think only the net.isr.dispatch value there is non-default.

Default values I see here are:

Tunable Name 	Description 	Value 	
net.inet.ip.portrange.first 		1024 	
net.inet.tcp.blackhole 	Do not send RST on segments to closed ports 	2 	
net.inet.udp.blackhole 	Do not send port unreachables for refused connects 	1 	
net.inet.ip.random_id 	Assign random ip_id values 	1 	
net.inet.tcp.drop_synfin 	Drop TCP packets with SYN+FIN set 	1 	
net.inet.ip.redirect 	Enable sending IP redirects 	1 	
net.inet6.ip6.redirect 	Send ICMPv6 redirects for unforwardable IPv6 packets 	1 	
net.inet6.ip6.use_tempaddr 	Create RFC3041 temporary addresses for autoconfigured addresses 	0 	
net.inet6.ip6.prefer_tempaddr 	Prefer RFC3041 temporary addresses in source address selection 	0 	
net.inet.tcp.syncookies 	Use TCP SYN cookies if the syncache overflows 	1 	
net.inet.tcp.recvspace 	Initial receive socket buffer size 	65228 	
net.inet.tcp.sendspace 	Initial send socket buffer size 	65228 	
net.inet.tcp.delayed_ack 	Delay ACK to try and piggyback it onto a data packet 	0 	
net.inet.udp.maxdgram 	Maximum outgoing UDP datagram size 	57344 	
net.link.bridge.pfil_onlyip 	Only pass IP packets when pfil is enabled 	0 	
net.link.bridge.pfil_member 	Packet filter on the member interface 	1 	
net.link.bridge.pfil_bridge 	Packet filter on the bridge interface 	0 	
net.link.tap.user_open 	Enable legacy devfs interface creation for all users 	1 	
net.link.vlan.mtag_pcp 	Retain VLAN PCP information as packets are passed up the stack 	1 	
kern.randompid 	Random PID modulus. Special values: 0: disable, 1: choose random value 	347 	
net.inet.ip.intr_queue_maxlen 	Maximum size of the IP input queue 	1000 	
hw.syscons.kbd_reboot 	enable keyboard reboot 	0 	
net.inet.tcp.log_debug 	Log errors caused by incoming TCP segments 	0 	
net.inet.tcp.tso 	Enable TCP Segmentation Offload 	1 	
net.inet.icmp.icmplim 	Maximum number of ICMP responses per second 	0 	
vfs.read_max 	Cluster read-ahead max block count 	32 	
kern.ipc.maxsockbuf 	Maximum socket buffer size 	4262144 	
net.inet.ip.process_options 	Enable IP options processing ([LS]SRR, RR, TS) 	0 	
kern.random.harvest.mask 	Entropy harvesting mask 	351 	
net.route.netisr_maxqlen 	maximum routing socket dispatch queue length 	1024 	
net.inet.udp.checksum 	compute udp checksum 	1 	
net.inet.icmp.reply_from_interface 	ICMP reply from incoming interface for non-local packets 	1 	
net.inet6.ip6.rfc6204w3 	Accept the default router list from ICMPv6 RA messages even when packet forwarding is enabled 	1 	
net.key.preferred_oldsa 		0 	
net.inet.carp.senderr_demotion_factor 	Send error demotion factor adjustment 	0 	
net.pfsync.carp_demotion_factor 	pfsync's CARP demotion factor adjustment 	0 	
net.raw.recvspace 		65536 	
net.raw.sendspace 		65536 	
net.inet.raw.recvspace 	Maximum space for incoming raw IP datagrams 	131072 	
net.inet.raw.maxdgram 	Maximum outgoing raw IP datagram size 	131072 	
kern.corefile 	Process corefile name format string 	/root/%N.core 	
kern.crypto.iimb.enable_aescbc 		1 	
kern.crypto.iimb.enable_multiq 		1 	
kern.crypto.iimb.use_task 		0 	
kern.crypto.iimb.arch 		auto 	
kern.crypto.iimb.prefetch 		1 	
kern.crypto.iimb.max_jobs 		256

Urgh, formatting fail!

RobbieTT

@stephenw10

Well I'm up and running on if_pppoe and I can see the pppoe load being spread across multiple cores. It works!

UK / Openreach FTTP / 1500MTU (baby-jumbos) / Xeon D-1736NT CPU @ 2.70GHz / HyperThreading Off / IPV4 & IPV6 / bidirectional fq_codel / 10 GbE LANs & VLANs

I think I will need to get the Netgate 6100 out and try this. Running pfSense+ on my Xeon provided no issues for single-core PPPoE anyway but it does seem to run at a slightly lower CPU load with the latest config.

No PPP logs or entries in the System logs, which I think has been mentioned already.

️

stephenw10

Ooo you got full size packets over pppoe working. For some reason I can't seem to set that. I'm wondering if I'm hitting some hardware restriction.

Did you have to do anything special?

RobbieTT

@stephenw10 said in New PPPoE backend, some feedback:

Ooo you got full size packets over pppoe working. For some reason I can't seem to set that. I'm wondering if I'm hitting some hardware restriction.

Did you have to do anything special?

Didn't change anything as I did an in-place update so my config was unchanged from before:

2025-04-24 at 22.24.37.png

@Smaug ~ % ping -D -s 1472 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 1472 data bytes
1480 bytes from 8.8.8.8: icmp_seq=0 ttl=119 time=8.628 ms
1480 bytes from 8.8.8.8: icmp_seq=1 ttl=119 time=8.637 ms
1480 bytes from 8.8.8.8: icmp_seq=2 ttl=119 time=8.457 ms
1480 bytes from 8.8.8.8: icmp_seq=3 ttl=119 time=8.303 ms
1480 bytes from 8.8.8.8: icmp_seq=4 ttl=119 time=8.383 ms
1480 bytes from 8.8.8.8: icmp_seq=5 ttl=119 time=8.600 ms
1480 bytes from 8.8.8.8: icmp_seq=6 ttl=119 time=8.442 ms
1480 bytes from 8.8.8.8: icmp_seq=7 ttl=119 time=8.582 ms
1480 bytes from 8.8.8.8: icmp_seq=8 ttl=119 time=8.845 ms
1480 bytes from 8.8.8.8: icmp_seq=9 ttl=119 time=8.423 ms

So everything worked for me out of the gate; I only checked it 'just to be sure' as it is a bit of a UK oddity.

I can compare a few things for you tomorrow if you like.

️

stephenw10

Yup, something odd Openreach are doing. I'll recheck....

stephenw10

Hmm, I have that set but the actual interface MTU is 1492 still:

pppoe1: flags=1008851<UP,POINTOPOINT,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1492
	description: BT
	options=0
	inet 86.191.X.X --> 172.16.13.252 netmask 0xffffffff
	inet6 fe80::201:21ff:fe01:6775%pppoe1 prefixlen 64 scopeid 0x10
	groups: pppoec
	nd6 options=123<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL,NO_DAD>

Do you have the parent set to 1508?

stephenw10

Ah forgot I'd added it to a bridge.

That worked once I set the parent to 1508.

[2.8.0-BETA][admin@pfsense.fire.box]/root: ifconfig igb1
igb1: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1508
	description: BT_MODEM
	options=48100b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWFILTER,HWSTATS,MEXTPG>
	ether 00:01:21:01:67:76
	inet 192.168.102.10 netmask 0xffffff00 broadcast 192.168.102.255
	inet6 fe80::201:21ff:fe01:6776%igb1 prefixlen 64 scopeid 0x2
	media: Ethernet autoselect (1000baseT <full-duplex>)
	status: active
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
[2.8.0-BETA][admin@pfsense.fire.box]/root: pppcfg pppoe1
	dev: igb1 svc: BTInfinity state: session
	sid: 0x1a99 PADI retries: 5 PADR retries: 0 time: 00:01:24
	sppp: phase network authproto auto authname "bthomehub@btbroadband.com" peerproto auto 
	dns: 81.139.56.100 81.139.57.100
[2.8.0-BETA][admin@pfsense.fire.box]/root: ifconfig pppoe1
pppoe1: flags=1008851<UP,POINTOPOINT,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
	description: BT
	options=0
	inet 86.191.X.X --> 172.16.13.252 netmask 0xffffffff
	inet6 fe80::201:21ff:fe01:6775%pppoe1 prefixlen 64 scopeid 0xf
	groups: pppoec
	nd6 options=123<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL,NO_DAD>

RobbieTT

@stephenw10 said in New PPPoE backend, some feedback:

Hmm, I have that set but the actual interface MTU is 1492 still:

pppoe1: flags=1008851<UP,POINTOPOINT,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1492
	description: BT
	options=0
	inet 86.191.X.X --> 172.16.13.252 netmask 0xffffffff
	inet6 fe80::201:21ff:fe01:6775%pppoe1 prefixlen 64 scopeid 0x10
	groups: pppoec
	nd6 options=123<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL,NO_DAD>

Do you have the parent set to 1508?

The link to the ONT? It is set to 1508 as you would expect - the final pipe needs to be a bit bigger to carry the PPPoE overhead and I am sure you used to have it set that way:

2025-04-24 at 22.54.23.png

️

RobbieTT

@stephenw10

Yeah, that would do it for sure!

Mine:

igc0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1508
	description: ONT
	options=4e020bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,WOL_MAGIC,RXCSUM_IPV6,TXCSUM_IPV6,HWSTATS,MEXTPG>
	ether 02:76:xx:xx:xx:81
	inet 10.0.0.1 netmask 0xffffff00 broadcast 10.0.0.255
	inet6 fe80::76:xxxx:fe00:xxxx%igc0 prefixlen 64 scopeid 0x5
	media: Ethernet autoselect (1000baseT <full-duplex>)
	status: active
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

️