New PPPoE backend, some feedback

stephenw10

Edit the cronjob and change the command to: /etc/rc.interfaces_wan_configure opt2

Where opt2 is the internal interface name for your pppoe interface. If that's actually WAN then use wan.

w0w

@stephenw10 said in New PPPoE backend, some feedback:

/etc/rc.interfaces_wan_configure

I'll ask something else while we're at it...

Regarding manually starting and stopping the WAN interface, which is PPPoE — what's the proper way to do it? Especially with the new backend...

/etc/rc.linkup stop wan
/etc/rc.linkup start wan

Or is there something newer now?

claudio69

@stephenw10

It worked fine.
Thanks for your help.

stephenw10

That should work. There's nothing newer I'm aware of. Because it's now a regular interface (mostly) you should be able to do regular interface things with it. If you want/need to.

RobbieTT

@stephenw10

Is there a definitive list of PPPoE performance tweaks that should be removed before using if_pppoe?

If so, it may be worth adding it to Jim's article too.

Apologies if it exists somewhere but I did look... and clearly I have forgotten all the tweaks I did apply to try and get the best PPPoE performance on the current backend.

️

w0w

Does anyone have packet loss on a new backend? The connection is not interrupted just loss of packets, about 5%, once a day or so.

MONITOR: WAN_PPPOE has packet loss

This could be something on isp or on my side also, because I haven't seen this before. IDK...

stephenw10

Not seen any significant loss here. Been solid since I switched both my edge PPPoE links to it:

Screenshot from 2025-04-24 19-20-59.png

And I'd been running it for months before that on one WAN in 25.03.

stephenw10

@RobbieTT said in New PPPoE backend, some feedback:

Is there a definitive list of PPPoE performance tweaks that should be removed before using if_pppoe?

There isn't. The testing we did though was without any of the previous pppoe tweaks AFAIK. So the default sysctls only.

RobbieTT

@stephenw10 said in New PPPoE backend, some feedback:

@RobbieTT said in New PPPoE backend, some feedback:

Is there a definitive list of PPPoE performance tweaks that should be removed before using if_pppoe?

There isn't. The testing we did though was without any of the previous pppoe tweaks AFAIK. So the default sysctls only.

I know, I just need a nudge on what I now need to remove (eg deleting the top entry for a start):

2025-04-24 at 20.24.17.png

️

stephenw10

I think only the net.isr.dispatch value there is non-default.

Default values I see here are:

Tunable Name 	Description 	Value 	
net.inet.ip.portrange.first 		1024 	
net.inet.tcp.blackhole 	Do not send RST on segments to closed ports 	2 	
net.inet.udp.blackhole 	Do not send port unreachables for refused connects 	1 	
net.inet.ip.random_id 	Assign random ip_id values 	1 	
net.inet.tcp.drop_synfin 	Drop TCP packets with SYN+FIN set 	1 	
net.inet.ip.redirect 	Enable sending IP redirects 	1 	
net.inet6.ip6.redirect 	Send ICMPv6 redirects for unforwardable IPv6 packets 	1 	
net.inet6.ip6.use_tempaddr 	Create RFC3041 temporary addresses for autoconfigured addresses 	0 	
net.inet6.ip6.prefer_tempaddr 	Prefer RFC3041 temporary addresses in source address selection 	0 	
net.inet.tcp.syncookies 	Use TCP SYN cookies if the syncache overflows 	1 	
net.inet.tcp.recvspace 	Initial receive socket buffer size 	65228 	
net.inet.tcp.sendspace 	Initial send socket buffer size 	65228 	
net.inet.tcp.delayed_ack 	Delay ACK to try and piggyback it onto a data packet 	0 	
net.inet.udp.maxdgram 	Maximum outgoing UDP datagram size 	57344 	
net.link.bridge.pfil_onlyip 	Only pass IP packets when pfil is enabled 	0 	
net.link.bridge.pfil_member 	Packet filter on the member interface 	1 	
net.link.bridge.pfil_bridge 	Packet filter on the bridge interface 	0 	
net.link.tap.user_open 	Enable legacy devfs interface creation for all users 	1 	
net.link.vlan.mtag_pcp 	Retain VLAN PCP information as packets are passed up the stack 	1 	
kern.randompid 	Random PID modulus. Special values: 0: disable, 1: choose random value 	347 	
net.inet.ip.intr_queue_maxlen 	Maximum size of the IP input queue 	1000 	
hw.syscons.kbd_reboot 	enable keyboard reboot 	0 	
net.inet.tcp.log_debug 	Log errors caused by incoming TCP segments 	0 	
net.inet.tcp.tso 	Enable TCP Segmentation Offload 	1 	
net.inet.icmp.icmplim 	Maximum number of ICMP responses per second 	0 	
vfs.read_max 	Cluster read-ahead max block count 	32 	
kern.ipc.maxsockbuf 	Maximum socket buffer size 	4262144 	
net.inet.ip.process_options 	Enable IP options processing ([LS]SRR, RR, TS) 	0 	
kern.random.harvest.mask 	Entropy harvesting mask 	351 	
net.route.netisr_maxqlen 	maximum routing socket dispatch queue length 	1024 	
net.inet.udp.checksum 	compute udp checksum 	1 	
net.inet.icmp.reply_from_interface 	ICMP reply from incoming interface for non-local packets 	1 	
net.inet6.ip6.rfc6204w3 	Accept the default router list from ICMPv6 RA messages even when packet forwarding is enabled 	1 	
net.key.preferred_oldsa 		0 	
net.inet.carp.senderr_demotion_factor 	Send error demotion factor adjustment 	0 	
net.pfsync.carp_demotion_factor 	pfsync's CARP demotion factor adjustment 	0 	
net.raw.recvspace 		65536 	
net.raw.sendspace 		65536 	
net.inet.raw.recvspace 	Maximum space for incoming raw IP datagrams 	131072 	
net.inet.raw.maxdgram 	Maximum outgoing raw IP datagram size 	131072 	
kern.corefile 	Process corefile name format string 	/root/%N.core 	
kern.crypto.iimb.enable_aescbc 		1 	
kern.crypto.iimb.enable_multiq 		1 	
kern.crypto.iimb.use_task 		0 	
kern.crypto.iimb.arch 		auto 	
kern.crypto.iimb.prefetch 		1 	
kern.crypto.iimb.max_jobs 		256

Urgh, formatting fail!

RobbieTT

@stephenw10

Well I'm up and running on if_pppoe and I can see the pppoe load being spread across multiple cores. It works!

UK / Openreach FTTP / 1500MTU (baby-jumbos) / Xeon D-1736NT CPU @ 2.70GHz / HyperThreading Off / IPV4 & IPV6 / bidirectional fq_codel / 10 GbE LANs & VLANs

I think I will need to get the Netgate 6100 out and try this. Running pfSense+ on my Xeon provided no issues for single-core PPPoE anyway but it does seem to run at a slightly lower CPU load with the latest config.

No PPP logs or entries in the System logs, which I think has been mentioned already.

️

stephenw10

Ooo you got full size packets over pppoe working. For some reason I can't seem to set that. I'm wondering if I'm hitting some hardware restriction.

Did you have to do anything special?

RobbieTT

@stephenw10 said in New PPPoE backend, some feedback:

Ooo you got full size packets over pppoe working. For some reason I can't seem to set that. I'm wondering if I'm hitting some hardware restriction.

Did you have to do anything special?

Didn't change anything as I did an in-place update so my config was unchanged from before:

2025-04-24 at 22.24.37.png

@Smaug ~ % ping -D -s 1472 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 1472 data bytes
1480 bytes from 8.8.8.8: icmp_seq=0 ttl=119 time=8.628 ms
1480 bytes from 8.8.8.8: icmp_seq=1 ttl=119 time=8.637 ms
1480 bytes from 8.8.8.8: icmp_seq=2 ttl=119 time=8.457 ms
1480 bytes from 8.8.8.8: icmp_seq=3 ttl=119 time=8.303 ms
1480 bytes from 8.8.8.8: icmp_seq=4 ttl=119 time=8.383 ms
1480 bytes from 8.8.8.8: icmp_seq=5 ttl=119 time=8.600 ms
1480 bytes from 8.8.8.8: icmp_seq=6 ttl=119 time=8.442 ms
1480 bytes from 8.8.8.8: icmp_seq=7 ttl=119 time=8.582 ms
1480 bytes from 8.8.8.8: icmp_seq=8 ttl=119 time=8.845 ms
1480 bytes from 8.8.8.8: icmp_seq=9 ttl=119 time=8.423 ms

So everything worked for me out of the gate; I only checked it 'just to be sure' as it is a bit of a UK oddity.

I can compare a few things for you tomorrow if you like.

️

stephenw10

Yup, something odd Openreach are doing. I'll recheck....

stephenw10

Hmm, I have that set but the actual interface MTU is 1492 still:

pppoe1: flags=1008851<UP,POINTOPOINT,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1492
	description: BT
	options=0
	inet 86.191.X.X --> 172.16.13.252 netmask 0xffffffff
	inet6 fe80::201:21ff:fe01:6775%pppoe1 prefixlen 64 scopeid 0x10
	groups: pppoec
	nd6 options=123<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL,NO_DAD>

Do you have the parent set to 1508?

stephenw10

Ah forgot I'd added it to a bridge.

That worked once I set the parent to 1508.

[2.8.0-BETA][admin@pfsense.fire.box]/root: ifconfig igb1
igb1: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1508
	description: BT_MODEM
	options=48100b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWFILTER,HWSTATS,MEXTPG>
	ether 00:01:21:01:67:76
	inet 192.168.102.10 netmask 0xffffff00 broadcast 192.168.102.255
	inet6 fe80::201:21ff:fe01:6776%igb1 prefixlen 64 scopeid 0x2
	media: Ethernet autoselect (1000baseT <full-duplex>)
	status: active
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
[2.8.0-BETA][admin@pfsense.fire.box]/root: pppcfg pppoe1
	dev: igb1 svc: BTInfinity state: session
	sid: 0x1a99 PADI retries: 5 PADR retries: 0 time: 00:01:24
	sppp: phase network authproto auto authname "bthomehub@btbroadband.com" peerproto auto 
	dns: 81.139.56.100 81.139.57.100
[2.8.0-BETA][admin@pfsense.fire.box]/root: ifconfig pppoe1
pppoe1: flags=1008851<UP,POINTOPOINT,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
	description: BT
	options=0
	inet 86.191.X.X --> 172.16.13.252 netmask 0xffffffff
	inet6 fe80::201:21ff:fe01:6775%pppoe1 prefixlen 64 scopeid 0xf
	groups: pppoec
	nd6 options=123<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL,NO_DAD>

RobbieTT

@stephenw10 said in New PPPoE backend, some feedback:

Hmm, I have that set but the actual interface MTU is 1492 still:

pppoe1: flags=1008851<UP,POINTOPOINT,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1492
	description: BT
	options=0
	inet 86.191.X.X --> 172.16.13.252 netmask 0xffffffff
	inet6 fe80::201:21ff:fe01:6775%pppoe1 prefixlen 64 scopeid 0x10
	groups: pppoec
	nd6 options=123<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL,NO_DAD>

Do you have the parent set to 1508?

The link to the ONT? It is set to 1508 as you would expect - the final pipe needs to be a bit bigger to carry the PPPoE overhead and I am sure you used to have it set that way:

2025-04-24 at 22.54.23.png

️

RobbieTT

@stephenw10

Yeah, that would do it for sure!

Mine:

igc0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1508
	description: ONT
	options=4e020bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,WOL_MAGIC,RXCSUM_IPV6,TXCSUM_IPV6,HWSTATS,MEXTPG>
	ether 02:76:xx:xx:xx:81
	inet 10.0.0.1 netmask 0xffffff00 broadcast 10.0.0.255
	inet6 fe80::76:xxxx:fe00:xxxx%igc0 prefixlen 64 scopeid 0x5
	media: Ethernet autoselect (1000baseT <full-duplex>)
	status: active
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

️

stephenw10

Yup and in fact you don't have to set the parent to 1508. If you set the pppoe interface to 1500 the parent will inherit that and be set to 1508.

I had the parent interface as part of an unassigned bridge and forgot. In that situation the bridge interface controls the MTU of all the members and was forcing it to 1500.

All good now though.

RobbieTT

@stephenw10 said in New PPPoE backend, some feedback:

Yup and in fact you don't have to set the parent to 1508. If you set the pppoe interface to 1500 the parent will inherit that and be set to 1508.

All good now though.

More things I didn't know!

I'll probably leave it set as 1508 just to help remind me what I am doing when I am tired and forgetting what I am supposed to be doing...

️