PPPoE: Problems getting an IPv6 address on reconnection and other problems

stephenw10

Hmm OK. And without the patch what behaviour were you seeing? Just failing to pull an IPv6 address?

w0w

@stephenw10
This need to be tested. Hope I'll do it soon.
Also I've seen that another one patch coming...

stephenw10

Yup, work on this is ongoing!

w0w

Some additional testing has been done. The uptime bug and the missing default route were probably caused by a custom script that I forgot to shut down. The issue seems impossible to reproduce when the script is disabled, or at least it seems not so easy to reproduce without this script.

The bad news is that IPv6 behaviour is still almost the same: most of the time I get only IPv4 or only IPv6 on the WAN interface, and then it disappears. However, almost every time I reboot the firewall I get both IPv4 and IPv6 running, so the problem occurs only when reconnecting.
And without the patch I don't see any difference.

I wonder, then, what the difference is between bringing up PPPoE during the firewall’s boot and restarting PPPoE manually or when it is triggered by other causes?

stephenw10

Ah, OK.
So, with our testing patch, what do you see in the logs after disconnecting and reconnecting the pppoe WAN in Status > Interfaces?

I forget if you were previously seeing RAs from your ISP? But the patch is expected to allow pulling an IPv6 lease when ISPs do not send an RA.

w0w

Disconnection log disconnect_masked.txt
Connection log connect_masked.txt
I don’t see anything unusual—perhaps only the CARP storming. I use CARP only on WAN2 and the LAN subnets, not on the PPPoE WAN.

@stephenw10 said in PPPoE: Problems getting an IPv6 address on reconnection and other problems:

I forget if you were previously seeing RAs from your ISP?

Every 3 seconds I receive RA packet from ISP. Before the early patch, I was getting a massive storm of log entries and all services kept restarting continuously, and so on, because of that. Now I have problem with receiving Ipv6 but only on reconnection. A clean boot simply works, and in rare cases, just saving the WAN interface settings and applying the changes works too. The same behavior first time occurred on the first 25.03 beta available, without new backend active.

stephenw10

Hmm, so if you disconnect the pppoe and reconnect it in Status > Interfaces what do you see logged?

w0w

@stephenw10
Yes

stephenw10

Um I was hoping for an actual log to look through.

w0w

@stephenw10
I just wasn’t fully awake when I read the message — for some reason, it seemed like the question was worded differently.
The last log I posted was actually from when I pressed disconnect and then arter it disconnected, connect.

stephenw10

Could be my turn to find more coffee then! Where is that log?

w0w

@w0w said in PPPoE: Problems getting an IPv6 address on reconnection and other problems:

Disconnection log disconnect_masked.txt
Connection log connect_masked.txt

My FF opens those attachments just fine.

stephenw10

Ha, OK. Definitely more coffee required! Somehow read straight past those.

w0w

pppoe0: flags=1008851<UP,POINTOPOINT,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1492
	description: WAN
	options=0
	inet xx.52.22.101 --> yyy.7.29.248 netmask 0xffffffff
	inet6 fe80::aab8:e0ff:fe02:6zz9%pppoe0 prefixlen 64 scopeid 0x11
	inet6 2001:1b28:b248:e39:NNNN:e0ff:fe02:6zz9 prefixlen 64 autoconf pltime 604800 vltime 2592000
	groups: pppoec
	nd6 options=123<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL,NO_DAD>

This is what I see when I get IPv6 only on WAN. Sometimes it shows also "detached" in Ipv6 options

If I run rtsol pppoe0 I am getting

rtsol: pppoe0 does not accept Router Advertisement.

So if I force it by issuing

ifconfig pppoe0 inet6 -ifdisabled -no_radr accept_rtadv
rtsol -F pppoe0

This results in the same outcome — I still receive an IPv6 address marked detached or not and after some time it disappears.

On reboot of firewall I am getting

pppoe0: flags=1008851<UP,POINTOPOINT,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1492
	description: WAN
	options=0
	inet xx.52.5.227 --> yyy.7.29.248 netmask 0xffffffff
	inet6 fe80::aab8:e0ff:fe02:6zz9%pppoe0 prefixlen 64 scopeid 0x10
	inet6 2001:1234:b248:e39:NNNN:e0ff:fe02:6zz9 prefixlen 64 detached autoconf pltime 604800 vltime 2592000
	inet6 2001:4321:d248:ffff:f8b3:abab:1435:769f prefixlen 128 pltime 86400 vltime 172800
	inet6 2001:4321:d248:ffff:49e6:baba:d204:50b8 prefixlen 128 pltime 86400 vltime 172800
	groups: pppoec
	nd6 options=123<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL,NO_DAD>

IPv6 works fine then.

I am not quite sure what it means, but…

EDIT:
It turns out that when I enable the “Do Not Wait for RA” option, the interface receives all addresses just fine.
Curiously, I recall having trouble obtaining addresses even with that option enabled. I probably need to run some more tests—maybe the other settings I tried in combination are now doing the trick.

stephenw10

Ah, OK. So, yes, if your ISP requires that then it should now work correctly with the new patch.

So with that set is it working as expected for you?

w0w

@stephenw10 said in PPPoE: Problems getting an IPv6 address on reconnection and other problems:

So, yes, if your ISP requires that then it should

It was not required until 23.05.
Of course, this doesn’t necessarily mean that it worked without that option in version 24.11, but it definitely works without it in 23.09. In fact, the need to enable this option in earlier alpha versions of 23.05 is what originally led to the creation of a patch, which was supposed to fix the issue that occurred without this option enabled. And logically, the problem—namely, log spam and endless restarts of services triggered by each RA received from the ISP—did stop, so I assumed I no longer needed the option.

Apologies for the slight confusion... But again, how is one supposed to make sense of all these modern connection and address assignment algorithms, when in one version of pfSense I need to enable this option, and in another, everything works fine without it?

Definitely will test later and report back with this option enabled.

stephenw10

Yup, this has been an interesting learning experience. Basically that mpd5/netgraph is doing a bunch of things I never thought it was so if_pppoe handling had to be improved.

w0w

I’m not sure what did the trick this time — whether it was rebooting the firewall or just the stars aligning — but everything’s back to how it was: getting an IPv6 address is still unreliable.

I ran an experiment. If I skip the GUI’s “WAN disconnect/connect” button and instead run ifconfig pppoe0 down / up, I have no issues getting an IPv6 address on either the WAN or LAN. During a reboot it stays that way too — still no address-assignment problems.

Another thing I’ve noticed: for as long as I can remember (definitely more than a couple of years), any PPPoE reconnect with my ISP via the same mpd5 used to require a handful of connection attempts before the link came up — say ten tries, sometimes more. Now I mostly see that behavior only on a reboot. If I break the connection through the GUI, wait until the “Connect” button is clickable again, the link comes up almost instantly — but IPv6 addressing becomes flaky. I don’t know what this means; maybe it’s unrelated and pfSense is simply closing the session cleanly when the GUI disconnects, which it might not have done before… No idea.

EDIT: Additional testing showed that if the link drops on the provider’s side or the WAN-side switch loses power, the connection comes back up normally and all addresses are obtained. For now it looks like the problem is only with the GUI button—and maybe with the rc.linkup script—and that’s perfectly fine by me since I almost never use that button anyway. Hopefully that’s the end of it.

stephenw10

Hmm, well I guess that's a good result? But I hate unexplained behaviour like that.

The connection now still loos like the connection log above?

That clearly shows the ISP does send an RA so I wouldn't expect to need 'do not wait' to be set:

2025-06-20 05:42:59.290097+03:00 	rtsold 	75589 	Received RA specifying route fe80::669e::IPV68 for interface wan(pppoe0)

I also note that appears to be part of an HA pair. Is that LAN side only?

As far as I know if_pppoe cannot work in an HA setup on the WAN side. The previous workaround setup no longer works.

w0w

@stephenw10 said in PPPoE: Problems getting an IPv6 address on reconnection and other problems:

Hmm, well I guess that's a good result? But I hate unexplained behaviour like that.

I can live with that

@stephenw10 said in PPPoE: Problems getting an IPv6 address on reconnection and other problems:

The connection now still loos like the connection log above?

Something like that, yes

@stephenw10 said in PPPoE: Problems getting an IPv6 address on reconnection and other problems:

I also note that appears to be part of an HA pair. Is that LAN side only?

The HA VIPs are configured only on WAN 2 (which has a static address) and on two LANs. However, even after I disabled HA, the same behavior persists.

@stephenw10 said in PPPoE: Problems getting an IPv6 address on reconnection and other problems:

As far as I know if_pppoe cannot work in an HA setup on the WAN side. The previous workaround setup no longer works.

Overall, the setup—HA only on one WAN (the non-PPPoE link) and on the LAN networks, plus active fail-over between the WANs—does work. The only thing I don’t quite understand is why HA triggers specifically when the PPPoE link drops: the master flips to backup and back again for a few seconds, basically flapping. I’m not sure whether it always behaved like that. In any case, I’ve reached the point where, even if the switchover isn’t completely seamless, the downtime is only about five seconds when, say, I power off one firewall—and that’s good enough for me.