Resolved: Did v23.05+ break IPv6?

RobbieTT

I've not changed my IPv6 settings since the update to 23.05 and 23.05.1. I receive a /48 from my ISP and distribute to three /64 subnets:

2023-07-13 at 12.21.43.png

The GUI does not show the delegations (going from memory, I recall this being populated before 23.05):

20230713-pfSense-DHCPv6 Leases-GUI-redacted.png

The routing table looks sparse:

[23.05.1-RELEASE][admin@Router-8.redacted.me]/root: netstat -6rn
Routing tables

Internet6:
Destination                       Gateway                       Flags     Netif Expire
default                           fe80::2a3:dead:feca:ae80%pppoe0 UG     pppoe0
::1                               link#10                       UHS         lo0
::10.10.10.1                      link#10                       UHS         lo0
2a02:reda:cted:1::/64              link#6                        U           ix1
2a02:reda:cted:1:92ec:dead:fe1b:70a9 link#10                     UHS         lo0
2a02:reda:cted:2::/64              link#13                       U      ix1.1003
2a02:reda:cted:2:92ec:dead:fe1b:70a9 link#10                     UHS         lo0
2a02:reda:cted:3::/64              link#1                        U          igc0
2a02:reda:cted:3:92ec:dead:fe1b:70aa link#10                     UHS         lo0
2a02:reda:feed:cted::/64           link#14                       U        pppoe0
2a02:reda:feed:cted:92ec:dead:fe1b:70aa link#10                  UHS         lo0
fe80::%igc0/64                    link#1                        U          igc0
fe80::1:1%lo0                     link#10                       UHS         lo0
fe80::92ec:dead:fe1b:70aa%lo0     link#10                       UHS         lo0
fe80::%igc3/64                    link#4                        U          igc3
fe80::92ec:dead:fe1b:70ad%lo0     link#10                       UHS         lo0
fe80::%ix1/64                     link#6                        U           ix1
fe80::1:1%lo0                     link#10                       UHS         lo0
fe80::92ec:dead:fe1b:70a9%lo0     link#10                       UHS         lo0
fe80::%lo0/64                     link#10                       U           lo0
fe80::1%lo0                       link#10                       UHS         lo0
fe80::%ix1.1003/64                link#13                       U      ix1.1003
fe80::1:1%lo0                     link#10                       UHS         lo0
fe80::92ec:dead:fe1b:70a9%lo0     link#10                       UHS         lo0
fe80::%pppoe0/64                  link#14                       U        pppoe0
fe80::92ec:dead:fe1b:70aa%lo0     link#10                       UHS         lo0
[23.05.1-RELEASE][admin@Router-8.redacted.me]/root:

Of particular note, the dhcpd.log is showing numerous entries warning that Unable to pick client prefix: no IPv6 pools on this shared network:


[23.05.1-RELEASE][admin@Router-8.redacted.me]/root: cat /var/log/dhcpd.log
Jul 13 09:15:20 Router-8 dhcpd[84408]: Solicit message from fe80::c77:d61c:dead:e7f1 port 546, transaction ID 0x1E58F500
Jul 13 09:15:20 Router-8 dhcpd[84408]: Unable to pick client prefix: no IPv6 pools on this shared network
Jul 13 09:15:20 Router-8 dhcpd[84408]: Sending Advertise to fe80::c77:d61c:dead:e7f1 port 546
Jul 13 09:30:03 Router-8 dhcpd[84408]: Solicit message from fe80::c4f:42d8:dead:772e port 546, transaction ID 0xA7B9C500
Jul 13 09:30:03 Router-8 dhcpd[84408]: Unable to pick client prefix: no IPv6 pools on this shared network
Jul 13 09:30:03 Router-8 dhcpd[84408]: Sending Advertise to fe80::c4f:42d8:dead:772e port 546
Jul 13 09:33:22 Router-8 dhcpd[84408]: Solicit message from fe80::1895:a21a:dead:b096 port 546, transaction ID 0xCBFC2500
Jul 13 09:33:22 Router-8 dhcpd[84408]: Unable to pick client prefix: no IPv6 pools on this shared network
[23.05.1-RELEASE][admin@Router-8.redacted.me]/root:

Did something break with IPv6 in the recent updates?

️

stephenw10

Hmm, I've not see that. I did hit this: https://redmine.pfsense.org/issues/14502
But that only applies if you're delegating prefixes internally.

RobbieTT

@stephenw10
No, not seen that issue but it does seem very similar. I don't think I am qualified to say if it is the same though, given the differences in configuration.

What would you expect in the dhcpd log for IPv6 solicit/advertise/client prefix given my Openreach-friendly IPv6 settings?

️

stephenw10

My edge is on 2.7 but it should be identical at this point.

I don't see that, or any errors. How do you have DHCPv6/RA configured?

Where is that request coming from? Is there actually a prefix delegated to that interface?

RobbieTT

@stephenw10 said in Did v23.05+ break IPv6?:

How do you have DHCPv6/RA configured?

2023-07-13 at 15.23.26.png

For your purposes, what is the best way of displaying that, beyond the image above?

Where is that request coming from? Is there actually a prefix delegated to that interface?

There are multiple FE80 addresses listed for the requests. Looking at a few of them = Apple stuff, HomePods, Apple TV etc which reside on my main LAN, tracking the WAN IPv6 interface with a Prefix ID of 1 (I have 3 subnets so prefix IDs of 1, 2 & 3).

️

stephenw10

The interface data from the the dhcpv6 section in the config should show it all. I'm using managed mode RA here which is probably why I'm not seeing it:

		<opt1>
			<range>
				<from>::1000</from>
				<to>::2000</to>
			</range>
			<prefixrange>
				<from></from>
				<to></to>
				<prefixlength>64</prefixlength>
			</prefixrange>
			<defaultleasetime></defaultleasetime>
			<maxleasetime></maxleasetime>
			<netmask></netmask>
			<domain></domain>
			<domainsearchlist></domainsearchlist>
			<enable></enable>
			<ddnsdomain></ddnsdomain>
			<ddnsdomainprimary></ddnsdomainprimary>
			<ddnsdomainkeyname></ddnsdomainkeyname>
			<ddnsdomainkeyalgorithm>hmac-md5</ddnsdomainkeyalgorithm>
			<ddnsdomainkey></ddnsdomainkey>
			<ddnsclientupdates>allow</ddnsclientupdates>
			<tftp></tftp>
			<ldap></ldap>
			<bootfile_url></bootfile_url>
			<dhcpv6leaseinlocaltime></dhcpv6leaseinlocaltime>
			<numberoptions></numberoptions>
			<ramode>managed</ramode>
			<rapriority>medium</rapriority>
			<rainterface></rainterface>
			<ravalidlifetime></ravalidlifetime>
			<rapreferredlifetime></rapreferredlifetime>
			<raminrtradvinterval></raminrtradvinterval>
			<ramaxrtradvinterval></ramaxrtradvinterval>
			<raadvdefaultlifetime></raadvdefaultlifetime>
			<radomainsearchlist></radomainsearchlist>
		</opt1>

Let me see if I can replicate it in SLAAC...

RobbieTT

@stephenw10

	</dhcpd>
	<dhcpdv6>
		<lan>
			<range>
				<from>::1000</from>
				<to>::2000</to>
			</range>
			<ramode>stateless_dhcp</ramode>
			<rapriority>medium</rapriority>
			<prefixrange>
				<from></from>
				<to></to>
				<prefixlength>64</prefixlength>
			</prefixrange>
			<defaultleasetime></defaultleasetime>
			<maxleasetime></maxleasetime>
			<netmask></netmask>
			<dhcp6c-dns>enabled</dhcp6c-dns>
			<domain></domain>
			<domainsearchlist></domainsearchlist>
			<ddnsdomain></ddnsdomain>
			<ddnsdomainprimary></ddnsdomainprimary>
			<ddnsdomainsecondary></ddnsdomainsecondary>
			<ddnsdomainkeyname></ddnsdomainkeyname>
			<ddnsdomainkeyalgorithm>hmac-md5</ddnsdomainkeyalgorithm>
			<ddnsdomainkey></ddnsdomainkey>
			<ddnsclientupdates>allow</ddnsclientupdates>
			<tftp></tftp>
			<ldap></ldap>
			<bootfile_url></bootfile_url>
			<dhcpv6leaseinlocaltime></dhcpv6leaseinlocaltime>
			<numberoptions></numberoptions>
			<rainterface></rainterface>
			<ravalidlifetime></ravalidlifetime>
			<rapreferredlifetime></rapreferredlifetime>
			<raminrtradvinterval></raminrtradvinterval>
			<ramaxrtradvinterval></ramaxrtradvinterval>
			<raadvdefaultlifetime></raadvdefaultlifetime>
			<radomainsearchlist></radomainsearchlist>
			<radvd-dns>enabled</radvd-dns>
			<enable></enable>
		</lan>
		<opt3>
			<ramode>stateless_dhcp</ramode>
			<rapriority>medium</rapriority>
			<rainterface></rainterface>
			<ravalidlifetime></ravalidlifetime>
			<rapreferredlifetime></rapreferredlifetime>
			<raminrtradvinterval></raminrtradvinterval>
			<ramaxrtradvinterval></ramaxrtradvinterval>
			<raadvdefaultlifetime></raadvdefaultlifetime>
			<radomainsearchlist></radomainsearchlist>
			<radvd-dns>enabled</radvd-dns>
			<range>
				<from>::1000</from>
				<to>::2000</to>
			</range>
			<prefixrange>
				<from></from>
				<to></to>
				<prefixlength>64</prefixlength>
			</prefixrange>
			<defaultleasetime></defaultleasetime>
			<maxleasetime></maxleasetime>
			<netmask></netmask>
			<dhcp6c-dns>enabled</dhcp6c-dns>
			<domain></domain>
			<domainsearchlist></domainsearchlist>
			<ddnsdomain></ddnsdomain>
			<ddnsdomainprimary></ddnsdomainprimary>
			<ddnsdomainsecondary></ddnsdomainsecondary>
			<ddnsdomainkeyname></ddnsdomainkeyname>
			<ddnsdomainkeyalgorithm>hmac-md5</ddnsdomainkeyalgorithm>
			<ddnsdomainkey></ddnsdomainkey>
			<ddnsclientupdates>allow</ddnsclientupdates>
			<tftp></tftp>
			<ldap></ldap>
			<bootfile_url></bootfile_url>
			<dhcpv6leaseinlocaltime></dhcpv6leaseinlocaltime>
			<numberoptions></numberoptions>
			<enable></enable>
		</opt3>
		<opt1>
			<ramode>stateless_dhcp</ramode>
			<rapriority>medium</rapriority>
			<rainterface></rainterface>
			<ravalidlifetime></ravalidlifetime>
			<rapreferredlifetime></rapreferredlifetime>
			<raminrtradvinterval></raminrtradvinterval>
			<ramaxrtradvinterval></ramaxrtradvinterval>
			<raadvdefaultlifetime></raadvdefaultlifetime>
			<radomainsearchlist></radomainsearchlist>
			<radvd-dns>enabled</radvd-dns>
			<range>
				<from>::1000</from>
				<to>::2000</to>
			</range>
			<prefixrange>
				<from></from>
				<to></to>
				<prefixlength>64</prefixlength>
			</prefixrange>
			<defaultleasetime></defaultleasetime>
			<maxleasetime></maxleasetime>
			<netmask></netmask>
			<dhcp6c-dns>enabled</dhcp6c-dns>
			<domain></domain>
			<domainsearchlist></domainsearchlist>
			<enable></enable>
			<ddnsdomain></ddnsdomain>
			<ddnsdomainprimary></ddnsdomainprimary>
			<ddnsdomainsecondary></ddnsdomainsecondary>
			<ddnsdomainkeyname></ddnsdomainkeyname>
			<ddnsdomainkeyalgorithm>hmac-md5</ddnsdomainkeyalgorithm>
			<ddnsdomainkey></ddnsdomainkey>
			<ddnsclientupdates>allow</ddnsclientupdates>
			<tftp></tftp>
			<ldap></ldap>
			<bootfile_url></bootfile_url>
			<dhcpv6leaseinlocaltime></dhcpv6leaseinlocaltime>
			<numberoptions></numberoptions>
		</opt1>
	</dhcpdv6>

Maybe I should be using a managed mode?

From my notes I did have it on 'Assisted' on my initial config. Cannot remember what prompted the change.

️

stephenw10

You would only change it from Assisted to disable DHCPv6. I set it to Managed to disable SLAAC because I had a badly behaved client on there.

RobbieTT

@stephenw10 said in Did v23.05+ break IPv6?:

You would only change it from Assisted to disable DHCPv6. I set it to Managed to disable SLAAC because I had a badly behaved client on there.

Ok, I guess I was hoping for a self-inflicted config snafu rather than a potential bug.

️

stephenw10

Are those clients actually failing to get IPv6 addresses in addition to the errors? I assume they are...

RobbieTT

@stephenw10

They seem to be. If I take this one as an example, which is a HomePod mini:

Jul 13 16:16:50 Router-8 dhcpd[88832]: Solicit message from fe80::c77:d61c:dead:e7f1 port 546, transaction ID 0x1E58F500
Jul 13 16:16:50 Router-8 dhcpd[88832]: Unable to pick client prefix: no IPv6 pools on this shared network
Jul 13 16:16:50 Router-8 dhcpd[88832]: Sending Advertise to fe80::c77:d61c:dead:e7f1 port 546

I find the 3 addresses I would expect in ndp:

2a02:reda:cted:1:44:85da:d2b:4e22     e0:2b:96:de:ad:e7    ix1 23h54m3s  S R
fe80::c77:d61c:dead:e7f1%ix1         e0:2b:96:de:ad:e7    ix1 23h59m49s S R
2a02:reda:cted:1:f838:ff81:92cb:b1f4  e0:2b:96:de:ad:e7    ix1 23h12m47s S R

Looking at the HomePod mini itself it only shows 2 IPv6 addresses:

fe80::c77:d61c:dead:e7f1
2a02:reda:cted:1:82:e074:5611:8f57

So they don't tie together neatly.

️

stephenw10

Hmm, not seeing that here after setting RA mode to Assisted:
(logs reveresed)

Jul 13 17:48:58 	dhcpd 	90545 	Sending Reply to fe80::d088:d265:1c85:243e port 546
Jul 13 17:48:58 	dhcpd 	90545 	Reply NA: address 2a00:reda:cted:6a02::1982 to client with duid 00:04:68:c4:1c:60:11:aa:96:67:78:74:1c:e1:f7:d0:35:e8 iaid = 227970921 valid for 7200 seconds
Jul 13 17:48:58 	dhcpd 	90545 	Request message from fe80::d088:d265:1c85:243e port 546, transaction ID 0x11886700
Jul 13 17:48:57 	dhcpd 	90545 	Sending Advertise to fe80::d088:d265:1c85:243e port 546
Jul 13 17:48:57 	dhcpd 	90545 	Advertise NA: address 2a00:reda:cted:6a02::1982 to client with duid 00:04:68:c4:1c:60:11:aa:96:67:78:74:1c:e1:f7:d0:35:e8 iaid = 227970921 valid for 7200 seconds
Jul 13 17:48:57 	dhcpd 	90545 	Picking pool address 2a00:reda:cted:6a02::1982
Jul 13 17:48:57 	dhcpd 	90545 	Solicit message from fe80::d088:d265:1c85:243e port 546, transaction ID 0x9163A000

Note that's inside the DHCPv6 pool I have defined. It's a DHCPv6 lease. You have a range defined but with RA mode set to stateless it may not actually be available.

It looks like your logs show DHCPv6 failure which might be expected with RA set to SLAAC only.

Steve

RobbieTT

@stephenw10
Thanks Steve, is there a suggested best-practice with this?

stephenw10

I would say use Assisted mode RA unless you have a good reason not to.

RobbieTT

@stephenw10
Thanks Steve, I'll give it a go.

Found my previous notes and apparently it all fell apart last time I tried Assisted Mode. Thinking back, I probably didn't leave enough time for all the revised IPv6 addresses to propagate.

Is there anything else to change in my config with Assisted Mode, or is as simple as changing the drop-down option?

️

stephenw10

I don't think anything else is needed but I will say it's not something I'm spent much time looking into.

I'm also not aware of anything that changed there since 23.01 so it's surprising you saw a change in behaviour.

RobbieTT

@stephenw10
Yep, it is all a bit odd - especially with the errors in the logs. It may be linked to the issue on Redmine but I am still not certain of it. I guess I will find out when that issue is closed.

Thanks for the help, again.

️

RobbieTT

@stephenw10

The Assisted Mode has IPv6 fully working again with no errors in the log.

As a 'promising' side-effect, my HomePods have suddenly remembered how to play an Apple Music playlist again; something that stopped a few Apple updates ago.

If I select SLAAC again, Apple Music becomes stupid again. I didn't think this issue would be IPv6 related, especially with IPv6 apparently working (at least for the most part).

I still have no idea what is up with SLAAC though, so this issue may still impact others, but Assisted Mode works for me.

Anyway, thanks Steve, 2 issues resolved for the price of 1.

️