IPv6 LAN Client DNS Address not Updating when IPv6 Subnet Changes from ISP
-
I have IPv6 working with DHCP6 on the WAN with a 64 prefix selected. LAN is set to track WAN interface and correctly receives a valid IP6 address and properly advertises to clients on my network. Clients get configured with valid IP6 addresses as well as a DNS entry for the IP6 address of my pfSense LAN interface. I am using 2.1.1-RELEASE.
Everything works well until the IP6 prefix changes from my ISP. When that happens, pfSense assigns a new IP6 address to the LAN interface on the new subnet but my clients think their preferred DNS server is still the old IP6 address of my pfSense box. It does not update. The clients get new IP6 addresses but still have old ones on the previous subnet which they seem to think are valid as the new ones do not take priority in routing. The result is I cannot route IP6 traffic anywhere from these clients until the old addresses go away. Also DNS lookups timeout on the IP6 DNS server and fall back to the IP4 DNS server which vastly increases the time to make any DNS lookups.
After a period of time this seems to resolve itself as the clients get the proper IP6 address set to their DNS server (maybe an hour or more) and the old subnet IPs go away on their interfaces. Is there something broken with how pfSense sends advertisements to the LAN when the WAN IP6 address changes? I think this may be related from the server log:
dhcpleases: Could not deliver signal HUP to process because its pidfile does not exist, No such process.
I checked and all the pid files in /var/run match the PIDs I see running but I'm guessing it is a critical step to send advertisement updates on this event.
I can reproduce it easily because on every reboot of my pfSense box I get a different IPv6 /64 subnet from my ISP. It seems to mainly fip flop between the same two subnets but the problems happen when it changes after clients are configured with one of them. I can fix a client if I cycle its network interface off/on which must force the solicit/response cycle to get IP6 info.
-
Have you filed a bug?
-
I have not yet. I was hoping to get a better handle on whether this is really a bug or just bad configuration.
My focus right now is around radvd. From the bootup sequence I noticed this in the routing log (read bottom to top):
radvd[43051]: resuming normal operation
radvd[43051]: attempting to reread config file
radvd[42441]: no auto-selected prefix on interface em0_vlan40, disabling advertisements
radvd[42441]: version 1.9.1 startedvlan40 is my lan.
After bootup, my radvd.conf looks good with the right IP6 info. Could it be that the config gets updated after radvd starts with an old/bad config? It seems that it starts up twice though - once with no prefix in the config and then again with a new pid that has no warning about the prefix.
Also, I tried sending a SIGHUP to the radvd process to hopefully trigger a reload of the config and start advertisements again. Doesn't seem to make a difference. Is there a simple way to see the advertisements in action?
-
When that happens, pfSense assigns a new IP6 address to the LAN interface on the new subnet but my clients think their preferred DNS server is still the old IP6 address of my pfSense box. It does not update.
After a period of time this seems to resolve itself as the clients get the proper IP6 address set to their DNS server (maybe an hour or more) …
I can fix a client if I cycle its network interface off/on which must force the solicit/response cycle to get IP6 info.
This is happening because the host's IPv6 DNS server is learned via DHCPv6. The host interface IPv6 address auto-configured from the prefix learned from the RA. You can see this in action with a IPv6 packet capture on the LAN interface.
So, if you wait, it will fix itself after the host checks in when the DHCPv6 lifetime counts down and it renews with the new address. Or, if you reboot the host it will do a new request and the new DNS (pfSense LAN interface) is learned.
This is not a issue with the host IPv6 interface address because it will change as soon as the updated RA is seen.
Having the old prefix still on the hosts interface is not unusual and not the cause of the problem. They tend to stick around until reboot on some OS'es.
I see no pfSense bug here. The real question is why your assigned prefix keeps changing. At one point, Comcast had that problem until they assigned different priorities to the two DHCP-PD responses.
-
I would expect that an address change on the LAN interface should prompt a "server-initiated configuration exchange" (see RFC 3315, section 19).
-
(The DNS info that's being advertised could change for any number of other reasons, so this would be an issue even if the prefix didn't change.)
-
I would expect that an address change on the LAN interface should prompt a "server-initiated configuration exchange" (see RFC 3315, section 19).
Wouldn't this be a deal breaker for that?
"… the use of a security mechanism is mandated in Reconfigure messages. The server MUST use DHCP authentication in the Reconfigure message."
( as in, Authentication for DHCP Messages - RFC 3118)... that seems to entail key exchange and the associated complexities.
-
I don't believe DHCP6 is part of this except for the WAN interface. All my hosts are being configured from RA messages with the RDNSS and DNSSL options. This appears to be how radvd is configured.
I believe the issue is either radvd not sending unsolicited messages on IP6 changes or my host ignoring them because the expiry time on all the info has not expired.
RFC 2641 makes it sound like the router should send an unsolicited RA from the old interface address with an expiry time of 0 and send a new RA for the correct new prefix and dns config. Otherwise IP6 will just break for hosts any time the prefix changes until everything expires or the host interfaces re-initialize and solicit the router. Since the WAN interface is dynamic it should be allowed to change and properly handled on all tracking interfaces.
-
I don't believe DHCP6 is part of this except for the WAN interface.
Easy enough to see. Run an IPv6 packet capture on the LAN, boot up a machine. You will see the link local make the request and pfSense will respond back via DHCPv6 with the DNS info, which is the LAN IPv6 address.
It's a common LAN configuration to use DHCPv6 for the DNS and domain name, and SLAAC for the interface addressing.
Yes, my LAN IPv6 address is dynamic DHCP-PD, track WAN, etc, etc.
Here is a capture taken at host boot time. DHCPv6 is requested and the only reply is with the DNS server info, which is the LAN interface address. Host interface addressing is from the RA.
-
Thanks for the info. I had no idea DHCPv6 was involved on my internal network. I thought since the RA daemon was configured with DNS also it was just using that. Either way, is this still the desired behavior that there is no automatic client reconfiguration if the WAN changes? It seems very fragile this way.
-
I seriously doubt relying on client time-outs is actually the best practice for this.
-
I agree. But in the meantime, what would be a reasonable way to configure around this?
First of all, am I now correct in understanding the components within pfSense involved are dhcpd (for ip6 dns config) and radvd (to advertise the prefix)? I'd be surprised if those packages are not robust enough to already handle this in some way.
-
I think I figured out why I was getting a new prefix on every reboot. I had the RAM disk option enabled for /var and /tmp because I want to limit writes to my ssd. After I disable that and reboot a few times I can confirm I start getting the same prefix. Can anyone else verify this is what causes it?
I'm not sure what in /var might be lost to screw up my next dhcp6 request but it's probably not in /var/dhcpd/var/db/ because I had the periodic backup of that folder enabled. Should this be logged as a bug?
I still think the auto address reconfiguration also needs to be addressed but keeping the same prefix on reboot will greatly limit the frustration.
-
Probably the DUID.