IPv6 changes aren't written to config.xml or dhcp6c.conf
-
@brado7274 happy it works for you. If your statement were true for pfSense 2.8.1 a lot more people would complain.
It does sounds very strange, you may trust the LLM a bit too much in my opinion in that case. They are known to give a lot of untrue information. It's certainly a good idea to cross check the information you get.
If the FreeBSD man pages are anything to go by neither
dhcp6snordhcp6care ISC software but from WIDE/KAME.The DHCP6 server is KEA (and KEA is also from the same company, ISC that created ISC DHCP) or ISC DHCP and the router advertisement daemon (RA) was never part of the ISC DHCP or KEA suite.
https://man.freebsd.org/cgi/man.cgi?query=dhcp6c
https://man.freebsd.org/cgi/man.cgi?query=dhcp6s
https://man.freebsd.org/cgi/man.cgi?query=radvdIf you want to help the pfSense devs you can backup your working config, switch to KEA and lets find the issue. By restoring your working config you can always get back.
Personally I'm not that good with reading long texts, listing with settings or screenshots are easier for me to process. But others will not have that limitation.
-
@patient0 said in IPv6 changes aren't written to config.xml or dhcp6c.conf:
I posted the resolution not so much for a debate, but to help anyone else experiencing something similar, and to let any devs that might be reading know and they could chase it down. But there are a few things perhaps I could give a little more color to:
If your statement were true for pfSense 2.8.1 a lot more people would complain.
I'm not sure that's necessarily the case. Switching from ISC to Kea alone might not cause an immediate problem, it might just lay a trip wire that isn't tripped until other IPv6 changes are made. I cannot say whether that's any IPv6 change or a specific one, but there's no doubt that there's a problem related to this. So just because folks upgraded to 2.8.x and are using Kea doesn't necessarily mean their system would immediately experience symptoms. They might not be making IPv6 changes immediately after. The system might use the same config files which were there prior, and so function would appear to be fine. But once the new config generation for IPv6 was tripped, perhaps that's what triggers the issue.
It does sounds very strange, you may trust the LLM a bit too much in my opinion in that case. They are known to give a lot of untrue information. It's certainly a good idea to cross check the information you get.
Bigger conversation, but AI is the most misapplied tech perhaps of our lifetimes. I don't trust any LLM -- but that said, it beats any search engine, and is great for grokking doc (which usually leaves much to be desired), APIs, libraries, etc. -- and narrowing the area to test and debug. Even when it is not completely right, perhaps b/c of version or feature variations, it can get you close, and in this case, its info squared exactly with the behavior.
I'm not the expert on the underlying pfSense code or the architecture under the hood, and whether the LLM explanation is 100% accurate, or if there are other nuances in play. Devs would be the right ones to speak to that -- but as there have been no other explanations provided, and this one squares perfectly with the behavior exhibited, until another explanation can explain it -- it stands.
If you want to help the pfSense devs you can backup your working config, switch to KEA and lets find the issue. By restoring your working config you can always get back.
That's basically what I did already -- I just didn't dig deeper to try to debug the issue at a code level. But it is certainly enough for those in that position run with it. There is absolutely a bug in pfSense, somewhere in 2.8.x apparently related to Kea DHCP and the generation of configuration. Using the UI admin console only should not be able to put pfSense in a state it cannot function properly or recover from.
I'm glad there was a way through it (factory reset, stick with ISC DHCP), and I hope it gets fixed. Without a workaround, pfSense would have been a no-op, I would have had to dump it immediately and switch to a different firewall, which I didn't want to do.
Thanks,
B -
Just as an FYI -- I went through this whole drill again, and reproduced the same behavior I've reported yet again. For any devs that might be reading, I may not have emphasized enough the aspect of multi-WAN in play, which appears to be the thing which throws it sideways.
As mentioned, I have two ISPs, AT&T Fiber, and Cox Cable Modem. Both work fine, both lines have been tested and are fine, getting expected bandwidth, and no noise on the line. Both support and route IPv6 just fine, when pfSense is removed from the equation. So the constituent parts of the equation are all proven to be working.
When resetting pfSense to factory settings, with only one WAN connection in play, IPv6 configured, resolved, and routed nearly by default. No problems at all. But as soon as you add the second WAN connection, and its associated LAN, that's when things don't work as advertised. No IPv6 config changes made on the second WAN / LAN take, and it appears this is true regardless of whether ISC DHCP or Kea DHCP is in play. Change IPv6 config on that second WAN / LAN, and those changes will never make it to the underlying OS config files. I looked at the config files manually -- they do not take, not when save and applying, nor after a restart of the box.
I'm not the expert on the internals under the hood, but hopefully this gets some devs in the vicinity of the issue. If the config generation functionality was indeed overhauled in 2.8.x as the LLM I consulted reported, it seems like whatever affect multi-WAN inflicts on that, it results in config for the second WAN doesn't take, and possibly with additional changes it ceases to write config for either WAN. But removing the second WAN / LAN connections and resetting cleans it up again.
I hope that helps. I've spent many days and hours testing this, and while there may be a way through, for all practical purposes, I consider pfSense no longer to have viable multi-WAN functionality as of 2.8.x (until a future fix materializes). I've had to run the second WAN/LAN around pfSense on different hardware.
Here's to hoping that both a definitive explanation of the cause and fix surfaces in the near future. I'd like to not have to punt pfSense entirely to support multi-WAN.
-
@brado7274 A separate unique WAN+LAN is probably pretty uncommon I’d think, compared to just multiple WANs and multiple internal networks. If I understood correctly.
If you can reproduce it you can file a report at Redmine.pfsense.org.
-
There's a related quirk there were depending on the config changes being done, you may need to re-save / apply the interface config for any WAN and LAN with related IPv6 settings. For example if LAN tracks WAN then changing the LAN config may also require re-saving WAN. That's the likely explanation for why you weren't seeing the prefix delegation config reflected in /var/etc/dhcp6c.conf.
That doesn't explain why you're now not seeing the second WAN's config at all, especially after a reboot. I've tested the setup on 25.07.1 which has a similar code base to 2.8.1 (i.e. any issues/fixes should affect both versions) and things behave as expected there.
-
@brado7274 said in IPv6 changes aren't written to config.xml or dhcp6c.conf:
Known symptom
In 2.8.x builds, if:
• The configctl binary is missing or broken (configctl: command not found — which you’ve seen),
• or the service mapping files under /usr/local/etc/configd/actions.d/ are missing/corrupted,Yeah, that is just plain wrong. Yet the LLM sounds very convincing, as it's designed to be.
But obviously that file should still be generated.
Just to be clear you initially said you tried testing with only one WAN and still didn't see the file correctly populated. But is that not in fact correct? You only see this with two WANs configured for IPv6?
-
@stephenw10 @marcosm Thanks for the replies. As testing this requires an outage, I’m going to have to wait for a window — probably this weekend. I’ll post back anything relevant. Thanks.
-
So far I'm unable to replicate this. The file is written as expected. It feels like you must have some unusual setting in place?
-
@stephenw10 I don’t know what to tell you, right off of factory reset — configure the second WAN interface, then the second LAN interface.
Let me ask this — does the pfSense alter any of its behavior or configuration based on response from what is connected to the WAN or LAN interface ports, or does pfSense do the same thing according to pfSense UI console configuration no matter what network traffic it detects on those ports?
-
The generated file is based only on the interface configuration not the status.
But you are now confirming it only happens with multiple dhcp6 clients configured?
-
@stephenw10 No. The first time it happened, which was the only time IPv6 changes were combined with a switch from ISC to Kea DHCP, it entered a state where no IPv6 change to either WAN or LAN took. The following two times I reset to factory settings, I did not switch to Kea DHCP (I stuck with the default ISC) and I did not make any changes to the main WAN and LAN interfaces, I only experimented with the second WAN and LAN interfaces. My concern at that point was not losing both WAN connections (there are few things that draw an immediate outcry in a household worse than a complete Internet outage.)
-
Right but just to confirm you've only ever seen this on a system with more than one dhcpv6 WAN configured?
-
@stephenw10 That is correct. The adding of the second WAN/LAN was what caused it. I have not encountered this with only one WAN/LAN in play, which is why I ultimately pulled the second WAN/LAN completely and am (for the temporary present) not running it through pfSense.
