Pfsense config becomes randomly corrupted on change
-
Good evening,
I have a weird issue I am not sure how to resolve, my pfsense comfig is fairly stable and has a range of standard features but then plugins like Tailscale, snort, Telegraf et al.
For some reason, when I make changes to settings in pfsense. (Could be a plugin, or even just a firewall or alias), something happens to the config file, and on next restart of the machine it loses all of its settings and wants me to reconfigure the LAN and WAN settings.
My resolution is to go back and mount the cf, and restore a few backups earlier which typically fixes it, but it’s hard to know what the issue is going to happen (for example it did it to me today, but I made changes yesterday so lots of the backups had the corruption).
Wondering what might be causing this and how I can resolve this as it has a big impact on family approval factor when on a restart the network ends up being down for 30 minutes instead of just rebooting.
-
Do you see any error messages printed to the console that might give a clue as to which part of the config is corrupt? Hard to guess what the issue might be without some hints via an error message.
The first and most likely possibility is failing hardware (eMMC card or disk drive). But there could be a faulty section of the
config.xmlfile that gets written, but hard to know that without either actually examining a "corrupt"config.xmlfile or perhaps getting a clue via any error messages logged to the console or system log.What type of hardware are you running pfSense on. Is it a Netgate appliance or a generic whitebox? And are you using free pfSense CE or the paid pfSense Plus version?
Are either of your NICs a USB device? Those are not always recognized in the same order during hardware initialization and that could lead to a situation like you describe. When pfSense asks you to reassign the interfaces, that usually means either the type of hardware device changed (that should not happen unless you install new cards) or some installed device failed to initialize during POST and thus the numbering and order of the NICs changes.
-
@bmeeks thanks for the response, I had two seperate issues one of which I resolved, the other I am not so sure on so I have removed all the plugins and will see if that resolves it.
First issue was a stupid one. I had created an Interface for Tailscale to add a static route, but since Tailscale was not running when it tried to add the interface on reboot it created an error and triggered the interface assignment.
The second issue is the config, it’s possible that it is the drive which is fairly old so I will try an inferential drive and see if that resolves the issue.
Thanks!
-
@TravisH said in Pfsense config becomes randomly corrupted on change:
First issue was a stupid one. I had created an Interface for Tailscale to add a static route, but since Tailscale was not running when it tried to add the interface on reboot it created an error and triggered the interface assignment.
Yep, that would do it. When the number, type, or name of interfaces discovered during a hardware scan at boot does not match what's stored in
config.xmlthat triggers the "reassign interfaces" logic.@TravisH said in Pfsense config becomes randomly corrupted on change:
The second issue is the config, it’s possible that it is the drive which is fairly old so I will try an inferential drive and see if that resolves the issue.
There was a bug with some settings creating bogus XML entries in the config, but that was fixed so far as I recall in pfSense Plus 24.11, and I don't really recall it manifesting in pfSense CE. But it's possible a version of the bug exists in CE and manifests in certain situations when the planets and stars align. Knowing the exact error that is happening would help narrow down the cause. Should be something printed in the system log (if you can get to it) or else it should scroll a message on the console during boot.
-
Just for reference the tailscale interface should never be assigned.
-
@stephenw10 Thanks, I had a weird issue where I could not route from a client inside the network to an external tailscale machine, I could route from the firewall / pfsense machine fine if I did a tracert, but if I did that from a client inside the network that was not the pfsense machine it would not work.
Assigning the network, and setting a static route allowed it to work, until next restart lol :(
-
Hmm, odd. The routes should be added by the daemon when it connects as long as they are defined in tailscale as I understand it.
But, yes, the tailscale interface is not expected to ever be assigned. It is not bypassed by the interfaces check at boot so will throw an error.
