Config restore broken in 2.3.2p1 ?
-
Just wondering if this is supposed to work or not, it certainly seems broken to me, which is a bit of a problem as I can't put my new system into service without a viable recovery procedure.
What's happened is that as part of my comissioning checks, I used the GUI Backup/Restore to restore good version of the system config from a couple of days ago. Restore are was 'All' and the file was unencrypted. After reboot I noticed the following:
In the GUI:
pfSense is booting, then packages will be reinstalled in the background. Do not make changes in the GUI until this is complete.On the console:
Waiting for Internet connection to update pkg metadata and finish package reinstallationUpdating pfSense-core repository catalogue...In the GUI under Status, all Logs are EMPTY !
No error conditions are shown in the Dashboard, but no IP forwarding seems to be happening on the firewall.
The default route is present, and I CAN ping the gateway (from the firewall).
Some of this seems similar to what folk have described Redmine bug #6594:
https://redmine.pfsense.org/issues/6594
BUT the title of #6594 is misleading and seems to contradict the description, the fault causes loss of 'internet connectivity'.
UPDATE, whilst writing this, after a delay of ~20minutes, the normal console menu appeared, and IP forwarding is functioning again.
HOWEVER, after getting back into the GUI, I'm seeing System/Package manager/Package Installer asking for "Confirmation Required to reinstall all packages".
FURTHER, there was the following Notice:
Package reinstall process was ABORTED due to lack of internet connectivity @ 2017-01-06 17:10:21.(a). I did not request a package re-installation, only a config restore.
(b). The only connectivity issue was inside pfSense until it sorted itself out after ~20mins.I'd be grateful for any advice, workarounds etc.
-
Packages reinstall after a config restore.
You need internet access for that.
Put WAN on an inside DHCP LAN or some other method of having internet while you do that.
-
Thanks, but there was no problem with the internet connection, the fault I saw involves an IP forwarding failure internal to pfSense. I had no way to 'give it' a WAN connection, it already had one.
Why the IP forwarding should fail alongside the spurious package re-install is naturally beyond me as a humble network admin, but they seem to have the same underlying cause.
To summarise, IP connectivity to/from the 'inside' LANs was OK, and I could ping the gateway from the firewall.
I was able to workaround it as follows (copy/paste from my notes):
UPDATE:
Console times out and menu appears after ~20mins. IP forwarding OK, but messages about package reinstall recur after a further reboot.Eventual workaround:
Under System/Package Manager/Package Installer
there should be a pending confirmation to 'reinstall all packages' (not clear if this is the add-on or base packages - would be nice if this was stated).
Click to confirm, and the task will FAIL with a message like:
Reinstalling pfSense-pkg-OpenBGPD ERROR: It was not possible to determine pfSense-pkg-OpenBGPD remote version Failed >>> Upgrading pfSense-pkg-OpenBGPD... Updating pfSense-core repository catalogue...Once this has failed, reboot again (3rd time), this time the 'package reinstall' messages will not recur. Checking packages installed and available in the GUI works OK, confirmed by 'pkg update' at the CLI. All seems normal again.
I've read that the devs are focussed on 2.4 so a fix for 2.3.2 is unlikely, maybe I should consider enabling the pre-production release if this has been fixed there ?
I would be nice to have some confidence that this has been fixed so any info to that effect would be useful.
PS. Oh, I don't like the sound of software having to be re-installed if I simply want to install a new running config.
It'd be really useful to have any pointers to do work on the config at the CLI. Is that covered on the official guidebook ?
-
Generally works. Looks to me like you have an issue with internet access from the firewall itself or possibly DNS from the firewall itself.
That might be completely different from internet access or DNS from hosts behind the firewall.
-
I appreciate your response, however I'm confident I understood what was going on.
I've been running networks since the 1980's and using unix of one sort or another since the early 90's, so I do have a bit of experience compared with some newbies on here.
I had this on both nodes of an HA pair, and I dealt with each one separately with connectivity working OK via the other node. The only problem was pfSense.
-
Ah. More details.
Was this on the units when they are not CARP master? Do they have public interface addresses or are you trying to game it by using private addresses on the interface with just one public CARP VIP? In that case something upstream would have to NAT for it when it is not CARP master.
I am glad you are so experienced, but your problem is indicative of not having DNS or internet access from the firewall itself at the time you restored the configuration. Or at least from the restored configuration during the package reinstall phase after the reboot.
If this was not the case the packages would have quickly downloaded and installed.
-
I've thought of a possible answer, IF we know (or assume) the scripts that run at restart after a config restore disable IP forwarding whilst they run, AND they also stop from running any add-on packages.
Because these systems also are the recursive nameservers, running the Bind package..
In which case my workaround is to configure each pfSense instance to use the other for recursive name resolution, and run such tasks one at a time.
I still don't like the internet dependency for critical tasks like a firewall config restore. Don't think they though that through properly.. Can this be done from the CLI, leaving the software well alone ?
-
No. Has to have internet access to the package repositories. You can restore and upgrade from the CLI but it generally does the same thing as the GUI.
Yeah, it is what it is. This is where we are at the moment.
-
OK, I suspected so, thanks for the clarification.
-
Just for completeness I should update this thread to say I was in error about the IP forwarding, I've checked the scenario out again, that actually continues OK.
The fault seems purely to be that the config restore script unloads the system nameserver (the Bind package), then barfs when it cannot resolve names.
