NetGate 4100 Boots Old Version After Maintenance Reboot

plfinch

Primary firewall NetGate 4100 was running pfSense 24.03 for a couple months without incident.

After maintenance reboot (rc.reboot) system unexpectedly reverted to a prior boot image running 23.05.1 which was the last version before updating to 24.03.

This old version is also running on the old configs and backups.

GUI reports an Uncaught TypeError and no pages are accessible.

I am currently running on backup firewall. What is my quickest path to recovery of the primary? At a minimum I'd like to get the last config file off it from before the reboot. Better would be to get the correct image booted.

Where do I start?

Peter

stephenw10

If it's running ZFS it probably reverted to the last good Boot Environment after the current one failed to boot for some reason.

If you have access to the CLI still you can use bectl list to see what BEs there are.

plfinch

Here are the boot environments.


[23.05.1-RELEASE][admin@gateway.local.lan]/root: bectl list
BE                          Active Mountpoint Space Created
auto-default-20230727155112 -      -          672M  2023-07-27 11:51
auto-default-20230727160107 -      -          980M  2023-07-27 12:01
auto-default-20240604112843 NR     /          3.56G 2024-06-04 11:28
default                     -      -          2.82G 2024-08-07 15:38
default_20240807153846      -      -          1.04G 2022-06-22 15:32

stephenw10

OK so the default BE is almost certainly what was running 24.03.

You can reactive that using: bectl activate default

If you're connected at the console you should be able to see why it fails to boot.

Another possibility would be to activate one of the older BEs then reupgrade that to 24.03. From there you can restore a config from a different BE to get back to where you were.

plfinch

Yeah, when I saw default wasn't activated I did so and rebooted. The expected 24.03 system booted. I did another reboot and that also came up fine.

[BEGIN DISREGARD]
Navigated around the GUI and everything seems working as it should except for System/Update which reports "Unable to check for updates" with the following log error:

Sep 13 11:17:44 gateway php-fpm[587]: /pkg_mgr_installed.php: The command '/usr/local/sbin/pfSense-repo-setup' returned exit code '1', the output was 'pfSense-repoc-static: failed to fetch the repo data failed to read the repo data. failed to update the repository settings!!!'

I will probably just do a clean reinstall and reload configs in case there are any other missed side effects.
[END]
The update issue resolved after a WAN reset.

I am still unclear as to how default boot environment deactivated and will it happen again?

Thanks for the help!

Peter

stephenw10

Hmm, interesting. I would have expected to see an alert confirming the BE roll back when it did.

Whenever I hit that it's usually because I've broken something completely so the firewall fails to boot entirely resulting in some pretty obvious errors.