23.09.a.20230919.1322 boot failure

jaltman

The upgrade via the serial console appeared to go smoothly

Installed packages to be REMOVED:
        pfSense-rc: 23.09.a.20230907.0600

Installed packages to be UPGRADED:
        pfSense: 23.09.a.20230907.0600 -> 23.09.a.20230919.1322 [pfSense]
        pfSense-base: 23.09.a.20230907.0600 -> 23.09.a.20230919.1322 [pfSense-core]
        pfSense-boot: 23.09.a.20230907.0600 -> 23.09.a.20230919.1322 [pfSense-core]
        pfSense-default-config-serial: 23.09.a.20230907.0600 -> 23.09.a.20230919.1322 [pfSense]
        pfSense-kernel-pfSense: 23.09.a.20230907.0600 -> 23.09.a.20230919.1322 [pfSense-core]
        pfSense-pkg-System_Patches: 2.2.5 -> 2.2.6 [pfSense]
        pfSense-repo: 23.09.a.20230907.0600 -> 23.09.a.20230919.1322 [pfSense]
        php82-openssl_x509_crl: 1.3_2 -> 1.3_3 [pfSense]

Number of packages to be removed: 1
Number of packages to be upgraded: 8

The process will require 91 MiB more space.
>>> Downloading pkg...

No packages are required to be fetched.
Integrity check was successful.
>>> Upgrading pfSense-boot...>>> Unmounting /boot/efi... done.
 done.
/usr/local/libexec/pfSense-upgrade: : Permission denied
>>> Removing vital flag from pfSense-rc... done.
>>> Upgrading pfSense kernel...
Checking integrity... done (0 conflicting)
The following 1 package(s) will be affected (of 0 checked):

Installed packages to be UPGRADED:
        pfSense-kernel-pfSense: 23.09.a.20230907.0600 -> 23.09.a.20230919.1322 [pfSense-core]

Number of packages to be upgraded: 1
[1/1] Upgrading pfSense-kernel-pfSense from 23.09.a.20230907.0600 to 23.09.a.20230919.1322...
[1/1] Extracting pfSense-kernel-pfSense-23.09.a.20230919.1322: .......... done
===> Keeping a copy of current kernel in /boot/kernel.old
>>> Removing unnecessary packages... done.
>>> Activating boot environment default... done.
System is going to be upgraded.  Rebooting in 10 seconds.

One package was updated

The following 1 package(s) will be affected (of 0 checked):

Installed packages to be UPGRADED:
        pfSense-pkg-System_Patches: 2.2.5 -> 2.2.6 [pfSense]

Number of packages to be upgraded: 1
[1/1] Upgrading pfSense-pkg-System_Patches from 2.2.5 to 2.2.6...
[1/1] Extracting pfSense-pkg-System_Patches-2.2.6: .......... done
Removing System_Patches components...
Menu items... done.
Loading package instructions...
Deinstall commands... done.
Saving updated package information...
overwrite!

but after starting a few packages

Writing configuration... done.
>>> Updating ldconfig... done.
>>> Removing unnecessary packages... done.
>>> Cleanup pkg cache... done.
 Starting package AWS VPC Wizard...done.
 Starting package IPsec Profile Wizard...done.
 Starting package acme...done.
 Starting package snort...done.
 Starting package iperf...done.
 Starting package Avahi...done.
 Starting package System Patches...done.
 Starting package Netgate Firmware Upgrade...done.
 Starting package pfBlockerNG...done.
 Starting /usr/local/etc/rc.d/pfb_dnsbl.sh...done.
 Starting /usr/local/etc/rc.d/pfb_filter.sh...done.
.: cannot open /Enter full pathname of shell or RETURN for /bin/sh:
# exit
/bin/sh: cannot open /etc/rc: No such file or directory
Enter full pathname of shell or RETURN for /bin/sh:
# exit
/bin/sh: cannot open /etc/rc: No such file or directory
Enter full pathname of shell or RETURN for /bin/sh: exit
#
# reboot

I recovered by booting into the prior image.

jaltman

I should have mentioned that the device is a Negate 4100

Similar experience rebuilding a prior backup unit (no-name 6-NIC), upgrading from 23.05.

Don't have a backup, so I'll resort to staying on 23.05 for now.

Edit: In my case the file "/etc/rc.be_functions.sh" could not be found.

Last six lines of console reads:
Starting syslog...done.
Starting CRON... done.
">>> Removing unnecessary Packages... done."
">>> Cleanup pkg cache... done."
.: cannot open /etc/rc.be_functions.sh: No such file or directory
Enter full Pathname of shell or RETURN for /bin/sh:

webConfigurator is active so GUI access exists until the unit is rebooted.

juanzelli

@jaltman Well, thank you for your sacrifice. I guess I'll be waiting for the next dev build before trying it on my 4100.

w0w

@rschell
This is just BEFORE reboot, after reboot you will have

@jaltman said in 23.09.a.20230919.1322 boot failure:

/bin/sh: cannot open /etc/rc: No such file or directory

at least it was in my case on both lab firewalls and virtual machine too

@rschell said in 23.09.a.20230919.1322 boot failure:

Don't have a backup, so I'll resort to staying on 23.05 for now.

This should be created automatically with boot environments magic.

PatRyan

@jaltman I had the same experience you did upgrading from 23.09.a.20230907 to 23.09.a.20230919 today. 23.09.a.20230907 seemed to be working OK but the upgrade to today's build hung just where yours did.

Snapshot 23.09.a.20230920-1314 boots successfully.

jaltman

@rschell Confirmed.

jimp

There was some breakage on the last few builds. The current build that finished a couple hours ago is good.

If you are stuck at that shell prompt at boot, press enter to start a shell and then do this:

UFS:

# mount -a -t ufs
# cp /etc/rc.pkgsave /etc/rc
# reboot

ZFS:

# mount -u /
# cp /etc/rc.pkgsave /etc/rc
# reboot

If your system is booted and running a snapshot from the last couple days, check that you have /etc/rc and /etc/pfSense-rc before updating or rebooting again. If either one is missing, you can copy them from /etc/rc.pkgsave or /etc/pfSense-rc.pkgsave if those files are present. If you don't have those backup copies, there are other ways to get them, such as by re-extracting the base package and/or pfSense package.

juanzelli

@rschell Also, confirmed good on my 4100. Plus, I think this build is using FreeBSD 14 Beta now

Viper_Rus

Hello.
But my situation is different. On 09.23.a.20230919.1322 everything was fine. Today I launched the update to 09.23.a.20230920-1314 and pfsense did not load.

entered the suggested commands:
mount -u /
cp /etc/rc.pkgsave /etc/rc
reboot

And the relboot was successful. Tell me, now everything is ok, then everything will be fine and nothing else needs to be done?

jimp

@Viper_Rus said in 23.09.a.20230919.1322 boot failure:

Hello.
But my situation is different. On 09.23.a.20230919.1322 everything was fine. Today I launched the update to 09.23.a.20230920-1314 and pfsense did not load.

Your situation is exactly what I mentioned in the last paragraph of my post above -- it booted OK that time but the file was missing, so the next reboot would fail even if it was a regular reboot, not just an upgrade.

And the relboot was successful. Tell me, now everything is ok, then everything will be fine and nothing else needs to be done?

You should be OK now if it's booting as expected. Anything else lingering that is wrong would be corrected by the next upgrade you do.

juanzelli

@juanzelli Subsequent bumps up to 23.09.a.20230922.1539 have been fine on my 4100