Upgrading pfSense 2.4.5.1 to 2.5.1 unknown file system

Exocomp

I'm upgrading from 2.4.5.1 to 2.5.1.

I'm using the web interface which is successful:

Installed packages to be UPGRADED:
	pfSense-rc: 2.4.5_1 -> 2.5.1 [pfSense-core]

Number of packages to be upgraded: 1
[1/1] Upgrading pfSense-rc from 2.4.5_1 to 2.5.1...
[1/1] Extracting pfSense-rc-2.5.1: ...... done
>>> Upgrading pfSense kernel... 
Checking integrity... done (0 conflicting)
The following 1 package(s) will be affected (of 0 checked):

Installed packages to be UPGRADED:
	pfSense-kernel-pfSense: 2.4.5_1 -> 2.5.1 [pfSense-core]

Number of packages to be upgraded: 1

The process will require 14 MiB more space.
[1/1] Upgrading pfSense-kernel-pfSense from 2.4.5_1 to 2.5.1...
[1/1] Extracting pfSense-kernel-pfSense-2.5.1: .......... done
===> Keeping a copy of current kernel in /boot/kernel.old
>>> Removing unnecessary packages... done.
System is going to be upgraded.  Rebooting in 10 seconds.
>>> Unlocking package pkg... done.
Success

However, after first boot I get the following:

I'm then stuck on mountroot.

Any help would be appreciated.

jimp

It's possible the ZFS module isn't loaded somehow.

Reboot and at the ASCII boot menu choose the option to go to the loader prompt, and at the OK prompt, type the following commands:

load opensolaris
load zfs
boot

Those should be automatically added to /boot/loader.conf but if you don't see any ZFS references in there, create or edit /boot/loader.conf.local and add:

opensolaris_load="YES"
zfs_load="YES"

Exocomp

@jimp thanks for the help, the load commands fail:

What command/interface can I use to edit /boot/loader.conf.local?

jimp

If they won't load from there, they wouldn't load from loader.conf.local.

Might be a disk issue. ZFS generally doesn't have problems like UFS with the filesystem becoming corrupt so I doubt it's related to that, but it is odd that it doesn't find those files from the loader.

The fastest way to recover is to reinstall and restore a backup at this point.

Exocomp

@jimp it's a vm and I recovered from a backup.

However, I've tried several times to upgrade which results in the same issue, looks like a bug.

Exocomp

@jimp I tried another box and it has the same issue.

jimp

Curious. I've upgraded several ZFS systems from 2.4.5-p1 to 2.5.x without issue in the past.

How were these installed? Is there just one disk or a mirror?

Exocomp

@jimp interesting. It's just one disk.

During original installation (v2.4.1) my notes show I selected

Auto (ZFS) Guided Root-on-ZFS
Leave everything as defaults
Stripe - No Redundancy

jimp

It's possible there was some difference in how the filesystem was made back on 2.4.1 that is part of the issue. Though I have a couple long-lived ZFS installs I'm not sure any were from that long ago at the moment since most get reinstalled now and then as they're in my lab.

What shows up in zpool status on these?

Exocomp

@jimp Here is the result:

zpool status

jimp

Maybe try updating the pool it to get around those errors:

zpool upgrade -a
gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 da0

Though I'd still say your safest bet is to reinstall and restore the configuration. You should even be able to use the "Recover config.xml" option in the installer to make it simpler.

Exocomp

@jimp after running those commands its not able to boot at all.

As you mentioned sounds like a complete reinstall is in order. Bummer.

jimp

I just installed a VM with 2.4.1 ZFS and upgraded to 2.4.5-p1 then 2.5.1 and it went without a hitch. There must be something off with those. Not sure what it might be, though.

Exocomp

@jimp I'll do some more testing out of curiosity.

v2.4.1
SHA256: 738C47B549284B158CC3F174288A5B47AD6F112728D5BAFD11412A1B77DA32F9

Above was the sha256 of v2.4.1 I installed from.

I don't have sha256 of v2.4.5.1 (since I used the web installer), can you provide it here and I'll download and do the same test as you but with my configuration.

Exocomp

@jimp I reinstalled 2.4.1 just to test, how I can I upgrade it to 2.4.5.1, is that possible through the system update web interface? Can you please direct me to the steps to do so.

jimp

It is kind of tricky due to the age of it, the easiest way is to go to a shell prompt and run pfSense-upgrade -c a few times in a row until it offers 2.4.5-p1, you may also need to pick the 2.4.5 update branch after running it a couple times.

The installer hashes wouldn't have anything to do with this, if it was wrong it would be much more broken.

Exocomp

@jimp I'm able to reproduce the issue with a fresh install of v2.4.5.1 then upgrade to v2.5.1.

I installed using pfSense-CE-2.4.5-RELEASE-p1-amd64.iso (sha256 0a09a7748419c86c665eb8d908f584e96d54859aa13f4eeb175a60548c70e228).

Then I used the web interface to upgrade to 2.5.1 which was successful then on first boot I get the same issue.

Is this the same way you're upgrading?

Exocomp

@jimp I did the same test but using ufs and the upgrade and boot up is fine.

So definitely related to zfs and fresh install of 2.4.5.1 to 2.5.1, at least for me.

Exocomp

@jimp I tried a fresh install of v2.5.1 and now I get a boot loop.

The following is broken on v2.5.1:

VM: Hyper-V Gen2 ZFS

Let me know what you need to file this as a bug.

I noticed others have reported a similar issue: https://forum.netgate.com/topic/161130/upgrading-to-2-5-0-on-hyperv-boot-loops

jimp

Probably nothing for us to do there, it's likely between FreeBSD and Hyper-V.

I have no problems with ZFS and that scenario on KVM or ESX.