PfSense+ 22.05 -> 23.01 Boot Failure after Upgrade

jimp

Also if you get those pkg errors, you might want to force a reinstall of pkg after selecting the previous stable/22.05 branch.

: pkg-static bootstrap -f
: pkg-static update -f

roach32

@steveits you rock! that worked great

roach32

@jimp

many thanks Jim. serial console isn't enabled. at any rate, I ordered a brand new server to replace this unit today since I have been wanting to upgrade to a proper powerful server to go the TNSR route and retire this one workstation-class box that isn't upgrading to 23.01. the 6 others will remain on pfsense +.

Snk

@jimp

I'm also having this problem with Hyper-V, the system won't finish booting.
I left it for more than 2 hours, before returning with the snapshot.
It simply hangs on the message: masks 0x00ff0000. 0x0000ff00. 0x000000ff. 0xff000000
And it doesn't go away for hours.

stephenw10

See: https://forum.netgate.com/topic/177220/latest-pfsense-23-01-beta-freezes-on-hyper-v

Snk

@stephenw10

My installation of pfsense has 15GB of disk, which leads me to believe that there is no space.
I just tried to update again and it failed again.
I'm completely tied hand and foot.

stephenw10

Like it filled at update or was already full?

if you roll back to an earlier snapshot how much is used?

Snk

@stephenw10

This is my gpart status

stephenw10

That doesn't show how much of the drive is in use. I assume you see a lot less used space in the hypervisor?

Snk

@stephenw10

Do you think if I increase the size from 20GB to 80GB it will solve this installation error?

stephenw10

Probably, if it;s simply filling the drive at upgrade.

Snk

@stephenw10

Would you have any tutorial on how to expand the space within pfsense?

jimp

In some cases, touch /root/force_growfs and then rebooting is sufficient. But that depends on which part of the disk is running out of space. That's quick and easy to try.

A clean install of a recent base is best if you can.

Snk

@jimp

That's what I thought... I'm preparing the host to do a clean install of the 23.01 version and later return my backup.

Snk

I did a fresh install straight from version 2.6.0 to version 22.01 and 22.05. When going to s version 23.01 the same problem continues. Locks in masks and that's all... It's difficult.

Snk

Guys, any news? Did you manage to install the 23.01 version normally in Hyper-V or still nothing?

stephenw10

No updates AFAIK. You're probably better posting in the linked thread above where we were looking at it during beta/RC.
This is probably some issue between the boot loader and the environment presented by Hyper-V. But it only affects some instances. Testing here it always booted and ran fine.
We are looking something similar we had to workaround in Azure that may also apply here. Unfortunately the workaround does not.

Steve

Snk

@stephenw10

Hello, thanks!
I'll follow up on the other thread then.
I redid the installation from scratch, but the problem still persists.

ttmcmurry

I'm experiencing a similar upgrade issue, but from pfSense CE 2.6.0 to 23.01, on Hyper-V 2022, VM configuration 10. There was a closed issue in redmine on this that I added more details to:

https://redmine.pfsense.org/issues/13875

I think there are two issues happening, and they seem unrelated, but they are exposed by the boot-time freeze as outlined in this thread:

/etc/hostid isn't been saved/set after a successful upgrade. this is solveable manually after upgrade while pfSense 23.01 is running with a command in the pfSense shell
Even if /etc/hostid is corrected, the VM freezes if being started from a power-off state, in the manner communicated.

I can reproduce this 100% of the time. The only thing that makes pfSense actually start up is resetting the VM several times until it boots.

stephenw10

The hostid error shown at boot is harmless:

can't find '/etc/hostid'

Unless you've found something different?