PfSense+ 22.05 -> 23.01 Boot Failure after Upgrade

stephenw10

See: https://forum.netgate.com/topic/177220/latest-pfsense-23-01-beta-freezes-on-hyper-v

Snk

My installation of pfsense has 15GB of disk, which leads me to believe that there is no space.
I just tried to update again and it failed again.
I'm completely tied hand and foot.

stephenw10

Like it filled at update or was already full?

if you roll back to an earlier snapshot how much is used?

Snk

@stephenw10

This is my gpart status

stephenw10

That doesn't show how much of the drive is in use. I assume you see a lot less used space in the hypervisor?

Snk

@stephenw10

Do you think if I increase the size from 20GB to 80GB it will solve this installation error?

stephenw10

Probably, if it;s simply filling the drive at upgrade.

Snk

@stephenw10

Would you have any tutorial on how to expand the space within pfsense?

jimp

In some cases, touch /root/force_growfs and then rebooting is sufficient. But that depends on which part of the disk is running out of space. That's quick and easy to try.

A clean install of a recent base is best if you can.

Snk

@jimp

That's what I thought... I'm preparing the host to do a clean install of the 23.01 version and later return my backup.

Snk

I did a fresh install straight from version 2.6.0 to version 22.01 and 22.05. When going to s version 23.01 the same problem continues. Locks in masks and that's all... It's difficult.

Snk

Guys, any news? Did you manage to install the 23.01 version normally in Hyper-V or still nothing?

stephenw10

No updates AFAIK. You're probably better posting in the linked thread above where we were looking at it during beta/RC.
This is probably some issue between the boot loader and the environment presented by Hyper-V. But it only affects some instances. Testing here it always booted and ran fine.
We are looking something similar we had to workaround in Azure that may also apply here. Unfortunately the workaround does not.

Steve

Snk

@stephenw10

Hello, thanks!
I'll follow up on the other thread then.
I redid the installation from scratch, but the problem still persists.

ttmcmurry

I'm experiencing a similar upgrade issue, but from pfSense CE 2.6.0 to 23.01, on Hyper-V 2022, VM configuration 10. There was a closed issue in redmine on this that I added more details to:

https://redmine.pfsense.org/issues/13875

I think there are two issues happening, and they seem unrelated, but they are exposed by the boot-time freeze as outlined in this thread:

/etc/hostid isn't been saved/set after a successful upgrade. this is solveable manually after upgrade while pfSense 23.01 is running with a command in the pfSense shell
Even if /etc/hostid is corrected, the VM freezes if being started from a power-off state, in the manner communicated.

I can reproduce this 100% of the time. The only thing that makes pfSense actually start up is resetting the VM several times until it boots.

stephenw10

The hostid error shown at boot is harmless:

can't find '/etc/hostid'

Unless you've found something different?

ttmcmurry

@stephenw10 if that's the case for hostid, understood.

Otherwise, this is a "me too" stating after the upgrade to 23.01, I'm also experiencing a boot failure as described above and in the Redmine post.

stephenw10

OK, also in Hyper-V?

ttmcmurry

@stephenw10 yes

swinn

I ran into this for the first time last night after running a Gen2 VM for many years. I haven't had time to dive into it, but here is what I noticed.

Windows Server 2022 Standard
Gen 2 VM, 4 cores, 4096MB RAM (not dynamic)
2 nics (WAN and LAN)
VM version 10
pfSense+ 23.05

First had an issue when I upgraded from 23.05 to 23.09. When it rebooted, it came up with the error. It sat like this for several hours until I could get to it physically.
In the boot menu, it had "kernel.old" as an option and that would allow it to boot. "kernel" would cause the error and hang.
I installed pfSense 2.7 fresh and after it rebooted, it hung with the error.
I noticed that if I messed around with the boot menu - just pressing keys to look at the options - when I pressed enter to continue booting, it would work. The next reboot would hang again.
Creating a Gen 1 VM worked fine.

I have been running this VM for years without any upgrade issues. The only thing I changed recently was upgrading the VM version from 9 to 10. I have a backup of it where it is still version 9, so I'll try restoring it and doing the upgrade to see if it still hangs.