pfSense 2.6 will not reboot in Proxmox :(

MrPete

@stephenw10
One is i7-3770 (giving 4 cores; has AES-NI, Active according to pfSense)
The other is i7-4770 (similar but faster)

stephenw10

Hmm, nope still reboots fine here with:
Screenshot from 2022-03-10 01-36-19.png

Maybe you have a non-default option?
Screenshot from 2022-03-10 01-36-40.png

qemu guest agent maybe? Seems most likely...

MrPete

@stephenw10 I definitely use qemu-guest-agent.

Just ran a set of tests:
..
BIOS: reboots fine, shuts down fine. (Screen is 25 lines ;) ). Either i440 or q35

UEFI: does not reboot, either i440 or q35. With i440 I do see "Rebooting..." but it doesn't.

Will check options... I know for sure I am setting UUID -- started out with identical UUID to primary CARP, which caused other trouble, so I set a different one.

I had OS type Linux (auto set for recent). Reset to "Other"... did not help.
Yes I had q-g-a...

stephenw10

Can you disable/remove the qemu agent as a test?

MrPete

@stephenw10

BUMMER. OK, duplicating close to yours, but with the one change I know breaks mine:

i440FX
UEFI (this breaks it for me)
pkg remove qemu-guest-agent**

Still no reboot.

**Note: don't know how it does it, but even though rclocal has enable q-g-e YES,... by disabling in Proxmox, when I looked in pf, it was NOT running. Not trusting that... I completely uninstalled it.

So:

On 2.5.2 I have had zero issues with this aspect of my config. I was running ZFS.in 2.5.2
I reinstalled 2.6 into the exact same VM.
On 2.6, on my host, if I use UEFI then pfSense will not reboot

I will test tomorrow to see if VM running BIOS is a viable workaround for my situation for now.

If you have further testing ideas, I'm all ears. Will do it early AM tomorrow. Now, gotta run. THANK YOU!!!!

stephenw10

What version of Proxmox are you running?

MrPete

@stephenw10

root@pve1:~# pveversion
pve-manager/7.1-10/6ddebafe (running kernel: 5.15.12-1-pve)

stephenw10

Hmm, OK I'm still running 6.4 but I know others here are running 7 and not hitting this.

Well I would be trying to replicate this on a clean install with a basic setup on your host. It's either something in the VM setup of the pfSense config and that would rule one of those out.

Steve

MrPete

@stephenw10 I'm on a close-to-clean install, as this was full reinstall for 2.6. Of course I reloaded my config.xml

I can do that in chunks, perhaps...

Plain install, no config at all. See if even that fails
Install, load config, no packages
Then start making smaller changes if we haven't nailed it.

MrPete

@stephenw10
Several lessons learned so far. This is from installing the 2.6 iso, using as "default" as I could, other than setting UEFI:

See screen grabs below for working almost-default settings.

ISO permissions must be set to 654 (Group executable) or it can't boot.
While your VM uses type "Other", that produced a bad VM for me (EFI partition too small, and would not boot for install.) Setting up VM at least initially as "Linux" (which it picks based on the ISO) produces better settings.
WIth UEFI, un-check "Pre-Enroll Keys" to avoid the Windows-style IOS security check, which will fail.

That's enough for install to work. :)

The good news: This setup does reboot.
The bad news: modifying my plain test install+reconfig setup to exactly match, does NOT reboot.

So on to phase two testing but no time today (during normal hours at least.)
pf2.6 proxmox uefi ok-1.jpg
pf2.6 proxmox uefi ok-2.jpg

stephenw10

Hmm, interesting.
I didn't do anything speci8al with the ISO. Simply picked the image for the drive.
I have tested both Other and Linux, both worked for me with UEFI.
I guess 'Pre-Enroll Keys' is a new option in 7, I don't see it in 6.4.

Steve

MrPete

@stephenw10 That makes sense.

One thing that was different: if I choose "Other" then the EFI partition is WAY too small for Linux.

More tests coming...

stephenw10

Yeah, that's good to know. This info is sure to help someone else who hits this.

MrPete

@stephenw10
I've run a nice starting set of additional tests, and ran into a roadblock.

For my initial 2.6 full reinstall,

I had a usb-bootable version of 2.6.
on the same usb, i placed my config.xml on the FAT32 partition
pfSense auto-installed that config during install

This time, I have 2.6 on a virtual DVD drive, and config.xml on a usb with a single FAT32 partition

The config is seemingly seen at boot (talks about validating)
YET is not loaded. At least not the LAN/VLAN/etc configuration

Is there a way to get the config to install? Without the basic lan setup, i can't even get to the gui :(

stephenw10

If it's in the root of a FAT32 formatted USB drive it should get pulled in and any boot:
https://docs.netgate.com/pfsense/en/latest/backup/restore-during-install.html#restore-using-the-external-configuration-locator-ecl

It will show 'config found' in the boot logs when that happens.

I've never tried doing that in Proxmox but it should be identical as long as the drive appears as daX.

Steve

MrPete

@stephenw10
Huh.
I'm doing this testing on my secondary CARP...
I went to shutdown the "real" secondary this AM to do some more testing on various vm's.

This time, it actually halted! I was a bit shocked ;)

Some more testing on the "real" one:

From the shell menu, option 6 to halt WORKS fully; option 5 to reboot gets stuck at rebooting.
From shell: if I do shutdown -h now it shuts down to a point of saying "press any key to reboot"... when I press a key, I get "rebooting..." and it is stuck.
Ahh.. option 5 and "reroot" works fully.

Not sure what is the difference.

stephenw10

Mmm, that seems expected. Opyion 6 form the menu would be shutdown -p now. '-h' just halts and seemingly fails to reboot like a normal reboot does.
The reroot option doesn't reload the kernel etc so doesn't touch any ACPI power functions. I would expect that to work.

Steve