Proxmox Backup Causes pfSense To Hang On Boot
-
TL;DR; Anybody experienced pfSense v2.7.0 hang on boot whilst loading Intel Ethernet Controller I226-V?
Problem: Every time my Proxmox all-VMs backup runs, pfSense hangs on (re)boot.
System: pfSense CE v2.7.0 installed as Proxmox v8.0.4 Virtual Machine with Proxmox Linux 6.2.16-10-pve kernel. 1GB RAM. 32GB SSD boot drive. WAN: Intel Ethernet NIC I226-V Port 1 PCI pass-through. LAN: Intel Ethernet NIC I226-V Ports 2-4 members of vmbr0 paravirtualised switch.
Behaviour: Backup starts, pfSense VM shuts down, backup successful, pfSense reboots <<ERROR>>, pfSense hangs on reboot. Normal shutdowns, reboots work fine otherwise, this only happens when a Proxmox backup is initiated.
System Log: full log attached, summary below...
... ... ... vgapci0: Boot video device virtio_pci0: <VirtIO PCI (legacy) Balloon adapter> port 0xf000-0xf03f mem 0xfd600000-0xfd603fff irq 11 at device 3.0 on pci0 vtballoon0: <VirtIO Balloon Adapter> on virtio_pci0 pcib1: <ACPI PCI-PCI bridge> mem 0xfea55000-0xfea550ff irq 10 at device 5.0 on pci0 pci1: <ACPI PCI bus> on pcib1 virtio_pci1: <VirtIO PCI (legacy) SCSI adapter> port 0xe000-0xe03f mem 0xfe600000-0xfe600fff,0xfd400000-0xfd403fff irq 10 at device 1.0 on pci1 vtscsi0: <VirtIO SCSI Adapter> on virtio_pci1 virtio_pci2: <VirtIO PCI (legacy) Console adapter> port 0xf040-0xf07f mem 0xfea56000-0xfea56fff,0xfd604000-0xfd607fff irq 11 at device 8.0 on pci0 vtcon0: <VirtIO Console Adapter> on virtio_pci2 igc0: <Intel(R) Ethernet Controller I226-V> mem 0xfe800000-0xfe8fffff,0xfea50000-0xfea53fff irq 11 at device 16.0 on pci0 <<<PFSENSE HANGS>>>Recovery: Restarting VM results in same issue, again and again, only recovery option is to power recycle the NUC.
Settings:
Has anybody else experienced this issue?
Any advice/guidance on troubleshooting this issue would be warmly received.
Thank you.
-
@tictag
Did you install the qemu-agent? -
@viragomann Yes.
-
I have now upgraded to pfSense+ v23.05.1-RELEASE in the hope that this would fix the issue, but there's no change. This happens in both pfSense CE and pfSense+.
-
does this still happen if you set things up to boot using uefi?
https://docs.netgate.com/pfsense/en/latest/recipes/virtualize-proxmox-ve.html#booting-uefi -
@ElTigreVerde said in Proxmox Backup Causes pfSense To Hang On Boot:
does this still happen if you set things up to boot using uefi?
Well, that was a 'journey'. No matter what I tried, I could not get pfSense to boot by changing its hardware configuration (changing BIOS, adding EFI disk, changing display etc). I finally assumed that pfSense must be installed on a UEFI bios for it to work properly. So I did this and ...
... it worked perfectly!!
I guess that fixes it for me but the fault still remains for installations with the default BIOS.
Would somebody from Netgate comment: is it still worth pursuing a resolution for this issue?
-
I am from Netgate ;) Glad you got it working
It was something specific to how you had it setup because I just migrated to a virtualized setup and was originally booting BIOS and didn't experience the same. Look through the proxmox documentation closely, there are some caveats in storage and drivers about what filesystem the vm disk is stored on and whether you are using raw format, etc... proxmoxsrvr/pve-docs/chapter-qm.html#qm_hard_disk
-
@ElTigreVerde OK, thank you.
In conclusion, I believe at the root of this issue is the Intel I226-V NIC, because pfSense (or rather FreeBSD) hung loading that driver every time [a proxmox backup was initiated]. For those reading this in the future, I can't offer you a fix, but can offer you a workaround: install pfSense on a UEFI BIOS.
-
so I haven't found this to be concrete, but intel says i226 is pcie3.1 interface (https://www.intel.com/content/www/us/en/products/details/ethernet/gigabit-network-adapters/i226-network-adapters/products.html) and being more familiar with pcie3.1 as something you see on SSD's (ala NVME) there may not be any BIOS implementation of the hardware and driver, any motherboard produced recently that would have that NIC onboard would be UEFI and use CSM if it needed BIOS to boot. a rabbit hole for another time...
-
Hi,
Thanks for sharing that issue. I've exactly the same. Took long time to define the issue.
So, on Proxmox 8 at least, when installing pfsense in a standard way without UEFI, we are able to start the firewall and so on. issue is that when restarting on Proxmox 8, pfsense doesn't restart (note that if trying multiple time, seems some of the versions may restart. So, consider it also as a random issue).
Point of attention may be that it is working fine under Proxmox 6. Absolutely no issues with that version.
