Hetzner KVM missing lan after upgrading to CE 2.8.0.b.20250414.1837
-
This is a test ce 2.7.2 install on hetzner cloud, which is kvm based.
Not much to tune regarding kvm options.
Did an upgrade from 2.7.2 to 2.8 beta. After upgrading, pf entered network configuration mode, because lan interface went missing.Comparing the two boot logs I see
pci7: <ACPI PCI bus> on pcib7 virtio_pci6: <VirtIO PCI (modern) Network adapter> mem 0xfdc80000-0xfdc80fff,0xe040000000-0xe040003fff irq 22 at device 0.0 on pci7 vtnet1: <VirtIO Networking Adapter> on virtio_pci6 vtnet1: Ethernet address: 86:00:00:c7:aa:ea vtnet1: netmap queues/slots: TX 1/256, RX 1/128 000.000087 [ 452] vtnet_netmap_attach vtnet attached txq=1, txd=256 rxq=1, rxd=128 pcib8: <ACPI PCI-PCI bridge> mem 0xfea18000-0xfea18fff irq 22 at device 2.7 on pci0 pci8: <ACPI PCI bus> on pcib8 pcib9: <ACPI PCI-PCI bridge> mem 0xfea19000-0xfea19fff irq 23 at device 3.0 on pci0 pci9: <ACPI PCI bus> on pcib9
Completely missing on 2.8 beta, thus no lan. However virtio vti0 is there, and WORKS!!!
2.7.2boot
Copyright (c) 1992-2023 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 14.0-CURRENT amd64 1400094 #1 RELENG_2_7_2-n255948-8d2b56da39c: Wed Dec 6 20:45:47 UTC 2023 root@freebsd:/var/jenkins/workspace/pfSense-CE-snapshots-2_7_2-main/obj/amd64/StdASW5b/var/jenkins/workspace/pfSense-CE-snapshots-2_7_2-main/sources/FreeBSD-src-RELENG_2_7_2/amd64.amd64/sys/pfSense amd64 FreeBSD clang version 16.0.6 (https://github.com/llvm/llvm-project.git llvmorg-16.0.6-0-g7cbf1a259152) VT(vga): text 80x25 CPU: AMD EPYC Processor (2495.45-MHz K8-class CPU) Origin="AuthenticAMD" Id=0x830f10 Family=0x17 Model=0x31 Stepping=0 Features=0x1783fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE,SSE2,HTT> Features2=0xfef83203<SSE3,PCLMULQDQ,SSSE3,FMA,CX16,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND,HV> AMD Features=0x2e500800<SYSCALL,NX,MMX+,FFXSR,Page1GB,RDTSCP,LM> AMD Features2=0xc003f3<LAHF,CMP,CR8,ABM,SSE4A,MAS,Prefetch,OSVW,Topology,PCXC> Structured Extended Features=0x219c01a9<FSGSBASE,BMI1,AVX2,SMEP,BMI2,RDSEED,ADX,SMAP,CLFLUSHOPT,CLWB,SHA> Structured Extended Features2=0x400004<UMIP,RDPID> Structured Extended Features3=0x20000000<ARCH_CAP> XSAVE Features=0x7<XSAVEOPT,XSAVEC,XINUSE> AMD Extended Feature Extensions ID EBX=0x100d205<CLZERO,XSaveErPtr,WBNOINVD,IBPB,IBRS,STIBP,SSBD> Hypervisor: Origin = "KVMKVMKVM" real memory = 8388608000 (8000 MB) avail memory = 8024604672 (7652 MB) Event timer "LAPIC" quality 600 ACPI APIC Table: <BOCHS BXPC > FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs FreeBSD/SMP: 1 package(s) x 4 core(s) random: registering fast source Intel Secure Key RNG random: fast provider: "Intel Secure Key RNG" random: unblocking device. ioapic0 <Version 1.1> irqs 0-23 Launching APs: 2 3 1 TCP_ratelimit: Is now initialized wlan: mac acl policy registered ipw_bss: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw.LICENSE. ipw_bss: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf. module_register_init: MOD_LOAD (ipw_bss_fw, 0xffffffff807475a0, 0) error 1 ipw_ibss: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw.LICENSE. ipw_ibss: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf. module_register_init: MOD_LOAD (ipw_ibss_fw, 0xffffffff80747650, 0) error 1 ipw_monitor: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw.LICENSE. ipw_monitor: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf. module_register_init: MOD_LOAD (ipw_monitor_fw, 0xffffffff80747700, 0) error 1 iwi_bss: You need to read the LICENSE file in /usr/share/doc/legal/intel_iwi.LICENSE. iwi_bss: If you agree with the license, set legal.intel_iwi.license_ack=1 in /boot/loader.conf. module_register_init: MOD_LOAD (iwi_bss_fw, 0xffffffff80765180, 0) error 1 iwi_ibss: You need to read the LICENSE file in /usr/share/doc/legal/intel_iwi.LICENSE. iwi_ibss: If you agree with the license, set legal.intel_iwi.license_ack=1 in /boot/loader.conf. module_register_init: MOD_LOAD (iwi_ibss_fw, 0xffffffff80765230, 0) error 1 iwi_monitor: You need to read the LICENSE file in /usr/share/doc/legal/intel_iwi.LICENSE. iwi_monitor: If you agree with the license, set legal.intel_iwi.license_ack=1 in /boot/loader.conf. module_register_init: MOD_LOAD (iwi_monitor_fw, 0xffffffff807652e0, 0) error 1 random: entropy device external interface kbd1 at kbdmux0 WARNING: Device "spkr" is Giant locked and may be deleted before FreeBSD 14.0. netgate0: <KVM Guest> kvmclock0: <KVM paravirtual clock> Timecounter "kvmclock" frequency 1000000000 Hz quality 975 kvmclock0: registered as a time-of-day clock, resolution 0.000001s vtvga0: <VT VGA driver> smbios0: <System Management BIOS> at iomem 0xf5270-0xf528e smbios0: Version: 2.8, BCD Revision: 2.8 acpi0: <BOCHS BXPC> acpi0: Power Button (fixed) cpu0: <ACPI CPU> on acpi0 atrtc0: <AT realtime clock> port 0x70-0x77 irq 8 on acpi0 atrtc0: registered as a time-of-day clock, resolution 1.000000s Event timer "RTC" frequency 32768 Hz quality 0 hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 100000000 Hz quality 950 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x608-0x60b on acpi0 pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0 pci0: <ACPI PCI bus> on pcib0 virtio_pci0: <VirtIO PCI (modern) GPU adapter> mem 0xfd000000-0xfd7fffff,0xe120000000-0xe120003fff,0xfea10000-0xfea10fff irq 21 at device 1.0 on pci0 pcib1: <ACPI PCI-PCI bridge> mem 0xfea11000-0xfea11fff irq 22 at device 2.0 on pci0 pci1: <ACPI PCI bus> on pcib1 virtio_pci1: <VirtIO PCI (modern) Network adapter> mem 0xfe880000-0xfe880fff,0xe100000000-0xe100003fff irq 22 at device 0.0 on pci1 vtnet0: <VirtIO Networking Adapter> on virtio_pci1 vtnet0: Ethernet address: 96:00:03:e3:ce:49 vtnet0: netmap queues/slots: TX 1/256, RX 1/128 000.000077 [ 452] vtnet_netmap_attach vtnet attached txq=1, txd=256 rxq=1, rxd=128 pcib2: <ACPI PCI-PCI bridge> mem 0xfea12000-0xfea12fff irq 22 at device 2.1 on pci0 pci2: <ACPI PCI bus> on pcib2 xhci0: <XHCI (generic) USB 3.0 controller> mem 0xfe600000-0xfe603fff irq 22 at device 0.0 on pci2 xhci0: 32 bytes context size, 64-bit DMA usbus0 on xhci0 usbus0: 5.0Gbps Super Speed USB v3.0 pcib3: <ACPI PCI-PCI bridge> mem 0xfea13000-0xfea13fff irq 22 at device 2.2 on pci0 pci3: <ACPI PCI bus> on pcib3 virtio_pci2: <VirtIO PCI (modern) Console adapter> mem 0xfe400000-0xfe400fff,0xe0c0000000-0xe0c0003fff irq 22 at device 0.0 on pci3 vtcon0: <VirtIO Console Adapter> on virtio_pci2 pcib4: <ACPI PCI-PCI bridge> mem 0xfea14000-0xfea14fff irq 22 at device 2.3 on pci0 pci4: <ACPI PCI bus> on pcib4 virtio_pci3: <VirtIO PCI (modern) Balloon adapter> mem 0xe0a0000000-0xe0a0003fff irq 22 at device 0.0 on pci4 vtballoon0: <VirtIO Balloon Adapter> on virtio_pci3 pcib5: <ACPI PCI-PCI bridge> mem 0xfea15000-0xfea15fff irq 22 at device 2.4 on pci0 pci5: <ACPI PCI bus> on pcib5 virtio_pci4: <VirtIO PCI (modern) Entropy adapter> mem 0xe080000000-0xe080003fff irq 22 at device 0.0 on pci5 pcib6: <ACPI PCI-PCI bridge> mem 0xfea16000-0xfea16fff irq 22 at device 2.5 on pci0 pci6: <ACPI PCI bus> on pcib6 virtio_pci5: <VirtIO PCI (modern) SCSI adapter> mem 0xfde00000-0xfde00fff,0xe060000000-0xe060003fff irq 22 at device 0.0 on pci6 vtscsi0: <VirtIO SCSI Adapter> on virtio_pci5 pcib7: <ACPI PCI-PCI bridge> mem 0xfea17000-0xfea17fff irq 22 at device 2.6 on pci0 pci7: <ACPI PCI bus> on pcib7 virtio_pci6: <VirtIO PCI (modern) Network adapter> mem 0xfdc80000-0xfdc80fff,0xe040000000-0xe040003fff irq 22 at device 0.0 on pci7 vtnet1: <VirtIO Networking Adapter> on virtio_pci6 vtnet1: Ethernet address: 86:00:00:c7:aa:ea vtnet1: netmap queues/slots: TX 1/256, RX 1/128 000.000087 [ 452] vtnet_netmap_attach vtnet attached txq=1, txd=256 rxq=1, rxd=128 pcib8: <ACPI PCI-PCI bridge> mem 0xfea18000-0xfea18fff irq 22 at device 2.7 on pci0 pci8: <ACPI PCI bus> on pcib8 pcib9: <ACPI PCI-PCI bridge> mem 0xfea19000-0xfea19fff irq 23 at device 3.0 on pci0 pci9: <ACPI PCI bus> on pcib9 isab0: <PCI-ISA bridge> at device 31.0 on pci0 isa0: <ISA bus> on isab0 ahci0: <Intel ICH9 AHCI SATA controller> port 0xc040-0xc05f mem 0xfea1a000-0xfea1afff irq 16 at device 31.2 on pci0 ahci0: AHCI v1.00 with 6 1.5Gbps ports, Port Multiplier not supported ahcich0: <AHCI channel> at channel 0 on ahci0 ahcich1: <AHCI channel> at channel 1 on ahci0 ahcich2: <AHCI channel> at channel 2 on ahci0 ahcich3: <AHCI channel> at channel 3 on ahci0 ahcich4: <AHCI channel> at channel 4 on ahci0 ahcich5: <AHCI channel> at channel 5 on ahci0 acpi_syscontainer0: <System Container> on acpi0 acpi_syscontainer1: <System Container> port 0xcd8-0xce3 on acpi0 acpi_syscontainer2: <System Container> port 0x620-0x62f on acpi0 acpi_syscontainer3: <System Container> port 0xcc0-0xcd7 on acpi0 uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0 atkbd0: <AT Keyboard> irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] psm0: <PS/2 Mouse> irq 12 on atkbdc0 psm0: [GIANT-LOCKED] WARNING: Device "psm" is Giant locked and may be deleted before FreeBSD 14.0. psm0: model IntelliMouse Explorer, device ID 4 orm0: <ISA Option ROM> at iomem 0xe7800-0xeffff pnpid ORM0000 on isa0 vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff pnpid PNP0900 on isa0 attimer0: <AT timer> at port 0x40 on isa0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 Timecounters tick every 10.000 msec ZFS filesystem version: 5 ZFS storage pool version: features support (5000) Trying to mount root from zfs:pfSense/ROOT/default []... ugen0.1: <(0x1b36) XHCI root HUB> at usbus0 uhub0 on usbus0 uhub0: <(0x1b36) XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0 uhub0: 8 ports with 8 removable, self powered Root mount waiting for: usbus0 CAM ugen0.2: <QEMU QEMU USB Tablet> at usbus0 uhid0 on uhub0 uhid0: <QEMU QEMU USB Tablet, class 0/0, rev 2.00/0.00, addr 1> on usbus0 Root mount waiting for: CAM Root mount waiting for: CAM Root mount waiting for: CAM Root mount waiting for: CAM Root mount waiting for: CAM Root mount waiting for: CAM Root mount waiting for: CAM Root mount waiting for: CAM da0 at vtscsi0 bus 0 scbus0 target 0 lun 0 da0: <QEMU QEMU HARDDISK 2.5+> Fixed Direct Access SPC-3 SCSI device da0: 300.000MB/s transfers da0: Command Queueing enabled da0: 156252MB (320004096 512 byte sectors) cd0 at ahcich0 bus 0 scbus1 target 0 lun 0 cd0: <QEMU QEMU DVD-ROM 2.5+> Removable CD-ROM SCSI device cd0: Serial Number QM00001 cd0: 150.000MB/s transfers (SATA 1.x, UDMA5, ATAPI 12bytes, PIO 8192bytes) cd0: Attempt to query device size failed: NOT READY, Medium not present CPU: AMD EPYC Processor (2495.31-MHz K8-class CPU) Origin="AuthenticAMD" Id=0x830f10 Family=0x17 Model=0x31 Stepping=0 Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT> Features2=0xfef83203<SSE3,PCLMULQDQ,SSSE3,FMA,CX16,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND,HV> AMD Features=0x2e500800<SYSCALL,NX,MMX+,FFXSR,Page1GB,RDTSCP,LM> AMD Features2=0xc003f3<LAHF,CMP,CR8,ABM,SSE4A,MAS,Prefetch,OSVW,Topology,PCXC> Structured Extended Features=0x219c01a9<FSGSBASE,BMI1,AVX2,SMEP,BMI2,RDSEED,ADX,SMAP,CLFLUSHOPT,CLWB,SHA> Structured Extended Features2=0x400004<UMIP,RDPID> Structured Extended Features3=0x20000000<ARCH_CAP> XSAVE Features=0x7<XSAVEOPT,XSAVEC,XINUSE> AMD Extended Feature Extensions ID EBX=0x100d205<CLZERO,XSaveErPtr,WBNOINVD,IBPB,IBRS,STIBP,SSBD> Hypervisor: Origin = "KVMKVMKVM"
2,8 beta boot
Copyright (c) 1992-2024 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 15.0-CURRENT #0 RELENG_2_8_0-n256077-f05c84494d5f: Mon Apr 14 19:20:19 UTC 2025 root@freebsd:/var/jenkins/workspace/pfSense-CE-snapshots-2_8_0-main/obj/amd64/660qS3Wa/var/jenkins/workspace/pfSense-CE-snapshots-2_8_0-main/sources/FreeBSD-src-RELENG_2_8_0/amd64.amd64/sys/pfSense amd64 FreeBSD clang version 19.1.5 (https://github.com/llvm/llvm-project.git llvmorg-19.1.5-0-gab4b5a2db582) VT(vga): text 80x25 CPU: AMD EPYC Processor (2495.45-MHz K8-class CPU) Origin="AuthenticAMD" Id=0x830f10 Family=0x17 Model=0x31 Stepping=0 Features=0x1783fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE,SSE2,HTT> Features2=0xfef83203<SSE3,PCLMULQDQ,SSSE3,FMA,CX16,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND,HV> AMD Features=0x2e500800<SYSCALL,NX,MMX+,FFXSR,Page1GB,RDTSCP,LM> AMD Features2=0xc003f3<LAHF,CMP,CR8,ABM,SSE4A,MAS,Prefetch,OSVW,Topology,PCXC> Structured Extended Features=0x219c01a9<FSGSBASE,BMI1,AVX2,SMEP,BMI2,RDSEED,ADX,SMAP,CLFLUSHOPT,CLWB,SHA> Structured Extended Features2=0x400004<UMIP,RDPID> Structured Extended Features3=0x20000000<ARCH_CAP> XSAVE Features=0x7<XSAVEOPT,XSAVEC,XINUSE> AMD Extended Feature Extensions ID EBX=0x100d205<CLZERO,XSaveErPtr,WBNOINVD,IBPB,IBRS,STIBP,SSBD> Hypervisor: Origin = "KVMKVMKVM" real memory = 8388608000 (8000 MB) avail memory = 8031084544 (7659 MB) Event timer "LAPIC" quality 600 ACPI APIC Table: <BOCHS BXPC > FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs FreeBSD/SMP: 1 package(s) x 4 core(s) random: registering fast source Intel Secure Key RNG random: fast provider: "Intel Secure Key RNG" random: unblocking device. ioapic0 <Version 1.1> irqs 0-23 Launching APs: 1 3 2 TCP_ratelimit: Is now initialized ipw_bss: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw.LICENSE. ipw_bss: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf. module_register_init: MOD_LOAD (ipw_bss_fw, 0xffffffff80758a00, 0) error 1 ipw_ibss: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw.LICENSE. ipw_ibss: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf. module_register_init: MOD_LOAD (ipw_ibss_fw, 0xffffffff80758ab0, 0) error 1 ipw_monitor: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw.LICENSE. ipw_monitor: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf. module_register_init: MOD_LOAD (ipw_monitor_fw, 0xffffffff80758b60, 0) error 1 iwi_bss: You need to read the LICENSE file in /usr/share/doc/legal/intel_iwi.LICENSE. iwi_bss: If you agree with the license, set legal.intel_iwi.license_ack=1 in /boot/loader.conf. module_register_init: MOD_LOAD (iwi_bss_fw, 0xffffffff807781d0, 0) error 1 iwi_ibss: You need to read the LICENSE file in /usr/share/doc/legal/intel_iwi.LICENSE. iwi_ibss: If you agree with the license, set legal.intel_iwi.license_ack=1 in /boot/loader.conf. module_register_init: MOD_LOAD (iwi_ibss_fw, 0xffffffff80778280, 0) error 1 iwi_monitor: You need to read the LICENSE file in /usr/share/doc/legal/intel_iwi.LICENSE. iwi_monitor: If you agree with the license, set legal.intel_iwi.license_ack=1 in /boot/loader.conf. module_register_init: MOD_LOAD (iwi_monitor_fw, 0xffffffff80778330, 0) error 1 random: entropy device external interface wlan: mac acl policy registered kbd1 at kbdmux0 WARNING: Device "spkr" is Giant locked and may be deleted before FreeBSD 15.0. netgate0: <KVM Guest> netgate0: version: 0.1 vtvga0: <VT VGA driver> kvmclock0: <KVM paravirtual clock> Timecounter "kvmclock" frequency 1000000000 Hz quality 975 kvmclock0: registered as a time-of-day clock, resolution 0.000001s smbios0: <System Management BIOS> at iomem 0xf5270-0xf528e smbios0: Version: 2.8, BCD Revision: 2.8 acpi0: <BOCHS BXPC> acpi0: Power Button (fixed) cpu0: <ACPI CPU> on acpi0 atrtc0: <AT realtime clock> port 0x70-0x77 irq 8 on acpi0 atrtc0: registered as a time-of-day clock, resolution 1.000000s Event timer "RTC" frequency 32768 Hz quality 0 hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 100000000 Hz quality 950 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x608-0x60b on acpi0 pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0 pci0: <ACPI PCI bus> on pcib0 virtio_pci0: <VirtIO PCI (modern) GPU adapter> mem 0xfd000000-0xfd7fffff,0xe120000000-0xe120003fff,0xfea10000-0xfea10fff irq 21 at device 1.0 on pci0 pcib1: <ACPI PCI-PCI bridge> mem 0xfea11000-0xfea11fff irq 22 at device 2.0 on pci0 pci1: <ACPI PCI bus> on pcib1 virtio_pci1: <VirtIO PCI (modern) Network adapter> mem 0xfe880000-0xfe880fff,0xe100000000-0xe100003fff irq 22 at device 0.0 on pci1 vtnet0: <VirtIO Networking Adapter> on virtio_pci1 vtnet0: Ethernet address: 96:00:03:e3:ce:49 vtnet0: netmap queues/slots: TX 1/256, RX 1/128 000.000077 [ 452] vtnet_netmap_attach vtnet attached txq=1, txd=256 rxq=1, rxd=128 pcib2: <ACPI PCI-PCI bridge> mem 0xfea12000-0xfea12fff irq 22 at device 2.1 on pci0 pci2: <ACPI PCI bus> on pcib2 xhci0: <XHCI (generic) USB 3.0 controller> mem 0xfe600000-0xfe603fff irq 22 at device 0.0 on pci2 xhci0: 32 bytes context size, 64-bit DMA usbus0 on xhci0 usbus0: 5.0Gbps Super Speed USB v3.0 pcib3: <ACPI PCI-PCI bridge> mem 0xfea13000-0xfea13fff irq 22 at device 2.2 on pci0 pci3: <ACPI PCI bus> on pcib3 virtio_pci2: <VirtIO PCI (modern) Console adapter> mem 0xfe400000-0xfe400fff,0xe0c0000000-0xe0c0003fff irq 22 at device 0.0 on pci3 vtcon0: <VirtIO Console Adapter> on virtio_pci2 pcib4: <ACPI PCI-PCI bridge> mem 0xfea14000-0xfea14fff irq 22 at device 2.3 on pci0 pci4: <ACPI PCI bus> on pcib4 virtio_pci3: <VirtIO PCI (modern) Balloon adapter> mem 0xe0a0000000-0xe0a0003fff irq 22 at device 0.0 on pci4 vtballoon0: <VirtIO Balloon Adapter> on virtio_pci3 pcib5: <ACPI PCI-PCI bridge> mem 0xfea15000-0xfea15fff irq 22 at device 2.4 on pci0 pci5: <ACPI PCI bus> on pcib5 virtio_pci4: <VirtIO PCI (modern) Entropy adapter> mem 0xe080000000-0xe080003fff irq 22 at device 0.0 on pci5 pcib6: <ACPI PCI-PCI bridge> mem 0xfea16000-0xfea16fff irq 22 at device 2.5 on pci0 pci6: <ACPI PCI bus> on pcib6 virtio_pci5: <VirtIO PCI (modern) SCSI adapter> mem 0xfde00000-0xfde00fff,0xe060000000-0xe060003fff irq 22 at device 0.0 on pci6 vtscsi0: <VirtIO SCSI Adapter> on virtio_pci5 pcib7: <ACPI PCI-PCI bridge> mem 0xfea17000-0xfea17fff irq 22 at device 2.6 on pci0 pcib8: <ACPI PCI-PCI bridge> mem 0xfea18000-0xfea18fff irq 22 at device 2.7 on pci0 pcib9: <ACPI PCI-PCI bridge> mem 0xfea19000-0xfea19fff irq 23 at device 3.0 on pci0 isab0: <PCI-ISA bridge> at device 31.0 on pci0 isa0: <ISA bus> on isab0 ahci0: <Intel ICH9 AHCI SATA controller> port 0xc040-0xc05f mem 0xfea1a000-0xfea1afff irq 16 at device 31.2 on pci0 ahci0: AHCI v1.00 with 6 1.5Gbps ports, Port Multiplier not supported ahcich0: <AHCI channel> at channel 0 on ahci0 ahcich1: <AHCI channel> at channel 1 on ahci0 ahcich2: <AHCI channel> at channel 2 on ahci0 ahcich3: <AHCI channel> at channel 3 on ahci0 ahcich4: <AHCI channel> at channel 4 on ahci0 ahcich5: <AHCI channel> at channel 5 on ahci0 acpi_syscontainer0: <System Container> on acpi0 acpi_syscontainer1: <System Container> port 0xcd8-0xce3 on acpi0 acpi_syscontainer2: <System Container> port 0x620-0x62f on acpi0 acpi_syscontainer3: <System Container> port 0xcc0-0xcd7 on acpi0 uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0 atkbd0: <AT Keyboard> irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] psm0: <PS/2 Mouse> irq 12 on atkbdc0 psm0: [GIANT-LOCKED] WARNING: Device "psm" is Giant locked and may be deleted before FreeBSD 15.0. psm0: model IntelliMouse Explorer, device ID 4 orm0: <ISA Option ROM> at iomem 0xe7800-0xeffff pnpid ORM0000 on isa0 vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff pnpid PNP0900 on isa0 attimer0: <AT timer> at port 0x40 on isa0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 Timecounters tick every 10.000 msec ugen0.1: <(0x1b36) XHCI root HUB> at usbus0 uhub0 on usbus0 uhub0: <(0x1b36) XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0 ZFS filesystem version: 5 ZFS storage pool version: features support (5000) Trying to mount root from zfs:pfSense/ROOT/default []... uhub0: 8 ports with 8 removable, self powered ugen0.2: <QEMU QEMU USB Tablet> at usbus0 uhid0 on uhub0 uhid0: <QEMU QEMU USB Tablet, class 0/0, rev 2.00/0.00, addr 1> on usbus0 Root mount waiting for: CAM Root mount waiting for: CAM Root mount waiting for: CAM Root mount waiting for: CAM Root mount waiting for: CAM Root mount waiting for: CAM Root mount waiting for: CAM Root mount waiting for: CAM da0 at vtscsi0 bus 0 scbus0 target 0 lun 0 da0: <QEMU QEMU HARDDISK 2.5+> Fixed Direct Access SPC-3 SCSI device da0: 300.000MB/s transfers da0: Command Queueing enabled da0: 156252MB (320004096 512 byte sectors) cd0 at ahcich0 bus 0 scbus1 target 0 lun 0 cd0: <QEMU QEMU DVD-ROM 2.5+> Removable CD-ROM SCSI device cd0: Serial Number QM00001 cd0: 150.000MB/s transfers (SATA 1.x, UDMA5, ATAPI 12bytes, PIO 8192bytes) cd0: Attempt to query device size failed: NOT READY, Medium not present CPU: AMD EPYC Processor (2495.31-MHz K8-class CPU) Origin="AuthenticAMD" Id=0x830f10 Family=0x17 Model=0x31 Stepping=0 Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT> Features2=0xfef83203<SSE3,PCLMULQDQ,SSSE3,FMA,CX16,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND,HV> AMD Features=0x2e500800<SYSCALL,NX,MMX+,FFXSR,Page1GB,RDTSCP,LM> AMD Features2=0xc003f3<LAHF,CMP,CR8,ABM,SSE4A,MAS,Prefetch,OSVW,Topology,PCXC> Structured Extended Features=0x219c01a9<FSGSBASE,BMI1,AVX2,SMEP,BMI2,RDSEED,ADX,SMAP,CLFLUSHOPT,CLWB,SHA> Structured Extended Features2=0x400004<UMIP,RDPID> Structured Extended Features3=0x20000000<ARCH_CAP> XSAVE Features=0x7<XSAVEOPT,XSAVEC,XINUSE> AMD Extended Feature Extensions ID EBX=0x100d205<CLZERO,XSaveErPtr,WBNOINVD,IBPB,IBRS,STIBP,SSBD> Hypervisor: Origin = "KVMKVMKVM"
-
-
@netblues After raising a ticket with hetzner, asking for the emulated machine to be changed to 440fx, all virtio interfaces now work.
2.7.2 works with q35, 2.8 requires 440fx. That was also the case in 2.4 if I recall. It got fixed, and now is rolled back. :( Lots of legacy entries in the boot log but at least it works.
Any performance issues need to be researched.Copyright (c) 1992-2024 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 15.0-CURRENT #0 RELENG_2_8_0-n256077-f05c84494d5f: Mon Apr 14 19:20:19 UTC 2025 root@freebsd:/var/jenkins/workspace/pfSense-CE-snapshots-2_8_0-main/obj/amd64/660qS3Wa/var/jenkins/workspace/pfSense-CE-snapshots-2_8_0-main/sources/FreeBSD-src-RELENG_2_8_0/amd64.amd64/sys/pfSense amd64 FreeBSD clang version 19.1.5 (https://github.com/llvm/llvm-project.git llvmorg-19.1.5-0-gab4b5a2db582) VT(vga): text 80x25 CPU: AMD EPYC Processor (2495.43-MHz K8-class CPU) Origin="AuthenticAMD" Id=0x830f10 Family=0x17 Model=0x31 Stepping=0 Features=0x1783fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE,SSE2,HTT> Features2=0xfef83203<SSE3,PCLMULQDQ,SSSE3,FMA,CX16,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND,HV> AMD Features=0x2e500800<SYSCALL,NX,MMX+,FFXSR,Page1GB,RDTSCP,LM> AMD Features2=0xc003f3<LAHF,CMP,CR8,ABM,SSE4A,MAS,Prefetch,OSVW,Topology,PCXC> Structured Extended Features=0x219c01a9<FSGSBASE,BMI1,AVX2,SMEP,BMI2,RDSEED,ADX,SMAP,CLFLUSHOPT,CLWB,SHA> Structured Extended Features2=0x400004<UMIP,RDPID> Structured Extended Features3=0x20000000<ARCH_CAP> XSAVE Features=0x7<XSAVEOPT,XSAVEC,XINUSE> AMD Extended Feature Extensions ID EBX=0x100d205<CLZERO,XSaveErPtr,WBNOINVD,IBPB,IBRS,STIBP,SSBD> Hypervisor: Origin = "KVMKVMKVM" real memory = 8388608000 (8000 MB) avail memory = 8058458112 (7685 MB) Event timer "LAPIC" quality 600 ACPI APIC Table: <BOCHS BXPC > FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs FreeBSD/SMP: 1 package(s) x 4 core(s) random: registering fast source Intel Secure Key RNG random: fast provider: "Intel Secure Key RNG" random: unblocking device. ioapic0 <Version 1.1> irqs 0-23 Launching APs: 2 3 1 TCP_ratelimit: Is now initialized ipw_bss: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw.LICENSE. ipw_bss: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf. module_register_init: MOD_LOAD (ipw_bss_fw, 0xffffffff80758a00, 0) error 1 ipw_ibss: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw.LICENSE. ipw_ibss: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf. module_register_init: MOD_LOAD (ipw_ibss_fw, 0xffffffff80758ab0, 0) error 1 ipw_monitor: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw.LICENSE. ipw_monitor: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf. module_register_init: MOD_LOAD (ipw_monitor_fw, 0xffffffff80758b60, 0) error 1 iwi_bss: You need to read the LICENSE file in /usr/share/doc/legal/intel_iwi.LICENSE. iwi_bss: If you agree with the license, set legal.intel_iwi.license_ack=1 in /boot/loader.conf. module_register_init: MOD_LOAD (iwi_bss_fw, 0xffffffff807781d0, 0) error 1 iwi_ibss: You need to read the LICENSE file in /usr/share/doc/legal/intel_iwi.LICENSE. iwi_ibss: If you agree with the license, set legal.intel_iwi.license_ack=1 in /boot/loader.conf. module_register_init: MOD_LOAD (iwi_ibss_fw, 0xffffffff80778280, 0) error 1 iwi_monitor: You need to read the LICENSE file in /usr/share/doc/legal/intel_iwi.LICENSE. iwi_monitor: If you agree with the license, set legal.intel_iwi.license_ack=1 in /boot/loader.conf. module_register_init: MOD_LOAD (iwi_monitor_fw, 0xffffffff80778330, 0) error 1 random: entropy device external interface wlan: mac acl policy registered kbd1 at kbdmux0 WARNING: Device "spkr" is Giant locked and may be deleted before FreeBSD 15.0. netgate0: <KVM Guest> netgate0: version: 0.1 vtvga0: <VT VGA driver> kvmclock0: <KVM paravirtual clock> Timecounter "kvmclock" frequency 1000000000 Hz quality 975 kvmclock0: registered as a time-of-day clock, resolution 0.000001s smbios0: <System Management BIOS> at iomem 0xf5250-0xf526e smbios0: Version: 2.8, BCD Revision: 2.8 acpi0: <BOCHS BXPC> acpi0: Power Button (fixed) cpu0: <ACPI CPU> on acpi0 atrtc0: <AT realtime clock> port 0x70-0x77 irq 8 on acpi0 atrtc0: registered as a time-of-day clock, resolution 1.000000s Event timer "RTC" frequency 32768 Hz quality 0 hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 100000000 Hz quality 950 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x608-0x60b on acpi0 pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0 pci0: <ACPI PCI bus> on pcib0 isab0: <PCI-ISA bridge> at device 1.0 on pci0 isa0: <ISA bus> on isab0 atapci0: <Intel PIIX3 WDMA2 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xc120-0xc12f at device 1.1 on pci0 ata0: <ATA channel> at channel 0 on atapci0 ata1: <ATA channel> at channel 1 on atapci0 pci0: <bridge> at device 1.3 (no driver attached) vgapci0: <VGA-compatible display> mem 0xfd000000-0xfdffffff,0xfeb94000-0xfeb94fff at device 2.0 on pci0 vgapci0: Boot video device virtio_pci0: <VirtIO PCI (legacy) Network adapter> port 0xc100-0xc11f mem 0xfeb95000-0xfeb95fff,0xe000000000-0xe000003fff irq 11 at device 3.0 on pci0 vtnet0: <VirtIO Networking Adapter> on virtio_pci0 vtnet0: Ethernet address: 96:00:03:e3:ce:49 vtnet0: netmap queues/slots: TX 1/256, RX 1/128 000.000759 [ 452] vtnet_netmap_attach vtnet attached txq=1, txd=256 rxq=1, rxd=128 virtio_pci1: <VirtIO PCI (legacy) SCSI adapter> port 0xc000-0xc03f mem 0xfeb96000-0xfeb96fff,0xe000004000-0xe000007fff irq 11 at device 4.0 on pci0 vtscsi0: <VirtIO SCSI Adapter> on virtio_pci1 virtio_pci2: <VirtIO PCI (legacy) Balloon adapter> port 0xc040-0xc07f mem 0xe000008000-0xe00000bfff irq 10 at device 5.0 on pci0 vtballoon0: <VirtIO Balloon Adapter> on virtio_pci2 virtio_pci3: <VirtIO PCI (legacy) Console adapter> port 0xc080-0xc0bf mem 0xfeb97000-0xfeb97fff,0xe00000c000-0xe00000ffff irq 10 at device 6.0 on pci0 vtcon0: <VirtIO Console Adapter> on virtio_pci3 xhci0: <XHCI (generic) USB 3.0 controller> mem 0xfeb90000-0xfeb93fff irq 11 at device 7.0 on pci0 xhci0: 32 bytes context size, 64-bit DMA usbus0 on xhci0 usbus0: 5.0Gbps Super Speed USB v3.0 virtio_pci4: <VirtIO PCI (legacy) Network adapter> port 0xc0c0-0xc0ff mem 0xfeb98000-0xfeb98fff,0xe000010000-0xe000013fff irq 10 at device 10.0 on pci0 vtnet1: <VirtIO Networking Adapter> on virtio_pci4 vtnet1: Ethernet address: 86:00:00:c7:aa:ea vtnet1: netmap queues/slots: TX 1/256, RX 1/128 000.000766 [ 452] vtnet_netmap_attach vtnet attached txq=1, txd=256 rxq=1, rxd=128 acpi_syscontainer0: <System Container> on acpi0 acpi_syscontainer1: <System Container> port 0xaf00-0xaf0b on acpi0 acpi_syscontainer2: <System Container> port 0xafe0-0xafe3 on acpi0 acpi_syscontainer3: <System Container> port 0xae00-0xae17 on acpi0 uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0 atkbd0: <AT Keyboard> irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] psm0: <PS/2 Mouse> irq 12 on atkbdc0 psm0: [GIANT-LOCKED] WARNING: Device "psm" is Giant locked and may be deleted before FreeBSD 15.0. psm0: model IntelliMouse Explorer, device ID 4 fdc0: <floppy drive controller (FDE)> port 0x3f2-0x3f5,0x3f7 irq 6 drq 2 on acpi0 fdc0: does not respond device_attach: fdc0 attach returned 6 orm0: <ISA Option ROM> at iomem 0xe7800-0xeffff pnpid ORM0000 on isa0 vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff pnpid PNP0900 on isa0 attimer0: <AT timer> at port 0x40 on isa0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 Timecounters tick every 10.000 msec ugen0.1: <(0x1b36) XHCI root HUB> at usbus0 uhub0 on usbus0 uhub0: <(0x1b36) XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0 ZFS filesystem version: 5 ZFS storage pool version: features support (5000) Trying to mount root from zfs:pfSense/ROOT/default []... uhub0: 8 ports with 8 removable, self powered ugen0.2: <QEMU QEMU USB Tablet> at usbus0 uhid0 on uhub0 uhid0: <QEMU QEMU USB Tablet, class 0/0, rev 2.00/0.00, addr 1> on usbus0 Root mount waiting for: CAM Root mount waiting for: CAM Root mount waiting for: CAM Root mount waiting for: CAM Root mount waiting for: CAM Root mount waiting for: CAM Root mount waiting for: CAM Root mount waiting for: CAM Root mount waiting for: CAM cd cd0 at ata1 bus 0 scbus1 target 0 lun 0 cd0: <QEMU QEMU DVD-ROM 2.5+> Removable CD-ROM SCSI device cd0: Serial Number QM00003 cd0: 16.700MB/s transfers (WDMA2, ATAPI 12bytes, PIO 65534bytes) cd0: Attempt to query device size failed: NOT READY, Medium not present da0 at vtscsi0 bus 0 scbus2 target 0 lun 0 da0: <QEMU QEMU HARDDISK 2.5+> Fixed Direct Access SPC-3 SCSI device da0: 300.000MB/s transfers da0: Command Queueing enabled da0: 156252MB (320004096 512 byte sectors) CPU: AMD EPYC Processor (2495.31-MHz K8-class CPU) Origin="AuthenticAMD" Id=0x830f10 Family=0x17 Model=0x31 Stepping=0 Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT> Features2=0xfef83203<SSE3,PCLMULQDQ,SSSE3,FMA,CX16,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND,HV> AMD Features=0x2e500800<SYSCALL,NX,MMX+,FFXSR,Page1GB,RDTSCP,LM> AMD Features2=0xc003f3<LAHF,CMP,CR8,ABM,SSE4A,MAS,Prefetch,OSVW,Topology,PCXC> Structured Extended Features=0x219c01a9<FSGSBASE,BMI1,AVX2,SMEP,BMI2,RDSEED,ADX,SMAP,CLFLUSHOPT,CLWB,SHA> Structured Extended Features2=0x400004<UMIP,RDPID> Structured Extended Features3=0x20000000<ARCH_CAP> XSAVE Features=0x7<XSAVEOPT,XSAVEC,XINUSE> AMD Extended Feature Extensions ID EBX=0x100d205<CLZERO,XSaveErPtr,WBNOINVD,IBPB,IBRS,STIBP,SSBD> Hypervisor: Origin = "KVMKVMKVM" pfSense is developed and maintained by Netgate. © ESF 2004 - 2025 View license.
-
Mmm, yet to replicate that in Proxmox here. There the legacy devices are the default even in Q35 and performance doesn't seem to be an issue.
Really we need to see what the actual vm conf used is that creates it. But I imagine that could be tough to get.
-
@stephenw10
You mean virsh dumpxml?In any case proxmox is a debian derivative, alma, oracle linux and what hetzner runs is redhat.
There must be something that controls how the virt devices appear in vm's, apart from changing the complete machine emulation.
-
I mean the qemu conf file for the VM which I expect to look pretty much the same in any host OS. Like:
root@pve:~# cat /etc/pve/qemu-server/119.conf args: -global virtio-net-pci.disable-legacy=on -global virtio-scsi-pci.disable-legacy=on -global virtio-ballon-pci.disable-legacy=on balloon: 1024 boot: order=scsi0;ide2;net0 cores: 1 ide2: local:iso/netgate-installer-v1.1-DEVELOPMENT-amd64-20250318-0600.iso,media=cdrom,size=1062414K machine: q35,viommu=virtio memory: 2048 meta: creation-qemu=7.2.0,ctime=1713818950 name: pfSense10 net0: virtio=8A:1C:48:8A:25:E7,bridge=vmbr0,firewall=1 net1: virtio=BC:24:11:CC:61:EA,bridge=vmbr1,firewall=1 numa: 0 ostype: l26 parent: snap3 scsi0: HDIMG:119/vm-119-disk-0.qcow2,iothread=1,size=10G scsihw: virtio-scsi-single serial0: socket smbios1: uuid=ac41daa9-526a-42b9-84d4-0904a44b7468 sockets: 1 vmgenid: 72cd897c-0186-4e1d-aaae-f41810d575d7
It does look like we might be on to something upstream though. But first we need to replicate the failure which is proving difficult.