OSProvisioningTimedOut error when trying to deploy pfsense from Azure marketplace
-
I am trying to create a new Azure VM using "pfSense Plus Public Cloud Firewall/VPN/Router" from the Azure marketplace.
I've tried 3 times now and received the same error every time...
The virtual machine is in a failed state.
OS Provisioning for VM 'StackpolePfSenseFWRouter' did not finish in the allotted time. The VM may still finish provisioning successfully. Please check provisioning state later. For details on how to check current provisioning state of Windows VMs, refer to https://aka.ms/WindowsVMLifecycle and Linux VMs, refer to https://aka.ms/LinuxVMLifecycle.Learn more about common virtual machine error codes.Details:
Error code: OSProvisioningTimedOut
Provisioning state: Failed
Provisioning state error code: ProvisioningState/failed/OSProvisioningTimedOut
Guest agent status: VM status blob is found but not yet populated.I've waited over 2 hours and it is still unresponsive.
Azure portal shows that the VM is running, but I cannot connect via SSH or HTTP(s).
If I try to restart the VM it just hangs, then errors out.Thank you.
-
What do you see on the virtual serial console? Or on a screenshot of the VM?
What machine size are you using?
Steve
-
Just tested using B1ms and it deployed fine:
Microsoft Azure - Netgate Device ID: 6a0453bc947ff4xxxxxx *** Welcome to Netgate pfSense Plus 24.03-RELEASE (amd64) on pfSense-24.03-test-SW1 *** WAN (wan) -> hn0 -> v4/DHCP4: 10.1.7.17/24 0) Logout (SSH only) 9) pfTop 1) Assign Interfaces 10) Filter Logs 2) Set interface(s) IP address 11) Restart GUI 3) Reset admin account and password 12) PHP shell + Netgate pfSense Plus tools 4) Reset to factory defaults 13) Update from console 5) Reboot system 14) Disable Secure Shell (sshd) 6) Halt system 15) Restore recent configuration 7) Ping host 16) Restart PHP-FPM 8) Shell Enter an option:
It does show a bunch of output from the Azure agent at the console when you initially connect.
-
I cannot connect via SSH, it simply times out.
Port 22 IS open and I tried the private and public IP.
The size I'm using is DS1 v2. I don't think I even had B1ms as an option when I ran the deployment. -
If you look at the serial console or diagnostic screenshot in Azure what does it show?
-
Here is the output from the serial console...
Welcome to Netgate pfSense Plus 24.03-RELEASE...
...ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/compat/pkg /usr/local/lib/compat/pkg /usr/local/lib/ipsec /usr/local/lib/perl5/5.36/mach/CORE
32-bit compatibility ldconfig path:
done.
3014
Config.xml is corrupted and is 0 bytes. Could not restore a previous backup.Launching the init system...Updating CPU Microcode...
CPU: Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz (2593.91-MHz K8-class CPU)
Origin="GenuineIntel" Id=0x50657 Family=0x6 Model=0x55 Stepping=7
Features=0xf8bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,SS>
Features2=0xfeda3203<SSE3,PCLMULQDQ,SSSE3,FMA,CX16,PCID,SSE4.1,SSE4.2,MOVBE,POPCNT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND,HV>
AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM>
AMD Features2=0x121<LAHF,ABM,Prefetch>
Structured Extended Features=0xd09f2fb9<FSGSBASE,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,NFPUSG,AVX512F,AVX512DQ,RDSEED,ADX,SMAP,CLFLUSHOPT,AVX512CD,AVX512BW,AVX512VL>
Structured Extended Features3=0x400<MD_CLEAR>
XSAVE Features=0xb<XSAVEOPT,XSAVEC,XSAVES>
Hypervisor: Origin = "Microsoft Hv"
Done.
Config.xml is corrupted and is 0 bytes. Could not restore a previous backup.Starting CRON... done.
Config.xml is corrupted and is 0 bytes. Could not restore a previous backup.Netgate pfSense Plus 24.03-RELEASE amd64 20240418-2354
Bootup completeFreeBSD/amd64 (Amnesiac) (ttyu0)
login: hn0: got notify, nvs type 128
pcib0: <Hyper-V PCI Express Pass Through> on vmbus0
pcib0: PCI VMBus using version 0x10004
pci0: <PCI bus> on pcib0
mlx5_core0: <mlx5_core> at device 2.0 on pci0
mlx5: Mellanox Core driver 3.7.1 (November 2021)mlx5_core0: WARN: mlx5_init_once:994:(pid 0): Unable to find vendor specific capabilities
mce0: Ethernet address: 60:45:bd:37:4b:11
mce0: link state changed to DOWN
hn0: link state changed to DOWN
mlx5_core0: WARN: mlx5_fwdump_prep:91:(pid 0): Unable to find vendor-specific capability, error 2
hvkvp0: detached
hvkvp0: <Hyper-V KVP> on vmbus0 -
Hmm, the mce NIC should not appear there. You are just launching the image in DS1 v2? Any specific zone? Anthing custom in the options?
-
This is in the US central zone, and no, no custom options.
Also, it will not let me log in with the credentials I entered during setup.
I AM able to get in with root, and when I list /etc/passwd, the username I specified during setup is not listed, nor is admin. -
Here is the initial output from startup...
Loading /boot/loader.conf.local Loading kernel... /boot/kernel/kernel text=0x19eec0 text=0xff4c38 text=0x17e3db4 data=0x180 data=0x22d718+0x3d18e8 0x8+0x1cb0f0+0x8+0x1da290/ Loading configured modules... can't find '/boot/entropy' can't find '/etc/hostid' Booting [/boot/kernel/kernel]... staging 0x36c00000-0x3aa7d000 (not copying) tramp 0x3aa7d000 PT4 0x3aa7e000 Start @ 0xffffffff8039f000 ... EFI framebuffer information: addr, size 0x40000000, 0x800000 dimensions 1024 x 768 stride 1024 masks 0x00ff0000, 0x0000ff00, 0x000000ff, 0xff000000 GDB: no debug ports present KDB: debugger backends: ddb KDB: current backend: ddb ---<<BOOT>>--- Copyright (c) 1992-2024 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 15.0-CURRENT #0 plus-RELENG_24_03-n256311-e71f834dd81: Fri Apr 19 00:28:14 UTC 2024 root@freebsd:/var/jenkins/workspace/pfSense-Plus-snapshots-24_03-main/obj/amd64/Y4MAEJ2R/var/jenkins/workspace/pfSense-Plus-snapshots-24_03-main/sources/FreeBSD-src-plus-RELENG_24_03/amd64.amd64/sys/pfSense amd64 FreeBSD clang version 17.0.6 (https://github.com/llvm/llvm-project.git llvmorg-17.0.6-0-g6009708b4367) SRAT: Ignoring memory at addr 0x1a0200000 SRAT: Ignoring memory at addr 0x1000000000 SRAT: Ignoring memory at addr 0x10000200000 SRAT: Ignoring memory at addr 0x20000200000 SRAT: Ignoring memory at addr 0x40000200000 SRAT: Ignoring memory at addr 0x80000200000 VT(efifb): resolution 1024x768 Hyper-V Version: 10.0.20348 [SP1] Features0x2e7f<VPRUNTIME,TMREFCNT,SYNIC,SYNTM,APIC,HYPERCALL,VPINDEX,REFTSC,IDLE,TMFREQ> PM Features=0x0 [C2] Features3=0xed7b2<DEBUG,XMMHC,IDLE,NUMA,TMFREQ,SYNCMC,CRASH,NPIEP> Timecounter "Hyper-V frequency 10000000 Hz quality 2000 CPU: Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz (2593.90-MHz K8-class CPU) Origin="GenuineIntel" Id=0x50657 Family=0x6 Model=0x55 Stepping=7 Features=0xf83fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE,SSE2,SS> Features2=0xfeda3203<SSE3,PCLMULQDQ,SSSE3,FMA,CX16,PCID,SSE4.1,SSE4.2,MOVBE,POPCNT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND,HV> AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM> AMD Features2=0x121<LAHF,ABM,Prefetch> Structured Extended Features=0xd09f2fb9<FSGSBASE,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,NFPUSG,AVX512F,AVX512DQ,RDSEED,ADX,SMAP,CLFLUSHOPT,AVX512CD,AVX512BW,AVX512VL> Structured Extended Features3=0x400<MD_CLEAR> XSAVE Features=0xb<XSAVEOPT,XSAVEC,XSAVES> Hypervisor: Origin = "Microsoft Hv" real memory = 3758096384 (3584 MB) avail memory = 3504218112 (3341 MB) Event timer "LAPIC" quality 100 ACPI APIC Table: <VRTUAL MICROSFT> random: registering fast source Intel Secure Key RNG random: fast provider: "Intel Secure Key RNG" arc4random: WARNING: initial seeding bypassed the cryptographic random device because it was not yet seeded and the knob 'bypass_before_seeding' was enabled. ioapic0 <Version 1.1> irqs 0-23 TCP_ratelimit: Is now initialized ipw_bss: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw.LICENSE. ipw_bss: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf. module_register_init: MOD_LOAD (ipw_bss_fw, 0xffffffff80750310, 0) error 1 ipw_ibss: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw.LICENSE. ipw_ibss: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf. module_register_init: MOD_LOAD (ipw_ibss_fw, 0xffffffff807503c0, 0) error 1 ipw_monitor: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw.LICENSE. ipw_monitor: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf. module_register_init: MOD_LOAD (ipw_monitor_fw, 0xffffffff80750470, 0) error 1 iwi_bss: You need to read the LICENSE file in /usr/share/doc/legal/intel_iwi.LICENSE. iwi_bss: If you agree with the license, set legal.intel_iwi.license_ack=1 in /boot/loader.conf. module_register_init: MOD_LOAD (iwi_bss_fw, 0xffffffff80770010, 0) error 1 iwi_ibss: You need to read the LICENSE file in /usr/share/doc/legal/intel_iwi.LICENSE. iwi_ibss: If you agree with the license, set legal.intel_iwi.license_ack=1 in /boot/loader.conf. module_register_init: MOD_LOAD (iwi_ibss_fw, 0xffffffff807700c0, 0) error 1 iwi_monitor: You need to read the LICENSE file in /usr/share/doc/legal/intel_iwi.LICENSE. iwi_monitor: If you agree with the license, set legal.intel_iwi.license_ack=1 in /boot/loader.conf. module_register_init: MOD_LOAD (iwi_monitor_fw, 0xffffffff80770170, 0) error 1 ranom: entropy devicl policy registered Tiounter "Hyper-V-T00000 Hz quality 3000 kbd0 at kbdmux0 WARNING: Device "spkr" Giant locked and may be deleted e FreeBSD 15.0. efirtc0: <EFI Realtime Clock> efirtc0: registered as a time-of-day clock, resolution 1.000000s netgate0: <Microsoft Azure> smbios0: <System Management BIOS> at iomem 0x3ff83000-0x3ff83017 smbios0: Version: 3.1 acpi0: <VRTUAL MICROSFT> atrtc0: <AT realtime clock> port 0x70-0x71 irq 8 on acpi0 atrtc0: registered as a time-of-day clock, resolution 1.000000s Event timer "RTC" frequency 32768 Hz quality 0 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <32-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 cpu0: <ACPI CPU> on acpi0 acpi_syscontainer0: <System Container> on acpi0 vmbus0: <Hyper-V Vmbus> on acpi_syscontainer0 uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart0: console (115200,n,8,1) uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 vmgenc0: <VM Generation Counter> on acpi0 vmbus_res0: <Hyper-V Vmbus Resource> irq 5 on acpi0 Timecounter "TSC-low" frequency 1296953099 Hz quality 800 Timecounters tick every 10.000 msec vmbus0: version 4.0 hvet0: <Hyper-V event timer> on vmbus0 Event timer "Hyper-V" frequency 10000000 Hz quality 1000 hvkbd0: <Hyper-V KBD> on vmbus0 kbd1 at hvkbd0 hvheartbeat0: <Hyper-V Heartbeat> on vmbus0 hvkvp0: <Hyper-V KVP> on vmbus0 hvshutdown0: <Hyper-V Shutdown> on vmbus0 hvtimesync0: <Hyper-V Timesync> on vmbus0 hvtimesync0: RTT hn0: <Hyper-V Netrandom: unblocking device. hn0: Ethernet address: 60:45:bd:37:4b:11 storvsc0: <Hyper-V SCSI> on vmbus0 hn0: link state cstorvsc1: <Hyper-V SCSI> on vmbus0 Trying to mount root from ufs:/dev/gpt/pfSense [rw]... da0 at storvsc0 bus 0 scbus0 target 0 lun 0 da0: <Msft Virtual Disk 1.0> Fixed Direct Access SPC-3 SCSI device da0: 300.000MB/s transfers da0: Command Queueing enabled da0: 10241MB (20973568 512 byte sectors) da1 at storvsc0 bus 0 scbus0 target 0 lun 1 da1: <Msft Virtual Disk 1.0> Fixed Direct Access SPC-3 SCSI device da1: 300.000MB/s transfers da1: Command Queueing enabled da1: 7168MB (14680064 512 byte sectors) WARNING: / was not properly dismounted Configuring crash dumps... No suitable dump device was found. Growing root partition to fill device da0 recovering is not needed da0p3 resized growfs: requested size 9.8GB is equal to the current filesystem size 9.8GB ** SU+J Recovering /dev/gpt/pfSense ** Reading 82247680 byte journal from inode 4. ** Building recovery table. ** Resolving unreferenced inode list. ** Processing journal entries. ** 365 journal records in 17920 bytes for 65.18% utilization ** Freed 20 inodes (4 dirs) 6 blocks, and 41 frags. /dev/gpt/pfSense: **** FILE SYSTEM MARKED CLEAN **** Filesystems are clean, continuing... Mounting filesystems... __ _ __ / _|___ ___ _ __ ___ ___ _ | '_ \| |_/ __|/ _ \ '_ \/ __|/ _ \ _| |_ | |_) | _\__ \ __/ | | \__ \ __/ |_ _| | .__/|_| |___/\___|_| |_|___/\___| |_| |_| Welcome to Netgate pfSense Plus 24.03-RELEASE...
-
Hmm, I can't replicate that.
Try deploying it without 'accelerated networking' enabled if it was enabled previously. Though in testing here it deploys fine with or without it.
-
I was able to deploy the old fashioned way... download the iso, create a virtual disk and vm in Hyper-v Manager, and upload to Azure. The VM is up and running and I am configuring it.
I'm going to abandon the marketplace option for now, but thank you very much for all your help.
-
Hmm, what image from the Marketplace exactly were you testing?
-
"pfSense Plus Public Cloud Firewall/VPN/Router"