Boot loop
-
Hello folks!
Because my pfsense hardware was already EOL, I deceided to replace it to a new one. I m happy with pfsense, but in the past I faced sometimes when I switched off the power problems, that after the firewall is not able to boot anymore. A friend recommended a other file format for the pfsense, that I used with new hardware.
However the new hardware is now since 1,5 month in use, I faced again a downtime because of pfsense. She is boot looping and show the following error messages on the console:
cing command kernel pid 0 tid 100052 td 0xfffff800046d4740
sched_switch() at sched_switch+0x630/frame 0xfffffe000045e9a0
mi_switch() at mi_switch+0xd4/frame 0xfffffe000045e9d0
sleepq_wait() at sleepq_wait+0x2c/frame 0xfffffe000045ea00
_sleep() at _sleep+0x253/frame 0xfffffe000045ea80
taskqueue_thread_loop() at taskqueue_thread_loop+0xe9/frame 0xfffffe000045eab0
fork_exit() at fork_exit+0x7e/frame 0xfffffe000045eaf0
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000045eaf0
--- trap 0, rip = 0, rsp = 0, rbp = 0 ---Tracing command kernel pid 0 tid 100053 td 0xfffff800046d4000
sched_switch() at sched_switch+0x630/frame 0xfffffe00004639a0
mi_switch() at mi_switch+0xd4/frame 0xfffffe00004639d0
sleepq_wait() at sleepq_wait+0x2c/frame 0xfffffe0000463a00
_sleep() at _sleep+0x253/frame 0xfffffe0000463a80
taskqueue_thread_loop() at taskqueue_thread_loop+0xe9/frame 0xfffffe0000463ab0
fork_exit() at fork_exit+0x7e/frame 0xfffffe0000463af0
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000463af0
--- trap 0, rip = 0, rsp = 0, rbp = 0 ---Tracing command kernel pid 0 tid 100054 td 0xfffff80004929740
sched_switch() at sched_switch+0x630/frame 0xfffffe00004689a0
mi_switch() at mi_switch+0xd4/frame 0xfffffe00004689d0
sleepq_wait() at sleepq_wait+0x2c/frame 0xfffffe0000468a00
_sleep() at _sleep+0x253/frame 0xfffffe0000468a80
taskqueue_thread_loop() at taskqueue_thread_loop+0xe9/frame 0xfffffe0000468ab0
fork_exit() at fork_exit+0x7e/frame 0xfffffe0000468af0
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000468af0
--- trap 0, rip = 0, rsp = 0, rbp = 0 ---Tracing command kernel pid 0 tid 100055 td 0xfffff80004929000
sched_switch() at sched_switch+0x630/frame 0xfffffe000046d9a0
mi_switch() at mi_switch+0xd4/frame 0xfffffe000046d9d0
sleepq_wait() at sleepq_wait+0x2c/frame 0xfffffe000046da00
_sleep(0xfffffe000051da00
_sleep() at _sleep+0x253/frame 0xfffffe000051da80
taskqueue_thread_loop() at taskqueue_thread_loop+0xe9/frame 0xfffffe000051dab0
fork_exit() at fork_exit+0x7e/frame 0xfffffe000051daf0
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe000051daf0
--- trap 0, rip = 0, rsp = 0, rbp = 0 ---
db:0:kdb.enter.default> capture off
db:0:kdb.enter.default> textdump dump
Textdump complete.
db:0:kdb.enter.default> reset
cpu_reset: Restarting BSPWhat is the issue? Can I fix this without again reinstall the firewall? Is it really required to use a USV in front of pfsense?
greetings
-
@interessierter
Ouch. That looks like something got very corrupted.
Others will likely have more input, but I'd say you're going to want to reinstall.
What hardware is it running on? A Negtate device, a homebuilt device, a VM?
What version of pfSense did you install?
What file format are you using, ZFS or UFS?
I have always run my critical infrastructure (even at home) behind a UPS; it just makes sense to do that at least to me. -
Hello!
It s this device: https://www.ipu-system.de/produkte/ipu660.html
It s installed on ZFSThis device was brand new installed 1,5 month ago. The old pfsense I have installed 3 times in a row only because of disable the power one time. (Power OFF / new install & this 3 times)
It s "always" better to have a USV sure, but honestly, this device must be able to handle that. Sure sometimes thinks go wrong, but this is not sometimes anymore. Every Windows Server is more rebust than this.
-
@interessierter said in Boot loop:
https://www.ipu-system.de/produkte/ipu660.html
That's a nice looking device, should be fine.
ZFS should be fine too.I think the latest version of the CE line is 2.5.2, is that what you installed? Just double checking because it helps rule out bugs in old code or incompatibility with your hardware.
Agree on yes it should be more robust on hard power off.
Have you checked memory? Sometimes bad or failing memory causes weird issues.
How about heating? Is it getting good airflow?
If you boot with console attached, have you stopped and checked any BIOS settings, make sure not overclocking RAM? Sometimes setting thing back to default helps because of conservative memory timings.
Thats about all I can think of to look at, hopefully others more familiar with that hardware can help more.
-
That's just the very end of the kernel panic. We need to see the full crash report or at least the begging of the panic with the panic string and back-trace.
ZFS is a lot more resilient than UFS but it's not indestructible. You should never just remove power without shutting down.
Steve
-
@stephenw10 Sorry but when there is a power outage, I don t know it before. Also at this downtime, there was never a powercut. It simply starts on his own:
Loading kernel... /boot/kernel/kernel text=0x1a5477c data=0x140 data=0x1b61790 syms=[0x8+0x1c0f38+0x8+0x1c7041] Loading configured modules... /boot/entropy size=0x1000 can't find '/etc/hostid' Start @ 0xffffffff803a0000 ... EFI framebuffer information: addr, size 0xc0000000, 0x1d4c00 dimensions 800 x 600 stride 800 masks 0x00ff0000, 0x0000ff00, 0x000000ff, 0xff000000 KDB: debugger backends: ddb KDB: current backend: ddb ---<<BOOT>>--- Copyright (c) 1992-2020 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 12.2-STABLE fd0f54f44b5c(RELENG_2_5_0) pfSense amd64 FreeBSD clang version 10.0.1 (git@github.com:llvm/llvm-project.git llvmorg-10.0.1-0-gef32c611aa2) VT(efifb): resolution 800x600 CPU: Intel(R) Core(TM) i3-6006U CPU @ 2.00GHz (1992.10-MHz K8-class CPU) Origin="GenuineIntel" Id=0x406e3 Family=0x6 Model=0x4e Stepping=3 Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE> Features2=0x7ffafbbf<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,SDBG,FMA,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND> AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM> AMD Features2=0x121<LAHF,ABM,Prefetch> Structured Extended Features=0x29c67af<FSGSBASE,TSCADJ,SGX,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,NFPUSG,MPX,RDSEED,ADX,SMAP,CLFLUSHOPT,PROCTRACE> Structured Extended Features3=0xc000000<IBPB,STIBP> XSAVE Features=0xf<XSAVEOPT,XSAVEC,XINUSE,XSAVES> VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID TSC: P-state invariant, performance statistics real memory = 17179869184 (16384 MB) avail memory = 16494395392 (15730 MB) Event timer "LAPIC" quality 600 ACPI APIC Table: <ALASKA A M I > FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs FreeBSD/SMP: 1 package(s) x 2 core(s) x 2 hardware threads random: unblocking device. ioapic0 <Version 2.0> irqs 0-119 on motherboard Launching APs: 1 3 2 Timecounter "TSC" frequency 1992098554 Hz quality 1000 wlan: mac acl policy registered random: entropy device external interface ipw_bss: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw.LICENSE. ipw_bss: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf. module_register_init: MOD_LOAD (ipw_bss_fw, 0xffffffff8073dd40, 0) error 1 ipw_ibss: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw.LICENSE. ipw_ibss: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf. module_register_init: MOD_LOAD (ipw_ibss_fw, 0xffffffff8073ddf0, 0) error 1 ipw_monitor: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw.LICENSE. ipw_monitor: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf. module_register_init: MOD_LOAD (ipw_monitor_fw, 0xffffffff8073dea0, 0) error 1 iwi_bss: You need to read the LICENSE file in /usr/share/doc/legal/intel_iwi.LICENSE. iwi_bss: If you agree with the license, set legal.intel_iwi.license_ack=1 in /boot/loader.conf. module_register_init: MOD_LOAD (iwi_bss_fw, 0xffffffff80765730, 0) error 1 iwi_ibss: You need to read the LICENSE file in /usr/share/doc/legal/intel_iwi.LICENSE. iwi_ibss: If you agree with the license, set legal.intel_iwi.license_ack=1 in /boot/loader.conf. module_register_init: MOD_LOAD (iwi_ibss_fw, 0xffffffff807657e0, 0) error 1 iwi_monitor: You need to read the LICENSE file in /usr/share/doc/legal/intel_iwi.LICENSE. iwi_monitor: If you agree with the license, set legal.intel_iwi.license_ack=1 in /boot/loader.conf. module_register_init: MOD_LOAD (iwi_monitor_fw, 0xffffffff80765890, 0) error 1 000.000018 [4336] netmap_init netmap: loaded module module_register_init: MOD_LOAD (vesa, 0xffffffff8140c3e0, 0) error 19 kbd0 at kbdmux0 random: registering fast source Intel Secure Key RNG random: fast provider: "Intel Secure Key RNG" [ath_hal] loaded mlx5en: Mellanox Ethernet driver 3.5.2 (September 2019) nexus0 efirtc0: <EFI Realtime Clock> on motherboard efirtc0: registered as a time-of-day clock, resolution 1.000000s cryptosoft0: <software crypto> on motherboard padlock0: No ACE support. acpi0: <ALASKA A M I > on motherboard acpi0: Power Button (fixed) cpu0: <ACPI CPU> on acpi0 hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 24000000 Hz quality 950 Event timer "HPET" frequency 24000000 Hz quality 550 Event timer "HPET1" frequency 24000000 Hz quality 440 Event timer "HPET2" frequency 24000000 Hz quality 440 Event timer "HPET3" frequency 24000000 Hz quality 440 Event timer "HPET4" frequency 24000000 Hz quality 440 atrtc0: <AT realtime clock> port 0x70-0x77 irq 8 on acpi0 atrtc0: Warning: Couldn't map I/O. atrtc0: registered as a time-of-day clock, resolution 1.000000s Event timer "RTC" frequency 32768 Hz quality 0 attimer0: <AT timer> port 0x40-0x43,0x50-0x53 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1808-0x180b on acpi0 pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0 pci0: <ACPI PCI bus> on pcib0 vgapci0: <VGA-compatible display> port 0xf000-0xf03f mem 0xde000000-0xdeffffff,0xc0000000-0xcfffffff irq 16 at device 2.0 on pci0 vgapci0: Boot video device xhci0: <Intel Sunrise Point-LP USB 3.0 controller> mem 0xdf610000-0xdf61ffff irq 16 at device 20.0 on pci0 xhci0: 32 bytes context size, 64-bit DMA usbus0 on xhci0 usbus0: 5.0Gbps Super Speed USB v3.0 pci0: <simple comms> at device 22.0 (no driver attached) ahci0: <Intel Sunrise Point-LP AHCI SATA controller> port 0xf090-0xf097,0xf080-0xf083,0xf060-0xf07f mem 0xdf628000-0xdf629fff,0xdf62d000-0xdf62d0ff,0xdf62c000-0xdf62c7ff irq 16 at device 23.0 on pci0 ahci0: AHCI v1.31 with 3 6Gbps ports, Port Multiplier not supported ahcich0: <AHCI channel> at channel 0 on ahci0 ahcich1: <AHCI channel> at channel 1 on ahci0 ahcich2: <AHCI channel> at channel 2 on ahci0 pcib1: <ACPI PCI-PCI bridge> irq 16 at device 28.0 on pci0 pci1: <ACPI PCI bus> on pcib1 igb0: <Intel(R) PRO/1000 PCI-Express Network Driver> port 0xe000-0xe01f mem 0xdf500000-0xdf51ffff,0xdf520000-0xdf523fff irq 16 at device 0.0 on pci1 igb0: Using 1024 TX descriptors and 1024 RX descriptors igb0: Using 2 RX queues 2 TX queues igb0: Using MSI-X interrupts with 3 vectors igb0: Ethernet address: 40:62:31:08:2a:43 igb0: netmap queues/slots: TX 2/1024, RX 2/1024 pcib2: <ACPI PCI-PCI bridge> irq 17 at device 28.1 on pci0 pci2: <ACPI PCI bus> on pcib2 igb1: <Intel(R) PRO/1000 PCI-Express Network Driver> port 0xd000-0xd01f mem 0xdf400000-0xdf41ffff,0xdf420000-0xdf423fff irq 17 at device 0.0 on pci2 igb1: Using 1024 TX descriptors and 1024 RX descriptors igb1: Using 2 RX queues 2 TX queues igb1: Using MSI-X interrupts with 3 vectors igb1: Ethernet address: 40:62:31:08:2a:44 igb1: netmap queues/slots: TX 2/1024, RX 2/1024 pcib3: <ACPI PCI-PCI bridge> irq 18 at device 28.2 on pci0 pci3: <ACPI PCI bus> on pcib3 igb2: <Intel(R) PRO/1000 PCI-Express Network Driver> port 0xc000-0xc01f mem 0xdf300000-0xdf31ffff,0xdf320000-0xdf323fff irq 18 at device 0.0 on pci3 igb2: Using 1024 TX descriptors and 1024 RX descriptors igb2: Using 2 RX queues 2 TX queues igb2: Using MSI-X interrupts with 3 vectors igb2: Ethernet address: 40:62:31:08:2a:45 igb2: netmap queues/slots: TX 2/1024, RX 2/1024 pcib4: <ACPI PCI-PCI bridge> irq 19 at device 28.3 on pci0 pci4: <ACPI PCI bus> on pcib4 igb3: <Intel(R) PRO/1000 PCI-Express Network Driver> port 0xb000-0xb01f mem 0xdf200000-0xdf21ffff,0xdf220000-0xdf223fff irq 19 at device 0.0 on pci4 igb3: Using 1024 TX descriptors and 1024 RX descriptors igb3: Using 2 RX queues 2 TX queues igb3: Using MSI-X interrupts with 3 vectors igb3: Ethernet address: 40:62:31:08:2a:46 igb3: netmap queues/slots: TX 2/1024, RX 2/1024 pcib5: <ACPI PCI-PCI bridge> irq 16 at device 28.4 on pci0 pci5: <ACPI PCI bus> on pcib5 igb4: <Intel(R) PRO/1000 PCI-Express Network Driver> port 0xa000-0xa01f mem 0xdf100000-0xdf11ffff,0xdf120000-0xdf123fff irq 16 at device 0.0 on pci5 igb4: Using 1024 TX descriptors and 1024 RX descriptors igb4: Using 2 RX queues 2 TX queues igb4: Using MSI-X interrupts with 3 vectors igb4: Ethernet address: 40:62:31:08:2a:47 igb4: netmap queues/slots: TX 2/1024, RX 2/1024 pcib6: <ACPI PCI-PCI bridge> irq 17 at device 28.5 on pci0 pci6: <ACPI PCI bus> on pcib6 igb5: <Intel(R) PRO/1000 PCI-Express Network Driver> port 0x9000-0x901f mem 0xdf000000-0xdf01ffff,0xdf020000-0xdf023fff irq 17 at device 0.0 on pci6 igb5: Using 1024 TX descriptors and 1024 RX descriptors igb5: Using 2 RX queues 2 TX queues igb5: Using MSI-X interrupts with 3 vectors igb5: Ethernet address: 40:62:31:08:2a:48 igb5: netmap queues/slots: TX 2/1024, RX 2/1024 isab0: <PCI-ISA bridge> at device 31.0 on pci0 isa0: <ISA bus> on isab0 pci0: <memory> at device 31.2 (no driver attached) hdac0: <Intel Sunrise Point-LP HDA Controller> mem 0xdf620000-0xdf623fff,0xdf600000-0xdf60ffff irq 16 at device 31.3 on pci0 acpi_button0: <Sleep Button> on acpi0 acpi_button1: <Power Button> on acpi0 acpi_tz0: <Thermal Zone> on acpi0 acpi_tz1: <Thermal Zone> on acpi0 uart0: <16950 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart0: console (115200,n,8,1) uart1: <16950 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 orm0: <ISA Option ROM> at iomem 0xc0000-0xcffff pnpid ORM0000 on isa0 est0: <Enhanced SpeedStep Frequency Control> on cpu0 Timecounters tick every 1.000 msec hdacc0: <Intel Skylake HDA CODEC> at cad 2 on hdac0 hdaa0: <Intel Skylake Audio Function Group> at nid 1 on hdacc0 pcm0: <Intel Skylake (HDMI/DP 8ch)> at nid 3 on hdaa0 Trying to mount root from ufs:/dev/gptid/6990fa38-0058-11ec-b1f3-406231082a43 [rw,noatime]... Root mount waiting for: usbus0 CAM ugen0.1: <0x8086 XHCI root HUB> at usbus0 uhub0: <0x8086 XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0 uhub0: 18 ports with 18 removable, self powered Root mount waiting for: CAM Root mount waiting for: CAM Root mount waiting for: CAM Root mount waiting for: CAM Root mount waiting for: CAM Root mount waiting for: CAM Root mount waiting for: CAM Root mount waiting for: CAM Root mount waiting for: CAM ada0 at ahcich1 bus 0 scbus1 target 0 lun 0 ada0: <TS32GMSA370S S0903D> ACS-2 ATA SATA 3.x device ada0: Serial Number G525650835 ada0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 1024bytes) ada0: Command Queueing enabled ada0: 30533MB (62533296 512 byte sectors) mountroot: waiting for device /dev/gptid/6990fa38-0058-11ec-b1f3-406231082a43... WARNING: / was not properly dismounted Dual Console: Serial Primary, Video Secondary Configuring crash dumps... Using /dev/gptid/6991ddb9-0058-11ec-b1f3-406231082a43 for dump device. ** SU+J Recovering /dev/gptid/6990fa38-0058-11ec-b1f3-406231082a43 ** Reading 33554432 byte journal from inode 4. ** Building recovery table. ** Resolving unreferenced inode list. ** Processing journal entries. ** 36 journal records in 1536 bytes for 75.00% utilization ** Freed 0 inodes (0 dirs) 4 blocks, and 2 frags. ***** FILE SYSTEM MARKED CLEAN ***** Filesystems are clean, continuing... Mounting filesystems... __ _ __ / _|___ ___ _ __ ___ ___ | '_ \| |_/ __|/ _ \ '_ \/ __|/ _ \ | |_) | _\__ \ __/ | | \__ \ __/ | .__/|_| |___/\___|_| |_|___/\___| |_| Welcome to pfSense 2.5.2-RELEASE... savecore 84 - - reboot after panic: ffs_valloc: dup alloc savecore 84 - - writing core to /var/crash/textdump.tar.0 ...ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/compat/pkg /usr/local/lib/compat/pkg /usr/local/lib/ipsec /usr/local/lib/perl5/5.32/mach/CORE 32-bit compatibility ldconfig path: done. 2021-10-04T14:06:56.661434+02:00 php-fpm 330 - - /ecl.php: No config.xml found, attempting last known config restore. 2021-10-04T14:06:56.662507+02:00 php-fpm 330 - - /ecl.php: New alert found: No config.xml found, attempting last known config restore. mode = 0100666, inum = 321055, fs = / panic: ffs_valloc: dup alloc cpuid = 2 time = 1633349216 KDB: enter: panic [ thread pid 334 tid 100131 ] Stopped at kdb_enter+0x37: movq $0,0x28e3166(%rip) db:0:kdb.enter.default> textdump set textdump set db:0:kdb.enter.default> capture on db:0:kdb.enter.default> run lockinfo db:1:lockinfo> show locks No such command; use "help" to list available commands db:1:lockinfo> show alllocks No such command; use "help" to list available commands db:1:lockinfo> show lockedvnods Locked vnodes vnode 0xfffff8000a51b3c0: tag ufs, type VDIR usecount 1, writecount 0, refcount 4 mountedhere 0 flags (VI_ACTIVE) v_object 0xfffff8000a1d8800 ref 0 pages 1 cleanbuf 0 dirtybuf 1 lock type ufs: EXCL by thread 0xfffff800045ee740 (pid 334, php-cgi, tid 100131) ino 321024, on dev gptid/6990fa38-0058-11ec-b1f3-406231082a43 vnode 0xfffff8000a9bad20: tag ufs, type VREG usecount 1, writecount 0, refcount 1 flags (VI_ACTIVE) lock type ufs: EXCL by thread 0xfffff800045ee740 (pid 334, php-cgi, tid 100131) ino 321055, on dev gptid/6990fa38-0058-11ec-b1f3-406231082a43 db:0:kdb.enter.default> show pcpu cpuid = 2 dynamic pcpu = 0xfffffe0080d7f380 curthread = 0xfffff800045ee740: pid 334 tid 100131 "php-cgi" curpcb = 0xfffff800045eece0 fpcurthread = 0xfffff800045ee740: pid 334 "php-cgi" idlethread = 0xfffff80004619000: tid 100005 "idle: cpu2" curpmap = 0xfffff8000a531138 tssp = 0xffffffff837176f0 commontssp = 0xffffffff837176f0 rsp0 = 0xfffffe00005f5bc0 kcr3 = 0x800000000a61b08d ucr3 = 0x800000000a62488d scr3 = 0xa62488d gs32p = 0xffffffff8371df08 ldt = 0xffffffff8371df48 tss = 0xffffffff8371df38 tlb gen = 999 curvnet = 0 db:0:kdb.enter.default> bt Tracing pid 334 tid 100131 td 0xfffff800045ee740 kdb_enter() at kdb_enter+0x37/frame 0xfffffe00005f53d0 vpanic() at vpanic+0x197/frame 0xfffffe00005f5420 panic() at panic+0x43/frame 0xfffffe00005f5480 ffs_valloc() at ffs_valloc+0x8f3/frame 0xfffffe00005f5510 ufs_makeinode() at ufs_makeinode+0xa3/frame 0xfffffe00005f56a0 ufs_create() at ufs_create+0x34/frame 0xfffffe00005f56c0 VOP_CREATE_APV() at VOP_CREATE_APV+0x75/frame 0xfffffe00005f56f0 vn_open_cred() at vn_open_cred+0x2e6/frame 0xfffffe00005f5840 kern_openat() at kern_openat+0x213/frame 0xfffffe00005f59c0 amd64_syscall() at amd64_syscall+0x387/frame 0xfffffe00005f5af0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00005f5af0 --- syscall (5, FreeBSD ELF64, sys_open), rip = 0x800b2e7ea, rsp = 0x7fffffffdc28, rbp = 0x7fffffffe170 --- db:0:kdb.enter.default> ps pid ppid pgrp uid state wmesg wchan cmd 334 1 329 0 R CPU 2 php-cgi 332 29 29 0 S+ sbwait 0xfffff8000a0198a4 fcgicli 331 329 329 0 S accept 0xfffff8000a6a1ba0 php-fpm 330 329 329 0 R CPU 3 php-fpm 329 1 329 0 Ss kqread 0xfffff800045cde00 php-fpm 68 0 0 0 DL mdwait 0xfffff8000a1ca000 [md0] 29 1 29 0 Ss+ pause 0xfffff800047a15d8 sh 28 0 0 0 DL aldslp 0xffffffff8368dfc0 [ALQ Daemon] 27 0 0 0 DL syncer 0xffffffff836ed508 [syncer] 26 0 0 0 DL vlruwt 0xfffff80007f96530 [vnlru] 25 0 0 0 DL (threaded) [bufdaemon] 100090 D qsleep 0xffffffff836ec810 [bufdaemon] 100095 D - 0xffffffff81e0a900 [bufspacedaemon-0] 100096 D - 0xffffffff81e12bc0 [bufspacedaemon-1] 100098 D - 0xffffffff81e1ae80 [bufspacedaemon-2] 100099 D - 0xffffffff81e23140 [bufspacedaemon-3] 100100 D - 0xffffffff81e2b400 [bufspacedaemon-4] 100101 D - 0xffffffff81e336c0 [bufspacedaemon-5] 100102 D - 0xffffffff81e3b980 [bufspacedaemon-6] 100108 D sdflush 0xfffff80004086ee8 [/ worker] 24 0 0 0 DL psleep 0xffffffff83715008 [vmdaemon] 23 0 0 0 DL (threaded) [pagedaemon] 100088 D psleep 0xffffffff83709dd8 [dom0] 100091 D launds 0xffffffff83709de4 [laundry: dom0] 100093 D umarcl 0xffffffff811dea80 [uma] 22 0 0 0 DL - 0xffffffff834e5c58 [rand_harvestq] 21 0 0 0 DL tzpoll 0xffffffff833a8e48 [acpi_thermal] 20 0 0 0 DL pftm 0xffffffff81091f10 [pf purge] 19 0 0 0 DL waiting 0xffffffff836f2878 [sctp_iterator] 18 0 0 0 DL - 0xffffffff836ec21c [soaiod4] 17 0 0 0 DL - 0xffffffff836ec21c [soaiod3] 9 0 0 0 DL - 0xffffffff836ec21c [soaiod2] 8 0 0 0 DL - 0xffffffff836ec21c [soaiod1] 16 0 0 0 DL (threaded) [usb] 100063 D - 0xfffffe0004384440 [usbus0] 100064 D - 0xfffffe0004384498 [usbus0] 100065 D - 0xfffffe00043844f0 [usbus0] 100066 D - 0xfffffe0004384548 [usbus0] 100067 D - 0xfffffe00043845a0 [usbus0] 7 0 0 0 DL (threaded) [cam] 100061 D - 0xffffffff8339c5c0 [doneq0] 100086 D - 0xffffffff8339c490 [scanner] 6 0 0 0 DL crypto_ 0xfffff80004629d50 [crypto returns 3] 5 0 0 0 DL crypto_ 0xfffff80004629cf0 [crypto returns 2] 4 0 0 0 DL crypto_ 0xfffff80004629c90 [crypto returns 1] 3 0 0 0 DL crypto_ 0xfffff80004629c30 [crypto returns 0] 2 0 0 0 DL crypto_ 0xffffffff83707538 [crypto] 15 0 0 0 DL seqstat 0xfffff80004a14488 [sequencer 00] 14 0 0 0 DL (threaded) [geom] 100048 D - 0xffffffff8368c638 [g_event] 100049 D - 0xffffffff8368c640 [g_up] 100050 D - 0xffffffff8368c648 [g_down] 13 0 0 0 DL (threaded) [ng_queue] 100043 D sleep 0xffffffff8330f150 [ng_queue0] 100044 D sleep 0xffffffff8330f150 [ng_queue1] 100045 D sleep 0xffffffff8330f150 [ng_queue2] 100046 D sleep 0xffffffff8330f150 [ng_queue3] 12 0 0 0 WL (threaded) [intr] 100007 I [swi6: Giant taskq] 100010 I [swi5: fast taskq] 100013 I [swi6: task queue] 100034 I [swi3: vm] 100035 I [swi4: clock (0)] 100036 I [swi4: clock (1)] 100037 I [swi4: clock (2)] 100038 I [swi4: clock (3)] 100039 I [swi1: netisr 0] 100040 I [swi1: netisr 1] 100041 I [swi1: netisr 2] 100042 I [swi1: netisr 3] 100062 I [irq264: xhci0] 100068 I [irq265: ahci0] 100069 I [i [idle: cpu2] 100006 CanRun [idle: cpu3] 1 0 1 0 SLs wait 0xfffff8000461a530 [init] 10 0 0 0 DL audit_w 0xffffffff83707b18 [audit] 0 0 0 0 DLs (threaded) [kernel] 100000 D swapin 0xffffffff8368c7e8 [swapper] 100008 D - 0xfffff800045f9700 [thread taskq] 100009 D - 0xfffff800045f9500 [aiod_kick taskq] 100011 D - 0xfffff800045f9000 [kqueue_ctx taskq] 100012 D - 0xfffff800045fad00 [inm_free taskq] 100014 D - 0xfffff800045fa800 [in6m_free taskq] 100015 D - 0xfffff800045fa600 [linuxkpi_short_wq_0] 100016 D - 0xfffff800045fa600 [linuxkpi_short_wq_1] 100017 D - 0xfffff800045fa600 [linuxkpi_short_wq_2] 100018 D - 0xfffff800045fa600 [linuxkpi_short_wq_3] 100019 D - 0xfffff800045fa600 [linuxkpi_short_wq_4] 100020 D - 0xfffff800045fa000 [linuxkpi_long_wq_0] 100021 D - 0xfffff800045fa000 [linuxkpi_long_wq_1] 100022 D - 0xfffff800045fa000 [linuxkpi_long_wq_2] 100023 D - 0xfffff800045fa000 [linuxkpi_long_wq_3] 100024 D - 0xfffff800045fa000 [linuxkpi_long_wq_4] 100025 D - 0xfffff800045fb900 [softirq_0] 100026 D - 0xfffff800045fb700 [softirq_1] 100027 D - 0xfffff800045fb500 [softirq_2] 100028 D - 0xfffff800045fb300 [softirq_3] 100029 D - 0xfffff800045fb100 [if_config_tqg_0] 100030 D - 0xfffff80004905300 [if_io_tqg_0] 100031 D - 0xfffff80004905100 [if_io_tqg_1] 100032 D - 0xfffff80004906e00 [if_io_tqg_2] 100033 D - 0xfffff80004906c00 [if_io_tqg_3] 100047 D - 0xfffff80004713b00 [firmware taskq] 100052 D - 0xfffff80004714400 [crypto_0] 100053 D - 0xfffff80004714400 [crypto_1] 100054 D - 0xfffff80004714400 [crypto_2] 100055 D - 0xfffff80004714400 [crypto_3] 100071 D - 0xfffff80004db3800 [mca taskq] 100082 D - 0xfffff80007f5e200 [acpi_task_0] 100083 D - 0xfffff80007f5e200 [acpi_task_1] 100084 D - 0xfffff80007f5e200 [acpi_task_2] 100085 D - 0xfffff80004acb300 [CAM taskq] 100087 D - 0xfffff80007f60b00 [mlx4] db:0:kdb.enter.default> alltrace Tracing command php-cgi pid 334 tid 100131 td 0xfffff800045ee740 (CPU 2) kdb_enter() at kdb_enter+0x37/frame 0xfffffe00005f53d0 vpanic() at vpanic+0x197/frame 0xfffffe00005f5420 panic() at panic+0x43/frame 0xfffffe00005f5480 ffs_valloc() at ffs_valloc+0x8f3/frame 0xfffffe00005f5510 ufs_makeinode() at ufs_makeinode+0xa3/frame 0xfffffe00005f56a0 ufs_create() at ufs_create+0x34/frame 0xfffffe00005f56c0 VOP_CREATE_APV() at VOP_CREATE_APV+0x75/frame 0xfffffe00005f56f0 vn_open_cred() at vn_open_cred+0x2e6/frame 0xfffffe00005f5840 kern_openat() at kern_openat+0x213/frame 0xfffffe00005f59c0 amd64_syscall() at amd64_syscall+0x387/frame 0xfffffe00005f5af0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00005f5af0 --- syscall (5, FreeBSD ELF64, sys_open), rip = 0x800b2e7ea, rsp = 0x7fffff40 sched_switch() at sched_switch+0x630/frame 0xfffffe00005d2740 mi_switch() at mi_switch+0xd4/frame 0xfffffe00005d2770 sleepq_catch_signals() at sleepq_catch_signals+0x403/frame 0xfffffe00005d27c0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00005d27f0 _sleep() at _sleep+0x1de/frame 0xfffffe00005d2870 solisten_dequeue() at solisten_dequeue+0x72/frame 0xfffffe00005d28c0 kern_accept4() at kern_accept4+0x1bc/frame 0xfffffe00005d2960 accept1() at accept1+0x60/frame 0xfffffe00005d29c0 amd64_syscall() at amd64_syscall+0x387/frame 0xfffffe00005d2af0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00005d2af0 --- syscall (30, FreeBSD ELF64, sys_accept), rip = 0x800ae739a, rsp = 0x7ffffffee898, rbp = 0x7ffffffee8d0 --- Tracing command php-fpm pid 330 tid 100114 td 0xfffff80004889000 (CPU 3) cpustop_handler() at cpustop_handler+0x27/frame 0xfffffe00040f2e00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe00040f2e10 trap() at trap+0x44/frame 0xfffffe00040f2f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe00040f2f20 --- trap 0x13, rip = 0x800b34312, rsp = 0x7fffffffcd10, rbp = 0x7fffffffcd50 --- Tracing command php-fpm pid 329 tid 100129 td 0xfffff8000a506000 sched_switch() at sched_switch+0x630/frame 0xfffffe00005eb480 mi_switch() at mi_switch+0xd4/frame 0xfffffe00005eb4b0 sleepq_catch_signals() at sleepq_catch_signals+0x403/frame 0xfffffe00005eb500 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x14/frame 0xfffffe00005eb540 _sleep() at _sleep+0x1b3/frame 0xfffffe00005eb5c0 kqueue_kevent() at kqueue_kevent+0xaee/frame 0xfffffe00005eb890 kern_kevent() at kern_kevent+0xb5/frame 0xfffffe00005eb900 kern_kevent_generic() at kern_kevent_generic+0x70/frame 0xfffffe00005eb960 sys_kevent() at sys_kevent+0x61/frame 0xfffffe00005eb9c0 amd64_syscall() at amd64_syscall+0x387/frame 0xfffffe00005ebaf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00005ebaf0 --- syscall (560, FreeBSD ELF64, sys_kevent), rip = 0x800ae72da, rsp = 0x7fffffffe848, rbp = 0x7fffffffe890 --- Tracing command md0 pid 68 tid 100117 td 0xfffff8000a195000 sched_switch() at sched_switch+0x630/frame 0xfffffe00005af970 mi_switch() at mi_switch+0xd4/frame 0xfffffe00005af9a0 sleepq_wait() at sleepq_wait+0x2c/frame 0xfffffe00005af9d0 _sleep() at _sleep+0x253/frame 0xfffffe00005afa50 md_kthread() at md_kthread+0x11e/frame 0xfffffe00005afab0 fork_exit() at fork_exit+0x7e/frame 0xfffffe00005afaf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00005afaf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Tracing command sh pid 29 tid 100103 td 0xfffff80007f1d740 sched_switch() at sched_switch+0x630/frame 0xfffffe00005697f0 mi_switch() at mi_switch+0xd4/frame 0xfffffe0000569820 sleepq_catch_signals() at sleepq_catch_signals+0x403/frame 0xfffffe0000569870 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00005698a0 _sleep() at _sleep+0x1de/frame 0xfffffe0000569920 kern_sigsuspend() at kern_sigsuspend+0x164/frame 0xfffffe0000569990 sys_sigsuspend() at sys_sigsuspend+0x31/frame 0xfffffe00005699c0 amd64_syscall() at amd64_syscall+0x387/frame 0xfffffe0000569af0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0000569af0 --- syscall (4, FreeBSD ELF64, sys_write), rip = 0x8003eb19a, rsp = 0x7fffffffe578, rbp = 0x7fffffffe630 --- -
After these :
@interessierter said in Boot loop:
2021-10-04T14:06:56.661434+02:00 php-fpm 330 - - /ecl.php: No config.xml found, attempting last known config restore.
2021-10-04T14:06:56.662507+02:00 php-fpm 330 - - /ecl.php: New alert found: No config.xml found, attempting last known config restore.your system goes down hill fast.
Most probably :@interessierter said in Boot loop:
panic: ffs_valloc: dup alloc
The other week there was some one asking why PHP was complaining about reading that 1.5 Giga config file ....
Or, these files could 'weights' several Megabytes maximum.This is mine :
[2.5.2-RELEASE][admin@pfsense.local.tld]/root: ls -al /conf/config.xml -rw-r--r-- 1 root wheel 440560 Sep 30 16:41 /conf/config.xmlJust to be sure : consider the file system hosed. Re install.
Just to be sure : never ever rip out the power. Do this with your PC and your Windows won't boot any more for the same reason. Or MacOS, or any Linux distro.
Power outage are even worse : instead of a clean power cut, the power stays on, offering half the tension ... this makes the power converter panic, and it could output nothing, anything, or some random tension.
Power issues are normal. For don't care devices : you don't care.
Sensible devices : get an UPS. -
@gertjan My Windows Server are always coming up again. However this ti9me there was no power problems, and still run in this issue.
When I now preapre the usb stick and copy the backup in, is than the recovery without a keyboard imput possible?
-
@interessierter I have now tried every available image on two different USB Sticks, the device is not booting from USB. I have no option to change the boot order. Also no ask if I want to boot from USB
-
@interessierter That almost sounds like you've got a hardware failure that has been building.
-
@interessierter said in Boot loop:
mode = 0100666, inum = 321055, fs = /
panic: ffs_valloc: dup alloc
cpuid = 2
time = 1633349216
KDB: enter: panicYes, that is a filesystem damage error. But it will have been caused by something else previously. Whatever caused it to reboot initially is the real cause here.
Steve
