4200 crashed

Gertjan

The web server encountered an rrror processing

If this happens, you can't use the second most important interface(s) on your router = all the NICs.
You need the most important interface : the console.

You need the most obvious : Security Gateway Manual Netgate-4200.
and this one : Connect to the Console.

and of course the serial USB console that came with your 4200 as that cable is actually the most important item you've found in the box ^^

From there on, you can see the 4200 boot .... and see errors, and act upon them.

wgstarks

@Gertjan
Looks like a completely normal boot. No idea why. I haven't done anything to it. Attaching the console boot.

-          __                                               
    _ __  / _|___  ___ _ __  ___  ___      _                
   | '_ \| |_/ __|/ _ \ '_ \/ __|/ _ \   _| |_              
   | |_) |  _\__ \  __/ | | \__ \  __/  |_   _|             
   | .__/|_| |___/\___|_| |_|___/\___|    |_|               
   |_|                                                      
                                                            
                                                                              
 ╔════ Welcome to Netgate pfSense Plus ════╗      __________________________  
 ║                                         ║     /                       ___\ 
 ║  1. Boot Multi user [Enter]             ║    |                      /`     
 ║  2. Boot Single user                    ║    |                     /    :-|
 ║  3. Escape to loader prompt             ║    |      _________  ___/    /_ |
 ║  4. Reboot                              ║    |    /` ____   / /__    ___/ |
 ║  5. Cons: Serial                        ║    |   /  /   /  /    /   /     |
 ║                                         ║    |  /  /___/  /    /   /      |
 ║  Options:                               ║    | /   ______/    /   /  _    |
 ║  6. Kernel: default/kernel (1 of 2)     ║    |/   /          /   / _| |_  |
 ║  7. Boot Options                        ║        /          /___/ |_   _| |
 ║  8. Boot Environments                   ║       /                   |_|   |
 ║                                         ║      /_________________________/ 
 ╚═════════════════════════════════════════╝                                  |
   Autoboot in 0 seconds. [Space] to pause 

Loading kernel...
/boot/kernel/kernel text=0x19eec0 text=0xff4c38 text=0x17e3db4 data=0x180 data=0x22d718+0x3d18e8 0x8+0x1cb0f0+0x8+0x1da290\
Loading configured modules...
/boot/kernel/zfs.ko size 0x5ea9a0 at 0x35a6000
/boot/kernel/cryptodev.ko size 0x7718 at 0x3b91000
/boot/entropy size=0x1000
can't find '/etc/hostid'
/boot/kernel/opensolaris.ko size 0x1e2f0 at 0x3b9a000
staging 0x3fc00000-0x43f40000 (not copying) tramp 0x43f40000 PT4 0x43f41000
Start @ 0xffffffff8039f000 ...
GDB: no debug ports present
KDB: debugger backends: ddb
KDB: current backend: ddb
---<<BOOT>>---
Copyright (c) 1992-2024 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
        The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 15.0-CURRENT #0 plus-RELENG_24_03-n256311-e71f834dd81: Fri Apr 19 00:28:14 UTC 2024
    root@freebsd:/var/jenkins/workspace/pfSense-Plus-snapshots-24_03-main/obj/amd64/Y4MAEJ2R/var/jenkins/workspace/pfSense-Plus-snapshots-24_03-main/sources/FreeBSD-src-plus-RELENG_24_03/amd64.amd64/sys/pfSense amd64
FreeBSD clang version 17.0.6 (https://github.com/llvm/llvm-project.git llvmorg-17.0.6-0-g6009708b4367)
VT(vga): resolution 640x480
CPU: Genuine Intel(R) Atom C1110 processor (2112.00-MHz K8-class CPU)
  Origin="GenuineIntel"  Id=0x906a4  Family=0x6  Model=0x9a  Stepping=4
  Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
  Features2=0x7ffafbbf<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,SDBG,FMA,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND>
  AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM>
  AMD Features2=0x121<LAHF,ABM,Prefetch>
  Structured Extended Features=0x239ca7eb<FSGSBASE,TSCADJ,BMI1,AVX2,FDPEXC,SMEP,BMI2,ERMS,INVPCID,NFPUSG,PQE,RDSEED,ADX,SMAP,CLFLUSHOPT,CLWB,PROCTRACE,SHA>
  Structured Extended Features2=0x984007bc<UMIP,PKU,OSPKE,WAITPKG,GFNI,VAES,VPCLMULQDQ,RDPID,MOVDIRI,MOVDIR64B>
  Structured Extended Features3=0xfc184410<FSRM,MD_CLEAR,IBT,IBPB,STIBP,L1DFL,ARCH_CAP,CORE_CAP,SSBD>
  XSAVE Features=0xf<XSAVEOPT,XSAVEC,XINUSE,XSAVES>
  IA32_ARCH_CAPS=0x188fd6b<RDCL_NO,IBRS_ALL,SKIP_L1DFL_VME,MDS_NO,TAA_NO>
  VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID,VID,PostIntr
  TSC: P-state invariant, performance statistics
real memory  = 4294967296 (4096 MB)
avail memory = 3887554560 (3707 MB)
Event timer "LAPIC" quality 600
ACPI APIC Table: <ALASKA A M I >
WARNING: L3 data cache covers more APIC IDs than a package (7 > 3)
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
FreeBSD/SMP: 1 package(s) x 4 core(s)
random: registering fast source Intel Secure Key RNG
random: fast provider: "Intel Secure Key RNG"
random: unblocking device.
ioapic0 <Version 2.0> irqs 0-119
Launching APs: 2 3 1
TCP_ratelimit: Is now initialized
ipw_bss: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw.LICENSE.
ipw_bss: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf.
module_register_init: MOD_LOAD (ipw_bss_fw, 0xffffffff80750310, 0) error 1
ipw_ibss: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw.LICENSE.
ipw_ibss: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf.
module_register_init: MOD_LOAD (ipw_ibss_fw, 0xffffffff807503c0, 0) error 1
ipw_monitor: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw.LICENSE.
ipw_monitor: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf.
module_register_init: MOD_LOAD (ipw_monitor_fw, 0xffffffff80750470, 0) error 1
iwi_bss: You need to read the LICENSE file in /usr/share/doc/legal/intel_iwi.LICENSE.
iwi_bss: If you agree with the license, set legal.intel_iwi.license_ack=1 in /boot/loader.conf.
module_register_init: MOD_LOAD (iwi_bss_fw, 0xffffffff80770010, 0) error 1
iwi_ibss: You need to read the LICENSE file in /usr/share/doc/legal/intel_iwi.LICENSE.
iwi_ibss: If you agree with the license, set legal.intel_iwi.license_ack=1 in /boot/loader.conf.
module_register_init: MOD_LOAD (iwi_ibss_fw, 0xffffffff807700c0, 0) error 1
iwi_monitor: You need to read the LICENSE file in /usr/share/doc/legal/intel_iwi.LICENSE.
iwi_monitor: If you agree with the license, set legal.intel_iwi.license_ack=1 in /boot/loader.conf.
module_register_init: MOD_LOAD (iwi_monitor_fw, 0xffffffff80770170, 0) error 1
random: entropy device external interface
wlan: mac acl policy registered
kbd0 at kbdmux0
WARNING: Device "spkr" is Giant locked and may be deleted before FreeBSD 15.0.
efirtc0: <EFI Realtime Clock>
efirtc0: registered as a time-of-day clock, resolution 1.000000s
netgate0: <Netgate 4200>
vtvga0: <VT VGA driver>
smbios0: <System Management BIOS> at iomem 0x61c6c000-0x61c6c017
smbios0: Version: 3.5
acpi0: <ALASKA A M I >
acpi_ec1: <Embedded Controller: GPE 0x6e, ECDT> port 0x62,0x66 on acpi0
acpi0: Power Button (fixed)
hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 19200000 Hz quality 950
Event timer "HPET" frequency 19200000 Hz quality 550
Event timer "HPET1" frequency 19200000 Hz quality 440
Event timer "HPET2" frequency 19200000 Hz quality 440
Event timer "HPET3" frequency 19200000 Hz quality 440
Event timer "HPET4" frequency 19200000 Hz quality 440
atrtc1: <AT realtime clock> on acpi0
atrtc1: Warning: Couldn't map I/O.
atrtc1: registered as a time-of-day clock, resolution 1.000000s
Event timer "RTC" frequency 32768 Hz quality 0
attimer0: <AT timer> port 0x40-0x43,0x50-0x53 irq 0 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1808-0x180b on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib1: <ACPI PCI-PCI bridge> at device 6.0 on pci0
pci1: <ACPI PCI bus> on pcib1
nvme0: <Generic NVMe Device> mem 0x67c00000-0x67c03fff at device 0.0 on pci1
xhci0: <Intel Alder Lake-P Thunderbolt 4 USB controller> mem 0x4000040000-0x400004ffff at device 13.0 on pci0
xhci0: 32 bytes context size, 64-bit DMA
usbus0 on xhci0
usbus0: 5.0Gbps Super Speed USB v3.0
pci0: <simple comms, UART> at device 18.0 (no driver attached)
pci0: <mass storage> at device 18.7 (no driver attached)
xhci1: <Intel Alder Lake USB 3.2 controller> mem 0x4000020000-0x400002ffff at device 20.0 on pci0
xhci1: 32 bytes context size, 64-bit DMA
usbus1 on xhci1
usbus1: 5.0Gbps Super Speed USB v3.0
pci0: <memory, RAM> at device 20.2 (no driver attached)
pci0: <serial bus> at device 21.0 (no driver attached)
pci0: <serial bus> at device 21.1 (no driver attached)
pci0: <simple comms> at device 22.0 (no driver attached)
pci0: <serial bus> at device 25.0 (no driver attached)
pci0: <serial bus> at device 25.1 (no driver attached)
pcib2: <ACPI PCI-PCI bridge> at device 28.0 on pci0
pci2: <ACPI PCI bus> on pcib2
pcib3: <ACPI PCI-PCI bridge> at device 28.4 on pci0
pci3: <ACPI PCI bus> on pcib3
igc0: <Intel(R) Ethernet Controller I226-V> mem 0x67a00000-0x67afffff,0x67b00000-0x67b03fff at device 0.0 on pci3
igc0: Using 1024 TX descriptors and 1024 RX descriptors
igc0: Using 1 RX queues 1 TX queues
igc0: Using MSI-X interrupts with 2 vectors
igc0: Ethernet address: 90:ec:77:8e:5e:b1
igc0: netmap queues/slots: TX 1/1024, RX 1/1024
pcib4: <ACPI PCI-PCI bridge> at device 28.5 on pci0
pci4: <ACPI PCI bus> on pcib4
igc1: <Intel(R) Ethernet Controller I226-V> mem 0x67700000-0x677fffff,0x67800000-0x67803fff at device 0.0 on pci4
igc1: Using 1024 TX descriptors and 1024 RX descriptors
igc1: Using 1 RX queues 1 TX queues
igc1: Using MSI-X interrupts with 2 vectors
igc1: Ethernet address: 90:ec:77:8e:5e:b0
igc1: netmap queues/slots: TX 1/1024, RX 1/1024
pcib5: <ACPI PCI-PCI bridge> at device 28.6 on pci0
pci5: <ACPI PCI bus> on pcib5
igc2: <Intel(R) Ethernet Controller I226-V> mem 0x67400000-0x674fffff,0x67500000-0x67503fff at device 0.0 on pci5
igc2: Using 1024 TX descriptors and 1024 RX descriptors
igc2: Using 1 RX queues 1 TX queues
igc2: Using MSI-X interrupts with 2 vectors
igc2: Ethernet address: 90:ec:77:8e:5e:af
igc2: netmap queues/slots: TX 1/1024, RX 1/1024
pcib6: <ACPI PCI-PCI bridge> at device 28.7 on pci0
pci6: <ACPI PCI bus> on pcib6
igc3: <Intel(R) Ethernet Controller I226-V> mem 0x67100000-0x671fffff,0x67200000-0x67203fff at device 0.0 on pci6
igc3: Using 1024 TX descriptors and 1024 RX descriptors
igc3: Using 1 RX queues 1 TX queues
igc3: Using MSI-X interrupts with 2 vectors
igc3: Ethernet address: 90:ec:77:8e:5e:ae
igc3: netmap queues/slots: TX 1/1024, RX 1/1024
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
pci0: <serial bus> at device 31.5 (no driver attached)
acpi_acad0: <AC Adapter> on acpi0
ns8250: UART FCR is brokene> iomem 0xfe03e000-0xfe03e007 irq 16 on acpi0
uart2: console (115200,n,8,1)
acpi_button0: <Sleep Button> on acpi0
cpu0: <ACPI CPU> on acpi0
acpi_tz0: <Thermal Zone> on acpi0
acpi_42000: <Netgate 4200 LED controller SILC1001> on acpi0
atrtc0: <AT realtime clock> at port 0x70 irq 8 on isa0
atrtc0: Warning: Couldn't map I/O.
atrtc0: registered as a time-of-day clock, resolution 1.000000s
atrtc0: Can't map interrupt.
hwpstate_intel0: <Intel Speed Shift> on cpu0
hwpstate_intel1: <Intel Speed Shift> on cpu1
hwpstate_intel2: <Intel Speed Shift> on cpu2
hwpstate_intel3: <Intel Speed Shift> on cpu3
Timecounter "TSC" frequency 2112002998 Hz quality 1000
Timecounters tick every 1.000 msec
ugen1.1: <Intel XHCI root HUB> at usbus1
ugen0.1: <Intel XHCI root HUB> at usbus0
ZFS filesystem version: 5
ZFS storage pool version: features support (5000)
uhub0 on usbus1
uhub0: <Intel XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus1
uhub1 on usbus0
uhub1: <Intel XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0
Trying to mount root from zfs:pfSense/ROOT/default []...
uhub1: 3 ports with 3 removable, self powered
uhub0: 16 ports with 16 removable, self powered
Root mount waiting for: usbus1 CAM
Root mount waiting for: usbus1 CAM
ugen1.2: <Generic Ultra Fast Media> at usbus1
umass0 on uhub0
umass0: <Generic Ultra Fast Media, class 0/0, rev 2.00/1.98, addr 1> on usbus1
Root mount waiting for: CAM
Root mount waiting for: CAM
Root mount waiting for: CAM
Root mount waiting for: CAM
Root mount waiting for: CAM
Root mount waiting for: CAM
Root mount waiting for: CAM
nda0 at nvme0 bus 0 scbus0 target 0 lun 1
nda0: <Samsung SSD 980 PRO 500GB 5B2QGXA7 S5GYNX0W819320R>
nda0: Serial Number S5GYNX0W819320R
nda0: nvme version 1.3
nda0: 476940MB (976773168 512 byte sectors)
da0 at umass-sim0 bus 0 scbus1 target 0 lun 0
da0: <Generic Ultra HS-COMBO 1.98> Removable Direct Access SCSI device
da0: Serial Number 000000225001
da0: 40.000MB/s transfers
da0: 14952MB (30621696 512 byte sectors)
da0: quirks=0x2<NO_6_BYTE>
Configuring crash dumps...
Using /dev/nda0p3 for dump device.
/dev/gpt/efiboot0: 8 files, 257 MiB free (16467 clusters)
FIXED
/dev/gpt/efiboot0: MARKING FILE SYSTEM CLEAN
Filesystems are clean, continuing...
Mounting filesystems...
Mounting ZFS boot environment...done.

        __
 _ __  / _|___  ___ _ __  ___  ___      _
| '_ \| |_/ __|/ _ \ '_ \/ __|/ _ \   _| |_
| |_) |  _\__ \  __/ | | \__ \  __/  |_   _|
| .__/|_| |___/\___|_| |_|___/\___|    |_|
|_|


Welcome to Netgate pfSense Plus 24.03-RELEASE (Patch 1)...

Checking dump device /dev/nda0p3 for crash dumps ... no crash dumps on /dev/nda0p3.
...ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/compat/pkg /usr/local/lib/compat/pkg /usr/local/lib/ipsec /usr/local/lib/perl5/5.36/mach/CORE
32-bit compatibility ldconfig path:
done.
3379
>>> Removing vital flag from php83...done.
External config loader 1.0 is now starting... nda0p1 nda0p2 nda0p4
Launching the init system...Updating CPU Microcode...
CPU: Genuine Intel(R) Atom C1110 processor (2112.00-MHz K8-class CPU)
  Origin="GenuineIntel"  Id=0x906a4  Family=0x6  Model=0x9a  Stepping=4
  Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
  Features2=0x7ffafbbf<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,SDBG,FMA,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND>
  AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM>
  AMD Features2=0x121<LAHF,ABM,Prefetch>
  Structured Extended Features=0x239ca7eb<FSGSBASE,TSCADJ,BMI1,AVX2,FDPEXC,SMEP,BMI2,ERMS,INVPCID,NFPUSG,PQE,RDSEED,ADX,SMAP,CLFLUSHOPT,CLWB,PROCTRACE,SHA>
  Structured Extended Features2=0x984007bc<UMIP,PKU,OSPKE,WAITPKG,GFNI,VAES,VPCLMULQDQ,RDPID,MOVDIRI,MOVDIR64B>
  Structured Extended Features3=0xfc184410<FSRM,MD_CLEAR,IBT,IBPB,STIBP,L1DFL,ARCH_CAP,CORE_CAP,SSBD>
  XSAVE Features=0xf<XSAVEOPT,XSAVEC,XINUSE,XSAVES>
  IA32_ARCH_CAPS=0x188fd6b<RDCL_NO,IBRS_ALL,SKIP_L1DFL_VME,MDS_NO,TAA_NO>
  VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID,VID,PostIntr
  TSC: P-state invariant, performance statistics
Done.
 done.
Initializing.................... done.
Starting device manager (devd)...ig4iic0: <Intel Alder Lake-P I2C Controller-0> at device 21.0 on pci0
ig4iic0: Using MSI
iicbus0: <Philips I2C bus (ACPI-hinted)> on ig4iic0
iicbus0: <unknown card> at addr 0x14
ig4iic1: <Intel Alder Lake-P I2C Controller-1> at device 21.1 on pci0
ig4iic1: Using MSI
iicbus1: <Philips I2C bus (ACPI-hinted)> on ig4iic1
ig4iic2: <Intel Alder Lake-P I2C Controller-4> at device 25.0 on pci0
ig4iic2: Using MSI
iicbus2: <Philips I2C bus (ACPI-hinted)> on ig4iic2
ig4iic3: <Intel Alder Lake-P I2C Controller-5> at device 25.1 on pci0
ig4iic3: Using MSI
iicbus3: <Philips I2C bus (ACPI-hinted)> on ig4iic3
driver bug: Unable to set devclass (class: uart devname: (unknown))
driver bug: Unable to set devclass (class: atkbdc devname: (unknown))
ichsmb0: <Intel Alder Lake SMBus controller> port 0xefa0-0xefbf mem 0x4000064000-0x40000640ff at device 31.4 on pci0
smbus0: <System Management Bus> on ichsmb0
done.
Loading configuration....done.
Updating configuration...done.
Loading cryptographic accelerator drivers...aesni0: <AES-CBC,AES-CCM,AES-GCM,AES-ICM,AES-XTS,SHA1,SHA256>
done.
Setting up extended sysctls...done.
Executing early shell commands...done.
Setting timezone...done.
Configuring looplo0: link state changed to UP
back interface...done.
Starting syslog...done.
Setting up interfaces microcode...done.
Configuring loopback interface...done.
Configuring VLAN interfaces...done.
Configuring WAN interface...done.
Configuring VLAN1003_GUEST interface...done.
Configuring VLAN20_IOT interface...done.
Configuring VLAN30_CAMERAS interface...done.
Configuring LAN interface...done.
Configuring CARP settings...done.
Starting Secure Shell Services...done.
Syncing OpenVPN settings...done.
[fib_algo] inet.0 (bsearch4#38) rebuild_fd_flm: switching algo to radix4_lockless
Configuring firewall..load_dn_sched dn_sched FIFO loaded
load_dn_sched dn_sched QFQ loaded
load_dn_sched dn_sched RR loaded
load_dn_sched dn_sched WF2Q+ loaded
load_dn_sched dn_sched PRIO loaded
load_dn_sched dn_sched FQ_CODEL loaded
load_dn_sched dn_sched FQ_PIE loaded
load_dn_aqm dn_aqm CODEL loaded
load_dn_aqm dn_aqm PIE loaded
...0 addresses deleted.
0 addresses deleted.
.done.
Starting PFLOG...done.
Setting up gateway monitors...done.
Setting up static routes...done.
Setting up DNSs...
Starting DNS Resolver...done.
Synchronizing user settings...done.
Configuring CRON...done.
Bootstrapping clock...done.
Starting NTP Server...done.
Starting webConfigurator...done.
Starting DHCP service...done.
Configuring firewall.....0 addresses deleted.
0 addresses deleted.
0 addresses deleted.
0 addresses deleted.
.done.
gpio_open: No such file or directory
gpio_open: No such file or directory
gpio_open: No such file or directory
Generating RRD graphs...done.
Starting syslog...done.
Configuring filter for dynamic IPsec VPN hosts... done
Starting CRON... done.
 Starting package AWS VPC Wizard...done.
 Starting package IPsec Profile Wizard...done.
 Starting package Netgate Firmware Upgrade...done.
 Starting package apcupsd...done.
 Starting package Avahi...done.
 Starting package System Patches...done.
 Starting package Cron...done.
 Starting package OpenVPN Client Export Utility...done.
 Starting package Backup...done.
 Starting package acme...done.
 Starting package sudo...done.
 Starting package crowdsec...done.
 Starting package Filer...done.
 Starting /usr/local/etc/rc.d/crowdsec_firewall.sh...done.
Netgate pfSense Plus 24.03-RELEASE (Patch 1) amd64 20240424-1738
Bootup complete
Performing automatic boot verification...done.

FreeBSD/amd64 (heimdall.**********.me) (ttyu2)

login: admin
Password:
Netgate 4200 - Serial: ************ - Netgate Device ID: ****************

*** Welcome to Netgate pfSense Plus 24.03-RELEASE (amd64) on heimdall ***

  Current Boot Environment: default 
     Next Boot Environment: default 

 WAN (wan)             -> igc3      -> 
 LAN (lan)             -> igc2      -> v4: 10.0.1.1/24
 VLAN1003_GUEST (opt1) -> igc2.1003 -> v4: 10.0.10.1/24
 VLAN20_IOT (opt2)     -> igc2.20   -> v4: 10.0.20.1/24
 VLAN30_CAMERAS (opt3) -> igc2.30   -> v4: 10.0.30.1/24

 0) Logout (SSH only)                  9) pfTop
 1) Assign Interfaces                 10) Filter Logs
 2) Set interface(s) IP address       11) Restart GUI
 3) Reset admin account and password  12) PHP shell + Netgate pfSense Plus tools
 4) Reset to factory defaults         13) Update from console
 5) Reboot system                     14) Disable Secure Shell (sshd)
 6) Halt system                       15) Restore recent configuration
 7) Ping host                         16) Restart PHP-FPM
 8) Shell                             

Enter an option: 
Message from syslogd@heimdall at Aug 28 16:10:25 ...
php-fpm[97567]: /index.php: Successful login for user 'admin' from: 10.0.1.174 (Local Database)

Not sure why it is working now? I can successfully access the webUI. Could really use suggestions for how to be sure this doesn't repeat???

wgstarks

Up for about 30 minutes and then crashed again. Console just showed the login menu and then the console option screen after logging in. I was attempting to uninstall the crowdsec pkg at the time of the crash. The uninstall appeared to be hung. 10 minutes or so with no progress removing the pkg from the shell. Maybe that’s related??? Was able to complete the uninstall process after rebooting the 4200.

Maybe the problem is heat related? All sensors are showing between 69C and 70.1C. No idea what the normal range is for this appliance.

I’m leaving a laptop hooked to the console port for a bit. Maybe it’ll catch something if it crashes again.

stephenw10

A crash report should be in /var/crash if it created one.

What happens when you try to access the crash reporter from the link in the gui?

stephenw10

@wgstarks said in 4200 crashed:

All sensors are showing between 69C and 70.1C.

That is quite hot. How is the 4200 installed? Does it have adequate airflow?

wgstarks

@stephenw10 said in 4200 crashed:

A crash report should be in /var/crash if it created one.

What happens when you try to access the crash reporter from the link in the gui?

Nothing in var/crash. Nothing ever loaded clicking the link. Just timed out.

wgstarks

@stephenw10 said in 4200 crashed:

@wgstarks said in 4200 crashed:

All sensors are showing between 69C and 70.1C.

That is quite hot. How is the 4200 installed? Does it have adequate airflow?

Ordered a 120mm 50cfm fan on Amazon. Will lay it on top as an exhaust fan.

wgstarks

This post is deleted!

Gertjan

@wgstarks said in 4200 crashed:

0 addresses deleted

Not sure what this is. Doesn't look harmful.

But this :

@wgstarks said in 4200 crashed:

gpio_open: No such file or directory
gpio_open: No such file or directory
gpio_open: No such file or directory

The start up sequence shouldn't have / use commands that ... do not exist ?
It's probably something non essential, as "gpio_open" makes me thing of the "front panel led driver".

Maybe I shouldn't compare a boot sequence of my 4100 with your 4200

Temperatures. This is without vents, and its placed above a T350 PowerEdge (ancient server running R2 2008 and all that). Not ideal, but not 70+.

stephenw10

@wgstarks said in 4200 crashed:

gpio_open: No such file or directory
gpio_open: No such file or directory
gpio_open: No such file or directory

Mmm, that does seem unexpected. Do you have anything custom installed?

Though I wouldn't expect that to be an issue.

wgstarks

@stephenw10 said in 4200 crashed:

@wgstarks said in 4200 crashed:

gpio_open: No such file or directory
gpio_open: No such file or directory
gpio_open: No such file or directory

Mmm, that does seem unexpected. Do you have anything custom installed?

Though I wouldn't expect that to be an issue.

I have a custom LED control script that lets me set color and pattern of led lights based on system status. Can’t find the link to that forum thread right now though.

stephenw10

Ah was that imported from an older device? That could certainly be throwing those gpio errors. It shouldn't cause it to crash though.

wgstarks

@stephenw10 said in 4200 crashed:

Ah was that imported from an older device? That could certainly be throwing those gpio errors. It shouldn't cause it to crash

I had a similar script on my 3100 but I don’t think it was migrated to the 4200. Either it wasn’t included in the backup file that negate tech support modified to restore from on the 4200 or it was removed by them as a precaution.

I have installed a new modified version of the led control script though. Discussion starts here. Not sure if that’s what’s causing the boot entries though. It runs on a cron schedule. Wouldn’t think that would happen until boot was complete.

stephenw10

Hmm, well it sure seems like that script still has a reference to the 2100 gpio devices that don't exist on the 4200.

wgstarks

@stephenw10 said in 4200 crashed:

Hmm, well it sure seems like that script still has a reference to the 2100 gpio devices that don't exist on the 4200.

I’ll connect via cyberduck and run a search for the old file just to be sure there isn’t still an old copy or something.

wgstarks

@stephenw10 said in 4200 crashed:

Hmm, well it sure seems like that script still has a reference to the 2100 gpio devices that don't exist on the 4200.

I'm only finding one script to control the led's and I don't think its the culprit.

#!/bin/sh
echo 0 > /dev/led/blue_0
echo 0 > /dev/led/blue_1
echo 0 > /dev/led/blue_2
echo 0 > /dev/led/red_0
echo 0 > /dev/led/red_1
echo 0 > /dev/led/red_2
echo 0 > /dev/led/green_0
echo 0 > /dev/led/green_1
echo 0 > /dev/led/green_2

check_current_states=$( pfctl -vvss | grep -e "igc3 icmp" )
WAN=0
case "$check_current_states" in
  *icmp* ) WAN=1 ;;
esac
if [ $WAN = 0 ];
then
 echo f2 > /dev/led/red_0
 echo f2 > /dev/led/red_1
 echo f2 > /dev/led/red_2
else
  /usr/local/sbin/pfSense-led.sh ready

fi

Don't see anything that looks like it matches.

stephenw10

Mmm, wouldn't expect it from that.

Does that work as expected?

wgstarks

@stephenw10
Yes. Just lights a led red when WAN status is down.

There isn’t any other custom cron entry other than an update checker so even if there was a custom script buried somewhere I don’t see how it would be called.

tgl

@wgstarks said in 4200 crashed:

Maybe the problem is heat related? All sensors are showing between 69C and 70.1C. No idea what the normal range is for this appliance.

That definitely seems high. My 4200 shows readings right around 50C, except for the "Zone 0" sensor which is typically 58C or so. It's in a closet that doesn't have much ventilation, so ambient temp maybe 25C or a bit more.

wgstarks

Looks like all my openvpn clients got deleted too. No idea what is going on with this box. Realized the problem when I tried to connect remotely. After I got home I checked and OpenVPN clients tab is empty.

This isn’t really a big problem since I’ve been thinking about switching to Tailscale anyway but I don’t know if it’s another symptom of a bigger issue?