Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    21.02 Sudden lockup

    Scheduled Pinned Locked Moved Official Netgate® Hardware
    164 Posts 30 Posters 51.3k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • M
      mcury @kphillips
      last edited by

      @kphillips Hm, this is what I have, no more logs to provide you unfortunately, pfsense no longer responds, and to access the GUI, I have to Restart the webConfigurator, disable pfblocker, and restart..
      Doing so, Internet works, GUI responds, normal usage.

      dead on arrival, nowhere to be found.

      K 1 Reply Last reply Reply Quote 0
      • K
        kphillips Administrator Netgate @mcury
        last edited by

        @mcury Can you attach a USB cable to the firewall and see what the USB console is doing?

        Instructions for doing so can be found here:
        https://docs.netgate.com/pfsense/en/latest/solutions/sg-3100/connect-to-console.html

        Even if the webConfigurator is locked up, this should always respond.

        M F 2 Replies Last reply Reply Quote 0
        • F
          ffuentes
          last edited by

          I am in the same boat as many here. SG-3100 sudden lockup after upgrade with no console output.
          I am guessing a kernel panic as I am unable to see anything via serial console or logs.

          1 Reply Last reply Reply Quote 0
          • M
            mcury @kphillips
            last edited by

            @kphillips That log is from the console, connected through USB cable to pfsense, using putty serial. The problem start exactly here when configuring Firewall as per below, boot proceeds to the end, but GUI and Internet don't work.
            As per mentioned above, I have to restart the GUI through the console with the option 11, disable pfblockerng and reboot, then everything works.

            Syncing OpenVPN settings...done.
            Configuring firewall.Segmentation fault (core dumped) <<<
            Starting CRON... done.

            dead on arrival, nowhere to be found.

            K 1 Reply Last reply Reply Quote 0
            • F
              ffuentes @kphillips
              last edited by

              @kphillips It shows nothing for me.
              I cant even get to the console. This is why I suspect a kernel panic.

              1 Reply Last reply Reply Quote 0
              • K
                kphillips Administrator Netgate @mcury
                last edited by

                @mcury Can you provide the full output instead of just the provided snippet? I need everything, not just a chunk, to provide to our devs.

                M 2 Replies Last reply Reply Quote 0
                • M
                  mcury @kphillips
                  last edited by

                  @kphillips Sure, but now you are asking for the system log, right?
                  Because the console output is complete there, from the boot start to finish, ok?

                  dead on arrival, nowhere to be found.

                  1 Reply Last reply Reply Quote 0
                  • R
                    rloeb
                    last edited by

                    @kphillips The last few posts in this thread point to pfBlockerNG as a possible culprit. I have been running since noon yesterday with pfBlockerNG. I was also running Snort, but as reported in a previous message, it aborted just after midnight. Other than the fact that Snort isn't running, pfsense seems to be functioning correctly. I'd reboot and connect to the console, but I have a whole lot of people in Zoom calls who will kill me if I disrupt what they're doing.

                    T 1 Reply Last reply Reply Quote 0
                    • T
                      TommyG @rloeb
                      last edited by

                      @rloeb I have neither PFBlocker or Snort packages installed and I am also impacted by this freezing WAN/LAN issue on my SG3100.

                      1 Reply Last reply Reply Quote 0
                      • R
                        rloeb
                        last edited by

                        Confirms what I was trying to say. This is broader than an app.

                        K 1 Reply Last reply Reply Quote 1
                        • K
                          kphillips Administrator Netgate @rloeb
                          last edited by

                          @rloeb Our engineers have a half dozen different SG-3100s in various configurations running non-stop on 21.02. None of them have experienced lockups yet, so we're still trying to track down what is unique here. Full console output from a frozen device would be helpful.

                          1 Reply Last reply Reply Quote 0
                          • M
                            mcury @kphillips
                            last edited by mcury

                            @kphillips Kindly check and advise if this log can help you, or if is there something else that I can do

                            Feb 18 15:42:16 	kernel 		---<<BOOT>>---
Feb 18 15:42:16 	kernel 		Copyright (c) 1992-2020 The FreeBSD Project.
Feb 18 15:42:16 	kernel 		Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
Feb 18 15:42:16 	kernel 		The Regents of the University of California. All rights reserved.
Feb 18 15:42:16 	kernel 		FreeBSD is a registered trademark of The FreeBSD Foundation.
Feb 18 15:42:16 	kernel 		FreeBSD 12.2-STABLE 38a4c12973d(plus-devel-12) pfSense-SG-3100 arm
Feb 18 15:42:16 	kernel 		FreeBSD clang version 10.0.1 (git@github.com:llvm/llvm-project.git llvmorg-10.0.1-0-gef32c611aa2)
Feb 18 15:42:16 	kernel 		CPU: ARM Cortex-A9 r4p1 (ECO: 0x00000000)
Feb 18 15:42:16 	kernel 		CPU Features:
Feb 18 15:42:16 	kernel 		Multiprocessing, Thumb2, Security, VMSAv7, Coherent Walk
Feb 18 15:42:16 	kernel 		Optional instructions:
Feb 18 15:42:16 	kernel 		UMULL, SMULL, SIMD(ext)
Feb 18 15:42:16 	kernel 		LoUU:2 LoC:2 LoUIS:2
Feb 18 15:42:16 	kernel 		Cache level 1:
Feb 18 15:42:16 	kernel 		32KB/32B 4-way data cache WB Read-Alloc Write-Alloc
Feb 18 15:42:16 	kernel 		32KB/32B 4-way instruction cache Read-Alloc
Feb 18 15:42:16 	kernel 		real memory = 2147479552 (2047 MB)
Feb 18 15:42:16 	kernel 		avail memory = 2071031808 (1975 MB)
Feb 18 15:42:16 	kernel 		FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
Feb 18 15:42:16 	kernel 		arc4random: no preloaded entropy cache
Feb 18 15:42:16 	kernel 		ipw_ibss: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw.LICENSE.
Feb 18 15:42:16 	kernel 		ipw_ibss: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf.
Feb 18 15:42:16 	kernel 		module_register_init: MOD_LOAD (ipw_ibss_fw, 0xc014deb0, 0) error 1
Feb 18 15:42:16 	kernel 		ipw_monitor: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw.LICENSE.
Feb 18 15:42:16 	kernel 		ipw_monitor: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf.
Feb 18 15:42:16 	kernel 		module_register_init: MOD_LOAD (ipw_monitor_fw, 0xc014df60, 0) error 1
Feb 18 15:42:16 	kernel 		iwi_bss: You need to read the LICENSE file in /usr/share/doc/legal/intel_iwi.LICENSE.
Feb 18 15:42:16 	kernel 		iwi_bss: If you agree with the license, set legal.intel_iwi.license_ack=1 in /boot/loader.conf.
Feb 18 15:42:16 	kernel 		module_register_init: MOD_LOAD (iwi_bss_fw, 0xc0156f50, 0) error 1
Feb 18 15:42:16 	kernel 		iwi_ibss: You need to read the LICENSE file in /usr/share/doc/legal/intel_iwi.LICENSE.
Feb 18 15:42:16 	kernel 		iwi_ibss: If you agree with the license, set legal.intel_iwi.license_ack=1 in /boot/loader.conf.
Feb 18 15:42:16 	kernel 		module_register_init: MOD_LOAD (iwi_ibss_fw, 0xc0157000, 0) error 1
Feb 18 15:42:16 	kernel 		iwi_monitor: You need to read the LICENSE file in /usr/share/doc/legal/intel_iwi.LICENSE.
Feb 18 15:42:16 	kernel 		iwi_monitor: If you agree with the license, set legal.intel_iwi.license_ack=1 in /boot/loader.conf.
Feb 18 15:42:16 	kernel 		module_register_init: MOD_LOAD (iwi_monitor_fw, 0xc01570b0, 0) error 1
Feb 18 15:42:16 	kernel 		ipw_bss: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw.LICENSE.
Feb 18 15:42:16 	kernel 		ipw_bss: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf.
Feb 18 15:42:16 	kernel 		module_register_init: MOD_LOAD (ipw_bss_fw, 0xc014de00, 0) error 1
Feb 18 15:42:16 	kernel 		wlan: mac acl policy registered
Feb 18 15:42:16 	kernel 		random: entropy device external interface
Feb 18 15:42:16 	kernel 		[ath_hal] loaded
Feb 18 15:42:16 	kernel 		ofwbus0: <Open Firmware Device Tree>
Feb 18 15:42:16 	kernel 		simplebus0: <Flattened device tree simple bus> on ofwbus0
Feb 18 15:42:16 	kernel 		simplebus1: <Flattened device tree simple bus> on simplebus0
Feb 18 15:42:16 	kernel 		l2cache0: <PL310 L2 cache controller> mem 0x8000-0x8fff on simplebus1
Feb 18 15:42:16 	kernel 		l2cache0: cannot allocate IRQ, not using interrupt
Feb 18 15:42:16 	kernel 		l2cache0: Part number: 0x3, release: 0x9
Feb 18 15:42:16 	kernel 		l2cache0: L2 Cache enabled: 1024KB/32B 16 ways
Feb 18 15:42:16 	kernel 		gic0: <ARM Generic Interrupt Controller> mem 0xd000-0xdfff,0xc100-0xc1ff on simplebus1
Feb 18 15:42:16 	kernel 		gic0: pn 0x39, arch 0x1, rev 0x2, implementer 0x43b irqs 192
Feb 18 15:42:16 	kernel 		mpic0: <Marvell Integrated Interrupt Controller> mem 0x20a00-0x20ccf,0x21070-0x210c7 irq 17 on simplebus1
Feb 18 15:42:16 	kernel 		gpio0: <Marvell Integrated GPIO Controller> mem 0x18100-0x1813f,0x181c0-0x181c7 irq 9,10,11,12 on simplebus1
Feb 18 15:42:16 	kernel 		gpio0: 4 IRQs available
Feb 18 15:42:16 	kernel 		gpio0: Disable interrupts (offset = 0 + EDGE(0x18)
Feb 18 15:42:16 	kernel 		gpio0: Disable interrupts (offset = 0 + LEV(0x1C))
Feb 18 15:42:16 	kernel 		gpio0: Setup intr 0
Feb 18 15:42:16 	kernel 		gpio0: Setup intr 1
Feb 18 15:42:16 	kernel 		gpio0: Setup intr 2
Feb 18 15:42:16 	kernel 		gpio0: Setup intr 3
Feb 18 15:42:16 	kernel 		gpio0: Clear int status (offset = 0)
Feb 18 15:42:16 	kernel 		gpiobus0: <GPIO bus> on gpio0
Feb 18 15:42:16 	kernel 		gpio1: <Marvell Integrated GPIO Controller> mem 0x18140-0x1817f,0x181c8-0x181cf irq 13,14,15,16 on simplebus1
Feb 18 15:42:16 	kernel 		gpio1: 4 IRQs available
Feb 18 15:42:16 	kernel 		gpio1: Disable interrupts (offset = 0 + EDGE(0x18)
Feb 18 15:42:16 	kernel 		gpio1: Disable interrupts (offset = 0 + LEV(0x1C))
Feb 18 15:42:16 	kernel 		gpio1: Setup intr 0
Feb 18 15:42:16 	kernel 		gpio1: Setup intr 1
Feb 18 15:42:16 	kernel 		gpio1: Setup intr 2
Feb 18 15:42:16 	kernel 		gpio1: Setup intr 3
Feb 18 15:42:16 	kernel 		gpio1: Clear int status (offset = 0)
Feb 18 15:42:16 	kernel 		gpiobus1: <GPIO bus> on gpio1
Feb 18 15:42:16 	kernel 		mp_tmr0: <ARM MPCore Timers> mem 0xc200-0xc21f irq 3 on simplebus1
Feb 18 15:42:16 	kernel 		Timecounter "MPCore" frequency 800000000 Hz quality 800
Feb 18 15:42:16 	kernel 		mp_tmr1: <ARM MPCore Timers> mem 0xc600-0xc61f irq 4 on simplebus1
Feb 18 15:42:16 	kernel 		Event timer "MPCore" frequency 800000000 Hz quality 1000
Feb 18 15:42:16 	kernel 		twsi0: <Marvell Integrated I2C Bus Controller> mem 0x11000-0x1101f irq 5 on simplebus1
Feb 18 15:42:16 	kernel 		iicbus0: <OFW I2C bus> on twsi0
Feb 18 15:42:16 	kernel 		iic0: <I2C generic I/O> on iicbus0
Feb 18 15:42:16 	kernel 		gpio2: <NXP PCA9552 LED driver> at addr 0xc0 on iicbus0
Feb 18 15:42:16 	kernel 		device_attach: gpio2 attach returned 6
Feb 18 15:42:16 	kernel 		gpio2: <ISSI IS31FL3199 9 channel light effect LED driver> at addr 0xce on iicbus0
Feb 18 15:42:16 	kernel 		gpiobus2: <OFW GPIO bus> on gpio2
Feb 18 15:42:16 	kernel 		gpioc2: <GPIO controller> on gpio2
Feb 18 15:42:16 	kernel 		uart0: <16550 or compatible> mem 0x12000-0x120ff irq 7 on simplebus1
Feb 18 15:42:16 	kernel 		uart0: console (-1,n,8,1)
Feb 18 15:42:16 	kernel 		uart1: <16550 or compatible> mem 0x12100-0x121ff irq 8 on simplebus1
Feb 18 15:42:16 	kernel 		gpioc0: <GPIO controller> on gpio0
Feb 18 15:42:16 	kernel 		gpioc1: <GPIO controller> on gpio1
Feb 18 15:42:16 	kernel 		wdt0: <Marvell Watchdog Timer> mem 0x20300-0x20333,0x20704-0x20707,0x18260-0x18263 irq 24,25 on simplebus1
Feb 18 15:42:16 	kernel 		pmsu0: <Power Management Service Unit> mem 0x22000-0x22fff on simplebus1
Feb 18 15:42:16 	kernel 		mvneta0: <NETA controller> mem 0x30000-0x33fff irq 26 on simplebus1
Feb 18 15:42:16 	kernel 		mvneta0: version is 10
Feb 18 15:42:16 	kernel 		mvneta0: Ethernet address: 00:08:a2:0c:c4:1b
Feb 18 15:42:16 	kernel 		miibus0: <MII bus> on mvneta0
Feb 18 15:42:16 	kernel 		mv88e151x0: <Marvell 88E1512 Gigabit PHY> PHY 1 on miibus0
Feb 18 15:42:16 	kernel 		mv88e151x0: none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, 1000baseSX, 1000baseSX-FDX, auto
Feb 18 15:42:16 	kernel 		mvneta1: <NETA controller> mem 0x34000-0x37fff irq 27 on simplebus1
Feb 18 15:42:16 	kernel 		mvneta1: version is 10
Feb 18 15:42:16 	kernel 		mvneta1: Ethernet address: 00:08:a2:0c:c4:1c
Feb 18 15:42:16 	kernel 		mdio0: <MDIO> on mvneta1
Feb 18 15:42:16 	kernel 		e6000sw0: <Marvell 88E6141> on mdio0
Feb 18 15:42:16 	kernel 		e6000sw0: single-chip addressing mode
Feb 18 15:42:16 	kernel 		e6000sw0: PHY at port 1
Feb 18 15:42:16 	kernel 		miibus1: <MII bus> on e6000sw0
Feb 18 15:42:16 	kernel 		e1000phy0: <Marvell 88E1000 Gigabit PHY> PHY 17 on miibus1
Feb 18 15:42:16 	kernel 		e1000phy0: none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto
Feb 18 15:42:16 	kernel 		e6000sw0: PHY at port 2
Feb 18 15:42:16 	kernel 		miibus2: <MII bus> on e6000sw0
Feb 18 15:42:16 	kernel 		e1000phy1: <Marvell 88E1000 Gigabit PHY> PHY 18 on miibus2
Feb 18 15:42:16 	kernel 		e1000phy1: none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto
Feb 18 15:42:16 	kernel 		e6000sw0: PHY at port 3
Feb 18 15:42:16 	kernel 		miibus3: <MII bus> on e6000sw0
Feb 18 15:42:16 	kernel 		e1000phy2: <Marvell 88E1000 Gigabit PHY> PHY 19 on miibus3
Feb 18 15:42:16 	kernel 		e1000phy2: none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto
Feb 18 15:42:16 	kernel 		e6000sw0: PHY at port 4
Feb 18 15:42:16 	kernel 		miibus4: <MII bus> on e6000sw0
Feb 18 15:42:16 	kernel 		e1000phy3: <Marvell 88E1000 Gigabit PHY> PHY 20 on miibus4
Feb 18 15:42:16 	kernel 		e1000phy3: none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto
Feb 18 15:42:16 	kernel 		e6000sw0: CPU port at 5
Feb 18 15:42:16 	kernel 		e6000sw0: fixed port at 5
Feb 18 15:42:16 	kernel 		e6000sw0: switch is ready.
Feb 18 15:42:16 	kernel 		etherswitch0: <Switch controller> on e6000sw0
Feb 18 15:42:16 	kernel 		mvneta2: <NETA controller> mem 0x70000-0x73fff irq 28 on simplebus1
Feb 18 15:42:16 	kernel 		mvneta2: version is 10
Feb 18 15:42:16 	kernel 		mvneta2: Ethernet address: 00:08:a2:0c:c4:1d
Feb 18 15:42:16 	kernel 		miibus5: <MII bus> on mvneta2
Feb 18 15:42:16 	kernel 		mv88e151x1: <Marvell 88E1512 Gigabit PHY> PHY 0 on miibus5
Feb 18 15:42:16 	kernel 		mv88e151x1: none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, 1000baseSX, 1000baseSX-FDX, auto
Feb 18 15:42:16 	kernel 		ehci0: <Marvell Integrated USB 2.0 controller> mem 0x58000-0x584ff irq 29 on simplebus1
Feb 18 15:42:16 	kernel 		usbus0: EHCI version 1.0
Feb 18 15:42:16 	kernel 		usbus0 on ehci0
Feb 18 15:42:16 	kernel 		cesa0: <Marvell Cryptographic Engine and Security Accelerator> mem 0x90000-0x9ffff irq 30,31 on simplebus1
Feb 18 15:42:16 	kernel 		rtc0: <Marvell Integrated RTC> mem 0xa3800-0xa381f,0x184a0-0x184ab irq 32 on simplebus1
Feb 18 15:42:16 	kernel 		rtc0: registered as a time-of-day clock, resolution 1.000000s
Feb 18 15:42:16 	kernel 		ahci0: <Marvell AHCI Controller> mem 0xa8000-0xa9fff irq 33 on simplebus1
Feb 18 15:42:16 	kernel 		ahci0: AHCI v1.00 with 2 6Gbps ports, Port Multiplier supported with FBS
Feb 18 15:42:16 	kernel 		ahci0: quirks=0x200010<2CH,MRVL_SR_DEL>
Feb 18 15:42:16 	kernel 		ahcich0: <AHCI channel> at channel 0 on ahci0
Feb 18 15:42:16 	kernel 		ahcich1: <AHCI channel> at channel 1 on ahci0
Feb 18 15:42:16 	kernel 		armada_thermal0: <Armada380 Thermal Control> mem 0xe4078-0xe407b,0xe4070-0xe4077 on simplebus1
Feb 18 15:42:16 	kernel 		sdhci_fdt0: <ARMADA38X SDHCI controller> mem 0xd8000-0xd8fff,0xdc000-0xdc0ff,0x18454-0x18457 irq 36 on simplebus1
Feb 18 15:42:16 	kernel 		sdhci_fdt0: 1 slot(s) allocated
Feb 18 15:42:16 	kernel 		xhci0: <Marvell Integrated USB 3.0 controller> mem 0xf8000-0xfbfff,0xfc000-0xfffff irq 38 on simplebus1
Feb 18 15:42:16 	kernel 		xhci0: 32 bytes context size, 32-bit DMA
Feb 18 15:42:16 	kernel 		usbus1 on xhci0
Feb 18 15:42:16 	kernel 		cesa1: <Marvell Cryptographic Engine and Security Accelerator> mem 0x90000-0x9ffff irq 39,40 on simplebus1
Feb 18 15:42:16 	kernel 		spi0: <Marvell SPI controller> mem 0xf001000000010600-0xf00100000001064f irq 1 on simplebus0
Feb 18 15:42:16 	kernel 		cpulist0: <Open Firmware CPU Group> on ofwbus0
Feb 18 15:42:16 	kernel 		cpu0: <Open Firmware CPU> on cpulist0
Feb 18 15:42:16 	kernel 		cpu1: <Open Firmware CPU> on cpulist0
Feb 18 15:42:16 	kernel 		cryptosoft0: <software crypto>
Feb 18 15:42:16 	kernel 		Timecounters tick every 1.000 msec
Feb 18 15:42:16 	kernel 		mvneta1: link state changed to UP
Feb 18 15:42:16 	kernel 		spibus0: <OFW SPI bus> on spi0
Feb 18 15:42:16 	kernel 		mx25l0: <M25Pxx Flash Family> at cs 0 mode 0 on spibus0
Feb 18 15:42:16 	kernel 		mx25l0: device type n25q128, size 16384K in 256 sectors of 64K, erase size 4K
Feb 18 15:42:16 	kernel 		Release APs
Feb 18 15:42:16 	kernel 		e6000sw0port1: link state changed to DOWN
Feb 18 15:42:16 	kernel 		usbus0:
Feb 18 15:42:16 	kernel 		e6000sw0port2: link state changed to DOWN
Feb 18 15:42:16 	kernel 		480Mbps High Speed USB v2.0
Feb 18 15:42:16 	kernel 		e6000sw0port3: link state changed to DOWN
Feb 18 15:42:16 	kernel 		usbus1:
Feb 18 15:42:16 	kernel 		e6000sw0port4: link state changed to DOWN
Feb 18 15:42:16 	kernel 		5.0Gbps Super Speed USB v3.0
Feb 18 15:42:16 	kernel 		ugen0.1: <Marvell EHCI root HUB> at usbus0
Feb 18 15:42:16 	kernel 		uhub0: <Marvell EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus0
Feb 18 15:42:16 	kernel 		ugen1.1: <Marvell XHCI root HUB> at usbus1
Feb 18 15:42:16 	kernel 		uhub1: <Marvell XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus1
Feb 18 15:42:16 	kernel 		arc4random: no preloaded entropy cache
Feb 18 15:42:16 	kernel 		Trying to mount root from ufs:/dev/diskid/DISK-D4EC8498s2a [rw,noatime]...
Feb 18 15:42:16 	kernel 		Root mount waiting for: usbus0 CAM usbus1
Feb 18 15:42:16 	kernel 		uhub1: 2 ports with 2 removable, self powered
Feb 18 15:42:16 	kernel 		mmc0: <MMC/SD bus> on sdhci_fdt0
Feb 18 15:42:16 	kernel 		mmcsd0: 8GB <MMCHC 008GE0 0.0 SN D4EC8498 MFG 06/2015 by 17 0x0000> at mmc0 50.0MHz/8bit/65535-block
Feb 18 15:42:16 	kernel 		mmcsd0boot0: 4MB partition 1 at mmcsd0
Feb 18 15:42:16 	kernel 		mmcsd0boot1: 4MB partition 2 at mmcsd0
Feb 18 15:42:16 	kernel 		mmcsd0rpmb: 4MB partition 3 at mmcsd0
Feb 18 15:42:16 	kernel 		uhub0: 1 port with 1 removable, self powered
Feb 18 15:42:16 	kernel 		ugen1.2: <INNO TECH USB to Serial> at usbus1
Feb 18 15:42:16 	kernel 		e6000sw0port2: link state changed to UP
Feb 18 15:42:16 	kernel 		Root mount waiting for: CAM
Feb 18 15:42:16 	kernel 		Root mount waiting for:
Feb 18 15:42:16 	kernel 		e6000sw0port1: link state changed to UP
Feb 18 15:42:16 	kernel 		CAM
Feb 18 15:42:16 	kernel 		e6000sw0port3: link state changed to UP
Feb 18 15:42:16 	kernel 		e6000sw0port4: link state changed to UP
Feb 18 15:42:16 	kernel 		Root mount waiting for: CAM
Feb 18 15:42:16 	kernel 		Root mount waiting for: CAM
Feb 18 15:42:16 	kernel 		Root mount waiting for: CAM
Feb 18 15:42:16 	kernel 		Root mount waiting for: CAM
Feb 18 15:42:16 	kernel 		Root mount waiting for: CAM
Feb 18 15:42:16 	kernel 		Root mount waiting for: CAM
Feb 18 15:42:16 	kernel 		mountroot: waiting for device /dev/diskid/DISK-D4EC8498s2a...
Feb 18 15:42:16 	kernel 		random: unblocking device.
Feb 18 15:42:16 	kernel 		lo0: link state changed to UP
Feb 18 15:42:16 	check_reload_status 	362 	Linkup starting $e6000sw0port1
Feb 18 15:42:16 	kernel 		e6000sw0port1: link state changed to DOWN
Feb 18 15:42:16 	check_reload_status 	362 	Linkup starting $e6000sw0port2
Feb 18 15:42:16 	kernel 		e6000sw0port2: link state changed to DOWN
Feb 18 15:42:16 	check_reload_status 	362 	Linkup starting $e6000sw0port3
Feb 18 15:42:16 	kernel 		e6000sw0port3: link state changed to DOWN
Feb 18 15:42:16 	check_reload_status 	362 	Linkup starting $e6000sw0port4
Feb 18 15:42:16 	kernel 		e6000sw0port4: link state changed to DOWN
Feb 18 15:42:16 	kernel 		vlan0: changing name to 'mvneta1.100'
Feb 18 15:42:16 	kernel 		vlan1: changing name to 'mvneta1.10'
Feb 18 15:42:16 	check_reload_status 	362 	Linkup starting mvneta2
Feb 18 15:42:16 	kernel 		vlan2: changing name to 'mvneta1.20'
Feb 18 15:42:16 	kernel 		mvneta2: link state changed to UP
Feb 18 15:42:17 	sshd 	22090 	Server listening on :: port 22.
Feb 18 15:42:17 	sshd 	22090 	Server listening on 0.0.0.0 port 22.
Feb 18 15:42:17 	check_reload_status 	362 	rc.newwanip starting mvneta2
Feb 18 15:42:17 	php 	366 	rc.bootup: calling interface_dhcpv6_configure.
Feb 18 15:42:17 	php 	366 	rc.bootup: Accept router advertisements on interface mvneta2
Feb 18 15:42:17 	php 	366 	rc.bootup: Starting rtsold process
Feb 18 15:42:18 	php-fpm 	348 	/rc.newwanip: rc.newwanip: Info: starting on mvneta2.
Feb 18 15:42:18 	php-fpm 	348 	/rc.newwanip: rc.newwanip: on (IP address: X.X.X.X) (interface: WAN[wan]) (real interface: mvneta2).
Feb 18 15:42:18 	php-fpm 	348 	/rc.newwanip: Accept router advertisements on interface mvneta2
Feb 18 15:42:18 	php-fpm 	348 	/rc.newwanip: Starting rtsold process
Feb 18 15:42:19 	check_reload_status 	362 	Linkup starting $e6000sw0port2
Feb 18 15:42:19 	kernel 		e6000sw0port2: link state changed to UP
Feb 18 15:42:20 	kernel 		done.
Feb 18 15:42:20 	check_reload_status 	362 	Linkup starting $e6000sw0port1
Feb 18 15:42:20 	kernel 		e6000sw0port1: link state changed to UP
Feb 18 15:42:20 	kernel 		e6000sw0port3: link state changed to UP
Feb 18 15:42:20 	kernel 		e6000sw0port4: link state changed to UP
Feb 18 15:42:20 	check_reload_status 	362 	Linkup starting $e6000sw0port3
Feb 18 15:42:20 	check_reload_status 	362 	Linkup starting $e6000sw0port4
Feb 18 15:42:20 	php-fpm 	348 	/rc.newwanip: The command '/usr/sbin/rtsold -1 -p /var/run/rtsold_mvneta2.pid -M /var/etc/rtsold_mvneta2_script.sh -O /var/etc/rtsold_mvneta2_script.sh mvneta2' returned exit code '1', the output was 'rtsold: failed to open pidfile: File exists'
Feb 18 15:42:21 	check_reload_status 	362 	rc.newwanip starting mvneta2
Feb 18 15:42:21 	php 	366 	rc.bootup: calling interface_dhcpv6_configure.
Feb 18 15:42:21 	php 	366 	rc.bootup: Accept router advertisements on interface mvneta2
Feb 18 15:42:21 	php 	366 	rc.bootup: Killing running rtsold process
Feb 18 15:42:22 	php-fpm 	349 	/rc.newwanip: rc.newwanip: Info: starting on mvneta2.
Feb 18 15:42:22 	php-fpm 	349 	/rc.newwanip: rc.newwanip: on (IP address: X.X.X.X) (interface: WAN[wan]) (real interface: mvneta2).
Feb 18 15:42:22 	php-fpm 	349 	/rc.newwanip: Accept router advertisements on interface mvneta2
Feb 18 15:42:22 	php-fpm 	349 	/rc.newwanip: Starting rtsold process
Feb 18 15:42:23 	php 	366 	rc.bootup: Starting rtsold process
Feb 18 15:42:25 	php 	366 	rc.bootup: The command '/usr/sbin/rtsold -1 -p /var/run/rtsold_mvneta2.pid -M /var/etc/rtsold_mvneta2_script.sh -O /var/etc/rtsold_mvneta2_script.sh mvneta2' returned exit code '1', the output was 'rtsold: failed to open pidfile: File exists'
Feb 18 15:42:25 	php 	366 	rc.bootup: Resyncing OpenVPN instances.
Feb 18 15:42:25 	kernel 		pflog0: promiscuous mode enabled
Feb 18 15:42:25 	rtsold 	41791 	Received RA specifying route for interface wan(mvneta2)
Feb 18 15:42:25 	rtsold 	42392 	Starting dhcp6 client for interface wan(mvneta2)
Feb 18 15:42:27 	php-fpm 	348 	/rc.newwanipv6: rc.newwanipv6: Info: starting on mvneta2.
Feb 18 15:42:27 	php-fpm 	348 	/rc.newwanipv6: rc.newwanipv6: on (IP address: X.X.X.X.X.X.X) (interface: wan) (real interface: mvneta2).
Feb 18 15:42:28 	php-fpm 	348 	/rc.newwanipv6: Gateway, none 'available' for inet, use the first one configured. 'WAN_DHCP'
Feb 18 15:42:28 	php-fpm 	348 	/rc.newwanipv6: Default gateway setting Interface WAN_DHCP Gateway as default.
Feb 18 15:42:28 	php-fpm 	348 	/rc.newwanipv6: Gateway, none 'available' for inet6, use the first one configured. 'WAN_DHCP6'
Feb 18 15:42:39 	kernel 		pid 366 (php-cgi), jid 0, uid 0: exited on signal 11 (core dumped)
Feb 18 15:42:39 	php-fpm 	349 	/rc.start_packages: Restarting/Starting all packages.
Feb 18 15:42:39 	php-fpm 	349 	/rc.start_packages: Starting service nut
Feb 18 15:42:39 	upsmon 	64296 	Startup successful
Feb 18 15:42:40 	blazer_usb 	65265 	Startup successful
Feb 18 15:42:40 	upsd 	65608 	listening on 192.168.255.249 port 3493
Feb 18 15:42:40 	upsd 	65608 	listening on ::1 port 3493
Feb 18 15:42:40 	upsd 	65608 	listening on 127.0.0.1 port 3493
Feb 18 15:42:40 	upsd 	65608 	Connected to UPS [ups]: blazer_usb-ups
Feb 18 15:42:40 	upsd 	65816 	Startup successful
Feb 18 15:42:42 	upsmon 	66974 	Startup successful
Feb 18 15:42:43 	lighttpd_pfb 	74974 	[pfBlockerNG] DNSBL Webserver started
Feb 18 15:42:44 	upsd 	81219 	listening on 192.168.255.249 port 3493
Feb 18 15:42:44 	upsd 	81219 	listening on ::1 port 3493
Feb 18 15:42:44 	upsd 	81219 	listening on 127.0.0.1 port 3493
Feb 18 15:42:44 	upsd 	81219 	Can't connect to UPS [ups] (blazer_usb-ups): Connection refused
Feb 18 15:42:44 	upsd 	81333 	Startup successful
Feb 18 15:42:44 	php-fpm 	349 	[pfBlockerNG] Starting firewall filter daemon
Feb 18 15:42:44 	php 	79331 	[pfBlockerNG] DNSBL parser daemon started
Feb 18 15:42:44 	root 	89009 	Bootup complete
Feb 18 15:42:45 	lighttpd_pfb 	90034 	[pfBlockerNG] DNSBL Webserver started
Feb 18 15:42:45 	php 	88877 	[pfBlockerNG] filterlog daemon started
Feb 18 15:42:45 	php 	88309 	[pfBlockerNG] filterlog daemon started
Feb 18 15:42:46 	php 	91697 	[pfBlockerNG] DNSBL parser daemon started
Feb 18 15:42:46 	upsd 	81333 	User local-monitor@::1 logged into UPS [ups]
Feb 18 15:42:46 	upsmon 	67756 	Poll UPS [ups] failed - Driver not connected
Feb 18 15:42:46 	upsmon 	67756 	Communications with UPS ups lost
Feb 18 15:42:46 	login 	98062 	login on ttyu0 as root
Feb 18 15:42:48 	upsd 	81333 	User monuser@192.168.255.252 logged into UPS [ups]
Feb 18 15:42:48 	blazer_usb 	13450 	Startup successful
Feb 18 15:42:50 	upsd 	81333 	Connected to UPS [ups]: blazer_usb-ups
Feb 18 15:42:51 	upsmon 	67756 	Communications with UPS ups established
Feb 18 15:43:44 	upsd 	81333 	User monuser2@192.168.255.251 logged into UPS [ups]
Feb 18 15:46:10 	login 	29288 	login on ttyu0 as root
Feb 18 15:46:21 	php 	35698 	rc.restart_webgui: Creating rrd update script
Feb 18 15:46:33 	php-fpm 	348 	/index.php: Successful login for user 'admin' from: 192.168.255.250 (Local Database Fallback)
Feb 18 15:46:39 	check_reload_status 	362 	Syncing firewall
Feb 18 15:46:41 	check_reload_status 	362 	Syncing firewall
Feb 18 15:46:41 	check_reload_status 	362 	Syncing firewall
Feb 18 15:46:42 	check_reload_status 	362 	Reloading filter
Feb 18 15:46:42 	php-fpm 	3213 	[pfBlockerNG] Stopping firewall filter daemon
Feb 18 15:46:42 	check_reload_status 	362 	Syncing firewall
Feb 18 15:46:42 	check_reload_status 	362 	Syncing firewall
Feb 18 15:46:44 	upsmon 	67756 	Poll UPS [ups] failed - Write error: Permission denied
Feb 18 15:46:44 	upsmon 	67756 	Communications with UPS ups lost
Feb 18 15:46:49 	upsd 	81333 	User local-monitor@::1 logged into UPS [ups]
Feb 18 15:46:49 	upsmon 	67756 	Communications with UPS ups established
Feb 18 15:46:52 	php 	64330 	rc.initial.reboot: Stopping all packages.
Feb 18 15:46:52 	upsmon 	67756 	Signal 15: exiting
Feb 18 15:46:52 	upsd 	81333 	User local-monitor@::1 logged out from UPS [ups]
Feb 18 15:46:52 	upsd 	81333 	mainloop: Interrupted system call
Feb 18 15:46:52 	upsd 	81333 	Signal 15: exiting
Feb 18 15:46:52 	blazer_usb 	13450 	Signal 15: exiting
Feb 18 15:46:55 	reboot 	73891 	rebooted by root
Feb 18 15:46:55 	syslogd 		exiting on signal 15
Feb 18 15:47:48 	syslogd 		kernel boot file is /boot/kernel/kernel
Feb 18 15:47:48 	kernel 		---<<BOOT>>---
Feb 18 15:47:48 	kernel 		Copyright (c) 1992-2020 The FreeBSD Project.
Feb 18 15:47:48 	kernel 		Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
Feb 18 15:47:48 	kernel 		The Regents of the University of California. All rights reserved.
Feb 18 15:47:48 	kernel 		FreeBSD is a registered trademark of The FreeBSD Foundation.
Feb 18 15:47:48 	kernel 		FreeBSD 12.2-STABLE 38a4c12973d(plus-devel-12) pfSense-SG-3100 arm

                            dead on arrival, nowhere to be found.

                            K 1 Reply Last reply Reply Quote 0
                            • K
                              kphillips Administrator Netgate @mcury
                              last edited by

                              @mcury Thank you! Was the last reboot you initiated after it had froze up?

                              M 1 Reply Last reply Reply Quote 0
                              • R
                                rloeb
                                last edited by

                                Understood. Tough to find a phantom bug! Some of the reports above say that there wasn't any output when it locked up. What is a signal 11 and what creates it? Is the core dump from a signal 11 of any use? (I have no idea how to retrieve it, but I thought I'd ask.)

                                F 1 Reply Last reply Reply Quote 0
                                • M
                                  mcury @kphillips
                                  last edited by

                                  @kphillips Yes, this is the first reboot after installing pfblockerng-devel.

                                  Installed the program, ran the wizard, and reboot, and the problem happens.
                                  To fix, got into the console, pressed 11 to restart the GUI, connected to it and disabled pfblockerng.
                                  Then reboot and it's all fine.

                                  dead on arrival, nowhere to be found.

                                  K 1 Reply Last reply Reply Quote 0
                                  • K
                                    kphillips Administrator Netgate @mcury
                                    last edited by

                                    @mcury OK thank you. So to confirm, the webConfigurator is inaccessible and you cannot pass traffic at all when this happens? What if you ping by IP address rather than by hostname from a device attached to the LAN? If you ping 8.8.8.8, for example, does it succeed but DNS queries fail? Since pfBlockerNG ties into unbound (DNS Resolver) I'm wondering if its crashing the DNS Resolver causing your issue.

                                    M 1 Reply Last reply Reply Quote 0
                                    • stephenw10S
                                      stephenw10 Netgate Administrator
                                      last edited by

                                      If you're still able to connect to the command line at the console but nothing else try to connect out. Can you ping anything local on any interface?

                                      What does ifconfig -va show?

                                      What about netstat -rn do you still have valid routes?

                                      Or etherswitchcfg is the switch still responding?

                                      Steve

                                      K 1 Reply Last reply Reply Quote 0
                                      • K
                                        kphillips Administrator Netgate @stephenw10
                                        last edited by

                                        @stephenw10 Important to note: Do this from the USB serial from the option 8 shell and after the device has locked up BEFORE rebooting it, please.

                                        1 Reply Last reply Reply Quote 0
                                        • M
                                          mcury @kphillips
                                          last edited by mcury

                                          @kphillips said in 21.02 Sudden lockup:

                                          @mcury OK thank you. So to confirm, the webConfigurator is inaccessible and you cannot pass traffic at all when this happens? What if you ping by IP address rather than by hostname from a device attached to the LAN? If you ping 8.8.8.8, for example, does it succeed but DNS queries fail? Since pfBlockerNG ties into unbound (DNS Resolver) I'm wondering if its crashing the DNS Resolver causing your issue.

                                          Yes, webConfigurator is inaccessible and I cannot pass traffic.
                                          Ping to 8.8.8.8 fails, or any other IP on the internet (it's not DNS).

                                          I'll perform the tests through the console.

                                          I'll reproduce the problem, give me a few minutes.
                                          1 - enable pfblockerng-devel again
                                          2 - Reboot
                                          3 - Test again the ping to 8.8.8.8, ping the pfsense LAN interface.

                                          Then, I'll perform the tests @stephenw10 asked

                                          1 - perform ifconfig -va
                                          2 - netstat -rn
                                          3 - etherswitchcfg to check the switch

                                          dead on arrival, nowhere to be found.

                                          M 1 Reply Last reply Reply Quote 0
                                          • F
                                            ffuentes @rloeb
                                            last edited by

                                            @rloeb no output for me when it locked up.
                                            Once it reboots I can see the normal boot process.

                                            @kphillips Anything I can capture to help out?

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.