Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    netgate APU4 bricked after initiating update in web console

    Scheduled Pinned Locked Moved Official Netgate® Hardware
    6 Posts 2 Posters 573 Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • B
      benkillin
      last edited by

      I have a Netgate APU4 that was working just fine since 2015 and I initiated an update to the latest version today. I forget the exact version number but it was _2 that I had and I was upgrading to the latest _3 release listed in the updater in the web administration panel of the device. Based on the version of the freebsd kernel that is listed in the boot output from the serial console (below) it was the upgrade from 11.2-RELEASE-p4 to 11.2-RELEASE-p10 that I installed. So probably pfsense 2.4.4-p2 to 2.4.4-p3.

      It seemed like the update was going well but the device never came back up after restarting.

      Since I had to restart my computer to get a serial port installed to be able to see the console of the device I no longer have the web console output. Below is the output of the device booting from the serial console.

      I would super appreciate it for any help that could be provided to resolve this issue.

      It hangs indefinitely at this point:

      Trying to mount root from ufs:/dev/ufsid/5358b0d792733dcf [rw]...
      WARNING: / was not properly dismounted
      uhub1: 4 ports with 4 removable, self powered
      uhub0: 5 ports with 5 removable, self powered
      ugen6.2: <Generic Flash Card ReaderWriter> at usbus6
      umass0 on uhub1
      umass0: <Generic Flash Card ReaderWriter, class 0/0, rev 2.01/1.00, addr 2> on usbus6
      uhub6: 5 ports with 5 removable, self powered
      da0 at umass-sim0 bus 0 scbus6 target 0 lun 0
      da0: <Multiple Card  Reader 1.00> Removable Direct Access SPC-2 SCSI device
      da0: Serial Number 058F63666485
      da0: 40.000MB/s transfers
      da0: Attempt to query device size failed: NOT READY, Medium not present
      da0: quirks=0x2<NO_6_BYTE>
      random: unblocking device.
      

      This is the full output of the boot from the serial console:

      PC Engines APU BIOS build date: Apr  5 2014
      Reading data from file [bootorder]
      SeaBIOS (version ?-20140405_120742-frink)
      SeaBIOS (version ?-20140405_120742-frink)
      Found coreboot cbmem console @ df150400
      Found mainboard PC Engines APU
      Relocating init from 0x000e8e71 to 0xdf1065e0 (size 39259)
      Found CBFS header at 0xfffffb90
      found file "bootorder" in cbmem
      CPU Mhz=1000
      Found 27 PCI devices (max PCI bus is 05)
      Copying PIR from 0xdf160400 to 0x000f27a0
      Copying MPTABLE from 0xdf161400/df161410 to 0x000f25b0 with length 1ec
      Copying ACPI RSDP from 0xdf162400 to 0x000f2590
      Copying SMBIOS entry point from 0xdf16d800 to 0x000f2570
      Using pmtimer, ioport 0x808
      Scan for VGA option rom
      EHCI init on dev 00:12.2 (regs=0xf7f08420)
      Found 1 lpt ports
      Found 2 serial ports
      AHCI controller at 11.0, iobase f7f08000, irq 11
      EHCI init on dev 00:13.2 (regs=0xf7f08520)
      EHCI init on dev 00:16.2 (regs=0xf7f08620)
      Searching bootorder for: /rom@img/setup
      Searching bootorder for: /rom@img/memtest
      Searching bootorder for: /pci@i0cf8/*@11/drive@0/disk@0
      AHCI/0: registering: "AHCI/0: INTEL SSDMCEAC030B3 ATA-9 Hard-Disk (28626 MiBytes                                                                                                                                                             )"
      OHCI init on dev 00:12.0 (regs=0xf7f04000)
      OHCI init on dev 00:13.0 (regs=0xf7f05000)
      OHCI init on dev 00:14.5 (regs=0xf7f06000)
      OHCI init on dev 00:16.0 (regs=0xf7f07000)
      Searching bootorder for: /pci@i0cf8/usb@16,2/storage@1/*@0/*@0,0
      Searching bootorder for: /pci@i0cf8/usb@16,2/usb-*@1
      USB MSC vendor='Multiple' product='Card  Reader' rev='1.00' type=0 removable=1
      Device reports MEDIUM NOT PRESENT
      scsi_is_ready returned -1
      Unable to configure USB MSC drive.
      Unable to configure USB MSC device.
      All threads complete.
      Scan for option roms
      Running option rom at c000:0003
      
      
      iPXE (http://ipxe.org) 00:00.0 C000 PCI2.10 PnP PMMpmm call arg1=1
      pmm call arg1=0
      +DF0F07E0pmm call arg1=1
      pmm call arg1=0
      +DF04C510 C000
      
      
      
      Searching bootorder for: /rom@genroms/pxeboot.rom
      
      Build date: Apr  5 2014
      System memory size: 4592 MB
      
      Press F12 for boot menu.
      
      Searching bootorder for: HALT
      drive 0x000f2500: PCHS=16383/16/63 translation=lba LCHS=1024/255/63 s=58626288
      Space available for UMB: c1000-ee800, f0000-f2500
      Returned 253952 bytes of ZoneHigh
      e820 map has 7 items:
        0: 0000000000000000 - 000000000009fc00 = 1 RAM
        1: 000000000009fc00 - 00000000000a0000 = 2 RESERVED
        2: 00000000000f0000 - 0000000000100000 = 2 RESERVED
        3: 0000000000100000 - 00000000df14e000 = 1 RAM
        4: 00000000df14e000 - 00000000e0000000 = 2 RESERVED
        5: 00000000f8000000 - 00000000f9000000 = 2 RESERVED
        6: 0000000100000000 - 000000011f000000 = 1 RAM
      enter handle_19:
        NULL
      Booting from Hard Disk...
      Booting from 0000:7c00
      
      F1  pfSense
      
      F6 PXE
      Boot:  F1
      //bboooott//ccoonnffiigg::  --SS111155220000  --DD
      
      Consoles: internal video/keyboard  serial port
      BIOS drive C: is disk0
      BIOS 639kB/3653944kB available memory
      
      FreeBSD/x86 bootstrap loader, Revision 1.1
      (Wed Nov 21 11:25:53 EST 2018 root@buildbot2.nyi.netgate.com)
      Error: stack underflow
      Error while including /boot/loader.4th, in the line:
      -
      
      /boot/kernel/kernel text=0x17c75d8 data=0xb949e0+0x557b28 syms=[0x8+0x1983a8+0x8+0x1990f6]
      Hit [Enter] to boot immediately, or any other key for command prompt.
      Booting [/boot/kernel/kernel]...
      KDB: debugger backends: ddb
      KDB: current backend: ddb
      Copyright (c) 1992-2018 The FreeBSD Project.
      Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
              The Regents of the University of California. All rights reserved.
      FreeBSD is a registered trademark of The FreeBSD Foundation.
      FreeBSD 11.2-RELEASE-p10 #9 10fea60fdde(factory-RELENG_2_4_4): Thu May 16 06:55:01 EDT 2019
          root@buildbot1-nyi.netgate.com:/build/factory-crossbuild-244/obj/amd64/bwlyKhYx/build/factory-crossbuild-244/pfSense/tmp/FreeBSD-src/sys/pfSense amd64
      FreeBSD clang version 6.0.0 (tags/RELEASE_600/final 326565) (based on LLVM 6.0.0)
      VT(vga): resolution 640x480
      CPU: AMD G-T40E Processor (1000.02-MHz K8-class CPU)
        Origin="AuthenticAMD"  Id=0x500f20  Family=0x14  Model=0x2  Stepping=0
        Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT>
        Features2=0x802209<SSE3,MON,SSSE3,CX16,POPCNT>
        AMD Features=0x2e500800<SYSCALL,NX,MMX+,FFXSR,Page1GB,RDTSCP,LM>
        AMD Features2=0x35ff<LAHF,CMP,SVM,ExtAPIC,CR8,ABM,SSE4A,MAS,Prefetch,IBS,SKINIT,WDT>
        SVM: NP,NRIP,NAsids=8
        TSC: P-state invariant, performance statistics
      real memory  = 4815060992 (4592 MB)
      avail memory = 4054708224 (3866 MB)
      Event timer "LAPIC" quality 600
      ACPI APIC Table: <CORE   COREBOOT>
      FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
      FreeBSD/SMP: 1 package(s) x 2 core(s)
      ioapic0 <Version 2.1> irqs 0-23 on motherboard
      SMP: AP CPU #1 Launched!
      Timecounter "TSC" frequency 1000023140 Hz quality 800
      ipw_bss: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw.LICENSE.
      ipw_bss: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf.
      module_register_init: MOD_LOAD (ipw_bss_fw, 0xffffffff80685700, 0) error 1
      random: entropy device external interface
      ipw_ibss: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw.LICENSE.
      ipw_ibss: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf.
      module_register_init: MOD_LOAD (ipw_ibss_fw, 0xffffffff806857b0, 0) error 1
      ipw_monitor: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw.LICENSE.
      ipw_monitor: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf.
      module_register_init: MOD_LOAD (ipw_monitor_fw, 0xffffffff80685860, 0) error 1
      iwi_bss: You need to read the LICENSE file in /usr/share/doc/legal/intel_iwi.LICENSE.
      iwi_bss: If you agree with the license, set legal.intel_iwi.license_ack=1 in /boot/loader.conf.
      module_register_init: MOD_LOAD (iwi_bss_fw, 0xffffffff806ac730, 0) error 1
      iwi_ibss: You need to read the LICENSE file in /usr/share/doc/legal/intel_iwi.LICENSE.
      iwi_ibss: If you agree with the license, set legal.intel_iwi.license_ack=1 in /boot/loader.conf.
      module_register_init: MOD_LOAD (iwi_ibss_fw, 0xffffffff806ac7e0, 0) error 1
      iwi_monitor: You need to read the LICENSE file in /usr/share/doc/legal/intel_iwi.LICENSE.
      iwi_monitor: If you agree with the license, set legal.intel_iwi.license_ack=1 in /boot/loader.conf.
      module_register_init: MOD_LOAD (iwi_monitor_fw, 0xffffffff806ac890, 0) error 1
      wlan: mac acl policy registered
      kbd0 at kbdmux0
      netmap: loaded module
      module_register_init: MOD_LOAD (vesa, 0xffffffff8120faa0, 0) error 19
      nexus0
      vtvga0: <VT VGA driver> on motherboard
      cryptosoft0: <software crypto> on motherboard
      padlock0: No ACE support.
      acpi0: <CORE COREBOOT> on motherboard
      acpi0: Power Button (fixed)
      cpu0: <ACPI CPU> on acpi0
      cpu1: <ACPI CPU> on acpi0
      atrtc0: <AT realtime clock> port 0x70-0x71 irq 8 on acpi0
      atrtc0: registered as a time-of-day clock, resolution 1.000000s
      Event timer "RTC" frequency 32768 Hz quality 0
      attimer0: <AT timer> port 0x40-0x43 irq 0 on acpi0
      Timecounter "i8254" frequency 1193182 Hz quality 0
      Event timer "i8254" frequency 1193182 Hz quality 100
      Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
      acpi_timer0: <32-bit timer at 3.579545MHz> port 0x808-0x80b on acpi0
      hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
      Timecounter "HPET" frequency 14318180 Hz quality 950
      Event timer "HPET" frequency 14318180 Hz quality 550
      Event timer "HPET1" frequency 14318180 Hz quality 450
      pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
      pci0: <ACPI PCI bus> on pcib0
      pcib1: <ACPI PCI-PCI bridge> irq 16 at device 4.0 on pci0
      pci1: <ACPI PCI bus> on pcib1
      re0: <RealTek 8168/8111 B/C/CP/D/DP/E/F/G PCIe Gigabit Ethernet> port 0x1000-0x10ff mem 0xf7a00000-0xf7a00fff,0xf7900000-0xf7903fff irq 16 at device 0.0 on pci1
      re0: Using 1 MSI-X message
      re0: ASPM disabled
      re0: Chip rev. 0x2c000000
      re0: MAC rev. 0x00200000
      miibus0: <MII bus> on re0
      rgephy0: <RTL8169S/8110S/8211 1000BASE-T media interface> PHY 1 on miibus0
      rgephy0:  none, 10baseT, 10baseT-FDX, 10baseT-FDX-flow, 100baseTX, 100baseTX-FDX, 100baseTX-FDX-flow, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, 1000baseT-FDX-flow, 1000baseT-FDX-flow-master, auto, auto-flow
      re0: Using defaults for TSO: 65518/35/2048
      re0: Ethernet address: 00:0d:b9:3a:4c:f8
      re0: netmap queues/slots: TX 1/256, RX 1/256
      pcib2: <ACPI PCI-PCI bridge> irq 17 at device 5.0 on pci0
      pci2: <ACPI PCI bus> on pcib2
      re1: <RealTek 8168/8111 B/C/CP/D/DP/E/F/G PCIe Gigabit Ethernet> port 0x2000-0x20ff mem 0xf7c00000-0xf7c00fff,0xf7b00000-0xf7b03fff irq 17 at device 0.0 on pci2
      re1: Using 1 MSI-X message
      re1: ASPM disabled
      re1: Chip rev. 0x2c000000
      re1: MAC rev. 0x00200000
      miibus1: <MII bus> on re1
      rgephy1: <RTL8169S/8110S/8211 1000BASE-T media interface> PHY 1 on miibus1
      rgephy1:  none, 10baseT, 10baseT-FDX, 10baseT-FDX-flow, 100baseTX, 100baseTX-FDX, 100baseTX-FDX-flow, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, 1000baseT-FDX-flow, 1000baseT-FDX-flow-master, auto, auto-flow
      re1: Using defaults for TSO: 65518/35/2048
      re1: Ethernet address: 00:0d:b9:3a:4c:f9
      re1: netmap queues/slots: TX 1/256, RX 1/256
      pcib3: <ACPI PCI-PCI bridge> irq 18 at device 6.0 on pci0
      pci3: <ACPI PCI bus> on pcib3
      re2: <RealTek 8168/8111 B/C/CP/D/DP/E/F/G PCIe Gigabit Ethernet> port 0x3000-0x30ff mem 0xf7e00000-0xf7e00fff,0xf7d00000-0xf7d03fff irq 18 at device 0.0 on pci3
      re2: Using 1 MSI-X message
      re2: ASPM disabled
      re2: Chip rev. 0x2c000000
      re2: MAC rev. 0x00200000
      miibus2: <MII bus> on re2
      rgephy2: <RTL8169S/8110S/8211 1000BASE-T media interface> PHY 1 on miibus2
      rgephy2:  none, 10baseT, 10baseT-FDX, 10baseT-FDX-flow, 100baseTX, 100baseTX-FDX, 100baseTX-FDX-flow, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, 1000baseT-FDX-flow, 1000baseT-FDX-flow-master, auto, auto-flow
      re2: Using defaults for TSO: 65518/35/2048
      re2: Ethernet address: 00:0d:b9:3a:4c:fa
      re2: netmap queues/slots: TX 1/256, RX 1/256
      ahci0: <AMD SB7x0/SB8x0/SB9x0 AHCI SATA controller> port 0x4010-0x4017,0x4020-0x4023,0x4018-0x401f,0x4024-0x4027,0x4000-0x400f mem 0xf7f08000-0xf7f083ff irq 19 at device 17.0 on pci0
      ahci0: AHCI v1.20 with 6 6Gbps ports, Port Multiplier supported
      ahci0: quirks=0x22000<ATI_PMP_BUG,1MSI>
      ahcich0: <AHCI channel> at channel 0 on ahci0
      ahcich1: <AHCI channel> at channel 1 on ahci0
      ahcich2: <AHCI channel> at channel 2 on ahci0
      ahcich3: <AHCI channel> at channel 3 on ahci0
      ahcich4: <AHCI channel> at channel 4 on ahci0
      ahcich5: <AHCI channel> at channel 5 on ahci0
      ohci0: <AMD SB7x0/SB8x0/SB9x0 USB controller> mem 0xf7f04000-0xf7f04fff irq 18 at device 18.0 on pci0
      usbus0 on ohci0
      usbus0: 12Mbps Full Speed USB v1.0
      ehci0: <AMD SB7x0/SB8x0/SB9x0 USB 2.0 controller> mem 0xf7f08400-0xf7f084ff irq 17 at device 18.2 on pci0
      usbus1: EHCI version 1.0
      usbus1 on ehci0
      usbus1: 480Mbps High Speed USB v2.0
      ohci1: <AMD SB7x0/SB8x0/SB9x0 USB controller> mem 0xf7f05000-0xf7f05fff irq 18 at device 19.0 on pci0
      usbus2 on ohci1
      usbus2: 12Mbps Full Speed USB v1.0
      ehci1: <AMD SB7x0/SB8x0/SB9x0 USB 2.0 controller> mem 0xf7f08500-0xf7f085ff irq 17 at device 19.2 on pci0
      usbus3: EHCI version 1.0
      usbus3 on ehci1
      usbus3: 480Mbps High Speed USB v2.0
      isab0: <PCI-ISA bridge> at device 20.3 on pci0
      isa0: <ISA bus> on isab0
      pcib4: <ACPI PCI-PCI bridge> at device 20.4 on pci0
      pci4: <ACPI PCI bus> on pcib4
      ohci2: <AMD SB7x0/SB8x0/SB9x0 USB controller> mem 0xf7f06000-0xf7f06fff irq 18 at device 20.5 on pci0
      usbus4 on ohci2
      usbus4: 12Mbps Full Speed USB v1.0
      pcib5: <ACPI PCI-PCI bridge> at device 21.0 on pci0
      pci5: <ACPI PCI bus> on pcib5
      ohci3: <AMD SB7x0/SB8x0/SB9x0 USB controller> mem 0xf7f07000-0xf7f07fff at device 22.0 on pci0
      usbus5 on ohci3
      usbus5: 12Mbps Full Speed USB v1.0
      ehci2: <AMD SB7x0/SB8x0/SB9x0 USB 2.0 controller> mem 0xf7f08600-0xf7f086ff at device 22.2 on pci0
      usbus6: EHCI version 1.0
      usbus6 on ehci2
      usbus6: 480Mbps High Speed USB v2.0
      acpi_button0: <Power Button> on acpi0
      orm0: <ISA Option ROM> at iomem 0xee800-0xeffff on isa0
      acpi_throttle0: <ACPI CPU Throttling> on cpu0
      acpi_throttle1: <ACPI CPU Throttling> on cpu1
      acpi_throttle1: failed to attach P_CNT
      device_attach: acpi_throttle1 attach returned 6
      Timecounters tick every 1.000 msec
      ugen1.1: <ATI EHCI root HUB> at usbus1
      ugen6.1: <ATI EHCI root HUB> at usbus6
      uhub0: <ATI EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus1
      uhub1: <ATI EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus6
      ugen5.1: <ATI OHCI root HUB> at usbus5
      ugen2.1: <ATI OHCI root HUB> at usbus2
      uhub2: <ATI OHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus5
      uhub3: <ATI OHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus2
      ugen0.1: <ATI OHCI root HUB> at usbus0
      ugen4.1: <ATI OHCI root HUB> at usbus4
      uhub4: <ATI OHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0
      uhub5: <ATI OHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus4
      ugen3.1: <ATI EHCI root HUB> at usbus3
      uhub6: <ATI EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus3
      ada0 at ahcich0 bus 0 scbus0 target 0 lun 0
      uhub3: 5 ports with 5 removable, self powered
      uhub5: 2 ports with 2 removable, self powered
      uhub4: 5 ports with 5 removable, self powered
      ada0: <INTEL SSDMCEAC030B3 LLLi> ACS-2 ATA SATA 3.x device
      uhub2: ada0: Serial Number CVLI446200X7030H
      4 ports with 4 removable, self powered
      ada0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes)
      ada0: Command Queueing enabled
      ada0: 28626MB (58626288 512 byte sectors)
      Trying to mount root from ufs:/dev/ufsid/5358b0d792733dcf [rw]...
      WARNING: / was not properly dismounted
      uhub1: 4 ports with 4 removable, self powered
      uhub0: 5 ports with 5 removable, self powered
      ugen6.2: <Generic Flash Card ReaderWriter> at usbus6
      umass0 on uhub1
      umass0: <Generic Flash Card ReaderWriter, class 0/0, rev 2.01/1.00, addr 2> on usbus6
      uhub6: 5 ports with 5 removable, self powered
      da0 at umass-sim0 bus 0 scbus6 target 0 lun 0
      da0: <Multiple Card  Reader 1.00> Removable Direct Access SPC-2 SCSI device
      da0: Serial Number 058F63666485
      da0: 40.000MB/s transfers
      da0: Attempt to query device size failed: NOT READY, Medium not present
      da0: quirks=0x2<NO_6_BYTE>
      random: unblocking device.
      
      
      1 Reply Last reply Reply Quote 0
      • stephenw10S
        stephenw10 Netgate Administrator
        last edited by

        Hmm, that looks a lot like the wrong console type set as primary. Did you import the config into the APU from something else originally?

        Do you have a backup config file you can check? Or use to re-install if needed.

        Steve

        1 Reply Last reply Reply Quote 0
        • B
          benkillin
          last edited by

          @stephenw10 said in netgate APU4 bricked after initiating update in web console:

          nothing was changed in the system. this came as a preinstalled pfsense image from netgate. the only thing I have ever done was press the upgrade button in the web console. no config import was performed. backup config - none unless pfsense itself does one.

          1 Reply Last reply Reply Quote 0
          • stephenw10S
            stephenw10 Netgate Administrator
            last edited by

            Hmm, you could try booting into single user mode from the loader menu. But to be honest if you have a backup of the config file it's going to be quickest to reinstall. And it will guarantee you a clean filesystem.

            Steve

            1 Reply Last reply Reply Quote 0
            • B
              benkillin
              last edited by

              I tried booting to single user mode by specifying boot -s at the kernel loader shell, but the same problem occurred where console output stopped. I worked with some people on the freebsd IRC channel on freenode and it appears somehow multiple files were corrupted. /boot/loader.4th was corrupted totally and /etc/ttys was truncated. I ended up doing a fresh install after taking a forensic backup of the drive, which is unfortunate that I lost all my settings.

              1 Reply Last reply Reply Quote 0
              • stephenw10S
                stephenw10 Netgate Administrator
                last edited by

                You were not able to recover the config during the install? It must have been a significantly damaged file system if so.

                Steve

                1 Reply Last reply Reply Quote 0
                • First post
                  Last post
                Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.