My power went out and now pfSense is rebooting constantly
-
My power went out and when it came back on, my NetGte 3100 pfSense firewall is booting and as soon as it finishes it auto reboots. The lights on the front are blue blinking slowly one then two then three.
If I hook up to the console I see
***** FILE SYSTEM MARKED CLEAN ***** Filesystems are clean, continuing... Mounting filesystems... __ _ __ / _|___ ___ _ __ ___ ___ _ | '_ \| |_/ __|/ _ \ '_ \/ __|/ _ \ _| |_ | |_) | _\__ \ __/ | | \__ \ __/ |_ _| | .__/|_| |___/\___|_| |_|___/\___| |_| |_| Welcome to Netgate pfSense Plus 21.05.2-RELEASE... ...ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/compat/pkg /usr/lib/engines /usr/local/lib/compat/pkg /usr/local/lib/ipsec /usr/local/lib/perl5/5.32/mach/CORE Soft Float compatibility ldconfig path: done. External config loader 1.0 is now starting... Launching the init system...mode = 0100666, inum = 45895, fs = / panic: ffs_valloc: dup alloc cpuid = 0 time = 1643400592 Uptime: 16s Automatic reboot in 15 seconds - press a key on the console to abort --> Press a key on the console to reboot, --> or switch off the system now.
I found this posting
https://forum.netgate.com/topic/134042/sg-1000-stuck-in-boot-loop-after-initial-configuration/3I tried what it said there, now I'm stuck in UBOOT
Marvell>>
and cannot get out.
-
@mcaughey
If I were you , I would contact pfSense tech support.
Seems like you need some tech help.Maybe @stephenw10 can help out here
/Bingo
-
@mcaughey
I'd assume that your filesystem is dirty. I guess, it's an UFS?
You can try to repair it.To do so connect to the console and boot the box into the single user mode and enter the prompt. Then enter
/sbin/fsck -y -t ufs
This will take a few minutes. You should see a success message when it's finished.
-
@viragomann It won't come out of UBOOT. I just get a prompt
Marvell>>
-
@mcaughey
Don't you get this screen at the beginning:
I don't know the 3100. Maybe it haves differently.
-
@mcaughey said in My power went out and now pfSense is rebooting constantly:
@viragomann It won't come out of UBOOT. I just get a prompt
Marvell>>
Carefully follow the instructions here from the Netgate docs for the SG-3100: https://docs.netgate.com/pfsense/en/latest/troubleshooting/filesystem-check.html.
When you get the
Marvell>>
prompt, that means you did not catch the boot-up and interrupt it at the right point.Netgate appliances are full-fledged PCs with an active filesystem. They really do not like losing power. You should put your firewall on a UPS. You can then install either the
nut
orapcupsd
package to monitor the UPS status and remaining battery life and automatically shutdown the firewall when power fails and the UPS battery is near exhaustion. That will prevent disk corruption. -
@mcaughey said in My power went out and now pfSense is rebooting constantly:
panic: ffs_valloc: dup alloc
That indicates a file system error from which you should be able to recover by running fsck.
At the Marvell>> prompt you should be able to just enter 'boot' to continue. It should boot past that automatically though. What error do you see there?
If you cannot get past that you can always re-install. Open a ticket with us to get a recovery image: https://www.netgate.com/tac-support-request
Steve
-
@bmeeks Thanks for this information. However, I cannot get to a prompt no matter what I do.
@stephenw10 If I type
boot
at the prompt it just boots back to the Marvell UBoot.
It just keeps coming up to Marvel prompt. I found a manual and I found the help function. It looks like I'm gong to have to read the manual and just be ok with being down for a week or so.
If I could get to a single user boot, I could do what the article, which is the same thing others have said.
The following just keep cycling. I press any key then I get to the prompt below.
General initialization - Version: 1.0.0 AVS selection from EFUSE disabled (Skip reading EFUSE values) Overriding default AVS value to: 0x23 Detected Device ID 6820 High speed PHY - Version: 2.0 Init Customer board board SerDes lanes topology details: | Lane # | Speed| Type | ------------------------------| | 0 | 3 | SATA0 | | 1 | 5 | PCIe0 | | 2 | 3 | SATA1 | | 3 | 4 | SGMII2 | | 4 | 5 | PCIe1 | | 5 | 5 | USB3 HOST1 | ------------------------------- PCIe Ref. Clock Buffer Control is 0xf00015bf, setting to 0xf00015bf SOC_CONTROL_REG1 is 0xf00015bf, setting to 0x0707c0f3 PCIe, Idx 0: detected no link PCIe, Idx 1: detected no link High speed PHY - Ended Successfully mv_ddr: mv_ddr-armada-17.06.1-g07f8294 (Oct 8 2018 - 12:59:22) DDR4 Training Sequence - Switching XBAR Window to FastPath Window mv_ddr: completed successfully ā __ __ _ _ | \/ | __ _ _ ____ _____| | | | |\/| |/ _` | '__\ \ / / _ \ | | | | | | (_| | | \ V / __/ | | |_| |_|\__,_|_| \_/ \___|_|_| _ _ ____ _ | | | | | __ ) ___ ___ | |_ | | | |___| _ \ / _ \ / _ \| __| | |_| |___| |_) | (_) | (_) | |_ \___/ |____/ \___/ \___/ \__| ** LOADER ** U-Boot 2013.01-02879-geb5cbf6642 (Oct 08 2018 - 12:59:19) Marvell version: devel-17.06.0 Board: Rogue-1 SoC: MV88F6820 Rev A0 running 2 CPUs CPU: ARM Cortex A9 MPCore (Rev 1) LE CPU 0 CPU @ 1600 [MHz] L2 @ 800 [MHz] TClock @ 250 [MHz] DDR4 @ 800 [MHz] DDR4 32 Bit Width,FastPath Memory Access, DLB Enabled, ECC Disabled DRAM: 2 GiB MMC: mv_sdh: 0 DBG: Calling spi_flash_probe from env_relocate_spec() SF: Probing bus 0 cs 0 @ 20000000Hz mode 3 SF: Detected W25Q32JV with page size 4 KiB, total 4 MiB PCI-e 0: Detected No Link. PCI-e 1: Detected No Link. USB2.0 0: Host Mode USB3.0 1: Host Mode Map: Code: 0x7fedc000:0x7ff9741c BSS: 0x7ffef600 Stack: 0x7f4cbf20 Heap: 0x7f4cc000:0x7fedc000 U-Boot Environment: 0x00100000:0x00110000 (SPI) Board configuration detected: Net: | port | Interface | PHY address | |--------|-----------|--------------| | egiga0 | RGMII | 0x00 | | egiga1 | RGMII | 0x01 | | egiga2 | SGMII | In-Band | egiga0 [PRIME], egiga1, egiga2 Hit any key to stop autoboot: 0 Marvell>> Marvell>> Marvell>> Marvell>>
-
I don't see where you typed boot there so I'm not sure exactly what happened.
This is what I expect to see there:
Hit any key to stop autoboot: 0 Marvell>> boot Reset SCSI AHCI init for unit0 Target spinup took 0 ms. SATA link 1 timeout. Error: SCSI Controller(s) 1B4B:9215 1B4B:9235 not found scanning bus for devices... Device 0: (0:0) Vendor: ATA Prod.: NT-32 Rev: 1.09 Type: Hard Disk Capacity: 30533.8 MB = 29.8 GB (62533296 x 512) Found 1 device(s). reading ubldr.bin 228648 bytes read in 10 ms (21.8 MiB/s) ## Starting application at 0x00200000 ... Consoles: U-Boot console Compatible U-Boot API signature found @0x7f4dc280 FreeBSD/armv6 U-Boot loader, Revision 1.2 (Tue Jul 10 10:26:23 EDT 2018 root@buildbot3) DRAM: 2048MB Number of U-Boot devices: 4 U-Boot env: loaderdev not set, will probe all devices. Found U-Boot device: disk Probing all disk devices... Checking unit=0 slice=<auto> partition=<auto>...disk0: read failed, error=1 Checking unit=1 slice=<auto> partition=<auto>... good. Booting from disk1s2a: Loading /boot/defaults/loader.conf console comconsole is invalid! no valid consoles! Available consoles: uboot /boot/kernel/kernel text=0xa9a600 data=0x11d4be8 data=0x0+0x54000 syms=[0x4+0xa9240+0x4+0xf4222] \ Hit [Enter] to boot immediately, or any other key for command prompt. Booting [/boot/kernel/kernel]... /boot/dtb/armada-385-80300-0148-G00-X100.dtb size=0x60ed Loaded DTB from file 'armada-385-80300-0148-G00-X100.dtb'. Kernel entry at 0x400180... Kernel args: (null) SOC: Marvell 88F6828, TClock 250MHz, Frequency 1600MHz Instruction cache prefetch enabled, data cache prefetch disabled ---<<BOOT>>--- Copyright (c) 1992-2021 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 12.3-STABLE plus-RELENG_22_01-n202637-97cc5c23e13 pfSense-3100 arm FreeBSD clang version 10.0.1 (git@github.com:llvm/llvm-project.git llvmorg-10.0.1-0-gef32c611aa2)
-
@stephenw10 said in My power went out and now pfSense is rebooting constantly:
Hi I am having an issue with an SG-3100 that I have, we never used this unit, it was kept as a spare device for majority of its life we purchased it back in 2019 or 2020 I believe. Today when I was updating it to the latest version 21.02.2 it updated fine but didn't come back after a restart.
Now its stuck in UBoot and I don't see the boot prompt to switch to single user mode.
General initialization - Version: 1.0.0 AVS selection from EFUSE disabled (Skip reading EFUSE values) Overriding default AVS value to: 0x23 Detected Device ID 6820 High speed PHY - Version: 2.0 Init Customer board board SerDes lanes topology details: | Lane # | Speed| Type | ------------------------------| | 0 | 3 | SATA0 | | 1 | 5 | PCIe0 | | 2 | 3 | SATA1 | | 3 | 4 | SGMII2 | | 4 | 5 | PCIe1 | | 5 | 5 | USB3 HOST1 | ------------------------------- PCIe Ref. Clock Buffer Control is 0xf00015bf, setting to 0xf00015bf SOC_CONTROL_REG1 is 0xf00015bf, setting to 0x0707c0f3 General initialization - Version: 1.0.0 AVS selection from EFUSE disabled (Skip reading EFUSE values) Overriding default AVS value to: 0x23 Detected Device ID 6820 High speed PHY - Version: 2.0 Init Customer board board SerDes lanes topology details: | Lane # | Speed| Type | ------------------------------| | 0 | 3 | SATA0 | | 1 | 5 | PCIe0 | | 2 | 3 | SATA1 | | 3 | 4 | SGMII2 | | 4 | 5 | PCIe1 | | 5 | 5 | USB3 HOST1 | ------------------------------- PCIe Ref. Clock Buffer Control is 0xf00015bf, setting to 0xf00015bf SOC_CONTROL_REG1 is 0xf00015bf, setting to 0x0707c0f3 PCIe, Idx 0: detected no link PCIe, Idx 1: detected no link High speed PHY - Ended Successfully mv_ddr: mv_ddr-armada-17.06.1-g07f8294 (Oct 8 2018 - 12:59:22) DDR4 Training Sequence - Switching XBAR Window to FastPath Window mv_ddr: completed successfully __ __ _ _ | \/ | __ _ _ ____ _____| | | | |\/| |/ _` | '__\ \ / / _ \ | | | | | | (_| | | \ V / __/ | | |_| |_|\__,_|_| \_/ \___|_|_| _ _ ____ _ | | | | | __ ) ___ ___ | |_ | | | |___| _ \ / _ \ / _ \| __| | |_| |___| |_) | (_) | (_) | |_ \___/ |____/ \___/ \___/ \__| ** LOADER ** U-Boot 2013.01-02879-geb5cbf6642 (Oct 08 2018 - 12:59:19) Marvell version: devel -17.06.0 Board: Rogue-1 SoC: MV88F6820 Rev A0 running 2 CPUs CPU: ARM Cortex A9 MPCore (Rev 1) LE CPU 0 CPU @ 1600 [MHz] L2 @ 800 [MHz] TClock @ 250 [MHz] DDR4 @ 800 [MHz] DDR4 32 Bit Width,FastPath Memory Access, DLB Enabled, ECC Disabled DRAM: 2 GiB MMC: mv_sdh: 0 DBG: Calling spi_flash_probe from env_relocate_spec() SF: Probing bus 0 cs 0 @ 20000000Hz mode 3 SF: Detected W25Q32JV with page size 4 KiB, total 4 MiB PCI-e 0: Detected No Link. PCI-e 1: Detected No Link. USB2.0 0: Host Mode USB3.0 1: Host Mode Map: Code: 0x7fedc000:0x7ff9741c BSS: 0x7ffef600 Stack: 0x7f4cbf20 Heap: 0x7f4cc000:0x7fedc000 U-Boot Environment: 0x00100000:0x00110000 (SPI) Board configuration detected: Net: | port | Interface | PHY address | |--------|-----------|--------------| | egiga0 | RGMII | 0x00 | | egiga1 | RGMII | 0x01 | | egiga2 | SGMII | In-Band | egiga0 [PRIME], egiga1, egiga2 Hit any key to stop autoboot: 0 reading ubldr.bin ** Unable to read file ubldr.bin ** ## Starting application at 0x00200000 ...
It gets stuck here.
General initialization - Version: 1.0.0 AVS selection from EFUSE disabled (Skip reading EFUSE values) Overriding default AVS value to: 0x23 Detected Device ID 6820 High speed PHY - Version: 2.0 Init Customer board board SerDes lanes topology details: | Lane # | Speed| Type | ------------------------------| | 0 | 3 | SATA0 | | 1 | 5 | PCIe0 | | 2 | 3 | SATA1 | | 3 | 4 | SGMII2 | | 4 | 5 | PCIe1 | | 5 | 5 | USB3 HOST1 | ------------------------------- PCIe Ref. Clock Buffer Control is 0xf00015bf, setting to 0xf00015bf SOC_CONTROL_REG1 is 0xf00015bf, setting to 0x0707c0f3 PCIe, Idx 0: detected no link PCIe, Idx 1: detected no link High speed PHY - Ended Successfully mv_ddr: mv_ddr-armada-17.06.1-g07f8294 (Oct 8 2018 - 12:59:22) General initialization - Version: 1.0.0 AVS selection from EFUSE disabled (Skip reading EFUSE values) Overriding default AVS value to: 0x23 Detected Device ID 6820 High speed PHY - Version: 2.0 Init Customer board board SerDes lanes topology details: | Lane # | Speed| Type | ------------------------------| | 0 | 3 | SATA0 | | 1 | 5 | PCIe0 | | 2 | 3 | SATA1 | | 3 | 4 | SGMII2 | | 4 | 5 | PCIe1 | | 5 | 5 | USB3 HOST1 | ------------------------------- PCIe Ref. Clock Buffer Control is 0xf00015bf, setting to 0xf00015bf SOC_CONTROL_REG1 is 0xf00015bf, setting to 0x0707c0f3 PCIe, Idx 0: detected no link PCIe, Idx 1: detected no link High speed PHY - Ended Successfully mv_ddr: mv_ddr-armada-17.06.1-g07f8294 (Oct 8 2018 - 12:59:22) DDR4 Training Sequence - Switching XBAR Window to FastPath Window mv_ddr: completed successfully __ __ _ _ | \/ | __ _ _ ____ _____| | | | |\/| |/ _` | '__\ \ / / _ \ | | | | | | (_| | | \ V / __/ | | |_| |_|\__,_|_| \_/ \___|_|_| _ _ ____ _ | | | | | __ ) ___ ___ | |_ | | | |___| _ \ / _ \ / _ \| __| | |_| |___| |_) | (_) | (_) | |_ \___/ |____/ \___/ \___/ \__| ** LOADER ** U-Boot 2013.01-02879-geb5cbf6642 (Oct 08 2018 - 12:59:19) Marvell version: devel-17.06.0 Board: Rogue-1 SoC: MV88F6820 Rev A0 running 2 CPUs CPU: ARM Cortex A9 MPCore (Rev 1) LE CPU 0 CPU @ 1600 [MHz] L2 @ 800 [MHz] TClock @ 250 [MHz] DDR4 @ 800 [MHz] DDR4 32 Bit Width,FastPath Memory Access, DLB Enabled, ECC Disabled DRAM: 2 GiB MMC: mv_sdh: 0 DBG: Calling spi_flash_probe from env_relocate_spec() SF: Probing bus 0 cs 0 @ 20000000Hz mode 3 SF: Detected W25Q32JV with page size 4 KiB, total 4 MiB PCI-e 0: Detected No Link. PCI-e 1: Detected No Link. USB2.0 0: Host Mode USB3.0 1: Host Mode Map: Code: 0x7fedc000:0x7ff9741c BSS: 0x7ffef600 Stack: 0x7f4cbf20 Heap: 0x7f4cc000:0x7fedc000 U-Boot Environment: 0x00100000:0x00110000 (SPI) Board configuration detected: Net: | port | Interface | PHY address | |--------|-----------|--------------| | egiga0 | RGMII | 0x00 | | egiga1 | RGMII | 0x01 | | egiga2 | SGMII | In-Band | egiga0 [PRIME], egiga1, egiga2 Hit any key to stop autoboot: 0 Marvell>>A Unknown command '' - try 'help' Marvell>> boot -s boot - boot default, i.e., run 'bootcmd' Usage: boot
What do I do? how do I restart from the
Marvell>>
prompt? or exit out of it? -
@aasimenator 21.02 is actually a few versions old. 22.01 is latest. Did yours lose power also? (per this thread)
I had an issue upgrading an older 3100 recently where it did not offer the latest...it took the first version and then the second upgrade said it would upgrade to 21.05 but half installed 22.01 and I had to reinstall:
https://docs.netgate.com/pfsense/en/latest/solutions/sg-3100/reinstall-pfsense.html -
Yeah I would just open a ticket to get the latest firmware image and then install that clean.
-
@stephenw10
@steveits
No, It didn't lose power during the update, it just didn't come back online after the update.I am getting the firmware reinstalled as of this message. no other way of this error sadly other than a full wipe / fresh install