Reboot loop

[zonenoire]

My SG-3100 pfSense Security Gateway Appliance is now in a reboot loop:

    __

_ __ / |__ ___ _ __ ___ ___
| '_ | |/ _|/ _ \ ' / __|/ _
| |) | _ \ / | | _ \ /
| ./|| |/_|| ||/_|
|_|

Welcome to pfSense 2.4.4-RELEASE...

...ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/ipsec /usr/local/lib/mysql /usr/local/lib/perl5/5.26/mach/CORE
Soft Float compatibility ldconfig path:
done.
panic: ffs_write: type 0xc6671000 0 (0,12017)
cpuid = 0
Uptime: 12s
Automatic reboot in 15 seconds - press a key on the console to abort

[stephenw10]

The filesystem has an error that cannot be repaired on a single fsck pass. Do this

Running a file system consistency check on the SG-3100:

1. At the SG-3100 console press any key to interrupt the boot loader when you see:
   Hit [Enter] to boot immediately, or any other key for command prompt.

   At the prompt enter: boot -s

   That will boot in single user mode to a question asking for a path to the shell, just press return to reach the # prompt.

2. At the # prompt run the following command: fsck -y /

   Run the fsck command at least 3 times; Repeat the command until no errors are reported, even if fsck claims the filesystem has been marked "clean".

3. Reboot by running: reboot

It should boot normally and with no residual long-term issues

Steve

[zonenoire]

Hi Stephen:

I can't get the the # prompt. I can, however get to the Marvell>> and loader> prompts as follows:

---

| / | __ _ _ ____ | | |
| |/| |/ _` | '\ \ / / _ \ | |
| | | | (| | | \ V / __/ | |
|| ||_,|| _/ _|||
_ _ ____ _
| | | | | __ ) ___ ___ | |
| | | || _ \ / _ \ / _ | __|
| || || |) | () | () | |
_/ |____/ _/ __/ __|
** LOADER **

U-Boot 2013.01-02873-g5e4ac3c (Aug 11 2017 - 13:12:40) Marvell version: devel-17.06.0

Board: Rogue-1
SoC: MV88F6820 Rev A0
running 2 CPUs
CPU: ARM Cortex A9 MPCore (Rev 1) LE
CPU 0
CPU @ 1600 [MHz]
L2 @ 800 [MHz]
TClock @ 250 [MHz]
DDR4 @ 800 [MHz]
DDR4 32 Bit Width,FastPath Memory Access, DLB Enabled, ECC Disabled
DRAM: 2 GiB
MMC: mv_sdh: 0
DBG: Calling spi_flash_probe from env_relocate_spec()
SF: Probing bus 0 cs 0 @ 20000000Hz mode 3
SF: Detected N25Q128 with page size 64 KiB, total 16 MiB
PCI-e 0: Detected No Link.
PCI-e 1: Detected No Link.
USB2.0 0: Host Mode
USB3.0 1: Host Mode

Map: Code: 0x7fedc000:0x7ff975a8
BSS: 0x7ffef600
Stack: 0x7f4cbf20
Heap: 0x7f4cc000:0x7fedc000
U-Boot Environment: 0x00100000:0x00110000 (SPI)

Board configuration detected:
Net:

port	Interface	PHY address
egiga0	RGMII	0x00
egiga1	RGMII	0x01
egiga2	SGMII	In-Band

egiga0 [PRIME], egiga1, egiga2
Hit any key to stop autoboot: 0
Marvell>>
Marvell>> bootd

Reset SCSI
AHCI init for unit0
SATA link 0 timeout.
Target spinup took 0 ms.

Error: SCSI Controller(s) 1B4B:9215 1B4B:9235 not found
scanning bus for devices...
Device 0: (1:0) Vendor: ATA Prod.: NT-32 Rev: Q092
Type: Hard Disk
Capacity: 30029.2 MB = 29.3 GB (61500000 x 512)
Found 1 device(s).
reading ubldr.bin
228648 bytes read in 11 ms (19.8 MiB/s)

Starting application at 0x00200000 ...

Consoles: U-Boot console
Compatible U-Boot API signature found @0x7f4dc280

FreeBSD/armv6 U-Boot loader, Revision 1.2
(Wed Sep 5 09:32:58 EDT 2018 root@buildbot3)

DRAM: 2048MB
Number of U-Boot devices: 4
U-Boot env: loaderdev not set, will probe all devices.
Found U-Boot device: disk
Probing all disk devices...
Checking unit=0 slice=<auto> partition=<auto>...disk0: read failed, error=1

Checking unit=1 slice=<auto> partition=<auto>... good.
Booting from disk1s2a:
Loading /boot/defaults/loader.conf
console comconsole is invalid!
no valid consoles!
Available consoles:
uboot
/boot/kernel/kernel data=0x11788d8+0x77728 syms=[0x4+0x97d60+0x4+0xd850d]
/
Hit [Enter] to boot immediately, or any other key for command prompt.

Type '?' for a list of commands, 'help' for more detailed help.
loader> ?
Available commands:
boot boot a file or loaded kernel
autoboot boot automatically after a delay
help detailed help
? list commands
show show variable(s)
set set a variable
unset unset a variable
echo echo arguments
read read input from the terminal
more show contents of a file
lsdev list all devices
include read commands from a file
ls list files
load load a kernel or module
unload unload all modules
lsmod list loaded modules
heap show heap usage
reboot reboot the system
devinfo show U-Boot devices
sysinfo show U-Boot system info
ubenv show or import U-Boot env vars
fdt flattened device tree handling
boot-conf load kernel and modules, then autoboot
read-conf read a configuration file
enable-module enable loading of a module
disable-module disable loading of a module
toggle-module toggle loading of a module
show-module show module load data
try-include try to load/interpret files
loader>

[stephenw10]

@zonenoire said in Reboot loop:

Hit [Enter] to boot immediately, or any other key for command prompt.

At the loader> prompt there enter boot -s.

Then you will reach the shell question and the # prompt.

Steve

[zonenoire]

Thanks Stephen.
SUCCESS!
Here is a synopsis of what I did in case someone else encounters this problem:

Marvell>>boot
...
loader>boot -s
...
Trying to mount root from ufs:/dev/diskid/DISK-CEF032182700058s2a [rw,noatime]...
WARNING: / was not properly dismounted
Enter full pathname of shell or RETURN for /bin/sh:

fsck -y

...

fsck -y

...

fsck -y

...
***** FILE SYSTEM MARKED CLEAN *****
#reboot
...

Netgate SG-3100 - Serial: 1130180696 - Netgate Device ID: bdfbc7c1c133fb11d042

*** Welcome to pfSense 2.4.4-RELEASE (arm) on pfSense ***

WAN (wan) -> mvneta2 -> v4/DHCP4: 192.168.1.9/24
LAN (lan) -> mvneta1 -> v4: 192.168.3.1/31
OPT1 (opt1) -> mvneta0 -> v4: 192.168.100.1/32

0. Logout (SSH only) 9) pfTop
1. Assign Interfaces 10) Filter Logs
2. Set interface(s) IP address 11) Restart webConfigurator
3. Reset webConfigurator password 12) PHP shell + pfSense tools
4. Reset to factory defaults 13) Update from console
5. Reboot system 14) Enable Secure Shell (sshd)
6. Halt system 15) Restore recent configuration
7. Ping host 16) Restart PHP-FPM
8. Shell

Enter an option: