Reboot loop

zonenoire

My SG-3100 pfSense Security Gateway Appliance is now in a reboot loop:

__

_ __ / |__ ___ _ __ ___ ___
| '_ | |/ _|/ _ \ ' / __|/ _
| |) | _ \ / | | _ \ /
| ./|| |/_|| ||/_|
|_|

Welcome to pfSense 2.4.4-RELEASE...

...ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/ipsec /usr/local/lib/mysql /usr/local/lib/perl5/5.26/mach/CORE
Soft Float compatibility ldconfig path:
done.
panic: ffs_write: type 0xc6671000 0 (0,12017)
cpuid = 0
Uptime: 12s
Automatic reboot in 15 seconds - press a key on the console to abort

stephenw10

The filesystem has an error that cannot be repaired on a single fsck pass. Do this

Running a file system consistency check on the SG-3100:

At the SG-3100 console press any key to interrupt the boot loader when you see:
Hit [Enter] to boot immediately, or any other key for command prompt.

At the prompt enter: boot -s

That will boot in single user mode to a question asking for a path to the shell, just press return to reach the # prompt.
At the # prompt run the following command: fsck -y /

Run the fsck command at least 3 times; Repeat the command until no errors are reported, even if fsck claims the filesystem has been marked "clean".
Reboot by running: reboot

It should boot normally and with no residual long-term issues

Steve

zonenoire

Hi Stephen:

I can't get the the # prompt. I can, however get to the Marvell>> and loader> prompts as follows:

| / | __ _ _ ____ | | |
| |/| |/ _` | '\ \ / / _ \ | |
| | | | (| | | \ V / __/ | |
|| ||_,|| _/ _|||
_ _ ____ _
| | | | | __ ) ___ ___ | |
| | | || _ \ / _ \ / _ | __|
| || || |) | () | () | |_
_/ |____/ _/ ___/ __|
** LOADER **

U-Boot 2013.01-02873-g5e4ac3c (Aug 11 2017 - 13:12:40) Marvell version: devel-17.06.0

Board: Rogue-1
SoC: MV88F6820 Rev A0
running 2 CPUs
CPU: ARM Cortex A9 MPCore (Rev 1) LE
CPU 0
CPU @ 1600 [MHz]
L2 @ 800 [MHz]
TClock @ 250 [MHz]
DDR4 @ 800 [MHz]
DDR4 32 Bit Width,FastPath Memory Access, DLB Enabled, ECC Disabled
DRAM: 2 GiB
MMC: mv_sdh: 0
DBG: Calling spi_flash_probe from env_relocate_spec()
SF: Probing bus 0 cs 0 @ 20000000Hz mode 3
SF: Detected N25Q128 with page size 64 KiB, total 16 MiB
PCI-e 0: Detected No Link.
PCI-e 1: Detected No Link.
USB2.0 0: Host Mode
USB3.0 1: Host Mode

Map: Code: 0x7fedc000:0x7ff975a8
BSS: 0x7ffef600
Stack: 0x7f4cbf20
Heap: 0x7f4cc000:0x7fedc000
U-Boot Environment: 0x00100000:0x00110000 (SPI)

Board configuration detected:
Net:

port	Interface	PHY address
egiga0	RGMII	0x00
egiga1	RGMII	0x01
egiga2	SGMII	In-Band
egiga0 [PRIME], egiga1, egiga2
Hit any key to stop autoboot: 0
Marvell>>
Marvell>> bootd

Reset SCSI
AHCI init for unit0
SATA link 0 timeout.
Target spinup took 0 ms.

Error: SCSI Controller(s) 1B4B:9215 1B4B:9235 not found
scanning bus for devices...
Device 0: (1:0) Vendor: ATA Prod.: NT-32 Rev: Q092
Type: Hard Disk
Capacity: 30029.2 MB = 29.3 GB (61500000 x 512)
Found 1 device(s).
reading ubldr.bin
228648 bytes read in 11 ms (19.8 MiB/s)

Starting application at 0x00200000 ...

Consoles: U-Boot console
Compatible U-Boot API signature found @0x7f4dc280

FreeBSD/armv6 U-Boot loader, Revision 1.2
(Wed Sep 5 09:32:58 EDT 2018 root@buildbot3)

DRAM: 2048MB
Number of U-Boot devices: 4
U-Boot env: loaderdev not set, will probe all devices.
Found U-Boot device: disk
Probing all disk devices...
Checking unit=0 slice=<auto> partition=<auto>...disk0: read failed, error=1

Checking unit=1 slice=<auto> partition=<auto>... good.
Booting from disk1s2a:
Loading /boot/defaults/loader.conf
console comconsole is invalid!
no valid consoles!
Available consoles:
uboot
/boot/kernel/kernel data=0x11788d8+0x77728 syms=[0x4+0x97d60+0x4+0xd850d]
/
Hit [Enter] to boot immediately, or any other key for command prompt.

Type '?' for a list of commands, 'help' for more detailed help.
loader> ?
Available commands:
boot boot a file or loaded kernel
autoboot boot automatically after a delay
help detailed help
? list commands
show show variable(s)
set set a variable
unset unset a variable
echo echo arguments
read read input from the terminal
more show contents of a file
lsdev list all devices
include read commands from a file
ls list files
load load a kernel or module
unload unload all modules
lsmod list loaded modules
heap show heap usage
reboot reboot the system
devinfo show U-Boot devices
sysinfo show U-Boot system info
ubenv show or import U-Boot env vars
fdt flattened device tree handling
boot-conf load kernel and modules, then autoboot
read-conf read a configuration file
enable-module enable loading of a module
disable-module disable loading of a module
toggle-module toggle loading of a module
show-module show module load data
try-include try to load/interpret files
loader>

stephenw10

@zonenoire said in Reboot loop:

Hit [Enter] to boot immediately, or any other key for command prompt.

At the loader> prompt there enter boot -s.

Then you will reach the shell question and the # prompt.

Steve

zonenoire

Thanks Stephen.
SUCCESS!
Here is a synopsis of what I did in case someone else encounters this problem:

Marvell>>boot
...
loader>boot -s
...
Trying to mount root from ufs:/dev/diskid/DISK-CEF032182700058s2a [rw,noatime]...
WARNING: / was not properly dismounted
Enter full pathname of shell or RETURN for /bin/sh:

fsck -y

...

fsck -y

...

fsck -y

...
***** FILE SYSTEM MARKED CLEAN *****
#reboot
...

Netgate SG-3100 - Serial: 1130180696 - Netgate Device ID: bdfbc7c1c133fb11d042

*** Welcome to pfSense 2.4.4-RELEASE (arm) on pfSense ***

WAN (wan) -> mvneta2 -> v4/DHCP4: 192.168.1.9/24
LAN (lan) -> mvneta1 -> v4: 192.168.3.1/31
OPT1 (opt1) -> mvneta0 -> v4: 192.168.100.1/32

Logout (SSH only) 9) pfTop
Assign Interfaces 10) Filter Logs
Set interface(s) IP address 11) Restart webConfigurator
Reset webConfigurator password 12) PHP shell + pfSense tools
Reset to factory defaults 13) Update from console
Reboot system 14) Enable Secure Shell (sshd)
Halt system 15) Restore recent configuration
Ping host 16) Restart PHP-FPM
Shell

Enter an option: