Pfsense Can't boot after power failure - kernel panic

triangleman

Hi guys,
PFSense got scrambled by two quick power failures… and now it boots to a kernel panic.
I can't even figure out how to run FSCK.

It boots until it reads:

WARNING: / was not properly dismounted.
***** FILE SYSTEM MARKED CLEAN *****

The "Mounting Filesystems…" PFSense ASCII graphic loads.

savecore: reboot after panic: ufs_dirbad: /: bad dir ino 722308 at offset 512: m
angled entry
savecore: writing core to /var/crash/textdump.tar.0
creating symlinks... panic: ufs_dirbad: /: bad dir ino 722308 at offset 512: m
led entry
cpuid = 0
KDB: enter: panic

A long trace then begins and system reboots.

I get the "Can't find kernel" error if i try to use option 5 to enter the kernel or kernel.old.

Can anyone help me?

Harvy66

Probably easier to re-install and restore your config from backup. You backup, right? And using ZFS would probably help prevent this in the future, at the cost of more memory.

triangleman

@Harvy66:

Probably easier to re-install and restore your config from backup. You backup, right? And using ZFS would probably help prevent this in the future, at the cost of more memory.

I'm all-for preventing this from happening again and will look into ZFS,
but re-installing after a mere power failure seems a little much.

Surely there's a way to use the 'recovery mode' from the CD to run FSCK on the disk right?
I looked into it, FSCK does run, but i don't know how use it for this.

Derelict

Single user mode

Press RETURN for /bin/sh

/sbin/fsck -y /

Run that repeatedly until it comes back completely clean

/sbin/reboot

triangleman

thanks Derelict,

~~Now i'm getting this error:
(This is from booting off the installer CD in single user mode in shell)

~~/sbin/fsck -y /

fsck: exec fsck_cd9660 for /dev/iso9660/PFSENSE in /sbin:/usr/sbin: No such file or directory~~

I'm not too familiar with BSD, but i imagine you might have meant i need to mount the filesystem first and run this on that directory?~~

I just got it working from the normal install, will try this until it fixes. thanks again

Harvy66

@triangleman:

@Harvy66:

Probably easier to re-install and restore your config from backup. You backup, right? And using ZFS would probably help prevent this in the future, at the cost of more memory.

I'm all-for preventing this from happening again and will look into ZFS,
but re-installing after a mere power failure seems a little much.

Surely there's a way to use the 'recovery mode' from the CD to run FSCK on the disk right?
I looked into it, FSCK does run, but i don't know how use it for this.

I'm not familiar with UFS. Most modern filesystems don't trash committed data, only data or metadata changes that are in-flight run the risk of corruption. My pfSense box has experienced 3 unexpected power failures in the many years. Never had an issue, but could just be lucky or you could be unlucky or something pathological.