Brand new SG2440, 503 error on web gui

mer

Got a brand new SG2440 Friday, finally had a chance to try it out today. Was able to connect to the web gui once, now getting continual 503 service unavailable errors. Hooked up serial console, get the standard menu, option 16 to restart php-fpm, errors on "no group wheel". Dr Google offers some advice, go to look at /etc/group and /etc/passwd, /etc/master.passwd and intead of text files, they are some text and some binary. file /etc/passwd returns "data". /etc/master.passwd looks to be partially crontab, a couple lines of /etc/group. Other files in /etc look odd, crontab looks like mostly /etc/group, master.passwd has crontab plus binary data.
Any ideas other than download pfsense to memstick and reinstall?

thanks
m

doktornotor

@mer:

Any ideas other than download pfsense to memstick and reinstall?

No, of course NOT. Completely pointless to mess with a box that has core files damaged due to filesystem corruption.

mer

Ok, that's what I was thinking too. Thanks.
More fun.

mer

Poking at the SG a bit more, it looks like passwd, groups, crontab, master.passwd (probably spwd.db and pwd.db) are corrupted, chunks of NULL characters in them, plus contents that shouldn't be there. Remainder of files looks fine (all same date at least in ls -ltr). I grabbed the live cd ISO, pulled corresponding files off and am going to see what happens by copying them over and doing a pwd_mkdb.

Update:
Well, that seems to have worked. Looks like it's all functioning again.

Well, that seems to have worked. Looks like it's all functioning again.

And can we also know what does the trick?

mer

@BlueKobold:

Well, that seems to have worked. Looks like it's all functioning again.

And can we also know what does the trick?

Sorry, of course.
Full disclosure: after some email with customer support at pfSense, I probably caused the original problem; pulled power without doing a shutdown first, causing fsck to get overly aggressive on reboot and making a mess of things. Lesson learned, do a full backup of the device before mucking with it.

What I did:
Grabbed the LiveCD ISO for amd64 from the pfSense support site, stuck it on my FreeBSD desktop. Unzipped, did the following:
mdconfig -f <diskimage.iso>-u 0
mount -t cd9660 /dev/md0 /mnt
find etc in the /mnt tree
copy passwd, master.passwd, group and crontab from the /mnt to a flash drive
umount /mnt

Boot the SG2440 with the serial console attached (I used a putty session), stick in the flash drive, then do:
mount -t msdosfs /dev/da1s1 /mnt (da1 was my device)
copy passwd, master.passwd, group and crontab from flash to SG2440 /etc
cd /etc
pwd_mkdb -p /etc/master.passwd
umount /flash
I then did passwd admin just to make sure I knew what the password was, rebooted and all was happy again.

The trick was finding install versions of passwd, master.passwd, group and crontab, LiveCD ISO was best shot at that, then it was digging back in my brain about mounting the ISO and pulling the files off. If you can mount an ISO file and get to the files on it, this should work just about anywhere.</diskimage.iso>