23.05.1 -> 23.09 - PKG is broken - SG5100

jimp

Those fsck errors are worrisome but there isn't enough detail to say if it's from an unclean shutdown/crash type FS issue or hardware. It's hard to tell on the mmc since it doesn't report data like SSDs do with SMART.

Was that fsck run from single user mode? It didn't appear to fix anything even though you used -y.

You might be able to use the reboot menu option to have it automatically run the fsck for you during boot.

Dyspareunia

@jimp said in 23.05.1 -> 23.09 - PKG is broken - SG5100:

Was that fsck run from single user mode? It didn't appear to fix anything even though you used -y.

No, I believe that requires I reboot into this mode using a console connection.

I WFH and need to keep things up for now, I'll look at this as soon as I can.

jimp

Before you do that you might want to get ahold of TAC and make sure you have installer media for 23.09 in case the filesystem damage is too severe to be recovered. There is a chance running fsck may make it worse in some ways even though it is repairing damage.

Hopefully it's just filesystem corruption, though.

Honestly you may be better off reinstalling 23.09 clean and going to ZFS as it is a lot more robust against these sorts of problems.

Dyspareunia

@jimp said in 23.05.1 -> 23.09 - PKG is broken - SG5100:

Honestly you may be better off reinstalling 23.09 clean and going to ZFS as it is a lot more robust against these sorts of problems.

Yeah way ahead of you, I contacted TAC yesterday and got the 23.09 image. I've been wanting to switch to ZFS for a while now as well as upgrade the storage on the device. Seems like now may be the time.

stephenw10

You should check the eMMC write life of the drive:
https://docs.netgate.com/pfsense/en/latest/troubleshooting/disk-lifetime.html#emmc

On the 5100 if the eMMC fails entirely it can cause problems booting at all. So if it's close to the write-life limit you should fit an SSD as soon as possible.

Steve

Dyspareunia

@stephenw10 said in 23.05.1 -> 23.09 - PKG is broken - SG5100:

On the 5100 if the eMMC fails entirely it can cause problems booting at all. So if it's close to the write-life limit you should fit an SSD as soon as possible.

Thanks for this, I took a look and I'm now properly terrified.

# mmc extcsd read /dev/mmcsd0rpmb | egrep 'LIFE|EOL'

eMMC Life Time Estimation A [EXT_CSD_DEVICE_LIFE_TIME_EST_TYP_A]: 0x0b
eMMC Life Time Estimation B [EXT_CSD_DEVICE_LIFE_TIME_EST_TYP_B]: 0x0b
eMMC Pre EOL information [EXT_CSD_PRE_EOL_INFO]: 0x01

According to the link you gave me:

0x0b
The disk has used 100%-110% of its estimated life time.

I've ordered a new SSD on Amazon, will arrive in the next few days.

stephenw10

If you don't have a bunch of heavy packages installed I'd suggest enabling ram disks to prevent any significant further writes before the SSD is installed. That does require a reboot.

Dyspareunia

@stephenw10 said in 23.05.1 -> 23.09 - PKG is broken - SG5100:

If you don't have a bunch of heavy packages installed I'd suggest enabling ram disks to prevent any significant further writes before the SSD is installed. That does require a reboot.

Thanks, done!

rloeb

@stephenw10 I am experiencing the same problems, but it's a near new 4100 running ZFS. Tried installing 23.09 from memstick (multiple times). Each time I assign the WAN and LAN ip addresses, but the device doesn't appear on the LAN an is not reachable.

stephenw10

You're seeing ssl library or pkg issues after a clean re-install?