Update to 25.11 failed on both 4100, one is kind of bricked, help needed :)

nicpic19

I updated pfSense on both my Netgate 4100 and it went really bad. It got stuck while upgrading but not at the same step.

One finally rebooted successfully with the old version (25.07) but one is kind of bricked (solid blue LED on the left, other LEDs off, no web interface access). I can still connect using the console cable and SSH but I have the following message:

/etc/rc.initial: No such file or directory.

Any help appreciated.

SteveITS

@nicpic19

stuck while upgrading

Can you be more specific?

Solid blue is just power on:
https://docs.netgate.com/pfsense/en/latest/solutions/netgate-4100/io-ports.html#led-patterns

nicpic19

@SteveITS Hi Steve and thank you for your quick answer.

Well when I plug the power, the left LED goes solid amber then solid blue but the router never gets to the "standard all 3 LEDs blinking blue" booting step, there is not much more information I could give to you unfortunately as I have very limited access, you could even say no access at all.

SteveITS

@nicpic19 I meant, on getting stuck. Did you wait more than say 15 minutes?

nicpic19

Yep it is like that for more than one week !

For now I'm using my ISP box for Internet access, the routeur is not connected to the WAN anymore.

Gertjan

@nicpic19 said in Update to 25.11 failed on both 4100, one is kind of bricked, help needed :):

I can still connect using the console cable and SSH

Console cable : when powering up, there is no 'SSH', because no OS, no 'file system', the '4100' doesn't even know i has NICs, a drive or anything. It just capable of testing and detecting stuff like : a serial interface, you saw the output on your screen, and device known as disks.
The console at that moment, is the BIOS talking to you.
If that works, there are big hopes.

When you say : "and SSH", the that means you use a NIC, the LAN IP probably to talk to a process called SSH, so the OS = pfSense, has booted an is running. That's miles deeper in the startup process.

If the console access works, you can do destructive or not, drive tests. If you don't have the MAX 4100, but the 'normal' one, do some search (research) here on the forum. You could add a SSD yourself, install pfSense on that drive, and you'll be good. (edit : IF the drive is bad)
(but please, don't de-solder the nv disk with a flame thrower)

Btw : for me : more then one LED lit up, and blinking : that means the processor is running something, so the system works.

I've a 4100 MAX myself.

SteveITS

@nicpic19 So if you started the upgrade, did nothing to it, and a week later it’s at that error then it failed somehow. You’d need to post the upgrade log but realistically you could just reinstall to recover, and restore from backup.
https://docs.netgate.com/pfsense/en/latest/solutions/netgate-4100/reinstall-pfsense.html

nicpic19

@Gertjan Thank you for helping me Gertjan.

SSH was an abuse of language from me, actually I connected to the router using Putty and the console cable with serial interface, no SSH involved. But the router asked for credentials and they worked.

I have a standard 4100.

My hope actually would be to push a 25.07 image to the router, I'm pretty sure that's the only thing I can do to have it back to life.

@SteveITS Actually I asked for a reboot while I still had access to the web console and then the shit happened lol. I had no other option anyway but I'm aware I may have screwed up.

SteveITS

@nicpic19 most likely then it just wasn’t done yet. A reinstall is probably the safest way to resolve it.

patient0

@nicpic19 said in Update to 25.11 failed on both 4100, one is kind of bricked, help needed :):

My hope actually would be to push a 25.07 image to the router, I'm pretty sure that's the only thing I can do to have it back to life.

If the 4100 is using ZFS as its filesystem you can revert to 25.07/another snapshot at the boot prompt (option 8). On every upgrade pfSense+ creates a snapshot.

https://docs.netgate.com/pfsense/en/latest/backup/zfsbe/loader.html

nicpic19

Hi guys

I have very good news this morning, the router is back to life! I reinstalled pfSense from the USB key with the procedure SteveITS sent to me, it went fine (reinstalled 25.07). I had a configuratioin backup so almost everything went back with the restore.

For now I noticed that I lost DHCP reservations, custom DNS entries and the pfBlocker package had to be reinstalled but all of the interfaces assignements, the firewall rules and the site-to-site VPN tunnel were up and running.

So big thanks to all of you for all your precious tips, it helped me a lot to do the correct thing and not screw up more. :)