Upgrading from 25.11.1 to 26.03 failed

kibade

I have a test router I use before doing anything to my production routers that is model 1100. I have 5 x 2100 production systems I need to upgrade from 25.11.1 to 26.03 once I see this upgrade working on the test router.

My test 1100 during the upgrade fails early at this point:
[ 24/178] Upgrading boost-libs from 1.88.0_2 to 1.89.0_2...
Failed

It has the following free space :
/ 1.2G 3.4G
34% of 3.4G (zfs)

In the general logs I see this error at the point it fails:
php 20268 Standard input code: New alert found: pkg: "Upgrading packages in cloned boot environment default" returned error code 137

I'm not clear what that error means, or how to fix this.
Can you help, please?
Thanks!

Mission-Ghost

Similar to my experience. It may have run out of memory.

Remove or at lease shut off optional packages especially hogs like pfblocker. Then retry. Doing so from the console menu while not logged on to the gui can save some more memory.

The 2100s have more memory so not all of this may be necessary on those. The 1100 is under-resourced for all but the most basic pfsense.

SteveITS

@kibade related: https://forum.netgate.com/post/1240763

kibade

@Mission-Ghost

I uninstalled:
Zabbix agent 6
nrpe (for nagios).

The following are still installed:
acme
aws-wizard
ipsec-profile-wizard
Nexus
openvpn-client-export

I checked before running the upgrade and the memory was using 21% of 965 MiB
It failed at the exact same point via the GUI. It will be a few hours before I can try using the console method. Fingers crossed that works.
Thanks

Mission-Ghost

@kibade

I suggest you remove all of the packages except Nexus, which is required for pfSense to work properly.

I had about 19% memory usage when update failed. On the 1100 you've got to strip it down to the frame to update successfully.

When it's done, you can reinstall all your packages and they should retain their settings and data from prior to deinstallation.

kibade

@Mission-Ghost
OK, I'll give that a try.
Shame I need to do this to get the updates to work.

Mission-Ghost

@kibade said in Upgrading from 25.11.1 to 26.03 failed:

Shame I need to do this to get the updates to work.

Trade off, I think, for such a cheap box. A friend's 2100 updated ok, except he had to be sure to remove all the inactive zfs snapshots so the box would have sufficient storage.

These low-end ARM boxes are too resource-constrained.

kibade

@Mission-Ghost
I just uninstalled everything except : Nexus
Mem usage has gone up, but I hope a reboot will fix.
28% of 965 MiB

I'll let you know how it goes.

kibade

@Mission-Ghost

Its all working now.

I rebooted and tried again via the gui but it failed at the same point.
I then connected the console cable and upgraded (Option 13) and it worked, then I noticed that I was still logged in via my browser and only got kicked off when there was a reboot. So I don't think it was memory issue because I was using the GUI.
Summary:
Only upgrade on a 1100 to V26.03 via the console.

I'll make sure I'm onsite when I try the upgrade on my first 2100, just to be sure that doesn't have the same issues. That's why I test first to see what might happen. I wish I had a 2100 to test instead of just 1100. It's on my wish list.

Thanks again for your advice. :-)

Mission-Ghost

@kibade You're welcome. I'm happy I could help. This forum has really helped me a ton and I'm glad I can pay it forward.

That said, I'm sure glad I'm not responsible for a large organization's network anymore. It was rewarding work, but stressful.