23.01 Upgrade failed - dead in the water
-
@jonb said in 23.01 Upgrade failed - dead in the water:
I have no IP access to the machine
Yep. pfSEnse says the same thing.
It can't go out, looking for updates etc.Redo your VM setup : assign NICs etc.
When pfSense boots, and it find NICs, you'll be able to assign them, like 'em0' == WAN, em1 is LAN etc.
And yes, the gettext error means something is really bad, as the gettext function is part of PHP.
-
@gertjan
I can't assign any nics through the usual menu. After upgrading it drops me to shell. -
@jimp
That output was after doing an update throught the web GUI. I though the same thing about the issue with the logs showing the old version.When the system reboots it just presents me the shell and to login. The last SS was running pfSense-upgrade when it was broken.
I am trying to post the rest of the log but the forum has a limit. So here the file.upgrade log.txt
-
@jonb If
/conf/upgrade_log.latest.txtis there it may also have more information. So far nothing in what you've posted looks out of the ordinary. -
@jimp said in 23.01 Upgrade failed - dead in the water:
/conf/upgrade_log.latest.txt
The second SS is /conf/upgrade_log.latest.txt
-
I have just tried an update through console instead of GUI. After reboot I get:
Anything else I can provide to help find the issue?
-
Whatever happened would be up much higher in the boot output than that. I'm not sure if it works on your version of vmware, but try pressing the Scroll Lock key and then Page Up as far as you can and see if you can see the start of the errors.
So far the
pfSense-upgradeoutput you've showed looked OK but incomplete, and nothing after that has been anything but errors that aren't suggesting any particular root cause. -
after upgrading to 23.01 and rebooting, good that I have cronjob saving my configs once a day -
@djwopasadjlk said in 23.01 Upgrade failed - dead in the water:
after upgrading to 23.01 and rebooting, good that I have cronjob saving my configs once a dayThat's unrelated to anything else going on in this thread. That looks like an issue some people have seen with Hyper-V failing to boot. There are already threads for that around.
-
@jimp it is happening on proxmox ve and I've tested it a couple of times, the vm is working on 2.6 common then upgraded to 22.01 and as soon as I upgrade to 23.01 and reboot I'm getting the kernel panic, so it should be update related.
-
@djwopasadjlk said in 23.01 Upgrade failed - dead in the water:
@jimp it is happening on proxmox ve and I've tested it a couple of times, the vm is working on 2.6 common then upgraded to 22.01 and as soon as I upgrade to 23.01 and reboot I'm getting the kernel panic, so it should be update related.
Then that's still going to need to be its own thread, it isn't even close to what's happening here.
-
I have cloned the VM and upgraded it after a factory reset. The upgrade went fine. So I am concluding it is a package causing the failure.
I am wondering if it is acme but haven't tested as yet.
-
I have ACME on every VM in my lab and they've all upgraded fine, so it's doubtful that would be contributing to it. If it's a package, it's more likely something heavier like pfBlockerNG, snort, suricata, squid, that sort of thing.
-
@jimp
Found it is squid or OpenVPN exporter.Based on the above SS I would suggest squid.
-
I have an almost identical issue running on a custom built physical hardware.
Upgrade paths:
pfsense CE 2.6 to pfsense+ 22.01 = success
pfsense+ 22.01 to 23.01 = fail
I followed by a clean install of CE 2.6 on an identical spare unit. Restored config, upgraded to +22.01 again, tried upgrading to 23.01 and failed with the same error.
Ended up doing another clean reinstall and restore (the Netgate support guys must have reactivated my activation token three times that night!) and tried upgrading only to 22.05.
This time limited success because the upgrade did not finish completely and left some packages (namely OpenVPN) in a limbo state. The updater on the dashboard kept saying there was a 22.05 update available even though I was already running it.
This luckily got fixed by running:
pfSense-upgrade -dy
To spare myself all the above misery I would have loved to do a clean install of pfsense+ 23.01 but the support guys said there is no memstick package available for whitebox hardware and the only way to get there is via the upgrades. So really, what's the point of migrating to pfsense+ on whitebox and virtual hardware if the upgrade path is broken and there isn't an option to install directly.
-
We're working on an installer for white box hardware and hope to have it ready with 22.05.
That said, there are quite a lot of people who have upgraded successfully. We still don't have information on vital pieces of the portion of the upgrade that failed here resulting the the errors that you and a small number of others have encountered. The posts so far have included the parts before the upgrade reboot and then these errors at the end, but what we need is the output in between where the first fatal error happened. Some errors from PHP are normal during the upgrade (they are unavoidable side effects of changing versions in-place) the errors we need to find in these failures are the errors that make the upgrade process terminate or get stuck in some other way.
Until we know what happened it's difficult to speculate about what will fix it. Unfortunately with only a video console it's very difficult to capture that part of the upgrade output, it would be simple to record it if you have a serial console available (and set as the primary console).
If you try again, I would definitely remove all packages first before performing the upgrade. You can reinstall them afterward.
Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!
Need help fast? Netgate Global Support!
Do not Chat/PM for help!
-
Hi, thanks for replying. I'm not sure I can repeat this on the production firewall anytime soon. I think I'll wait until the pfsense+ 22.05 installer is out. I don't won't to have to go through the re-registration process again.
-
@morgenstern That is the exact same error screen I get on my Dell Optiplex PC. I guess the next time when I try the upgrade again I'll remove ALL the packages first and then give it a go. I did removed most of the packages and got that same screen.
-
Here is mine. This is after I removed ALL packages. Rebooted a couple of times before upgrading to make sure any weird stuffs left behind are removed. Something to do with my openssl cert. It never get past this point.
So for now I've reverted back to the previous version.
-
@darkk Where are you using that openssl cert. That is a different error to the one I posted which was caused by squid. I would be guessing openVPN.
