23.01 Upgrade failed - dead in the water
-
@jimp it is happening on proxmox ve and I've tested it a couple of times, the vm is working on 2.6 common then upgraded to 22.01 and as soon as I upgrade to 23.01 and reboot I'm getting the kernel panic, so it should be update related.
-
@djwopasadjlk said in 23.01 Upgrade failed - dead in the water:
@jimp it is happening on proxmox ve and I've tested it a couple of times, the vm is working on 2.6 common then upgraded to 22.01 and as soon as I upgrade to 23.01 and reboot I'm getting the kernel panic, so it should be update related.
Then that's still going to need to be its own thread, it isn't even close to what's happening here.
-
I have cloned the VM and upgraded it after a factory reset. The upgrade went fine. So I am concluding it is a package causing the failure.
I am wondering if it is acme but haven't tested as yet.
-
I have ACME on every VM in my lab and they've all upgraded fine, so it's doubtful that would be contributing to it. If it's a package, it's more likely something heavier like pfBlockerNG, snort, suricata, squid, that sort of thing.
-
@jimp
Found it is squid or OpenVPN exporter.Based on the above SS I would suggest squid.
-
I have an almost identical issue running on a custom built physical hardware.
Upgrade paths:
pfsense CE 2.6 to pfsense+ 22.01 = success
pfsense+ 22.01 to 23.01 = fail
I followed by a clean install of CE 2.6 on an identical spare unit. Restored config, upgraded to +22.01 again, tried upgrading to 23.01 and failed with the same error.
Ended up doing another clean reinstall and restore (the Netgate support guys must have reactivated my activation token three times that night!) and tried upgrading only to 22.05.
This time limited success because the upgrade did not finish completely and left some packages (namely OpenVPN) in a limbo state. The updater on the dashboard kept saying there was a 22.05 update available even though I was already running it.
This luckily got fixed by running:
pfSense-upgrade -dy
To spare myself all the above misery I would have loved to do a clean install of pfsense+ 23.01 but the support guys said there is no memstick package available for whitebox hardware and the only way to get there is via the upgrades. So really, what's the point of migrating to pfsense+ on whitebox and virtual hardware if the upgrade path is broken and there isn't an option to install directly.
-
We're working on an installer for white box hardware and hope to have it ready with 22.05.
That said, there are quite a lot of people who have upgraded successfully. We still don't have information on vital pieces of the portion of the upgrade that failed here resulting the the errors that you and a small number of others have encountered. The posts so far have included the parts before the upgrade reboot and then these errors at the end, but what we need is the output in between where the first fatal error happened. Some errors from PHP are normal during the upgrade (they are unavoidable side effects of changing versions in-place) the errors we need to find in these failures are the errors that make the upgrade process terminate or get stuck in some other way.
Until we know what happened it's difficult to speculate about what will fix it. Unfortunately with only a video console it's very difficult to capture that part of the upgrade output, it would be simple to record it if you have a serial console available (and set as the primary console).
If you try again, I would definitely remove all packages first before performing the upgrade. You can reinstall them afterward.
Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!
Need help fast? Netgate Global Support!
Do not Chat/PM for help!
-
Hi, thanks for replying. I'm not sure I can repeat this on the production firewall anytime soon. I think I'll wait until the pfsense+ 22.05 installer is out. I don't won't to have to go through the re-registration process again.
-
@morgenstern That is the exact same error screen I get on my Dell Optiplex PC. I guess the next time when I try the upgrade again I'll remove ALL the packages first and then give it a go. I did removed most of the packages and got that same screen.
-
Here is mine. This is after I removed ALL packages. Rebooted a couple of times before upgrading to make sure any weird stuffs left behind are removed. Something to do with my openssl cert. It never get past this point.
So for now I've reverted back to the previous version.
-
@darkk Where are you using that openssl cert. That is a different error to the one I posted which was caused by squid. I would be guessing openVPN.
-
Hi,
22.05 to 23.01 FAIL.
- install clean 2.6.0 CE
- Upgrade clean to 22.05 OK
- Upgrade clean to 23.01 OK
- with 23.01 clean, run backup xml. OK.
-
Seems I ran into the same/similar situation here. I get the same error as in this post by Jonb.
Device (whitebox) is already registered for pfSense+
Clean install CE 2.6.0, with restoring the previous configuration, went ok.
Upgrade to 23.01 failed; tried twice (clean install in between).I reverted to CE 2.6.0 and now I'm thinking to remove some or all the packages and try the upgrade once more.
The packages I have are these:
For some reason I suspect pfBlockerNG-devel to be the culprit.
pfBlockerNG, ntopng and Wireguard have an explicit setting to retain their configuration.
Acme and HAproxy though do not have these options. Do you know if their configuration is retained upon re-installation?Cheers!
-
Dead here too. Followed steps. Upon boot 23.01 can't find the boot drive.
Back to square one, install 2.7 Feb release, now having this kind of error message during boot: "pfSense_plus-v23_01_amd64-pfsense_plus_v23_0//packagesite.pkg: Authentication error." Also I can no longer check for updates.
-
I have the same issue. Previously tried to upgrade (a few weeks ago) from a current 2.7 Dev version without uninstalling packages. Had mismatch issues with packages.
To remove the PHP v 8.1 vs 8.2 potential problem yesterday I reinstalled the 230215 build of 2.7. Then was able to see the upgrade choice to 23.01. Ran the upgrade which seemed to go fine until it rebooted and stopped at the same point as shown in the above posts. It stops at a login: prompt and won't go any further.
Reinstalled the current 2.7 Dev version from yesterday and tried the upgrade again. Same result.
Finally, reinstalled the current 2.7 Dev version while waiting for further progress.
Is there anyway to capture the complete startup log to post here?
-
@patryan Right now the check for the 23.01 update fails, can’t check. Makes me think something is being addressed. I too am back at 2.7. Had to remember the lingo for installing the Realtek 1.98 driver, some other odds and ends but I’m up and running on 2.7.
-
Here is a screen shot where things stop during the upgrade. If I login and reboot then the boot drive is not recognized and I have to reinstall.
-
@patryan Same here....
-
I'm getting the same error.
I'm running on VMware ESXI 8.0U1.
I have tried 3 times to upgrade to pfSense Plus from CE 2.6.0 brand new install and it fails each time.
Why is it that we don't have a direct upgrade to pfSense Plus? Clearly the upgrade path doesn't work from CE 2.6.0.
On another note, I tried upgrading from CE 2.7.0 and the same result as well.
And lastly, I have noticed I get a SCSI error if there is a VMware snapshot on the machine during a boot of CE 2.6.0. If I remove the VMware snapshot and then reboot, no error. (This is separate from the certs.inc fatal error though and doesn't seem to affect it.)
Can anyone figure out what's causing this problem? Even NetGate's own hardware is failing with this error.
