23.01 -> 23.05 upgrade failed

[pdavis]

I saw this morning that version 23.05 was available, so I started the upgrade process. The upgrade failed with a server message, and now when I look at the GUI it says it is up to date on the latest version with 23.01.

Was 23.05 released, and then pulled back?

[jrey]

@pdavis

Not sure if same issue, however my 2100 amber/orange light blinking (update available) - GUI shows Current Stable Version (23.05) is available.

Being brave just starting the first coffee for the morning, hit the confirm update.

Several certificate verification errors displayed, output ending in

ERROR: It was not possible to determine pfSense-upgrade remote version
Upgrading pfSense-upgrade... failed.

Nothing downloaded or changed, but the System Update GUI now showed
in the drop down Current Stable Version (23.05) and below that
Current Base 23.01
Latest Base 23.01
with no update confirm button.

From the drop down I selected Previous Stable Version and let the screen refresh, then reselected Current Stable (23.05) - Confirm button reappeared.

this clearly requires more coffee. I may try the upgrade again later or just hold off for another day. The flashing amber light is both annoying and taunting however.

[s0m3f00l]

My sg-3100 still says there is an upgrade to 23.05 available. Looks like I won't be pulling the trigger

[jordanp123]

Im in the same boat here, two virtualized instances failed on upgrade with certificate errors (both still routing fine at the moment). Watching this thread to see if we can get a solutions.

[Gertjan]

@s0m3f00l said in 23.01 -> 23.05 upgrade failed:

Looks like I won't be pulling the trigger

Definitely a smart thing to do.

Wait for a Blog post to show up here first.
Then you have to read the Release notes ...
Then - my personnel advice - you check the forum regularly, and look for forum feedback.

Right now : don't update any pfSense packages (anymore).

And - my personnel advice - don't forgot to do a pfSense reboot edit : before upgrading, and follow this reboot from the console.

[jordanp123]

@gertjan
After refreshing the package list, and logging back in the update worked...weird.

[jrey]

For what its worth, the update may just not be available yet, even though the devices think it is.

There was this thread from back on Feb 19 talking about cert errors.

https://forum.netgate.com/topic/178063/can-t-upgrade-certificate-verification-failed?_=1684837002821

of course in my case that thread was a couple of days after I had successfully upgraded from 22.05 to 23.01 on the 16th

Even though the devices are saying there is an update, I haven't really seen an announcement (at least not yet). Guessing it could just timing at this point. There is always another coffee tomorrow morning. No rush.

[mcury]

@jrey said in 23.01 -> 23.05 upgrade failed:

Even though the devices are saying there is an update, I haven't really seen an announcement (

I got the firmware from Netgate TAC, it is available already, just open a ticket there requesting it.
I got it for the SG-4100, so I suppose the only thing missing is the official announcement..

[jrey]

@mcury

On the other hand, no real rush. There is actually enough on the to do plate for today anyway. So this upgrade can wait.

After the official announcement is made, then, and if the certificate errors still persist, can reach out to TAC then. For me, no point swamping them with requests at this point.

[mcury]

@jrey said in 23.01 -> 23.05 upgrade failed:

@mcury

On the other hand, no real rush. There is actually enough on the to do plate for today anyway. So this upgrade can wait.

After the official announcement is made, then, and if the certificate errors still persist, can reach out to TAC then. For me, no point swamping them with requests at this point.

They took 5 minutes to provide me the firmware link..
I usually don't upgrade without it, things can always go south and I like to have a firmware and backup config in hands.
You can just wait, I don't see a problem there.

[s0m3f00l]

@mcury This is a great idea that I never even considered. Thanks!

[demux]

This is the result of 23.01 -> 23.05:

Updating repositories metadata...
Updating pfSense-core repository catalogue...
Certificate verification failed for /C=US/ST=Texas/L=Austin/O=Rubicon Communications, LLC (Netgate)/OU=pfSense Plus/CN=pfsense-plus-pkg01.atx.netgate.com
34942771200:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-Plus-snapshots-23_05-main/sources/FreeBSD-src-plus-RELENG_23_05/crypto/openssl/ssl/statem/statem_clnt.c:1921:
Certificate verification failed for /C=US/ST=Texas/L=Austin/O=Rubicon Communications, LLC (Netgate)/OU=pfSense Plus/CN=pfsense-plus-pkg01.atx.netgate.com
34942771200:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-Plus-snapshots-23_05-main/sources/FreeBSD-src-plus-RELENG_23_05/crypto/openssl/ssl/statem/statem_clnt.c:1921:
...
Certificate verification failed for /C=US/ST=Texas/L=Austin/O=Rubicon Communications, LLC (Netgate)/OU=pfSense Plus/CN=pfsense-plus-pkg00.atx.netgate.com
34942771200:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-Plus-snapshots-23_05-main/sources/FreeBSD-src-plus-RELENG_23_05/crypto/openssl/ssl/statem/statem_clnt.c:1921:
pkg-static: https://pfsense-plus-pkg00.atx.netgate.com/pfSense_plus-v23_05_amd64-pfSense_plus_v23_05/packagesite.txz: Authentication error
Unable to update repository pfSense
Error updating repositories!
ERROR: It was not possible to determine pfSense-upgrade remote version
ERROR: It was not possible to determine pfSense-upgrade remote version
Upgrading pfSense-upgrade... failed.

And then it says that 23.01 is latest version.

[Raffi_]

I had the same issue with upgrade error through GUI and then the GUI showing I already have the latest install. I was able to solve this by SSHing into console and using "pfsense-upgrade".

Make sure you backup your current setup first.
Do not upgrade packages.
If you upgraded packages before performing the pfsense upgrade, you may have to do a completely fresh install. Been burned by that once and it was enough to learn the lesson. Thankfully, I had backups and was back up and running pretty quick!

[jrey]

@demux

and then if you set it back to previous (23.01) and let it sit a moment, then change it back to 23.05 it will offer the update again. I haven't retried it at this point, as other more important things on the table today.

However, the certificate issue, doesn't seem like a "me" problem, and hopefully next time a try the update, it will have been resolved by the official release.

In the meantime I did reach out to TAC to get the firmware download. So that's an entirely different set of issues for another day if needed. Like finding a console to hook up etc.

Better would be if the update just worked. So far nothing lost nothing changed running 23.01 as if nothing happened, and the update still showing.

[demux]

Thanks a lot.
I will wait until someone from Netgate says "fixed".
I don't want want to set it up again. It has i226 ports and it was a small horror to bring it to 23.01. I don't want to mess that.

[mark_lab_user]

Got the upgrade failed on first try. Then went back to Previous Stable Version 23.01 on the System/Update/System Update Gui page. and then rebooted.

Then 2nd try succeeded. Don't know why.

But now my Wireguard Tunnel and Peer to a privacy VPN is gone. The package, interface and WAN and LAN firewall rules exist for the VPN though.

I restored the XML backup file which does contain the correct wireguard keys and endpoint address so I don't know what to do next.

[demux]

@mark_lab_user I saw there is an update to the wireguard package. May it's for 23.01 only.

[jrey]

@demux

Just FYI, I completed the update this morning. No issues. So the certificate issue yesterday was simply the device was ready to update, the servers where not. -- So all good here.

[demux]

I normally update packages when I see that they are available. I read multiple times that you should not update packages before system update to a new release. Why? What can happen? I try to have an up-to-date system all the time, does that mean that I should/can never update the system?

[Gertjan]

@demux said in 23.01 -> 23.05 upgrade failed:

Why?

Example :

A pfSense GUI package uses PHP. PHP, among version, will be nearly identical, but new functionality can get added, old functionality can have been removed.
PHP is an interpreted language, so what if the updated package uses new functionality, available only in the the newer PHP version ?
The newly available pfSense package will 'depend' on the new PHP version, not installed on pfSense. So, it will also get pull in (== installed) this new PHP version ... overwriting the older, exiting PHP version.
Now you just broke the entire GUI ...

Another example :

Software is written with an OS version in mind.
Most system functions like 'open a file' and 'close a file' is functionality exposed by the system as system libraries.
Newer software needs newer library version.
So, when the package, not only a pfSense GUI package but also 'core' (executable) packages depend on newer libraries, these will get installed also.
Overwriting system core libraries
Now the entire system is broken.

Most complex, and easy to understand example :
I'll exaggerate : would you install Windows XP software on a Windows 11 version ?
Noop. If you need software that needs Windows 11, you first install Windows 11, and then you install your software.

To get more examples :
In stead of using the GUI, next time, use the console or (better) SSH access.
Use option 13.

This will list things to be updated, if any.
And if it does, it will list what it updates, and if it depends on something, it will also get these.
Most often, there isn't much to do.
But if you see (example) that the latest pfSense package X upgrade also upgrade system packages like PHP, you better think twice before you hit the 'Y' key to proceed with the upgrade.

World's most know actual show case :
I see this on my PC right now :

and I have also a rather big Office365 update/upgrade waiting for this PC, as it adds (probably ?) Windows 11 support, amongst others.

What will I do ?

Easy.
I wait before I install Windows 11 ("let it mature first") as this concerns an PC I use on my work, and it works pretty well up until now.
When it's time, I'll upgrade to Windows 11 - and only then I'll upgrade other apps.

Also :
pfSense has ZFS support with the recent versions.
So, you can crate a snap shot of your current pfSense version.
When done, select this snap shot and boot it.
Now, install the new pfSense version.
Then upgrade the packages - actually : you have nothing to do, as, when you upgrade pfSense, all packages get also re installed == to their latest version)

Test everything.

If there is the slights issue you can't resolve right away, you can boot the previous snap shot and you'll be back with a working pfSense.

Lasts words : package upgrade add functionality.
If the update concerns a security issue, you will find forum / blog posts about the issue, and you will find instructions about how to proceed.