Error "libssl.so.30" not found" when installing package

[eminence32]

@stephenw10 said in Error "libssl.so.30" not found" when installing package:

pfSense-upgrade -d

Hi all,

I am also seeing the same problems mentioned in the original post.

I'd like to avoid a factory reset if possible, so I'm trying to follow some of the instructions in this thread.

I've set my update branch in the GUI to "Previous Plus Version (23.05.1)" (which matches my currently running version "23.05-RELEASE").

From a ssh shell:

[23.05-RELEASE][root@pfs.local.mydomain.com]/root: pkg-static clean -ay ; pkg-static install -fy pkg pfSense-repo pfSense-upgrade
pkg-static: No active remote repositories configured
Checking integrity... done (0 conflicting)
The most recent versions of packages are already installed

[23.05-RELEASE][root@pfs.local.mydomain.com]/root: pkg help
ld-elf.so.1: Shared object "libssl.so.30" not found, required by "pkg"

Note that I do not have a pfSense.conf file as one of the above commenters showed (I'm not sure if this is expected or not)

[23.05-RELEASE][root@pfs.local.mydomain.com]/root: ls -l /usr/local/etc/pkg/repos/
total 4
-rw-r--r--  1 root  wheel  26 May 22 11:02 FreeBSD.conf

EDIT : Using the GUI to switch the update branch from "Previous Plus Version" to "Latest Stable" and then back to "Previous Plus Version" has fixed the missing pfSense.conf file:

[23.05-RELEASE][root@pfs.local.em32.net]/root: ls -l /usr/local/etc/pkg/repos/
total 4
-rw-r--r--  1 root  wheel  26 May 22 11:02 FreeBSD.conf
lrwxr-xr-x  1 root  wheel  63 Nov 18 15:13 pfSense.conf -> /usr/local/etc/pfSense/pkg/repos/pfSense-repo-stable-point.conf

Now a pkg-static install -fy pkg pfSense-repo pfSense-upgrade has worked. Now trying an upgrade to the latest 23.09...

[Grizwald]

Hi all

This is my first post on this forum and I thought I should also share what I did to fix this issue. I followed this guide another user offered https://docs.netgate.com/pfsense/en/latest/troubleshooting/upgrades.html#upgrade-not-offered-library-errors

My issue was the command returned this:

[2.7.0-RELEASE][admin@FW-EDGE.grizwald.lan]/root: pkg-static clean -ay ; pkg-static install -fy pkg pfSense-repo pfSense-upgrade
pkg-static: Repository pfSense-core missing. 'pkg update' required
pkg-static: No package database installed.  Nothing to do!
Updating pfSense-core repository catalogue...
pkg-static: An error occured while fetching package
pkg-static: An error occured while fetching package
repository pfSense-core has no meta file, using default settings
pkg-static: An error occured while fetching package
pkg-static: An error occured while fetching package
Unable to update repository pfSense-core
Updating pfSense repository catalogue...
pkg-static: An error occured while fetching package
pkg-static: An error occured while fetching package
repository pfSense has no meta file, using default settings
pkg-static: An error occured while fetching package
pkg-static: An error occured while fetching package
Unable to update repository pfSense
Error updating repositories!

I then attempted this command to force pkg update :

[2.7.0-RELEASE][admin@FW-EDGE.grizwald.lan]/root: pkg-static update -f
Updating pfSense-core repository catalogue...
pkg-static: An error occured while fetching package
pkg-static: An error occured while fetching package
repository pfSense-core has no meta file, using default settings
pkg-static: An error occured while fetching package
pkg-static: An error occured while fetching package
Unable to update repository pfSense-core
Updating pfSense repository catalogue...
pkg-static: An error occured while fetching package
pkg-static: An error occured while fetching package
repository pfSense has no meta file, using default settings
pkg-static: An error occured while fetching package
pkg-static: An error occured while fetching package
Unable to update repository pfSense
Error updating repositories!

I followed the document which states to use the command "certctl rehash" if "An error occured while fetching package"

"A general error that could have a few different causes. It may indicate that pkg does not trust the package servers. Try running certctl rehash from the console, a root shell prompt, or via Diagnostics > Command Prompt. This will allow pkg to utilize the system certificates until the next reboot." - from document

After running this command I was able to force update my pkg:

[2.7.0-RELEASE][admin@FW-EDGE.grizwald.lan]/root: pkg-static update -f
Updating pfSense-core repository catalogue...
Fetching meta.conf: 100%    163 B   0.2kB/s    00:01
Fetching packagesite.pkg: 100%    1 KiB   1.5kB/s    00:01
Processing entries: 100%
pfSense-core repository update completed. 4 packages processed.
Updating pfSense repository catalogue...
Fetching meta.conf: 100%    163 B   0.2kB/s    00:01
Fetching packagesite.pkg: 100%  157 KiB 160.6kB/s    00:01
Processing entries:   0%
Newer FreeBSD version for package xxhash:
To ignore this error set IGNORE_OSVERSION=yes
- package: 1400094
- running kernel: 1400085
Ignore the mismatch and continue? [y/N]: y
Processing entries: 100%
pfSense repository update completed. 549 packages processed.
All repositories are up to date.

And then finally upgrade to the latest version using

pkg-static install -fy pkg pfSense-repo pfSense-upgrade

Everything is working again. And sorry for not having this formatted correctly. It is my first post ever on the netgate forums., but hopefully this helps someone else.

[Sagy]

@Grizwald Mate, thank you for this, the rehash fixed all my pfsense issues. Made an account purely just to say thanks.

[scruzuser]

@stephenw10
So, I'm in a fine pickle...

Running a pkg-static update to get to 23.09.1 I ran into the same issue with missing files:

[19/110] Upgrading cyrus-sasl from 2.1.28 to 2.1.28_1...
*** Updated user `cyrus'.
[19/110] Extracting cyrus-sasl-2.1.28_1: 100%
ld-elf.so.1: Shared object "libcrypto.so.30" not found, required by "libsasl2.so.3"
ld-elf.so.1: Shared object "libcrypto.so.30" not found, required by "libsasl2.so.3"
WARNING: Failed to create /usr/local/etc/sasldb2.db
pkg-static: Cannot delete vital package: pfSense-rc!
pkg-static: If you are sure you want to remove pfSense-rc,
pkg-static: unset the 'vital' flag with: pkg set -v 0 pfSense-rc

And it then stopped. It didn't seem prudent to unset the vital flag.

I came across this thread and have tried messing with System/Update/Update Settings to go to "Previous Plus Version (23.05.1)". I get the following when trying to run the pkg-static update -f (many lines of the form):

Updating pfSense-core repository catalogue...
Certificate verification failed for /C=US/ST=Texas/L=Austin/O=Rubicon Communications, LLC (Netgate)/OU=pfSense Plus/CN=pfsense-plus-pkg01.atx.netgate.com
34946478080:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-Plus-snapshots-23_05_1-main/sources/FreeBSD-src-plus-RELENG_23_05_1/crypto/openssl/ssl/statem/statem_clnt.c:1921:
Certificate verification failed for /C=US/ST=Texas/L=Austin/O=Rubicon Communications, LLC (Netgate)/OU=pfSense Plus/CN=pfsense-plus-pkg01.atx.netgate.com
34946478080:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-Plus-snapshots-23_05_1-main/sources/FreeBSD-src-plus-RELENG_23_05_1/crypto/openssl/ssl/statem/statem_clnt.c:1921:
Certificate verification failed for /C=US/ST=Texas/L=Austin/O=Rubicon Communications, LLC (Netgate)/OU=pfSense Plus/CN=pfsense-plus-pkg01.atx.netgate.com
34946478080:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-Plus-snapshots-23_05_1-main/sources/FreeBSD-src-plus-RELENG_23_05_1/crypto/openssl/ssl/statem/statem_clnt.c:1921:
Certificate verification failed for /C=US/ST=Texas/L=Austin/O=Rubicon Communications, LLC (Netgate)/OU=pfSense Plus/CN=pfsense-plus-pkg01.atx.netgate.com
34946478080:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-Plus-snapshots-23_05_1-main/sources/FreeBSD-src-plus-RELENG_23_05_1/crypto/openssl/ssl/statem/statem_clnt.c:1921:
Certificate verification failed for /C=US/ST=Texas/L=Austin/O=Rubicon Communications, LLC (Netgate)/OU=pfSense Plus/CN=pfsense-plus-pkg01.atx.netgate.com
34946478080:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-Plus-snapshots-23_05_1-main/sources/FreeBSD-src-plus-RELENG_23_05_1/crypto/openssl/ssl/statem/statem_clnt.c:1921:
Certificate verification failed for /C=US/ST=Texas/L=Austin/O=Rubicon Communications, LLC (Netgate)/OU=pfSense Plus/CN=pfsense-plus-pkg01.atx.netgate.com
34946478080:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-Plus-snapshots-23_05_1-main/sources/FreeBSD-src-plus-RELENG_23_05_1/crypto/openssl/ssl/statem/statem_clnt.c:1921:
pkg-static: https://pfsense-plus-pkg01.atx.netgate.com/pfSense_plus-v23_05_1_amd64-core/meta.txz: Authentication error
repository pfSense-core has no meta file, using default settings

I have run the certctl rehash. has no effect.
I now find that /usr/local/etc/pkg/repos seems wrong based on what I see in this thread:
$ ls -l /usr/local/etc/pkg/repos
total 1
-rw-r--r-- 1 root wheel 26 Jun 27 20:55 FreeBSD.conf
$

I tried making a link manually (changing things back and forth in the System/Update/Update Settings as mentioned prior did not create the pfSense.conf file):
ln -s /usr/local/etc/pfSense/pkg/repos/pfSense-repo-stable-point.conf pfSense.conf
and then run pkg-static update and I get the above cert errors. The contents of my pfSense.conf file are:

FreeBSD: { enabled: no }

pfSense-core: {
url: "pkg+https://pfsense-plus-pkg.netgate.com/pfSense_plus-v23_05_1_amd64-core",
mirror_type: "srv",
signature_type: "fingerprints",
fingerprints: "/usr/local/share/pfSense/keys/pkg",
enabled: yes
}

pfSense: {
url: "pkg+https://pfsense-plus-pkg.netgate.com/pfSense_plus-v23_05_1_amd64-pfSense_plus_v23_05_1",
mirror_type: "srv",
signature_type: "fingerprints",
fingerprints: "/usr/local/share/pfSense/keys/pkg",
enabled: yes
}

And after I put the link in and try to run pkg-static clean -ay:
$ pkg-static clean -ay
pkg-static: Repository pfSense-core missing. 'pkg update' required
pkg-static: No package database installed. Nothing to do!

Any clue how to get passed the cert errors, get my pfSense.conf file to really appear and on to the pkg-static update -f to work?
Thanks.

[stephenw10]

What repo is it trying to reach when you run: pkg-static d update ?

[scruzuser]

@stephenw10 said in Error "libssl.so.30" not found" when installing package:

pkg-static d update

I believe you mean "pkg-static -d update" (missing the '-' above)?
$ pkg-static -d update
DBG(1)[36885]> pkg initialized
Updating pfSense-core repository catalogue...
DBG(1)[36885]> PkgRepo: verifying update for pfSense-core
DBG(1)[36885]> PkgRepo: need forced update of pfSense-core
DBG(1)[36885]> Pkgrepo, begin update of '/var/db/pkg/repo-pfSense-core.sqlite'
DBG(1)[36885]> Request to fetch pkg+https://pfsense-plus-pkg.netgate.com/pfSense_plus-v23_05_1_amd64-core/meta.conf
DBG(1)[36885]> opening libfetch fetcher
DBG(1)[36885]> Fetch > libfetch: connecting
DBG(1)[36885]> Fetch: fetching from: https://pfsense-plus-pkg00.atx.netgate.com/pfSense_plus-v23_05_1_amd64-core/meta.conf with opts "i"
Certificate verification failed for /C=US/ST=Texas/L=Austin/O=Rubicon Communications, LLC (Netgate)/OU=pfSense Plus/CN=pfsense-plus-pkg00.atx.netgate.com
34955853824:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-Plus-snapshots-23_05_1-main/sources/FreeBSD-src-plus-RELENG_23_05_1/crypto/openssl/ssl/statem/statem_clnt.c:1921:
DBG(1)[36885]> Fetch: fetching from: https://pfsense-plus-pkg00.atx.netgate.com/pfSense_plus-v23_05_1_amd64-core/meta.conf with opts "i"
Certificate verification failed for /C=US/ST=Texas/L=Austin/O=Rubicon Communications, LLC (Netgate)/OU=pfSense Plus/CN=pfsense-plus-pkg00.atx.netgate.com
34955853824:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-Plus-snapshots-23_05_1-main/sources/FreeBSD-src-plus-RELENG_23_05_1/crypto/openssl/ssl/statem/statem_clnt.c:1921:
DBG(1)[36885]> Fetch: fetching from: https://pfsense-plus-pkg00.atx.netgate.com/pfSense_plus-v23_05_1_amd64-core/meta.conf with opts "i"
Certificate verification failed for /C=US/ST=Texas/L=Austin/O=Rubicon Communications, LLC (Netgate)/OU=pfSense Plus/CN=pfsense-plus-pkg00.atx.netgate.com
34955853824:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-Plus-snapshots-23_05_1-main/sources/FreeBSD-src-plus-RELENG_23_05_1/crypto/openssl/ssl/statem/statem_clnt.c:1921:
DBG(1)[36885]> Request to fetch pkg+https://pfsense-plus-pkg.netgate.com/pfSense_plus-v23_05_1_amd64-core/meta.txz
DBG(1)[36885]> opening libfetch fetcher
DBG(1)[36885]> Fetch > libfetch: connecting
DBG(1)[36885]> Fetch: fetching from: https://pfsense-plus-pkg00.atx.netgate.com/pfSense_plus-v23_05_1_amd64-core/meta.txz with opts "i"
Certificate verification failed for /C=US/ST=Texas/L=Austin/O=Rubicon Communications, LLC (Netgate)/OU=pfSense Plus/CN=pfsense-plus-pkg00.atx.netgate.com
34955853824:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-Plus-snapshots-23_05_1-main/sources/FreeBSD-src-plus-RELENG_23_05_1/crypto/openssl/ssl/statem/statem_clnt.c:1921:
DBG(1)[36885]> Fetch: fetching from: https://pfsense-plus-pkg00.atx.netgate.com/pfSense_plus-v23_05_1_amd64-core/meta.txz with opts "i"
Certificate verification failed for /C=US/ST=Texas/L=Austin/O=Rubicon Communications, LLC (Netgate)/OU=pfSense Plus/CN=pfsense-plus-pkg00.atx.netgate.com
34955853824:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-Plus-snapshots-23_05_1-main/sources/FreeBSD-src-plus-RELENG_23_05_1/crypto/openssl/ssl/statem/statem_clnt.c:1921:
DBG(1)[36885]> Fetch: fetching from: https://pfsense-plus-pkg00.atx.netgate.com/pfSense_plus-v23_05_1_amd64-core/meta.txz with opts "i"
Certificate verification failed for /C=US/ST=Texas/L=Austin/O=Rubicon Communications, LLC (Netgate)/OU=pfSense Plus/CN=pfsense-plus-pkg00.atx.netgate.com
34955853824:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-Plus-snapshots-23_05_1-main/sources/FreeBSD-src-plus-RELENG_23_05_1/crypto/openssl/ssl/statem/statem_clnt.c:1921:
pkg-static: https://pfsense-plus-pkg00.atx.netgate.com/pfSense_plus-v23_05_1_amd64-core/meta.txz: Authentication error
repository pfSense-core has no meta file, using default settings
DBG(1)[36885]> Request to fetch pkg+https://pfsense-plus-pkg.netgate.com/pfSense_plus-v23_05_1_amd64-core/packagesite.pkg
DBG(1)[36885]> opening libfetch fetcher
DBG(1)[36885]> Fetch > libfetch: connecting
DBG(1)[36885]> Fetch: fetching from: https://pfsense-plus-pkg00.atx.netgate.com/pfSense_plus-v23_05_1_amd64-core/packagesite.pkg with opts "i"
Certificate verification failed for /C=US/ST=Texas/L=Austin/O=Rubicon Communications, LLC (Netgate)/OU=pfSense Plus/CN=pfsense-plus-pkg00.atx.netgate.com
34955853824:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-Plus-snapshots-23_05_1-main/sources/FreeBSD-src-plus-RELENG_23_05_1/crypto/openssl/ssl/statem/statem_clnt.c:1921:
DBG(1)[36885]> Fetch: fetching from: https://pfsense-plus-pkg00.atx.netgate.com/pfSense_plus-v23_05_1_amd64-core/packagesite.pkg with opts "i"
Certificate verification failed for /C=US/ST=Texas/L=Austin/O=Rubicon Communications, LLC (Netgate)/OU=pfSense Plus/CN=pfsense-plus-pkg00.atx.netgate.com
34955853824:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-Plus-snapshots-23_05_1-main/sources/FreeBSD-src-plus-RELENG_23_05_1/crypto/openssl/ssl/statem/statem_clnt.c:1921:
DBG(1)[36885]> Fetch: fetching from: https://pfsense-plus-pkg00.atx.netgate.com/pfSense_plus-v23_05_1_amd64-core/packagesite.pkg with opts "i"
Certificate verification failed for /C=US/ST=Texas/L=Austin/O=Rubicon Communications, LLC (Netgate)/OU=pfSense Plus/CN=pfsense-plus-pkg00.atx.netgate.com
34955853824:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-Plus-snapshots-23_05_1-main/sources/FreeBSD-src-plus-RELENG_23_05_1/crypto/openssl/ssl/statem/statem_clnt.c:1921:
pkg-static: https://pfsense-plus-pkg00.atx.netgate.com/pfSense_plus-v23_05_1_amd64-core/packagesite.pkg: Authentication error
DBG(1)[36885]> Request to fetch pkg+https://pfsense-plus-pkg.netgate.com/pfSense_plus-v23_05_1_amd64-core/packagesite.txz
DBG(1)[36885]> opening libfetch fetcher
DBG(1)[36885]> Fetch > libfetch: connecting
DBG(1)[36885]> Fetch: fetching from: https://pfsense-plus-pkg00.atx.netgate.com/pfSense_plus-v23_05_1_amd64-core/packagesite.txz with opts "i"
Certificate verification failed for /C=US/ST=Texas/L=Austin/O=Rubicon Communications, LLC (Netgate)/OU=pfSense Plus/CN=pfsense-plus-pkg00.atx.netgate.com
34955853824:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-Plus-snapshots-23_05_1-main/sources/FreeBSD-src-plus-RELENG_23_05_1/crypto/openssl/ssl/statem/statem_clnt.c:1921:
DBG(1)[36885]> Fetch: fetching from: https://pfsense-plus-pkg00.atx.netgate.com/pfSense_plus-v23_05_1_amd64-core/packagesite.txz with opts "i"
Certificate verification failed for /C=US/ST=Texas/L=Austin/O=Rubicon Communications, LLC (Netgate)/OU=pfSense Plus/CN=pfsense-plus-pkg00.atx.netgate.com
34955853824:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-Plus-snapshots-23_05_1-main/sources/FreeBSD-src-plus-RELENG_23_05_1/crypto/openssl/ssl/statem/statem_clnt.c:1921:
DBG(1)[36885]> Fetch: fetching from: https://pfsense-plus-pkg00.atx.netgate.com/pfSense_plus-v23_05_1_amd64-core/packagesite.txz with opts "i"
Certificate verification failed for /C=US/ST=Texas/L=Austin/O=Rubicon Communications, LLC (Netgate)/OU=pfSense Plus/CN=pfsense-plus-pkg00.atx.netgate.com
34955853824:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-Plus-snapshots-23_05_1-main/sources/FreeBSD-src-plus-RELENG_23_05_1/crypto/openssl/ssl/statem/statem_clnt.c:1921:
pkg-static: https://pfsense-plus-pkg00.atx.netgate.com/pfSense_plus-v23_05_1_amd64-core/packagesite.txz: Authentication error
Unable to update repository pfSense-core
Updating pfSense repository catalogue...
DBG(1)[36885]> PkgRepo: verifying update for pfSense
DBG(1)[36885]> PkgRepo: need forced update of pfSense
DBG(1)[36885]> Pkgrepo, begin update of '/var/db/pkg/repo-pfSense.sqlite'
DBG(1)[36885]> Request to fetch pkg+https://pfsense-plus-pkg.netgate.com/pfSense_plus-v23_05_1_amd64-pfSense_plus_v23_05_1/meta.conf
DBG(1)[36885]> opening libfetch fetcher
DBG(1)[36885]> Fetch > libfetch: connecting
DBG(1)[36885]> Fetch: fetching from: https://pfsense-plus-pkg00.atx.netgate.com/pfSense_plus-v23_05_1_amd64-pfSense_plus_v23_05_1/meta.conf with opts "i"
Certificate verification failed for /C=US/ST=Texas/L=Austin/O=Rubicon Communications, LLC (Netgate)/OU=pfSense Plus/CN=pfsense-plus-pkg00.atx.netgate.com
34955853824:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-Plus-snapshots-23_05_1-main/sources/FreeBSD-src-plus-RELENG_23_05_1/crypto/openssl/ssl/statem/statem_clnt.c:1921:
DBG(1)[36885]> Fetch: fetching from: https://pfsense-plus-pkg00.atx.netgate.com/pfSense_plus-v23_05_1_amd64-pfSense_plus_v23_05_1/meta.conf with opts "i"
Certificate verification failed for /C=US/ST=Texas/L=Austin/O=Rubicon Communications, LLC (Netgate)/OU=pfSense Plus/CN=pfsense-plus-pkg00.atx.netgate.com
34955853824:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-Plus-snapshots-23_05_1-main/sources/FreeBSD-src-plus-RELENG_23_05_1/crypto/openssl/ssl/statem/statem_clnt.c:1921:
DBG(1)[36885]> Fetch: fetching from: https://pfsense-plus-pkg00.atx.netgate.com/pfSense_plus-v23_05_1_amd64-pfSense_plus_v23_05_1/meta.conf with opts "i"
Certificate verification failed for /C=US/ST=Texas/L=Austin/O=Rubicon Communications, LLC (Netgate)/OU=pfSense Plus/CN=pfsense-plus-pkg00.atx.netgate.com
34955853824:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-Plus-snapshots-23_05_1-main/sources/FreeBSD-src-plus-RELENG_23_05_1/crypto/openssl/ssl/statem/statem_clnt.c:1921:
DBG(1)[36885]> Request to fetch pkg+https://pfsense-plus-pkg.netgate.com/pfSense_plus-v23_05_1_amd64-pfSense_plus_v23_05_1/meta.txz
DBG(1)[36885]> opening libfetch fetcher
DBG(1)[36885]> Fetch > libfetch: connecting
DBG(1)[36885]> Fetch: fetching from: https://pfsense-plus-pkg00.atx.netgate.com/pfSense_plus-v23_05_1_amd64-pfSense_plus_v23_05_1/meta.txz with opts "i"
Certificate verification failed for /C=US/ST=Texas/L=Austin/O=Rubicon Communications, LLC (Netgate)/OU=pfSense Plus/CN=pfsense-plus-pkg00.atx.netgate.com
34955853824:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-Plus-snapshots-23_05_1-main/sources/FreeBSD-src-plus-RELENG_23_05_1/crypto/openssl/ssl/statem/statem_clnt.c:1921:
DBG(1)[36885]> Fetch: fetching from: https://pfsense-plus-pkg00.atx.netgate.com/pfSense_plus-v23_05_1_amd64-pfSense_plus_v23_05_1/meta.txz with opts "i"
Certificate verification failed for /C=US/ST=Texas/L=Austin/O=Rubicon Communications, LLC (Netgate)/OU=pfSense Plus/CN=pfsense-plus-pkg00.atx.netgate.com
34955853824:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-Plus-snapshots-23_05_1-main/sources/FreeBSD-src-plus-RELENG_23_05_1/crypto/openssl/ssl/statem/statem_clnt.c:1921:
DBG(1)[36885]> Fetch: fetching from: https://pfsense-plus-pkg00.atx.netgate.com/pfSense_plus-v23_05_1_amd64-pfSense_plus_v23_05_1/meta.txz with opts "i"
Certificate verification failed for /C=US/ST=Texas/L=Austin/O=Rubicon Communications, LLC (Netgate)/OU=pfSense Plus/CN=pfsense-plus-pkg00.atx.netgate.com
34955853824:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-Plus-snapshots-23_05_1-main/sources/FreeBSD-src-plus-RELENG_23_05_1/crypto/openssl/ssl/statem/statem_clnt.c:1921:
pkg-static: https://pfsense-plus-pkg00.atx.netgate.com/pfSense_plus-v23_05_1_amd64-pfSense_plus_v23_05_1/meta.txz: Authentication error
repository pfSense has no meta file, using default settings
DBG(1)[36885]> Request to fetch pkg+https://pfsense-plus-pkg.netgate.com/pfSense_plus-v23_05_1_amd64-pfSense_plus_v23_05_1/packagesite.pkg
DBG(1)[36885]> opening libfetch fetcher
DBG(1)[36885]> Fetch > libfetch: connecting
DBG(1)[36885]> Fetch: fetching from: https://pfsense-plus-pkg00.atx.netgate.com/pfSense_plus-v23_05_1_amd64-pfSense_plus_v23_05_1/packagesite.pkg with opts "i"
Certificate verification failed for /C=US/ST=Texas/L=Austin/O=Rubicon Communications, LLC (Netgate)/OU=pfSense Plus/CN=pfsense-plus-pkg00.atx.netgate.com
34955853824:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-Plus-snapshots-23_05_1-main/sources/FreeBSD-src-plus-RELENG_23_05_1/crypto/openssl/ssl/statem/statem_clnt.c:1921:
DBG(1)[36885]> Fetch: fetching from: https://pfsense-plus-pkg00.atx.netgate.com/pfSense_plus-v23_05_1_amd64-pfSense_plus_v23_05_1/packagesite.pkg with opts "i"
Certificate verification failed for /C=US/ST=Texas/L=Austin/O=Rubicon Communications, LLC (Netgate)/OU=pfSense Plus/CN=pfsense-plus-pkg00.atx.netgate.com
34955853824:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-Plus-snapshots-23_05_1-main/sources/FreeBSD-src-plus-RELENG_23_05_1/crypto/openssl/ssl/statem/statem_clnt.c:1921:
DBG(1)[36885]> Fetch: fetching from: https://pfsense-plus-pkg00.atx.netgate.com/pfSense_plus-v23_05_1_amd64-pfSense_plus_v23_05_1/packagesite.pkg with opts "i"
Certificate verification failed for /C=US/ST=Texas/L=Austin/O=Rubicon Communications, LLC (Netgate)/OU=pfSense Plus/CN=pfsense-plus-pkg00.atx.netgate.com
34955853824:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-Plus-snapshots-23_05_1-main/sources/FreeBSD-src-plus-RELENG_23_05_1/crypto/openssl/ssl/statem/statem_clnt.c:1921:
pkg-static: https://pfsense-plus-pkg00.atx.netgate.com/pfSense_plus-v23_05_1_amd64-pfSense_plus_v23_05_1/packagesite.pkg: Authentication error
DBG(1)[36885]> Request to fetch pkg+https://pfsense-plus-pkg.netgate.com/pfSense_plus-v23_05_1_amd64-pfSense_plus_v23_05_1/packagesite.txz
DBG(1)[36885]> opening libfetch fetcher
DBG(1)[36885]> Fetch > libfetch: connecting
DBG(1)[36885]> Fetch: fetching from: https://pfsense-plus-pkg00.atx.netgate.com/pfSense_plus-v23_05_1_amd64-pfSense_plus_v23_05_1/packagesite.txz with opts "i"
Certificate verification failed for /C=US/ST=Texas/L=Austin/O=Rubicon Communications, LLC (Netgate)/OU=pfSense Plus/CN=pfsense-plus-pkg00.atx.netgate.com
34955853824:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-Plus-snapshots-23_05_1-main/sources/FreeBSD-src-plus-RELENG_23_05_1/crypto/openssl/ssl/statem/statem_clnt.c:1921:
DBG(1)[36885]> Fetch: fetching from: https://pfsense-plus-pkg00.atx.netgate.com/pfSense_plus-v23_05_1_amd64-pfSense_plus_v23_05_1/packagesite.txz with opts "i"
Certificate verification failed for /C=US/ST=Texas/L=Austin/O=Rubicon Communications, LLC (Netgate)/OU=pfSense Plus/CN=pfsense-plus-pkg00.atx.netgate.com
34955853824:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-Plus-snapshots-23_05_1-main/sources/FreeBSD-src-plus-RELENG_23_05_1/crypto/openssl/ssl/statem/statem_clnt.c:1921:
DBG(1)[36885]> Fetch: fetching from: https://pfsense-plus-pkg00.atx.netgate.com/pfSense_plus-v23_05_1_amd64-pfSense_plus_v23_05_1/packagesite.txz with opts "i"
Certificate verification failed for /C=US/ST=Texas/L=Austin/O=Rubicon Communications, LLC (Netgate)/OU=pfSense Plus/CN=pfsense-plus-pkg00.atx.netgate.com
34955853824:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-Plus-snapshots-23_05_1-main/sources/FreeBSD-src-plus-RELENG_23_05_1/crypto/openssl/ssl/statem/statem_clnt.c:1921:
pkg-static: https://pfsense-plus-pkg00.atx.netgate.com/pfSense_plus-v23_05_1_amd64-pfSense_plus_v23_05_1/packagesite.txz: Authentication error
Unable to update repository pfSense
Error updating repositories!

My System/Update/Update Settings is set to "Previous Plus Version (23.05.1)"

[stephenw10]

Sorry yes I typo'd the command.

If you set the branch to 23.09.1 what error do you get?

To be honest I would just reinstall at that point if you can do that .

[scruzuser]

@stephenw10

If I set the branch to 23.09.1 in System/Update/Update Settings, no file appears in /usr/local/etc/pkg/repos (and if I had a link there to the existing 23.05.1 file as noted above, it is removed), and then the command fails:

[23.05.1-RELEASE][admin@machine@host]/usr/local/etc/pkg/repos: pkg-static -d update
DBG(1)[63107]> pkg initialized
No active remote repositories configured.

I can restore from a backup, but I'm not sure that is enough, as that is "configuration" info of my firewall and I think I'm also missing some underlying certificates and I'm not knowledgeable enough to know if the restore will have the certs. I think I need to go all the way back to an installer image as described here:
https://docs.netgate.com/pfsense/en/latest/install/index.html

?

Or just will a "restore" be good enough?

I will try a full power cycle as suggested here sometime this week before I go restoring: https://forum.netgate.com/topic/165700/repo01-netgate-com-tls-cert-seems-invalid/11

I just can't take the system down now - too many users on it.

[stephenw10]

A config restore would do nothing here.

Is it installed ZFS? If so you should have a BE snapshot from before the upgrade was fist started you could roll back to.

[scruzuser]

@stephenw10 It seems to be installed as ZFS - whatever netgate does by default on a Netgate purchased 4100

$ df -t zfs
Filesystem 1K-blocks Used Avail Capacity Mounted on
pfSense/ROOT/default 9178497 1399296 7779201 15% /
pfSense/tmp 7780149 948 7779201 0% /tmp
pfSense/home 7779637 436 7779201 0% /home
pfSense/var 7784825 5624 7779201 0% /var
pfSense/var/log 7784069 4868 7779201 0% /var/log
pfSense/var/db 7783109 3908 7779201 0% /var/db
pfSense/var/tmp 7779313 112 7779201 0% /var/tmp
pfSense/var/cache 7779305 104 7779201 0% /var/cache
pfSense/ROOT/default/cf 7799629 20428 7779201 0% /cf
pfSense/ROOT/default/var_cache_pkg 7788689 9488 7779201 0% /var/cache/pkg
pfSense/ROOT/default/var_db_pkg 7785453 6252 7779201 0% /var/db/pkg

I've never used ZFS - but looking at System/Boot Environments - Here's what I see:

it sure is odd that a 23.05.1 BE was created on 2022-06-22, and somehow booted on 2023-12-14. The 23.05 BE from 2023-8-09 seems like one I want to go back to, but oddly, the only Action icons that I can select are the stars, none of the other are working. So just select the 2023-08-09 star and then reboot?

[stephenw10]

In a normal upgrade it should have created a new snapshot on 23.05.1 and then upgraded the current snap to 23.09.1. So it must have failed before that snap was created.

If your config hasn't changed significantly since that 23.05 snap you can certainly roll back to that. Then upgrade from there to 23.09 and then to 23.09.1.

I would backup your current config first though so you can restore that afterwards to replace anything that has changed.

[scruzuser]

@stephenw10
It is odd that the 23.05.1 creation date shows 2022-06-22 when 23.05.1 wasn't even released until 2023-06-29! I didn't even buy this device until May of 2023....

I will be preparing a recovery USB stick and I do have my old configuration backed up before I do anything - just in case.

Then I'll do the rollback to 23.05 snap.

I have to change an IPSEC tunnel I have to AWS as it has SHA1 (as per AWS' instructions - I've come to learn that they do support much more so I need to move to SHA256 before the 23.09 upgrade given deprecated support for SHA1). Then I'll do the upgrades... I'll report back later this week. Thanks again.

[stephenw10]

It's still booting the default BE which is the very first one created, potentially before the system clock was set.

Every time you update it creates a different BE from the current one but doesn't switch to that. So the creation date of the default remains.

[scruzuser]

@stephenw10

TL;DR: Up and running 23.09.1 by installing fresh and restoring saved off config.xml.

Story:
I was wondering why the device had only been up for four days during this process - I haven't rebooted it in a long time. Then someone mentioned we had a power glitch four days ago. Just momentarily. No odd weather, nothing to explain it. Just a quick glitch. I didn't know this when I started this process two days after the power glitch. It seems the power glitch corrupted the file system that we were running.

So I went back to the 23.05 BE and rebooted - things booted fine. Yeah! But I had made updates since then and had a configuration file - I had actually generated it from the "corrupt" system - I did compare it to something I had made a few weeks back (with tkdiff) and the differences were appropriate and the file was not corrupt, so I felt comfortable using it. So I thought, let's try to boot the BE that I had been running, just to see if I could. It wouldn't boot. It tried to start up, somehow thought it was a UFS file system, couldn't find /bin/rm and other basic command, all messed up. Just hung.. Quite a disaster.

I put in a USB stick with 23.09.1, booted it, reinstalled fresh, reloaded my saved config.xml file and everything is back to "normal" and upgraded. So thanks for such a great configuration saving mechanism. Thanks for your guidance and these forums. It really was a smooth reinstall. I'd call it "elegant". Thanks to Netgate TAC to responding with a request for 23.09.1 so quickly earlier in the day.

A new UPS is coming as well as I purchased a Netgate 1100 as a "backup" in case this hardware does die/get fried - I really should not be running w/o a spare. The 1100 will suffice for a few days in case the 4100 ever dies. I always have a UPS on my networking eqpt to ride out glitches and short outages, but just hadn't gotten to it yet. That will teach me to not dally on ordering UPS....
Once again, thanks to all who posted here and big shout out to @stephenw10. I also learned a lot more than I had bargained for on this journey - always worth it.

[Cleetus Antony]

@stephenw10
Getting this below error

pkg-static: An error occured while fetching package
Unable to update repository pfSense
Error updating repositories!

[stephenw10]

What are you running to generate that error?

What pfSense version are you running?