Package List Empty

stephenw10 · Mar 10, 2023, 4:40 PM

Visiting System->Update seems to cause the list of available packages to repopulate

If you have the webgui open in a tab for a long time the cert may have expired when you visit the package manager or if you run a pkg command at the CLI you might see that request error.
Visiting the Firmware Upgrade page or the dashboard with the update check enabled or running pfSense-upgrade -c will pull a new cert. So usually that's the first thing that happens when you login and you should always have a current cert.

Steve

dem · Mar 10, 2023, 5:18 PM

@stephenw10 I have the dashboard check disabled so that might explain why I've seen the empty list a few times.

Vollans · Mar 11, 2023, 2:00 AM

@dem Same here as it slows dashboard loading so much when you login.

mfld · Mar 11, 2023, 7:31 AM

@stephenw10 said in Package List Empty:

or running pfSense-upgrade -c will pull a new cert.

Weird. Some boxes are effected and what I see on these is

pfSense-upgrade -c
......ERROR: It was not possible to determine pkg remote version
>>> Updating repositories metadata... failed.
ERROR: It was not possible to determine pfSense remote version
ERROR: It was not possible to determine pfSense-base remote version
ERROR: It was not possible to determine pfSense-kernel-pfSense remote version
Your system is up to date

On a good box I see

pfSense-upgrade -c
>>> Updating repositories metadata... done.
Your system is up to date

pfSense-upgrade -c

did not seem to help.

pkg update
Updating pfSense-core repository catalogue...
Certificate verification failed for /C=US/ST=Texas/L=Austin/O=Rubicon Communications, LLC (Netgate)/OU=pfSense Plus/CN=pfsense-plus-pkg00.atx.netgate.com
35158245376:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-Plus-snapshots-23_01-main/sources/FreeBSD-src-plus-RELENG_23_01/crypto/openssl/ssl/statem/statem_clnt.c:1921:
Certificate verification failed for /C=US/ST=Texas/L=Austin/O=Rubicon Communications, LLC (Netgate)/OU=pfSense Plus/CN=pfsense-plus-pkg00.atx.netgate.com
35158245376:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-Plus-snapshots-23_01-main/sources/FreeBSD-src-plus-RELENG_23_01/crypto/openssl/ssl/statem/statem_clnt.c:1921:
Certificate verification failed for /C=US/ST=Texas/L=Austin/O=Rubicon Communications, LLC (Netgate)/OU=pfSense Plus/CN=pfsense-plus-pkg00.atx.netgate.com
35158245376:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-Plus-snapshots-23_01-main/sources/FreeBSD-src-plus-RELENG_23_01/crypto/openssl/ssl/statem/statem_clnt.c:1921:
Certificate verification failed for /C=US/ST=Texas/L=Austin/O=Rubicon Communications, LLC (Netgate)/OU=pfSense Plus/CN=pfsense-plus-pkg00.atx.netgate.com
[snip]
routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-Plus-snapshots-23_01-main/sources/FreeBSD-src-plus-RELENG_23_01/crypto/openssl/ssl/statem/statem_clnt.c:1921:
pkg: https://pfsense-plus-pkg00.atx.netgate.com/pfSense_plus-v23_01_amd64-core/meta.txz: **Authentication error**

It keeps happening to different boxes at different times.

mfld · Mar 11, 2023, 7:44 AM

@stephenw10 said in Package List Empty:

running pfSense-upgrade -c will pull a new cert.

Is that instant / real-time ? Because I see this on a working box:

mv /etc/ssl/pfSense-repo-custom.cert /etc/ssl/pfSense-repo-custom.cert.TESTING

pfSense-upgrade -c
>>> Updating repositories metadata... done.
Your system is up to date

It seemingly completed fine. But now there is no /etc/ssl/pfSense-repo-custom.cert

stephenw10 · Mar 12, 2023, 2:12 AM

That's the cert for the CE > Plus upgrade repo. The dynamic repo cert is passed with the repo details and, yes, it's updated close to instantly:

[23.05-DEVELOPMENT][admin@plusdev.stevew.lan]/root: ls -ls /usr/local/etc/pfSense/pkg/repos
total 29
1 -rw-r--r--  1 root  wheel    16 Mar 12 01:08 pfSense-repo-23.05-Devel.abi
1 -rw-r--r--  1 root  wheel    17 Mar 12 01:08 pfSense-repo-23.05-Devel.altabi
5 -rw-r--r--  1 root  wheel   547 Mar 12 01:08 pfSense-repo-23.05-Devel.conf
1 -rw-r--r--  1 root  wheel    36 Mar 12 01:08 pfSense-repo-23.05-Devel.descr
1 -rw-r--r--  1 root  wheel    11 Mar 12 01:08 pfSense-repo-23.05-Devel.name
1 -rw-r--r--  1 root  wheel    16 Mar 12 01:08 pfSense-repo-prev.abi
1 -rw-r--r--  1 root  wheel    17 Mar 12 01:08 pfSense-repo-prev.altabi
5 -rw-r--r--  1 root  wheel   498 Mar 12 01:08 pfSense-repo-prev.conf
1 -rw-r--r--  1 root  wheel    31 Mar 12 01:08 pfSense-repo-prev.descr
1 -rw-r--r--  1 root  wheel     4 Mar 12 01:08 pfSense-repo-prev.name
5 -rw-r--r--  1 root  wheel  2053 Mar 12 01:08 pfSense-repo-stable-cert.pem
5 -rw-------  1 root  wheel  3243 Mar 12 01:08 pfSense-repo-stable-key.pem
1 -rw-r--r--  1 root  wheel    16 Mar 12 01:08 pfSense-repo-stable.abi
1 -rw-r--r--  1 root  wheel    17 Mar 12 01:08 pfSense-repo-stable.altabi
5 -rw-r--r--  1 root  wheel   506 Mar 12 01:08 pfSense-repo-stable.conf
1 -rw-r--r--  1 root  wheel     1 Mar 12 01:08 pfSense-repo-stable.default
1 -rw-r--r--  1 root  wheel    29 Mar 12 01:08 pfSense-repo-stable.descr
1 -rw-r--r--  1 root  wheel     6 Mar 12 01:08 pfSense-repo-stable.name
[23.05-DEVELOPMENT][admin@plusdev.stevew.lan]/root: pfSense-upgrade -c
>>> Updating repositories metadata... done.
Your system is up to date
[23.05-DEVELOPMENT][admin@plusdev.stevew.lan]/root: ls -ls /usr/local/etc/pfSense/pkg/repos
total 29
1 -rw-r--r--  1 root  wheel    16 Mar 12 02:09 pfSense-repo-23.05-Devel.abi
1 -rw-r--r--  1 root  wheel    17 Mar 12 02:09 pfSense-repo-23.05-Devel.altabi
5 -rw-r--r--  1 root  wheel   547 Mar 12 02:09 pfSense-repo-23.05-Devel.conf
1 -rw-r--r--  1 root  wheel    36 Mar 12 02:09 pfSense-repo-23.05-Devel.descr
1 -rw-r--r--  1 root  wheel    11 Mar 12 02:09 pfSense-repo-23.05-Devel.name
1 -rw-r--r--  1 root  wheel    16 Mar 12 02:09 pfSense-repo-prev.abi
1 -rw-r--r--  1 root  wheel    17 Mar 12 02:09 pfSense-repo-prev.altabi
5 -rw-r--r--  1 root  wheel   498 Mar 12 02:09 pfSense-repo-prev.conf
1 -rw-r--r--  1 root  wheel    31 Mar 12 02:09 pfSense-repo-prev.descr
1 -rw-r--r--  1 root  wheel     4 Mar 12 02:09 pfSense-repo-prev.name
5 -rw-r--r--  1 root  wheel  2053 Mar 12 02:09 pfSense-repo-stable-cert.pem
5 -rw-------  1 root  wheel  3243 Mar 12 02:09 pfSense-repo-stable-key.pem
1 -rw-r--r--  1 root  wheel    16 Mar 12 02:09 pfSense-repo-stable.abi
1 -rw-r--r--  1 root  wheel    17 Mar 12 02:09 pfSense-repo-stable.altabi
5 -rw-r--r--  1 root  wheel   506 Mar 12 02:09 pfSense-repo-stable.conf
1 -rw-r--r--  1 root  wheel     1 Mar 12 02:09 pfSense-repo-stable.default
1 -rw-r--r--  1 root  wheel    29 Mar 12 02:09 pfSense-repo-stable.descr
1 -rw-r--r--  1 root  wheel     6 Mar 12 02:09 pfSense-repo-stable.name

Steve

mfld · Mar 12, 2023, 2:57 PM

@stephenw10 said in Package List Empty:

That's the cert for the CE > Plus upgrade repo. The dynamic repo cert is passed with the repo details and, yes, it's updated close to instantly:

Weird.

pfSense-upgrade -d -c
......ERROR: It was not possible to determine pkg remote version
>>> Updating repositories metadata...
Updating pfSense-core repository catalogue...
Certificate verification failed for /C=US/ST=Texas/L=Austin/O=Rubicon Communications, LLC (Netgate)/OU=pfSense Plus/CN=pfsense-plus-pkg00.atx.netgate.com
34937565184:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-Plus-snapshots-23_01-main/sources/FreeBSD-src-plus-RELENG_23_01/crypto/openssl/ssl/statem/statem_clnt.c:1921:
Certificate verification failed for /C=US/ST=Texas/L=Austin/O=Rubicon Communications, LLC (Netgate)/OU=pfSense Plus/CN=pfsense-plus-pkg00.atx.netgate.com
34937565184:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-Plus-snapshots-23_01-main/sources/FreeBSD-src-plus-RELENG_23_01/crypto/openssl/ssl/statem/statem_clnt.c:1921:
Certificate verification failed for /C=US/ST=Texas/L=Austin/O=Rubicon Communications, LLC (Netgate)/OU=pfSense Plus/CN=pfsense-plus-pkg00.atx.netgate.com
34937565184:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-Plus-snapshots-23_01-main/sources/FreeBSD-src-plus-RELENG_23_01/crypto/openssl/ssl/statem/statem_clnt.c:1921:
Certificate verification failed for /C=US/ST=Texas/L=Austin/O=Rubicon Communications, LLC (Netgate)/OU=pfSense Plus/CN=pfsense-plus-pkg00.atx.netgate.com
34937565184:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-Plus-snapshots-23_01-main/sources/FreeBSD-src-plus-RELENG_23_01/crypto/openssl/ssl/statem/statem_clnt.c:1921:
pkg-static: https://pfsense-plus-pkg00.atx.netgate.com/pfSense_plus-v23_01_amd64-core/meta.txz: Authentication error
repository pfSense-core has no meta file, using default settings
Certificate verification failed for /C=US/ST=Texas/L=Austin/O=Rubicon Communications, LLC (Netgate)/OU=pfSense Plus/CN=pfsense-plus-pkg00.atx.netgate.com
34937565184:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-Plus-snapshots-23_01-main/sources/FreeBSD-src-plus-RELENG_23_01/crypto/openssl/ssl/statem/statem_clnt.c:1921:
Certificate verification failed for /C=US/ST=Texas/L=Austin/O=Rubicon Communications, LLC (Netgate)/OU=pfSense Plus/CN=pfsense-plus-pkg00.atx.netgate.com
34937565184:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-Plus-snapshots-23_01-main/sources/FreeBSD-src-plus-RELENG_23_01/crypto/openssl/ssl/statem/statem_clnt.c:1921:
pkg-static: https://pfsense-plus-pkg00.atx.netgate.com/pfSense_plus-v23_01_amd64-core/packagesite.pkg: Authentication error
Certificate verification failed for /C=US/ST=Texas/L=Austin/O=Rubicon Communications, LLC (Netgate)/OU=pfSense Plus/CN=pfsense-plus-pkg00.atx.netgate.com
34937565184:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-Plus-snapshots-23_01-main/sources/FreeBSD-src-plus-RELENG_23_01/crypto/openssl/ssl/statem/statem_clnt.c:1921:
Certificate verification failed for /C=US/ST=Texas/L=Austin/O=Rubicon Communications, LLC (Netgate)/OU=pfSense Plus/CN=pfsense-plus-pkg00.atx.netgate.com
34937565184:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-Plus-snapshots-23_01-main/sources/FreeBSD-src-plus-RELENG_23_01/crypto/openssl/ssl/statem/statem_clnt.c:1921:
pkg-static: https://pfsense-plus-pkg00.atx.netgate.com/pfSense_plus-v23_01_amd64-core/packagesite.txz: Authentication error
Unable to update repository pfSense-core
Updating pfSense repository catalogue...
Certificate verification failed for /C=US/ST=Texas/L=Austin/O=Rubicon Communications, LLC (Netgate)/OU=pfSense Plus/CN=pfsense-plus-pkg00.atx.netgate.com
34937565184:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-Plus-snapshots-23_01-main/sources/FreeBSD-src-plus-RELENG_23_01/crypto/openssl/ssl/statem/statem_clnt.c:1921:
Certificate verification failed for /C=US/ST=Texas/L=Austin/O=Rubicon Communications, LLC (Netgate)/OU=pfSense Plus/CN=pfsense-plus-pkg00.atx.netgate.com
34937565184:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-Plus-snapshots-23_01-main/sources/FreeBSD-src-plus-RELENG_23_01/crypto/openssl/ssl/statem/statem_clnt.c:1921:
Certificate verification failed for /C=US/ST=Texas/L=Austin/O=Rubicon Communications, LLC (Netgate)/OU=pfSense Plus/CN=pfsense-plus-pkg00.atx.netgate.com
34937565184:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-Plus-snapshots-23_01-main/sources/FreeBSD-src-plus-RELENG_23_01/crypto/openssl/ssl/statem/statem_clnt.c:1921:
Certificate verification failed for /C=US/ST=Texas/L=Austin/O=Rubicon Communications, LLC (Netgate)/OU=pfSense Plus/CN=pfsense-plus-pkg00.atx.netgate.com
34937565184:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-Plus-snapshots-23_01-main/sources/FreeBSD-src-plus-RELENG_23_01/crypto/openssl/ssl/statem/statem_clnt.c:1921:
pkg-static: https://pfsense-plus-pkg00.atx.netgate.com/pfSense_plus-v23_01_amd64-pfSense_plus_v23_01/meta.txz: Authentication error
repository pfSense has no meta file, using default settings
Certificate verification failed for /C=US/ST=Texas/L=Austin/O=Rubicon Communications, LLC (Netgate)/OU=pfSense Plus/CN=pfsense-plus-pkg00.atx.netgate.com
34937565184:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-Plus-snapshots-23_01-main/sources/FreeBSD-src-plus-RELENG_23_01/crypto/openssl/ssl/statem/statem_clnt.c:1921:
Certificate verification failed for /C=US/ST=Texas/L=Austin/O=Rubicon Communications, LLC (Netgate)/OU=pfSense Plus/CN=pfsense-plus-pkg00.atx.netgate.com
34937565184:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-Plus-snapshots-23_01-main/sources/FreeBSD-src-plus-RELENG_23_01/crypto/openssl/ssl/statem/statem_clnt.c:1921:
pkg-static: https://pfsense-plus-pkg00.atx.netgate.com/pfSense_plus-v23_01_amd64-pfSense_plus_v23_01/packagesite.pkg: Authentication error
Certificate verification failed for /C=US/ST=Texas/L=Austin/O=Rubicon Communications, LLC (Netgate)/OU=pfSense Plus/CN=pfsense-plus-pkg00.atx.netgate.com
34937565184:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-Plus-snapshots-23_01-main/sources/FreeBSD-src-plus-RELENG_23_01/crypto/openssl/ssl/statem/statem_clnt.c:1921:
Certificate verification failed for /C=US/ST=Texas/L=Austin/O=Rubicon Communications, LLC (Netgate)/OU=pfSense Plus/CN=pfsense-plus-pkg00.atx.netgate.com
34937565184:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-Plus-snapshots-23_01-main/sources/FreeBSD-src-plus-RELENG_23_01/crypto/openssl/ssl/statem/statem_clnt.c:1921:
pkg-static: https://pfsense-plus-pkg00.atx.netgate.com/pfSense_plus-v23_01_amd64-pfSense_plus_v23_01/packagesite.txz: Authentication error
Unable to update repository pfSense
Error updating repositories!
ERROR: It was not possible to determine pfSense remote version
ERROR: It was not possible to determine pfSense-base remote version
ERROR: It was not possible to determine pfSense-kernel-pfSense remote version
Your system is up to date

But the time stamps of those certs DID update!

ls -ls /usr/local/etc/pfSense/pkg/repos
total 16
5 -rw-r--r--  1 root  wheel  2057 Mar 12 14:52 pfSense-repo-stable-cert.pem
5 -rw-------  1 root  wheel  3243 Mar 12 14:52 pfSense-repo-stable-key.pem
1 -rw-r--r--  1 root  wheel    16 Mar 12 14:52 pfSense-repo-stable.abi
1 -rw-r--r--  1 root  wheel    17 Mar 12 14:52 pfSense-repo-stable.altabi
5 -rw-r--r--  1 root  wheel   506 Mar 12 14:52 pfSense-repo-stable.conf
1 -rw-r--r--  1 root  wheel     1 Mar 12 14:52 pfSense-repo-stable.default
1 -rw-r--r--  1 root  wheel    29 Mar 12 14:52 pfSense-repo-stable.descr
1 -rw-r--r--  1 root  wheel     6 Mar 12 14:52 pfSense-repo-stable.name

Will DM you the NDI

mfld · Mar 24, 2023, 7:45 AM

Went nuclear after nothing worked.

mkdir /root/backup && cp -r /usr/local/etc/pfSense/pkg/repos /root/backup/ && rm -rf /usr/local/etc/pfSense/pkg/repos && pfSense-upgrade -c -d

Resolved it on that instance.

overlordchex · Mar 31, 2023, 1:25 AM

@stephenw10

I'm having this same issue as I mistakenly acquired a new token thinking I would need one after changing out the OS Drive. the system automatically activates itself with the old token and then drops the SSL certs after about a week. the registration option is grayed out so I have no means of fixing it

bavcon22 · Apr 4, 2023, 4:29 PM

This post is like I posted in, look at link below: https://forum.netgate.com/post/1097286

overlordchex · Apr 1, 2023, 4:36 PM

@bavcon22 , not quite. mine is automatically registered for PFsense+ and will install the PFsense+ upgrade after being reinstalled from disk, but after about a week it loses access to the PFsense+ package repo until I reinstall the entire thing again. additionally, because it thinks it's registered already it wont let me input a new activation key.

Thank you though!

stephenw10 · Mar 31, 2023, 1:22 PM

What error are you actually seeing when this happens?

If you send me your NDI in chat I can check it.

mfld · Apr 1, 2023, 4:37 PM

@overlordchex

Try this

mkdir /root/backup && cp -r /usr/local/etc/pfSense/pkg/repos /root/backup/ && rm -rf /usr/local/etc/pfSense/pkg/repos && pfSense-upgrade -c -d

overlordchex · Apr 2, 2023, 2:34 AM

@mfld

@stephenw10 was able to help me resolve it but i'm pretty sure this would have fixed it also.
Many thanks!

jasongreb · Apr 3, 2023, 9:45 PM

I am having this same issue. I'm building out a new pfSense setup, started on CE and installed a few packages. Applied my Plus configuration and upgraded to 23.01. Now I can no longer see any available packages. It's is resolving, but the certificate is apparently not being passed as I'm getting a 400 Bad Request error. Has this been resolved yet? Is there something I should be doing differently?

overlordchex · Apr 4, 2023, 4:28 PM

@jasongreb

try the above post from @mfld or the instructions on this page. https://redmine.pfsense.org/issues/14137

stephenw10 · Apr 4, 2023, 11:16 AM

Yup try the instructions on that bug report first, resave the update repo path. If you just upgraded that's probably the cause.

Steve

t41k2m3 · Apr 6, 2023, 3:35 PM

@stephenw10 said in Package List Empty:

What device do you have? If you send me your NDI in chat I can check it.
Experiencing same issue, will send NDI via chat as well if you could please help and check. Thank you.

jasongreb · Apr 6, 2023, 7:25 PM

Hey everyone, this was fixed for me by going to System / Update / Update Settings and clicking the Save button without making any changes. Once I did that, the package list came up (I may have forced a refresh too).

really2002 · Apr 10, 2023, 4:41 PM

@jasongreb it works!!!!