No packages in available packages list

HansWerner3dit

@stephenw10 said in No packages in available packages list:

Check: /usr/local/etc/pfSense/pkg/repos
Do you see entries for the available branches?

i do see some repository files in there...

[24.03-RELEASE][admin@pfSense.3dit.local]/usr/local/etc/pfSense/pkg/repos: ls /usr/local/etc/pfSense/pkg/repos
pfSense-repo-0000-cert.pem pfSense-repo-0000.default  pfSense-repo-0001-key.pem  pfSense-repo-0001.name
pfSense-repo-0000-key.pem  pfSense-repo-0000.descr    pfSense-repo-0001.abi      pfSense-repo-0001.version
pfSense-repo-0000.abi      pfSense-repo-0000.name     pfSense-repo-0001.altabi
pfSense-repo-0000.altabi   pfSense-repo-0000.version  pfSense-repo-0001.conf
pfSense-repo-0000.conf     pfSense-repo-0001-cert.pem pfSense-repo-0001.descr

the fingerprints entry points to /usr/local/share/pfSense/keys/pkg instead of the trusted subpath /usr/local/share/pfSense/keys/pkg/trusted. could that be an issue?

[24.03-RELEASE][admin@pfSense.3dit.local]/usr/local/etc/pfSense/pkg/repos: cat pfSense-repo-0000.conf 
FreeBSD: { enabled: no }

pfSense-core: {
    url: "pkg+https://pfsense-plus-pkg.netgate.com/pfSense_plus-v24_03_amd64-core",
    mirror_type: "srv",
    signature_type: "fingerprints",
    fingerprints: "/usr/local/share/pfSense/keys/pkg",
    enabled: yes
}

pfSense: {
    url: "pkg+https://pfsense-plus-pkg.netgate.com/pfSense_plus-v24_03_amd64-pfSense_plus_v24_03",
    mirror_type: "srv",
    signature_type: "fingerprints",
    fingerprints: "/usr/local/share/pfSense/keys/pkg",
    enabled: yes
}

Gertjan

@HansWerner3dit said in No packages in available packages list:

the fingerprints entry points to /usr/local/share/pfSense/keys/pkg instead of the trusted subpath /usr/local/share/pfSense/keys/pkg/trusted. could that be an issue?

I'm on 24.11, and see the same thing, so I presume that not an issue.

Your /usr/local/etc/pfSense/pkg/repos looks fine, but I have one file more :
pfSense-repo-0000.release
and it contains one character :

Important detail (that you didn't show) :

[24.11-RELEASE][root@pfSense.bhf.tld]/root: ll  /usr/local/etc/pfSense/pkg/repos/
total 17
-rw-r--r--  1 root wheel 2057 Feb  4 09:34 pfSense-repo-0000-cert.pem
-rw-r--r--  1 root wheel 3243 Feb  4 09:34 pfSense-repo-0000-key.pem
-rw-r--r--  1 root wheel   16 Feb  4 09:34 pfSense-repo-0000.abi
-rw-r--r--  1 root wheel   17 Feb  4 09:34 pfSense-repo-0000.altabi
-rw-r--r--  1 root wheel  506 Feb  4 09:34 pfSense-repo-0000.conf
-rw-r--r--  1 root wheel    1 Feb  4 09:34 pfSense-repo-0000.default
-rw-r--r--  1 root wheel   30 Feb  4 09:34 pfSense-repo-0000.descr
-rw-r--r--  1 root wheel    5 Feb  4 09:34 pfSense-repo-0000.name
-rw-r--r--  1 root wheel    1 Feb  4 09:34 pfSense-repo-0000.release
-rw-r--r--  1 root wheel   12 Feb  4 09:34 pfSense-repo-0000.version

= the files date and time. Mine are all pretty recent, as they get updated when "pkg" syncs with the package server. Last time that happened was today, at 09h34, or about 70 minutes ago.
You see the same time date ?

I don't have the file that start with
pfSense-repo-0001*

Files in /usr/local/share/pfSense/keys/pkg/trusted/
I have the same set of files. Same names, same file size (95 bytes).

HansWerner3dit

@Gertjan said in No packages in available packages list:

Important detail (that you didn't show) :

Sorry, i had them, but my post always got flagged as spam :/

[24.03-RELEASE][admin@pfSense.3dit.local]/usr/local/etc/pfSense/pkg/repos: ls -l /usr/local/etc/pfSense/pkg/repos
total 68
-rw-r--r--  1 root wheel 2053 Feb  4 10:03 pfSense-repo-0000-cert.pem
-rw-r--r--  1 root wheel 3243 Feb  4 10:03 pfSense-repo-0000-key.pem
-rw-r--r--  1 root wheel   16 Feb  4 10:03 pfSense-repo-0000.abi
-rw-r--r--  1 root wheel   17 Feb  4 10:03 pfSense-repo-0000.altabi
-rw-r--r--  1 root wheel  506 Feb  4 10:03 pfSense-repo-0000.conf
-rw-r--r--  1 root wheel    1 Feb  4 10:03 pfSense-repo-0000.default
-rw-r--r--  1 root wheel   31 Feb  4 10:03 pfSense-repo-0000.descr
-rw-r--r--  1 root wheel    9 Feb  4 10:03 pfSense-repo-0000.name
-rw-r--r--  1 root wheel   12 Feb  4 10:03 pfSense-repo-0000.version
-rw-r--r--  1 root wheel 2053 Feb  4 10:03 pfSense-repo-0001-cert.pem
-rw-r--r--  1 root wheel 3243 Feb  4 10:03 pfSense-repo-0001-key.pem
-rw-r--r--  1 root wheel   16 Feb  4 10:03 pfSense-repo-0001.abi
-rw-r--r--  1 root wheel   17 Feb  4 10:03 pfSense-repo-0001.altabi
-rw-r--r--  1 root wheel  506 Feb  4 10:03 pfSense-repo-0001.conf
-rw-r--r--  1 root wheel   30 Feb  4 10:03 pfSense-repo-0001.descr
-rw-r--r--  1 root wheel    5 Feb  4 10:03 pfSense-repo-0001.name
-rw-r--r--  1 root wheel   12 Feb  4 10:03 pfSense-repo-0001.version

i have been moving the 0001 repos out of the folder and created a .release file with content 1 ... but did had no effect when running the pkg -d update command.
furthermore, when changing the release branch, the folder structure was restored (with 0000 and 0001 repos)

in addition, i have been running the fsck command mulltiple times (like in the youtube netgate video shown) ... filesystem should be ok.

Gertjan

@HansWerner3dit said in No packages in available packages list:

i have been moving the 0001 repos out of the folder and created a .release file with content 1 ... but did had no effect when running the pkg -d update command.
furthermore, when changing the release branch, the folder structure was restored (with 0000 and 0001 repos)

The time / date is very recent, like minutes ago. So nothing awkward there.
I guess the "0001" files are there as they indicate the 'future' version ? (check content to be sure)

@HansWerner3dit said in No packages in available packages list:

i have been running the fsck command mulltiple times

From the console, while booting, right ?

stephenw10

Ok that all looks good.

Check:

[24.11-RELEASE][admin@4200.stevew.lan]/root: ls -ls /usr/local/etc/pkg/repos
total 1
1 -rw-r--r--  1 root wheel 25 Nov 21 21:00 FreeBSD.conf
1 lrwxr-xr-x  1 root wheel 55 Jan  7 16:45 pfSense.conf -> /usr/local/etc/pfSense/pkg/repos/pfSense-repo-0002.conf

That symlink is what gets changed when you set a different repo branch.

If it's pointing at /usr/local/share/pfSense/pkg/repos then check that: ls -ls /usr/local/share/pfSense/pkg/repos
That#s the old repo system that would only ever be used now if the system cannot access the dynamic repos.

HansWerner3dit

@Gertjan said in No packages in available packages list:

From the console, while booting, right ?

Unfortunately not yet from the console, but with the restart option to do a filecheck using 5 -> F

I was of the assumption this would run the fsck while booting. if not, i will take a look into fsck'ing in the console while booting.

Netgate pfSense Plus will reboot. This may take a few minutes, depending on your hardware.
Do you want to proceed?

    Y/y: Reboot normally
    R/r: Reroot (Stop processes, remount disks, re-run startup sequence)
    S: Reboot into Single User Mode (requires console access!)
    F: Reboot and run a filesystem check

Gertjan

@HansWerner3dit said in No packages in available packages list:

F: Reboot and run a filesystem check

Never tried that one myself.
A very basic file check is always done before the file system is 'mounted'.
The video shows the thorough one : all files, all folder, the entire thing. Like in the good old DOS (Windows) days. People were (not) educated to 'just flip the power switch whenever these see fit', and this really started to create a huge mess on their disks, everybody started to run CHKDSK /F every morning all day long, and losing files, entire folders or even the entire disk while doing so.
This is one of the reason the batteries in our phones are now soldered in place - we can remove them anymore. A phone has also a very complex file system, and breaking it will break the phone. A power down has to be user initiated, so it can shut down properly.

HansWerner3dit

@stephenw10 very much thanks for all your guidance and ideas.

i have been checking the symlink and it looks like this (not pointing to the old repo system).

[24.03-RELEASE][admin@pfSense.3dit.local]/usr/local/etc/pfSense: ls -ls /usr/local/etc/pkg/repos
total 4
4 -rw-r--r--  1 root wheel 25 Apr 19  2024 FreeBSD.conf
0 lrwxr-xr-x  1 root wheel 55 Feb  4 11:23 pfSense.conf -> /usr/local/etc/pfSense/pkg/repos/pfSense-repo-0001.conf

Gertjan

@HansWerner3dit

The "0001" references what pfSense version ?
Open them up to see whats in it.

HansWerner3dit

@Gertjan said in No packages in available packages list:

The "0001" references what pfSense version ?
Open them up to see whats in it.

it shows 24.11

[24.03-RELEASE][admin@pfSense.3dit.local]/usr/local/etc/pfSense: cat /usr/local/etc/pfSense/pkg/repos/pfSense-repo-0001.conf
FreeBSD: { enabled: no }

pfSense-core: {
    url: "pkg+https://pfsense-plus-pkg.netgate.com/pfSense_plus-v24_11_amd64-core",
    mirror_type: "srv",
    signature_type: "fingerprints",
    fingerprints: "/usr/local/share/pfSense/keys/pkg",
    enabled: yes
}

pfSense: {
    url: "pkg+https://pfsense-plus-pkg.netgate.com/pfSense_plus-v24_11_amd64-pfSense_plus_v24_11",
    mirror_type: "srv",
    signature_type: "fingerprints",
    fingerprints: "/usr/local/share/pfSense/keys/pkg",
    enabled: yes
}

Gertjan

@HansWerner3dit

( ) that matches 100 % with what I have ...

stephenw10

Hmm, yeah that really does all look fine. Hard to imagine how but perhaps the actual key is incorrect:

[24.11-RELEASE][admin@4200.stevew.lan]/root: cat /usr/local/share/pfSense/keys/pkg/trusted/pkg.pfsense.org.20160406 
function: sha256
fingerprint: 2af3a0ccc74829048add462f0fa348aafcaae2d270b775df0a67bd10ed1ad6df

HansWerner3dit

@stephenw10 said in No packages in available packages list:

Hmm, yeah that really does all look fine. Hard to imagine how but perhaps the actual key is incorrect:
[24.11-RELEASE][admin@4200.stevew.lan]/root: cat /usr/local/share/pfSense/keys/pkg/trusted/pkg.pfsense.org.20160406 
function: sha256
fingerprint: 2af3a0ccc74829048add462f0fa348aafcaae2d270b775df0a67bd10ed1ad6df

well well well... that is something.

[24.03-RELEASE][admin@pfSense.3dit.local]/root: cat /usr/local/share/pfSense/keys/pkg/trusted/pkg.pfsense.org.20160406
function: sha256
fingerprint: 30be9cc2e7b2b3ba1ff3b2be1795f3f0719ab9a65322695703dc7b8f004035a8

when using your fingerprint i do get a step further with the pkg -d update command. but i do not want to simply ignore the version mismatch at that point. or is it not an real issue here?

Fetching packagesite.pkg: 100%  192 KiB 196.9kB/s    00:01    
* Connection #0 to host pfsense-plus-pkg01.atx.netgate.com left intact
DBG(1)[70983]> PkgRepo: extracting packagesite.yaml of repo pfSense
DBG(1)[71924]> PkgRepo: extracting signature of repo in a sandbox
DBG(1)[70983]> Pkgrepo, reading new packagesite.yaml for '/var/db/pkg/repo-pfSense.sqlite'
Processing entries:   0%
Newer FreeBSD version for package zabbix7-agent:
To ignore this error set IGNORE_OSVERSION=yes
- package: 1500019
- running kernel: 1500012
Ignore the mismatch and continue? [y/N]: n
pkg: repository pfSense contains packages for wrong OS version: FreeBSD:15:amd64
Processing entries: 100%
Unable to update repository pfSense
Error updating repositories!

using the option 13) Update from console prints the following for the 24.03 branch.

Enter an option: 13

Messages:
Your Netgate device has pfSense+ as part of your device purchase.
libbe_init("") failed.
>>> Updating repositories metadata...
Updating pfSense-core repository catalogue...
Fetching meta.conf: . done
Fetching packagesite.pkg: . done
Processing entries: . done
pfSense-core repository update completed. 5 packages processed.
Updating pfSense repository catalogue...
Fetching meta.conf: . done
Fetching data.pkg: ......... done
Processing entries: .......... done
pfSense repository update completed. 736 packages processed.
All repositories are up to date.
>>> Upgrading pfSense-upgrade...done.
Messages:
Your Netgate device has pfSense+ as part of your device purchase.
>>> Setting vital flag on pfSense-upgrade...done.
libbe_init("") failed.
>>> Updating repositories metadata...
Updating pfSense-core repository catalogue...
Fetching meta.conf: . done
Fetching packagesite.pkg: . done
Processing entries: . done
pfSense-core repository update completed. 5 packages processed.
Updating pfSense repository catalogue...
Fetching meta.conf: . done
Fetching data.pkg: .......... done
Processing entries: .......... done
pfSense repository update completed. 736 packages processed.
All repositories are up to date.
>>> Setting vital flag on pkg...done.
Your packages are up to date
Netgate 5100 - Serial: NG201812003658 - Netgate Device ID: 2a5dafd7126cfa1f6d3c

*** Welcome to Netgate pfSense Plus 24.03-RELEASE (amd64) on pfSense ***

is the libbe_init("") failed message of any concern here?

and finally the Webinterface also shows the 24.11 upgrade

stephenw10

The libbe error is expected if you;re not running ZFS.

I would run pfSense-upgrade -d at the CLI to get full debug output during the upgrade.

Gertjan

@HansWerner3dit said in No packages in available packages list:

is the libbe_init("") failed message of any concern here?

As said above.

I add (not me, the forum et Google search) :
libbe_init("") failed
freebsd libbe_init("") failed

What I make of it : you are not (what ??) using ZFS. And the upgrader presumes you do (lol : a small bug).
So, that's bad and very good news : get your system cleaned up for good : ask TAC for a new recent 24.11 firmware, burn an USB drive (see Etcher pfSense documentation), and install from scratch.
Your issue will be done - promised.
The installer will propose the default ZFS file system, something you really (really! ) want to have.

Sorry .. this means that you have to install again - but hey, if you do this slooooowly, it will take 10 minutes or so, and it's always good to do it at least ones, so you know, in case of absolute stress, what to do.

HansWerner3dit

Thank you really much for your thorough assistance @Gertjan and @stephenw10 ...

The firewall is running again with the latest version.

Regarding the reinstall using zfs, i would want to first setup a virtual machine and have a look at the "restore" process for the current configuration.
I am not skilled enough to restore the current setup from scratch [Wireguard, WAN Backup, Gateways, VLAN, DHCP6 but no DHCP4 (managed by another server instance), firewall rules and aliases,...]. We got this box few years ago pre-configured by a certified company...

TL;DR for anyone having the same issue when trying to upgrade.

DBG(1)[95168]> PkgRepo: extracting signature of repo in a sandbox
pkg: No trusted public keys found
Unable to update repository pfSense

Check the public key / fingerprint -- mine was somehow altered (reason unknown).

[24.11-RELEASE][admin@4200.stevew.lan]/root: cat /usr/local/share/pfSense/keys/pkg/trusted/pkg.pfsense.org.20160406 
function: sha256
fingerprint: 2af3a0ccc74829048add462f0fa348aafcaae2d270b775df0a67bd10ed1ad6df

Gertjan

@HansWerner3dit said in No packages in available packages list:

I am not skilled enough to restore the current setup from scratch [Wireguard, WAN Backup, Gateways, VLAN, DHCP6

You are ^^ and it's easy to understand why.

Always keep a recent copy of the pfSense config : Diagnostics > Backup & Restore
When you re create or re install pfSense on the same device (same hardware, or to be more precise : same network interface NICs) then all you have to do is : import the pfSense config (same place Diagnostics > Backup & Restore), and your setup is done.

Example, one or two versions ago, the filesystem changed. This means the partitions on the drive being used have to be altered. A simple GUI or console 'upgrade' can't do this.
You need to get the firmware on an USB drive and see pfSense Documentation, look for the word 'Etcher'. Also find the doc about installing pfSense with 'ZFS'.
When pfSense starts, do a minimal WAN/LAN manual config.
Ones in the GUI, import your backed up config.
Done.