No packages in available packages list

HansWerner3dit

@stephenw10 no worries ;)

regarding the pkg -d update: it is exactly the same error for both branches.

see the diff for both commands on different branches

https://www.diffchecker.com/k3NqSkre/

Gertjan

@HansWerner3dit said in No packages in available packages list:

https://www.diffchecker.com/k3NqSkre/

Actually, everything look fine, except for the last 4,5 lines.

DBG(1)[95168]> PkgRepo: extracting signature of repo in a sandbox
pkg: No trusted public keys found
Unable to update repository pfSense

signatures are extract to a temporary location (before used).
What if this temporary location (aka sandbox) couldn't be created because of file system errors ?

Free disk space ok ?
File system ok ? See Netgate Youtube channel, there is a video that shopws you how to test the file system.

stephenw10

Check: /usr/local/share/pfSense/keys/pkg/trusted/
Do you see keys present?

Check: /usr/local/etc/pfSense/pkg/repos
Do you see entries for the available branches?

HansWerner3dit

@Gertjan said in No packages in available packages list:

Free disk space ok ?

space usage should not be a problem except the devfs (/dev) one?

[24.03-RELEASE][admin@pfSense.3dit.local]/root: df -h
Filesystem                     Size    Used   Avail Capacity  Mounted on
/dev/ufsid/5e9a0bc1bef83450    6.7G    2.5G    3.7G    40%    /
devfs                          1.0K      0B    1.0K     0%    /dev
tmpfs                           60M    288K     60M     0%    /tmp
tmpfs                          100M     16M     84M    16%    /var

HansWerner3dit

@stephenw10 said in No packages in available packages list:

Check: /usr/local/share/pfSense/keys/pkg/trusted/
Do you see keys present?

Yes, there are a few keys in there. but are those ok'ish?

[24.03-RELEASE][admin@pfSense.3dit.local]/root: ls -la /usr/local/share/pfSense/keys/pkg/trusted/
total 24
drwxr-xr-x  2 root wheel 512 Jan 13 12:06 .
drwxr-xr-x  4 root wheel 512 Apr 19  2024 ..
-rw-r--r--  1 root wheel  95 Apr 19  2024 beta.pfsense.org.20151223
-rw-r--r--  1 root wheel  95 Apr 19  2024 ce-beta.pfsense.org.20151223
-rw-r--r--  1 root wheel  95 Apr 19  2024 ce-pkg.pfsense.org.20160406
-rw-r--r--  1 root wheel  95 Jan 13 12:06 pkg.pfsense.org.20160406

HansWerner3dit

@stephenw10 said in No packages in available packages list:

Check: /usr/local/etc/pfSense/pkg/repos
Do you see entries for the available branches?

i do see some repository files in there...

[24.03-RELEASE][admin@pfSense.3dit.local]/usr/local/etc/pfSense/pkg/repos: ls /usr/local/etc/pfSense/pkg/repos
pfSense-repo-0000-cert.pem pfSense-repo-0000.default  pfSense-repo-0001-key.pem  pfSense-repo-0001.name
pfSense-repo-0000-key.pem  pfSense-repo-0000.descr    pfSense-repo-0001.abi      pfSense-repo-0001.version
pfSense-repo-0000.abi      pfSense-repo-0000.name     pfSense-repo-0001.altabi
pfSense-repo-0000.altabi   pfSense-repo-0000.version  pfSense-repo-0001.conf
pfSense-repo-0000.conf     pfSense-repo-0001-cert.pem pfSense-repo-0001.descr

the fingerprints entry points to /usr/local/share/pfSense/keys/pkg instead of the trusted subpath /usr/local/share/pfSense/keys/pkg/trusted. could that be an issue?

[24.03-RELEASE][admin@pfSense.3dit.local]/usr/local/etc/pfSense/pkg/repos: cat pfSense-repo-0000.conf 
FreeBSD: { enabled: no }

pfSense-core: {
    url: "pkg+https://pfsense-plus-pkg.netgate.com/pfSense_plus-v24_03_amd64-core",
    mirror_type: "srv",
    signature_type: "fingerprints",
    fingerprints: "/usr/local/share/pfSense/keys/pkg",
    enabled: yes
}

pfSense: {
    url: "pkg+https://pfsense-plus-pkg.netgate.com/pfSense_plus-v24_03_amd64-pfSense_plus_v24_03",
    mirror_type: "srv",
    signature_type: "fingerprints",
    fingerprints: "/usr/local/share/pfSense/keys/pkg",
    enabled: yes
}

Gertjan

@HansWerner3dit said in No packages in available packages list:

the fingerprints entry points to /usr/local/share/pfSense/keys/pkg instead of the trusted subpath /usr/local/share/pfSense/keys/pkg/trusted. could that be an issue?

I'm on 24.11, and see the same thing, so I presume that not an issue.

Your /usr/local/etc/pfSense/pkg/repos looks fine, but I have one file more :
pfSense-repo-0000.release
and it contains one character :

Important detail (that you didn't show) :

[24.11-RELEASE][root@pfSense.bhf.tld]/root: ll  /usr/local/etc/pfSense/pkg/repos/
total 17
-rw-r--r--  1 root wheel 2057 Feb  4 09:34 pfSense-repo-0000-cert.pem
-rw-r--r--  1 root wheel 3243 Feb  4 09:34 pfSense-repo-0000-key.pem
-rw-r--r--  1 root wheel   16 Feb  4 09:34 pfSense-repo-0000.abi
-rw-r--r--  1 root wheel   17 Feb  4 09:34 pfSense-repo-0000.altabi
-rw-r--r--  1 root wheel  506 Feb  4 09:34 pfSense-repo-0000.conf
-rw-r--r--  1 root wheel    1 Feb  4 09:34 pfSense-repo-0000.default
-rw-r--r--  1 root wheel   30 Feb  4 09:34 pfSense-repo-0000.descr
-rw-r--r--  1 root wheel    5 Feb  4 09:34 pfSense-repo-0000.name
-rw-r--r--  1 root wheel    1 Feb  4 09:34 pfSense-repo-0000.release
-rw-r--r--  1 root wheel   12 Feb  4 09:34 pfSense-repo-0000.version

= the files date and time. Mine are all pretty recent, as they get updated when "pkg" syncs with the package server. Last time that happened was today, at 09h34, or about 70 minutes ago.
You see the same time date ?

I don't have the file that start with
pfSense-repo-0001*

Files in /usr/local/share/pfSense/keys/pkg/trusted/
I have the same set of files. Same names, same file size (95 bytes).

HansWerner3dit

@Gertjan said in No packages in available packages list:

Important detail (that you didn't show) :

Sorry, i had them, but my post always got flagged as spam :/

[24.03-RELEASE][admin@pfSense.3dit.local]/usr/local/etc/pfSense/pkg/repos: ls -l /usr/local/etc/pfSense/pkg/repos
total 68
-rw-r--r--  1 root wheel 2053 Feb  4 10:03 pfSense-repo-0000-cert.pem
-rw-r--r--  1 root wheel 3243 Feb  4 10:03 pfSense-repo-0000-key.pem
-rw-r--r--  1 root wheel   16 Feb  4 10:03 pfSense-repo-0000.abi
-rw-r--r--  1 root wheel   17 Feb  4 10:03 pfSense-repo-0000.altabi
-rw-r--r--  1 root wheel  506 Feb  4 10:03 pfSense-repo-0000.conf
-rw-r--r--  1 root wheel    1 Feb  4 10:03 pfSense-repo-0000.default
-rw-r--r--  1 root wheel   31 Feb  4 10:03 pfSense-repo-0000.descr
-rw-r--r--  1 root wheel    9 Feb  4 10:03 pfSense-repo-0000.name
-rw-r--r--  1 root wheel   12 Feb  4 10:03 pfSense-repo-0000.version
-rw-r--r--  1 root wheel 2053 Feb  4 10:03 pfSense-repo-0001-cert.pem
-rw-r--r--  1 root wheel 3243 Feb  4 10:03 pfSense-repo-0001-key.pem
-rw-r--r--  1 root wheel   16 Feb  4 10:03 pfSense-repo-0001.abi
-rw-r--r--  1 root wheel   17 Feb  4 10:03 pfSense-repo-0001.altabi
-rw-r--r--  1 root wheel  506 Feb  4 10:03 pfSense-repo-0001.conf
-rw-r--r--  1 root wheel   30 Feb  4 10:03 pfSense-repo-0001.descr
-rw-r--r--  1 root wheel    5 Feb  4 10:03 pfSense-repo-0001.name
-rw-r--r--  1 root wheel   12 Feb  4 10:03 pfSense-repo-0001.version

i have been moving the 0001 repos out of the folder and created a .release file with content 1 ... but did had no effect when running the pkg -d update command.
furthermore, when changing the release branch, the folder structure was restored (with 0000 and 0001 repos)

in addition, i have been running the fsck command mulltiple times (like in the youtube netgate video shown) ... filesystem should be ok.

Gertjan

@HansWerner3dit said in No packages in available packages list:

i have been moving the 0001 repos out of the folder and created a .release file with content 1 ... but did had no effect when running the pkg -d update command.
furthermore, when changing the release branch, the folder structure was restored (with 0000 and 0001 repos)

The time / date is very recent, like minutes ago. So nothing awkward there.
I guess the "0001" files are there as they indicate the 'future' version ? (check content to be sure)

@HansWerner3dit said in No packages in available packages list:

i have been running the fsck command mulltiple times

From the console, while booting, right ?

stephenw10

Ok that all looks good.

Check:

[24.11-RELEASE][admin@4200.stevew.lan]/root: ls -ls /usr/local/etc/pkg/repos
total 1
1 -rw-r--r--  1 root wheel 25 Nov 21 21:00 FreeBSD.conf
1 lrwxr-xr-x  1 root wheel 55 Jan  7 16:45 pfSense.conf -> /usr/local/etc/pfSense/pkg/repos/pfSense-repo-0002.conf

That symlink is what gets changed when you set a different repo branch.

If it's pointing at /usr/local/share/pfSense/pkg/repos then check that: ls -ls /usr/local/share/pfSense/pkg/repos
That#s the old repo system that would only ever be used now if the system cannot access the dynamic repos.

HansWerner3dit

@Gertjan said in No packages in available packages list:

From the console, while booting, right ?

Unfortunately not yet from the console, but with the restart option to do a filecheck using 5 -> F

I was of the assumption this would run the fsck while booting. if not, i will take a look into fsck'ing in the console while booting.

Netgate pfSense Plus will reboot. This may take a few minutes, depending on your hardware.
Do you want to proceed?

    Y/y: Reboot normally
    R/r: Reroot (Stop processes, remount disks, re-run startup sequence)
    S: Reboot into Single User Mode (requires console access!)
    F: Reboot and run a filesystem check

Gertjan

@HansWerner3dit said in No packages in available packages list:

F: Reboot and run a filesystem check

Never tried that one myself.
A very basic file check is always done before the file system is 'mounted'.
The video shows the thorough one : all files, all folder, the entire thing. Like in the good old DOS (Windows) days. People were (not) educated to 'just flip the power switch whenever these see fit', and this really started to create a huge mess on their disks, everybody started to run CHKDSK /F every morning all day long, and losing files, entire folders or even the entire disk while doing so.
This is one of the reason the batteries in our phones are now soldered in place - we can remove them anymore. A phone has also a very complex file system, and breaking it will break the phone. A power down has to be user initiated, so it can shut down properly.

HansWerner3dit

@stephenw10 very much thanks for all your guidance and ideas.

i have been checking the symlink and it looks like this (not pointing to the old repo system).

[24.03-RELEASE][admin@pfSense.3dit.local]/usr/local/etc/pfSense: ls -ls /usr/local/etc/pkg/repos
total 4
4 -rw-r--r--  1 root wheel 25 Apr 19  2024 FreeBSD.conf
0 lrwxr-xr-x  1 root wheel 55 Feb  4 11:23 pfSense.conf -> /usr/local/etc/pfSense/pkg/repos/pfSense-repo-0001.conf

Gertjan

@HansWerner3dit

The "0001" references what pfSense version ?
Open them up to see whats in it.

HansWerner3dit

@Gertjan said in No packages in available packages list:

The "0001" references what pfSense version ?
Open them up to see whats in it.

it shows 24.11

[24.03-RELEASE][admin@pfSense.3dit.local]/usr/local/etc/pfSense: cat /usr/local/etc/pfSense/pkg/repos/pfSense-repo-0001.conf
FreeBSD: { enabled: no }

pfSense-core: {
    url: "pkg+https://pfsense-plus-pkg.netgate.com/pfSense_plus-v24_11_amd64-core",
    mirror_type: "srv",
    signature_type: "fingerprints",
    fingerprints: "/usr/local/share/pfSense/keys/pkg",
    enabled: yes
}

pfSense: {
    url: "pkg+https://pfsense-plus-pkg.netgate.com/pfSense_plus-v24_11_amd64-pfSense_plus_v24_11",
    mirror_type: "srv",
    signature_type: "fingerprints",
    fingerprints: "/usr/local/share/pfSense/keys/pkg",
    enabled: yes
}

Gertjan

@HansWerner3dit

( ) that matches 100 % with what I have ...

stephenw10

Hmm, yeah that really does all look fine. Hard to imagine how but perhaps the actual key is incorrect:

[24.11-RELEASE][admin@4200.stevew.lan]/root: cat /usr/local/share/pfSense/keys/pkg/trusted/pkg.pfsense.org.20160406 
function: sha256
fingerprint: 2af3a0ccc74829048add462f0fa348aafcaae2d270b775df0a67bd10ed1ad6df

HansWerner3dit

@stephenw10 said in No packages in available packages list:

Hmm, yeah that really does all look fine. Hard to imagine how but perhaps the actual key is incorrect:
[24.11-RELEASE][admin@4200.stevew.lan]/root: cat /usr/local/share/pfSense/keys/pkg/trusted/pkg.pfsense.org.20160406 
function: sha256
fingerprint: 2af3a0ccc74829048add462f0fa348aafcaae2d270b775df0a67bd10ed1ad6df

well well well... that is something.

[24.03-RELEASE][admin@pfSense.3dit.local]/root: cat /usr/local/share/pfSense/keys/pkg/trusted/pkg.pfsense.org.20160406
function: sha256
fingerprint: 30be9cc2e7b2b3ba1ff3b2be1795f3f0719ab9a65322695703dc7b8f004035a8

when using your fingerprint i do get a step further with the pkg -d update command. but i do not want to simply ignore the version mismatch at that point. or is it not an real issue here?

Fetching packagesite.pkg: 100%  192 KiB 196.9kB/s    00:01    
* Connection #0 to host pfsense-plus-pkg01.atx.netgate.com left intact
DBG(1)[70983]> PkgRepo: extracting packagesite.yaml of repo pfSense
DBG(1)[71924]> PkgRepo: extracting signature of repo in a sandbox
DBG(1)[70983]> Pkgrepo, reading new packagesite.yaml for '/var/db/pkg/repo-pfSense.sqlite'
Processing entries:   0%
Newer FreeBSD version for package zabbix7-agent:
To ignore this error set IGNORE_OSVERSION=yes
- package: 1500019
- running kernel: 1500012
Ignore the mismatch and continue? [y/N]: n
pkg: repository pfSense contains packages for wrong OS version: FreeBSD:15:amd64
Processing entries: 100%
Unable to update repository pfSense
Error updating repositories!

using the option 13) Update from console prints the following for the 24.03 branch.

Enter an option: 13

Messages:
Your Netgate device has pfSense+ as part of your device purchase.
libbe_init("") failed.
>>> Updating repositories metadata...
Updating pfSense-core repository catalogue...
Fetching meta.conf: . done
Fetching packagesite.pkg: . done
Processing entries: . done
pfSense-core repository update completed. 5 packages processed.
Updating pfSense repository catalogue...
Fetching meta.conf: . done
Fetching data.pkg: ......... done
Processing entries: .......... done
pfSense repository update completed. 736 packages processed.
All repositories are up to date.
>>> Upgrading pfSense-upgrade...done.
Messages:
Your Netgate device has pfSense+ as part of your device purchase.
>>> Setting vital flag on pfSense-upgrade...done.
libbe_init("") failed.
>>> Updating repositories metadata...
Updating pfSense-core repository catalogue...
Fetching meta.conf: . done
Fetching packagesite.pkg: . done
Processing entries: . done
pfSense-core repository update completed. 5 packages processed.
Updating pfSense repository catalogue...
Fetching meta.conf: . done
Fetching data.pkg: .......... done
Processing entries: .......... done
pfSense repository update completed. 736 packages processed.
All repositories are up to date.
>>> Setting vital flag on pkg...done.
Your packages are up to date
Netgate 5100 - Serial: NG201812003658 - Netgate Device ID: 2a5dafd7126cfa1f6d3c

*** Welcome to Netgate pfSense Plus 24.03-RELEASE (amd64) on pfSense ***

is the libbe_init("") failed message of any concern here?

and finally the Webinterface also shows the 24.11 upgrade

stephenw10

The libbe error is expected if you;re not running ZFS.

I would run pfSense-upgrade -d at the CLI to get full debug output during the upgrade.

Gertjan

@HansWerner3dit said in No packages in available packages list:

is the libbe_init("") failed message of any concern here?

As said above.

I add (not me, the forum et Google search) :
libbe_init("") failed
freebsd libbe_init("") failed

What I make of it : you are not (what ??) using ZFS. And the upgrader presumes you do (lol : a small bug).
So, that's bad and very good news : get your system cleaned up for good : ask TAC for a new recent 24.11 firmware, burn an USB drive (see Etcher pfSense documentation), and install from scratch.
Your issue will be done - promised.
The installer will propose the default ZFS file system, something you really (really! ) want to have.

Sorry .. this means that you have to install again - but hey, if you do this slooooowly, it will take 10 minutes or so, and it's always good to do it at least ones, so you know, in case of absolute stress, what to do.