Empty field "Firmware Branch" in <System/Update/Update Settings> after a fresh install v2.7.2

pfthbst · Dec 23, 2023, 8:57 PM

Hi pfSense Experts!

I get a blank "Firmware Branch" field in System/Update/Update Settings after a fresh install v2.7.2.

I see error messages from pfSense-repoc in System/Update/System Update:

Messages
pfSense-repoc: si_get_packages: failed to run the pkg info command: /usr/local/sbin/pkg-static info -R --raw-format json-compact pfSense-pkg-* 2>&1 pfSense-repoc: no pfSense packages installed pfSense-repoc: failed to fetch the repo data

Current Base System 2.7.2
Latest Base System 2.7.2
StatusUp to date.

Available Packages do not show any packages as a result.

My previous config v2.6.0 has restored very well, exept for apcups, radius, suricata, vnstat.

Help solve this problem please, I'm stuck in frustration and many hours trying to find solution.

The following are the output messages of some commands:

$ ls -l /var/db/pkg/
total 4794
-rw-r--r-- 1 root wheel 8994816 Dec 23 20:37 local.sqlite
-rw-r--r-- 1 root wheel 158 Dec 6 23:12 pfSense.meta
-rw-r--r-- 1 root wheel 1261568 Dec 6 23:12 repo-pfSense.sqlite
-rw-r--r-- 1 root wheel 0 Dec 6 23:12 repo-pfSense.sqlite-journal

$ /usr/local/sbin/pkg-static update -f
pkg-static: Unable to open '/usr/local/etc/pkg/repos//pfSense.conf':No such file or directory
No active remote repositories configured.

$ /usr/local/sbin/pkg-static install -yf pkg pfSense pfSense-kernel-pfSense
pkg-static: Unable to open '/usr/local/etc/pkg/repos//pfSense.conf':No such file or directory
pkg-static: No packages available to install matching 'pfSense' have been found in the repositories
pkg-static: No packages available to install matching 'pfSense-kernel-pfSense' have been found in the repositories

$ /etc/rc.php_ini_setup
pkg-static: Unable to open '/usr/local/etc/pkg/repos//pfSense.conf':No such file or directory
pkg-static: Unable to open '/usr/local/etc/pkg/repos//pfSense.conf':No such file or directory
pkg-static: Unable to open '/usr/local/etc/pkg/repos//pfSense.conf':No such file or directory
pkg-static: Unable to open '/usr/local/etc/pkg/repos//pfSense.conf':No such file or directory
15475
[: NaN: bad number

$ pkg-static clean -ay; pkg-static install -fy pkg pfSense-repo pfSense-upgrade
pkg-static: Unable to open '/usr/local/etc/pkg/repos//pfSense.conf':No such file or directory
pkg-static: No active remote repositories configured
pkg-static: Unable to open '/usr/local/etc/pkg/repos//pfSense.conf':No such file or directory
Checking integrity... done (0 conflicting)
The most recent versions of packages are already installed

$ pkg info -x pfSense
pkg: Unable to open '/usr/local/etc/pkg/repos//pfSense.conf':No such file or directory
pfSense-2.7.2
pfSense-Status_Monitoring-php82-1.8_3
pfSense-base-2.7.2
pfSense-boot-2.7.2
pfSense-composer-deps-0.1
pfSense-default-config-2.7.2
pfSense-kernel-pfSense-2.7.2
pfSense-repo-2.7.2
pfSense-repoc-20230912
pfSense-upgrade-1.2.1
php82-pfSense-module-0.95

$ pfSense-repoc
pfSense-repoc: si_get_packages: failed to run the pkg info command: /usr/local/sbin/pkg-static info -R --raw-format json-compact pfSense-pkg-* 2>&1
pfSense-repoc: no pfSense packages installed
pfSense-repoc: failed to fetch the repo data
failed to read the repo data.

$ pkg upgrade pfSense-repoc
pkg: Unable to open '/usr/local/etc/pkg/repos//pfSense.conf':No such file or directory
No active remote repositories configured.

$ pkg-static -d update
DBG(1)[60573]> pkg initialized
pkg-static: Unable to open '/usr/local/etc/pkg/repos//pfSense.conf':No such file or directory
No active remote repositories configured.

$ /usr/local/sbin/pkg-static info -R --raw-format json-compact pfSense-pkg-*
pkg-static: Unable to open '/usr/local/etc/pkg/repos//pfSense.conf':No such file or directory
pkg-static: No package(s) matching pfSense-pkg-*

$ pkg update -f
pkg: Unable to open '/usr/local/etc/pkg/repos//pfSense.conf':No such file or directory
No active remote repositories configured.

$ ls -l /usr/local/etc/pkg/repos/
total 1
-rw-r--r-- 1 root wheel 26 Dec 6 22:09 FreeBSD.conf
lrwxr-xr-x 1 root wheel 19 Dec 23 15:10 pfSense.conf -> /pfSense-repo-.conf

$ cat /usr/local/share/pfSense/pkg/repos/pfSense-repo.conf
FreeBSD: { enabled: no }

pfSense-core: {
url: "pkg+https://pkg.pfsense.org/pfSense_v2_7_2_amd64-core",
mirror_type: "srv",
signature_type: "fingerprints",
fingerprints: "/usr/local/share/pfSense/keys/pkg",
enabled: yes
}

pfSense: {
url: "pkg+https://pkg.pfsense.org/pfSense_v2_7_2_amd64-pfSense_v2_7_2",
mirror_type: "srv",
signature_type: "fingerprints",
fingerprints: "/usr/local/share/pfSense/keys/pkg",
enabled: yes
}

stephenw10 · Dec 25, 2023, 2:15 AM

@pfthbst said in Empty field "Firmware Branch" in <System/Update/Update Settings> after a fresh install v2.7.2:

pkg-static: Unable to open '/usr/local/etc/pkg/repos//pfSense.conf':No such file or directory

If you see that go to Sys > Update > Update Settings and resave the 2.7.2 branch. Then retest.

pfthbst · Dec 25, 2023, 11:26 AM

@stephenw10
Thank you for your response!

In the System/Update/Update Settings, I see an empty "Branch" field and cannot select anything.
It has nothing:

pfthbst · Dec 25, 2023, 4:44 PM

Sorry for my English.
2.5.2 was the last version with normal upgrade.

I am using pfSense for my family network, many thanks to all the pfSense developers.
For about 7-8 years I've been adding, changing hardware, and tweaking the config.

Right now I have multiple static ip addresses, several vlans, vpn and ipsec tunnels, smart UPS, wifi controller on Linux.
Now I have electrical problems very often, so communicating UPS with pfSense is just a vital necessity.
Rebuilding a config from scratch is a huge problem right now for me.
That's why I'm periodically trying and trying to upgrade.

But..
Upgrading from 2.5.2 to 2.6.0 has successfully failed from cli and from web.
Upgrading from 2.5.2 to 2.7.0 has successfully failed from cli and from web.

Freshly installed 2.7.0 failed to restore the 2.5.2 config.

I then did a fresh install of 2.6.0 and successfully restored the 2.5.2 config.

Then the upgrade from 2.6.0 to 2.7.0 has successfully failed from cli and from web.

All this time I have been trying to restore the full config with RRD and extra data included.
All this time I've been having some kind of problem with the pfSense repo.

I was thinking maybe it's a problem with some packages?

I installed a fresh version 2.6.0, successfully restored the config, removed all packages, made the config backup with RRD and extra data included.
Freshly installed 2.7.2 failed to restore the config with removed packages:

I'm now on fresh 2.7.2 with a partially restored 2.6.0 config and a pfSense-repo issue.

pfthbst · Dec 25, 2023, 11:31 AM

I did:

cp /usr/local/share/pfSense/pkg/repos/pfSense-repo.conf /usr/local/etc/pkg/repos/pfSense.conf

Then:

pkg-static bootstrap -f

pkg(8) is already installed. Forcing reinstallation through pkg(7).
The package management tool is not yet installed on your system.
Do you want to fetch and install it now? [y/N]: y
Bootstrapping pkg from pkg+https://pkg.pfsense.org/pfSense_v2_7_2_amd64-pfSense_v2_7_2, please wait...
Verifying signature with trusted certificate pkg.pfsense.org.20160406... done
Installing pkg-1.20.8_3...
package pkg is already installed, forced install
Extracting pkg-1.20.8_3: 100%
Scanning /usr/share/certs/untrusted for certificates...
Scanning /usr/share/certs/trusted for certificates...
Scanning /usr/local/share/certs for certificates..


pkg update

Updating pfSense-core repository catalogue...
Fetching meta.conf: 100%    163 B   0.2kB/s    00:01    
Fetching packagesite.pkg: 100%    1 KiB   1.5kB/s    00:01    
Processing entries: 100%
pfSense-core repository update completed. 4 packages processed.
Updating pfSense repository catalogue...
pkg: Repository pfSense has a wrong packagesite, need to re-create database
Fetching meta.conf: 100%    163 B   0.2kB/s    00:01    
Fetching packagesite.pkg: 100%  157 KiB 160.7kB/s    00:01    
Processing entries: 100%
pfSense repository update completed. 549 packages processed.
All repositories are up to date.


pkg-static clean -ay; pkg-static install -fy pkg pfSense-repo pfSense-upgrade

pkg-static: Invalid ABI
pkg-static: Warning: Major OS version upgrade detected.  Running "pkg bootstrap -f" recommended
The following package files will be deleted:
	/var/cache/pkg/pfSense-repo-2.7.2~3aa9dedf41.pkg
	/var/cache/pkg/pfSense-upgrade-1.2.1~7f3940b016.pkg
	/var/cache/pkg/pfSense-upgrade-1.2.1.pkg
	/var/cache/pkg/pkg-1.20.8_3~48cca6e983.pkg
	/var/cache/pkg/pkg-1.20.8_3.pkg
	/var/cache/pkg/pfSense-repo-2.7.2.pkg
The cleanup will free 11 MiB
Deleting files: 100%
All done
pkg-static: Invalid ABI
pkg-static: Warning: Major OS version upgrade detected.  Running "pkg bootstrap -f" recommended
Updating pfSense-core repository catalogue...
Fetching meta.conf:   0%
Fetching packagesite.pkg:   0%
pfSense-core repository is up to date.
Updating pfSense repository catalogue...
Fetching meta.conf:   0%
Fetching packagesite.pkg:   0%
pfSense repository is up to date.
All repositories are up to date.
Updating database digests format: 100%      1 B   0.0kB/s    00:01    
The following 3 package(s) will be affected (of 0 checked):

Installed packages to be REINSTALLED:
	pfSense-repo-2.7.2 [pfSense]
	pfSense-upgrade-1.2.1 [pfSense]
	pkg-1.20.8_3 [pfSense]

Number of packages to be reinstalled: 3

11 MiB to be downloaded.
[1/3] Fetching pfSense-repo-2.7.2.pkg: 100%    6 KiB   5.9kB/s    00:01    
[2/3] Fetching pkg-1.20.8_3.pkg: 100%   10 MiB   1.8MB/s    00:06    
[3/3] Fetching pfSense-upgrade-1.2.1.pkg: 100%   20 KiB  20.2kB/s    00:01    
Checking integrity... done (0 conflicting)
[1/3] Reinstalling pfSense-repo-2.7.2...
[1/3] Extracting pfSense-repo-2.7.2: 100%
cat: /pfSense-repo-.abi: No such file or directory
cat: /pfSense-repo-.altabi: No such file or directory
[2/3] Reinstalling pkg-1.20.8_3...
[2/3] Extracting pkg-1.20.8_3: 100%
Scanning /usr/share/certs/untrusted for certificates...
Scanning /usr/share/certs/trusted for certificates...
Scanning /usr/local/share/certs for certificates...
[3/3] Reinstalling pfSense-upgrade-1.2.1...
[3/3] Extracting pfSense-upgrade-1.2.1: 100%
You may need to manually remove /usr/local/etc/pkg.conf if it is no longer needed.

pfSense-upgrade -dc

pfSense-repoc-static: si_get_packages: failed to run the pkg info command: /usr/local/sbin/pkg-static info -R --raw-format json-compact pfSense-pkg-\* 2>&1
pfSense-repoc-static: no pfSense packages installed
pfSense-repoc-static: failed to fetch the repo data
failed to read the repo data.
failed to update the repository settings!!!
failed to update the repository settings!!!

pfthbst · Dec 25, 2023, 11:36 AM

Now Available Packages is shows all packages:

But I can't install anything.

I got errors:
pfSense-pkg-apcupsd installation failed!
Another instance of pfSense-upgrade is running. Try again later

System/Update/System Update is still showing an empty "Branch" field, with error message:

pfSense-repoc: si_get_packages: failed to run the pkg info command: /usr/local/sbin/pkg-static info -R --raw-format json-compact pfSense-pkg-* 2>&1 pfSense-repoc: no pfSense packages installed pfSense-repoc: failed to fetch the repo data

Any advice would be very helpful!

Merry Christmas to all the pfSense guys and their families!

stephenw10 · Dec 25, 2023, 4:44 PM

@pfthbst said in Empty field "Firmware Branch" in <System/Update/Update Settings> after a fresh install v2.7.2:

I installed a fresh version 2.6.0, successfully restored the config, removed all packages, made the config backup with RRD and extra data included.
Freshly installed 2.7.2 failed to restore the config with removed packages:

That looks like this: https://redmine.pfsense.org/issues/13132
That should be fixed but it's possible you found some variant on that. However you should be able to manually remove the rogue sshdata section from the config and then import that to 2.7.2.

pfthbst · Dec 26, 2023, 9:30 AM

@stephenw10
Thanks for the advice!

I removed redundant <sshdata> section from 2.6.0 no-packages config.

The fresh 2.7.2 successfully restored this config, but <System/Update/System Update> still has an empty "Branch" field.
With a new error message:
pfSense-repoc: failed to fetch the repo data

Available Packages still shows no packages available.

Now pfSense-repoc does not display messages:

pfSense-repoc: si_get_packages: failed to run the pkg info command: /usr/local/sbin/pkg-static info -R --raw-format json-compact pfSense-pkg-* 2>&1
pfSense-repoc: no pfSense packages installed

But it's still showing errors:

pfSense-repoc

pfSense-repoc: failed to fetch the repo data
failed to read the repo data.

pfthbst · Dec 26, 2023, 9:37 AM

I have a noname box with an Intel CPU.
I've noticed a couple things in the system.log

Dec 26 09:32:54 pfSense kernel: netgate0: <unknown hardware>
Strange things are happening with timestamps in the system.log. The Boot sequence starts with my actual UTC/GMT +2 timestamp, then the UTC/GMT timestamp appears:

Dec 26 09:32:54 pfSense syslogd: kernel boot file is /boot/kernel/kernel
Dec 26 09:32:54 pfSense kernel: ---<<BOOT>>---
..
Dec 26 07:38:53 pfSense php-fpm[48596]: /pkg_mgr.php: The command '/usr/local/sbin/pfSense-repo-setup' returned exit code '1', the output was 'pfSense-repoc-static: failed to fetch the repo data failed to read the repo data. failed to update the repository settings!!!'

Could this have any effect on the pfSense repo problems?

pfthbst · Dec 26, 2023, 10:30 AM

I did:
pkg-static bootstrap -f
pkg update
pkg-static clean -ay; pkg-static install -fy pkg pfSense-repo pfSense-upgrade

pkg-static bootstrap -f

pkg(8) is already installed. Forcing reinstallation through pkg(7).
The package management tool is not yet installed on your system.
Do you want to fetch and install it now? [y/N]: y
Bootstrapping pkg from pkg+https://pkg.pfsense.org/pfSense_v2_7_2_amd64-pfSense_v2_7_2, please wait...
Verifying signature with trusted certificate pkg.pfsense.org.20160406.pkgsave... done
Installing pkg-1.20.8_3...
package pkg is already installed, forced install
Extracting pkg-1.20.8_3: 100%
Scanning /usr/share/certs/untrusted for certificates...
Scanning /usr/share/certs/trusted for certificates...
Scanning /usr/local/share/certs for certificates...


pkg update

Updating pfSense-core repository catalogue...
Fetching meta.conf: 100%    163 B   0.2kB/s    00:01    
Fetching packagesite.pkg: 100%    1 KiB   1.5kB/s    00:01    
Processing entries: 100%
pfSense-core repository update completed. 4 packages processed.
Updating pfSense repository catalogue...
pkg: Repository pfSense has a wrong packagesite, need to re-create database
Fetching meta.conf: 100%    163 B   0.2kB/s    00:01    
Fetching packagesite.pkg: 100%  157 KiB 160.7kB/s    00:01    
Processing entries: 100%
pfSense repository update completed. 549 packages processed.
All repositories are up to date.


pkg-static clean -ay; pkg-static install -fy pkg pfSense-repo pfSense-upgrade

Nothing to do.
Updating pfSense-core repository catalogue...
Fetching meta.conf:   0%
Fetching packagesite.pkg:   0%
pfSense-core repository is up to date.
Updating pfSense repository catalogue...
Fetching meta.conf:   0%
Fetching packagesite.pkg:   0%
pfSense repository is up to date.
All repositories are up to date.
Updating database digests format: 100%      1 B   0.0kB/s    00:01    
The following 3 package(s) will be affected (of 0 checked):

Installed packages to be REINSTALLED:
	pfSense-repo-2.7.2 [pfSense]
	pfSense-upgrade-1.2.1 [pfSense]
	pkg-1.20.8_3 [pfSense]

Number of packages to be reinstalled: 3

11 MiB to be downloaded.
[1/3] Fetching pfSense-repo-2.7.2.pkg: 100%    6 KiB   5.9kB/s    00:01    
[2/3] Fetching pkg-1.20.8_3.pkg: 100%   10 MiB   1.4MB/s    00:08    
[3/3] Fetching pfSense-upgrade-1.2.1.pkg: 100%   20 KiB  20.2kB/s    00:01    
Checking integrity... done (0 conflicting)
[1/3] Reinstalling pfSense-repo-2.7.2...
[1/3] Extracting pfSense-repo-2.7.2: 100%
[2/3] Reinstalling pkg-1.20.8_3...
[2/3] Extracting pkg-1.20.8_3: 100%
Scanning /usr/share/certs/untrusted for certificates...
Scanning /usr/share/certs/trusted for certificates...
Scanning /usr/local/share/certs for certificates...
[3/3] Reinstalling pfSense-upgrade-1.2.1...
[3/3] Extracting pfSense-upgrade-1.2.1: 100%
You may need to manually remove /usr/local/etc/pkg.conf if it is no longer needed.

Then I rebooted.

Now I see a full list of Available Packages, but can install nothing.
I see error:
Another instance of pfSense-upgrade is running. Try again later

System/Update/System Update "Branch" field still remains empty.

The timestamps were weird before rebooting:

pkg-static has an actual UTC/GMT+2 timestamp
/diag_reboot.php has a UTC/GMT timestamp

Dec 26 11:47:10 pfSense pkg-static[82886]: pkg-1.20.8_3 installed
Dec 26 11:47:50 pfSense pkg-static[66562]: pfSense-repo reinstalled: 2.7.2 -> 2.7.2 
Dec 26 11:48:50 pfSense pkg-static[66562]: pkg reinstalled: 1.20.8_3 -> 1.20.8_3 
Dec 26 11:48:50 pfSense pkg-static[66562]: pfSense-upgrade reinstalled: 1.2.1 -> 1.2.1 
Dec 26 09:50:19 pfSense php-fpm[48596]: /diag_reboot.php: Stopping all packages.
Dec 26 11:50:21 pfSense reboot[6976]: rebooted by root

pfthbst · Dec 26, 2023, 11:14 AM

I've added localhost to the NTP Server Configuration->Interface
Now all booting logs have one actual timestamp.

I did again:
pkg-static bootstrap -f
pkg-static clean -ay; pkg-static install -fy pkg pfSense-repo pfSense-upgrade

Rebooted, and nothing changed.

/usr/local/sbin/pfSense-repo-setup
pfSense-repoc-static: failed to fetch the repo data
failed to read the repo data.
failed to update the repository settings!!!

System/Update/System Update "Branch" field remains empty.
I see a full list of Available Packages, but can install nothing.

stephenw10 · Dec 29, 2023, 2:23 PM

Hmm, I assume it was showing the branches in a clean install of 2.7.2 though?

Does the config file contain a branch setting? That shoudn't actually matter though.

pfthbst · Dec 30, 2023, 12:38 PM

@stephenw10
Thanks for the reply.

After numerous experiments, I found the cause of my repo problem.

A few years ago, I saw a solution for the annoying bug - https://redmine.pfsense.org/issues/8987
Under dns host override I redirected dns calls to ews.netgate.com to localhost.
That was the cause of my repo issues.

Prior to version 2.6.0 this didn't cause any problem.

pfthbst · Dec 30, 2023, 12:43 PM

I still see a few minor issues.

The tcpdump output have GMP timestamp, while Timezone seting is "Etc/GMT-2 (2 hours AHEAD of GMT)
All logs have the correct local timestamp, but not the tcpdump output.
2.7.2 can't restore rrddata from 2.6.0 config at all.
If I try to backup current 2.7.2 config with "Backup extra data" ticked, the config.xml is zero lenth.

Should I create a new topic, or can I continue here?

stephenw10 · Dec 30, 2023, 11:08 PM

@pfthbst said in Empty field "Firmware Branch" in <System/Update/Update Settings> after a fresh install v2.7.2:

Under dns host override I redirected dns calls to ews.netgate.com to localhost.

Ah, yes that would do it! Access to that is required to pull the dynamic repo data in 2.7.

Timestamps in tcpdump are actually in epoch time but by default it displays that as UTC. I don't think tcpdump itself can use a local timezone, I could be wrong. If so though that would require some recoding to make the page show it as such. It's not a bug but could be a missing feature.

Yes, it should be a new topic to discuss further.