Empty field "Firmware Branch" in <System/Update/Update Settings> after a fresh install v2.7.2

pfthbst

Sorry for my English.
2.5.2 was the last version with normal upgrade.

I am using pfSense for my family network, many thanks to all the pfSense developers.
For about 7-8 years I've been adding, changing hardware, and tweaking the config.

Right now I have multiple static ip addresses, several vlans, vpn and ipsec tunnels, smart UPS, wifi controller on Linux.
Now I have electrical problems very often, so communicating UPS with pfSense is just a vital necessity.
Rebuilding a config from scratch is a huge problem right now for me.
That's why I'm periodically trying and trying to upgrade.

But..
Upgrading from 2.5.2 to 2.6.0 has successfully failed from cli and from web.
Upgrading from 2.5.2 to 2.7.0 has successfully failed from cli and from web.

Freshly installed 2.7.0 failed to restore the 2.5.2 config.

I then did a fresh install of 2.6.0 and successfully restored the 2.5.2 config.

Then the upgrade from 2.6.0 to 2.7.0 has successfully failed from cli and from web.

All this time I have been trying to restore the full config with RRD and extra data included.
All this time I've been having some kind of problem with the pfSense repo.

I was thinking maybe it's a problem with some packages?

I installed a fresh version 2.6.0, successfully restored the config, removed all packages, made the config backup with RRD and extra data included.
Freshly installed 2.7.2 failed to restore the config with removed packages:

2.7.2_conf_restore_failed.jpg

I'm now on fresh 2.7.2 with a partially restored 2.6.0 config and a pfSense-repo issue.

pfthbst

I did:

cp /usr/local/share/pfSense/pkg/repos/pfSense-repo.conf /usr/local/etc/pkg/repos/pfSense.conf

Then:

pkg-static bootstrap -f

pkg(8) is already installed. Forcing reinstallation through pkg(7).
The package management tool is not yet installed on your system.
Do you want to fetch and install it now? [y/N]: y
Bootstrapping pkg from pkg+https://pkg.pfsense.org/pfSense_v2_7_2_amd64-pfSense_v2_7_2, please wait...
Verifying signature with trusted certificate pkg.pfsense.org.20160406... done
Installing pkg-1.20.8_3...
package pkg is already installed, forced install
Extracting pkg-1.20.8_3: 100%
Scanning /usr/share/certs/untrusted for certificates...
Scanning /usr/share/certs/trusted for certificates...
Scanning /usr/local/share/certs for certificates..


pkg update

Updating pfSense-core repository catalogue...
Fetching meta.conf: 100%    163 B   0.2kB/s    00:01    
Fetching packagesite.pkg: 100%    1 KiB   1.5kB/s    00:01    
Processing entries: 100%
pfSense-core repository update completed. 4 packages processed.
Updating pfSense repository catalogue...
pkg: Repository pfSense has a wrong packagesite, need to re-create database
Fetching meta.conf: 100%    163 B   0.2kB/s    00:01    
Fetching packagesite.pkg: 100%  157 KiB 160.7kB/s    00:01    
Processing entries: 100%
pfSense repository update completed. 549 packages processed.
All repositories are up to date.


pkg-static clean -ay; pkg-static install -fy pkg pfSense-repo pfSense-upgrade

pkg-static: Invalid ABI
pkg-static: Warning: Major OS version upgrade detected.  Running "pkg bootstrap -f" recommended
The following package files will be deleted:
	/var/cache/pkg/pfSense-repo-2.7.2~3aa9dedf41.pkg
	/var/cache/pkg/pfSense-upgrade-1.2.1~7f3940b016.pkg
	/var/cache/pkg/pfSense-upgrade-1.2.1.pkg
	/var/cache/pkg/pkg-1.20.8_3~48cca6e983.pkg
	/var/cache/pkg/pkg-1.20.8_3.pkg
	/var/cache/pkg/pfSense-repo-2.7.2.pkg
The cleanup will free 11 MiB
Deleting files: 100%
All done
pkg-static: Invalid ABI
pkg-static: Warning: Major OS version upgrade detected.  Running "pkg bootstrap -f" recommended
Updating pfSense-core repository catalogue...
Fetching meta.conf:   0%
Fetching packagesite.pkg:   0%
pfSense-core repository is up to date.
Updating pfSense repository catalogue...
Fetching meta.conf:   0%
Fetching packagesite.pkg:   0%
pfSense repository is up to date.
All repositories are up to date.
Updating database digests format: 100%      1 B   0.0kB/s    00:01    
The following 3 package(s) will be affected (of 0 checked):

Installed packages to be REINSTALLED:
	pfSense-repo-2.7.2 [pfSense]
	pfSense-upgrade-1.2.1 [pfSense]
	pkg-1.20.8_3 [pfSense]

Number of packages to be reinstalled: 3

11 MiB to be downloaded.
[1/3] Fetching pfSense-repo-2.7.2.pkg: 100%    6 KiB   5.9kB/s    00:01    
[2/3] Fetching pkg-1.20.8_3.pkg: 100%   10 MiB   1.8MB/s    00:06    
[3/3] Fetching pfSense-upgrade-1.2.1.pkg: 100%   20 KiB  20.2kB/s    00:01    
Checking integrity... done (0 conflicting)
[1/3] Reinstalling pfSense-repo-2.7.2...
[1/3] Extracting pfSense-repo-2.7.2: 100%
cat: /pfSense-repo-.abi: No such file or directory
cat: /pfSense-repo-.altabi: No such file or directory
[2/3] Reinstalling pkg-1.20.8_3...
[2/3] Extracting pkg-1.20.8_3: 100%
Scanning /usr/share/certs/untrusted for certificates...
Scanning /usr/share/certs/trusted for certificates...
Scanning /usr/local/share/certs for certificates...
[3/3] Reinstalling pfSense-upgrade-1.2.1...
[3/3] Extracting pfSense-upgrade-1.2.1: 100%
You may need to manually remove /usr/local/etc/pkg.conf if it is no longer needed.

pfSense-upgrade -dc

pfSense-repoc-static: si_get_packages: failed to run the pkg info command: /usr/local/sbin/pkg-static info -R --raw-format json-compact pfSense-pkg-\* 2>&1
pfSense-repoc-static: no pfSense packages installed
pfSense-repoc-static: failed to fetch the repo data
failed to read the repo data.
failed to update the repository settings!!!
failed to update the repository settings!!!

pfthbst

Now Available Packages is shows all packages:

But I can't install anything.

I got errors:
pfSense-pkg-apcupsd installation failed!
Another instance of pfSense-upgrade is running. Try again later

System/Update/System Update is still showing an empty "Branch" field, with error message:

pfSense-repoc: si_get_packages: failed to run the pkg info command: /usr/local/sbin/pkg-static info -R --raw-format json-compact pfSense-pkg-* 2>&1 pfSense-repoc: no pfSense packages installed pfSense-repoc: failed to fetch the repo data

Any advice would be very helpful!

Merry Christmas to all the pfSense guys and their families!

stephenw10

@pfthbst said in Empty field "Firmware Branch" in <System/Update/Update Settings> after a fresh install v2.7.2:

I installed a fresh version 2.6.0, successfully restored the config, removed all packages, made the config backup with RRD and extra data included.
Freshly installed 2.7.2 failed to restore the config with removed packages:

That looks like this: https://redmine.pfsense.org/issues/13132
That should be fixed but it's possible you found some variant on that. However you should be able to manually remove the rogue sshdata section from the config and then import that to 2.7.2.

pfthbst

@stephenw10
Thanks for the advice!

I removed redundant <sshdata> section from 2.6.0 no-packages config.

The fresh 2.7.2 successfully restored this config, but <System/Update/System Update> still has an empty "Branch" field.
With a new error message:
pfSense-repoc: failed to fetch the repo data

Available Packages still shows no packages available.

Now pfSense-repoc does not display messages:

pfSense-repoc: si_get_packages: failed to run the pkg info command: /usr/local/sbin/pkg-static info -R --raw-format json-compact pfSense-pkg-* 2>&1
pfSense-repoc: no pfSense packages installed

But it's still showing errors:

pfSense-repoc

pfSense-repoc: failed to fetch the repo data
failed to read the repo data.

pfthbst

I have a noname box with an Intel CPU.
I've noticed a couple things in the system.log

Dec 26 09:32:54 pfSense kernel: netgate0: <unknown hardware>
Strange things are happening with timestamps in the system.log. The Boot sequence starts with my actual UTC/GMT +2 timestamp, then the UTC/GMT timestamp appears:

Dec 26 09:32:54 pfSense syslogd: kernel boot file is /boot/kernel/kernel
Dec 26 09:32:54 pfSense kernel: ---<<BOOT>>---
..
Dec 26 07:38:53 pfSense php-fpm[48596]: /pkg_mgr.php: The command '/usr/local/sbin/pfSense-repo-setup' returned exit code '1', the output was 'pfSense-repoc-static: failed to fetch the repo data failed to read the repo data. failed to update the repository settings!!!'

Could this have any effect on the pfSense repo problems?

pfthbst

I did:
pkg-static bootstrap -f
pkg update
pkg-static clean -ay; pkg-static install -fy pkg pfSense-repo pfSense-upgrade

pkg-static bootstrap -f

pkg(8) is already installed. Forcing reinstallation through pkg(7).
The package management tool is not yet installed on your system.
Do you want to fetch and install it now? [y/N]: y
Bootstrapping pkg from pkg+https://pkg.pfsense.org/pfSense_v2_7_2_amd64-pfSense_v2_7_2, please wait...
Verifying signature with trusted certificate pkg.pfsense.org.20160406.pkgsave... done
Installing pkg-1.20.8_3...
package pkg is already installed, forced install
Extracting pkg-1.20.8_3: 100%
Scanning /usr/share/certs/untrusted for certificates...
Scanning /usr/share/certs/trusted for certificates...
Scanning /usr/local/share/certs for certificates...


pkg update

Updating pfSense-core repository catalogue...
Fetching meta.conf: 100%    163 B   0.2kB/s    00:01    
Fetching packagesite.pkg: 100%    1 KiB   1.5kB/s    00:01    
Processing entries: 100%
pfSense-core repository update completed. 4 packages processed.
Updating pfSense repository catalogue...
pkg: Repository pfSense has a wrong packagesite, need to re-create database
Fetching meta.conf: 100%    163 B   0.2kB/s    00:01    
Fetching packagesite.pkg: 100%  157 KiB 160.7kB/s    00:01    
Processing entries: 100%
pfSense repository update completed. 549 packages processed.
All repositories are up to date.


pkg-static clean -ay; pkg-static install -fy pkg pfSense-repo pfSense-upgrade

Nothing to do.
Updating pfSense-core repository catalogue...
Fetching meta.conf:   0%
Fetching packagesite.pkg:   0%
pfSense-core repository is up to date.
Updating pfSense repository catalogue...
Fetching meta.conf:   0%
Fetching packagesite.pkg:   0%
pfSense repository is up to date.
All repositories are up to date.
Updating database digests format: 100%      1 B   0.0kB/s    00:01    
The following 3 package(s) will be affected (of 0 checked):

Installed packages to be REINSTALLED:
	pfSense-repo-2.7.2 [pfSense]
	pfSense-upgrade-1.2.1 [pfSense]
	pkg-1.20.8_3 [pfSense]

Number of packages to be reinstalled: 3

11 MiB to be downloaded.
[1/3] Fetching pfSense-repo-2.7.2.pkg: 100%    6 KiB   5.9kB/s    00:01    
[2/3] Fetching pkg-1.20.8_3.pkg: 100%   10 MiB   1.4MB/s    00:08    
[3/3] Fetching pfSense-upgrade-1.2.1.pkg: 100%   20 KiB  20.2kB/s    00:01    
Checking integrity... done (0 conflicting)
[1/3] Reinstalling pfSense-repo-2.7.2...
[1/3] Extracting pfSense-repo-2.7.2: 100%
[2/3] Reinstalling pkg-1.20.8_3...
[2/3] Extracting pkg-1.20.8_3: 100%
Scanning /usr/share/certs/untrusted for certificates...
Scanning /usr/share/certs/trusted for certificates...
Scanning /usr/local/share/certs for certificates...
[3/3] Reinstalling pfSense-upgrade-1.2.1...
[3/3] Extracting pfSense-upgrade-1.2.1: 100%
You may need to manually remove /usr/local/etc/pkg.conf if it is no longer needed.

Then I rebooted.

Now I see a full list of Available Packages, but can install nothing.
I see error:
Another instance of pfSense-upgrade is running. Try again later

System/Update/System Update "Branch" field still remains empty.

The timestamps were weird before rebooting:

pkg-static has an actual UTC/GMT+2 timestamp
/diag_reboot.php has a UTC/GMT timestamp

Dec 26 11:47:10 pfSense pkg-static[82886]: pkg-1.20.8_3 installed
Dec 26 11:47:50 pfSense pkg-static[66562]: pfSense-repo reinstalled: 2.7.2 -> 2.7.2 
Dec 26 11:48:50 pfSense pkg-static[66562]: pkg reinstalled: 1.20.8_3 -> 1.20.8_3 
Dec 26 11:48:50 pfSense pkg-static[66562]: pfSense-upgrade reinstalled: 1.2.1 -> 1.2.1 
Dec 26 09:50:19 pfSense php-fpm[48596]: /diag_reboot.php: Stopping all packages.
Dec 26 11:50:21 pfSense reboot[6976]: rebooted by root

pfthbst

I've added localhost to the NTP Server Configuration->Interface
Now all booting logs have one actual timestamp.

I did again:
pkg-static bootstrap -f
pkg-static clean -ay; pkg-static install -fy pkg pfSense-repo pfSense-upgrade

Rebooted, and nothing changed.

/usr/local/sbin/pfSense-repo-setup
pfSense-repoc-static: failed to fetch the repo data
failed to read the repo data.
failed to update the repository settings!!!

System/Update/System Update "Branch" field remains empty.
I see a full list of Available Packages, but can install nothing.

stephenw10

Hmm, I assume it was showing the branches in a clean install of 2.7.2 though?

Does the config file contain a branch setting? That shoudn't actually matter though.

pfthbst

@stephenw10
Thanks for the reply.

After numerous experiments, I found the cause of my repo problem.

A few years ago, I saw a solution for the annoying bug - https://redmine.pfsense.org/issues/8987
Under dns host override I redirected dns calls to ews.netgate.com to localhost.
That was the cause of my repo issues.

Prior to version 2.6.0 this didn't cause any problem.

pfthbst

I still see a few minor issues.

The tcpdump output have GMP timestamp, while Timezone seting is "Etc/GMT-2 (2 hours AHEAD of GMT)
All logs have the correct local timestamp, but not the tcpdump output.
2.7.2 can't restore rrddata from 2.6.0 config at all.
If I try to backup current 2.7.2 config with "Backup extra data" ticked, the config.xml is zero lenth.

Should I create a new topic, or can I continue here?

stephenw10

@pfthbst said in Empty field "Firmware Branch" in <System/Update/Update Settings> after a fresh install v2.7.2:

Under dns host override I redirected dns calls to ews.netgate.com to localhost.

Ah, yes that would do it! Access to that is required to pull the dynamic repo data in 2.7.

Timestamps in tcpdump are actually in epoch time but by default it displays that as UTC. I don't think tcpdump itself can use a local timezone, I could be wrong. If so though that would require some recoding to make the page show it as such. It's not a bug but could be a missing feature.

Yes, it should be a new topic to discuss further.