[WORKAROUND] Unable to update and package install (ERROR: It was not possible to identify which pfSense kernel is installed)

NollipfSense

@sergei_shablovsky Should post in Development 2.7 section so developers are aware of issues.

Sergei_Shablovsky

@stephenw10 said in Unable to update and package install (ERROR: It was not possible to identify which pfSense kernel is installed):

What does the output of pkg info actually show?

A lot of installed packages.

As You suggest:

Ok, you are missing the pfSense-repo and pfSense-upgrade packages.
So I would run:
pkg-static clean -ay; pkg-static install -fy pkg pfSense-repo pfSense-upgrade
As shown here: https://docs.netgate.com/pfsense/en/latest/troubleshooting/upgrades.html#forced-pkg-reinstall

As a result:

6 .pkg files from /root/var/cache/pkg/ was deleted
3 packages reinstalled
after reboot I still cannot see installed before packages (achieved...)

But now for me MUCH IMPORTANT that after restoring on a fresh 2.7.0 install from backup, after all packages installed manually, SOME SETTINGS RESTORE, BUT SOME - NO (I not mean whole section not restored, I mean SOME SETTINGS VITHIN CATEGORY, as in my case, some Firewall rules for Interfaces Groups not restored, and this Interfaces Groups not restored...and may be something else, who knows?)

So now the main question are: how to manually ensure that settings from backup successfully restored ?, this mean step by step check each settings in backup config...

Sergei_Shablovsky

@nollipfsense said in Unable to update and package install (ERROR: It was not possible to identify which pfSense kernel is installed):

@sergei_shablovsky Should post in Development 2.7 section so developers are aware of issues.

After CE's developers make a public version with missing “;” at the end of string (see discussion, but conclusion are “there are no sufficient code control at all”), I am not sure that make big sense... :)

stephenw10

You should always restore the complete config if you restoring into a new install.

Restoring sections will only restore those parts and, more importantly, they do not get put through the config updater so you can only restore into the same config version.

Steve

Sergei_Shablovsky

@stephenw10 said in Unable to update and package install (ERROR: It was not possible to identify which pfSense kernel is installed):

You should always restore the complete config if you restoring into a new install.

Restoring sections will only restore those parts and, more importantly, they do not get put through the config updater so you can only restore into the same config version.

Thank You for explanation, mr. Steve!

Please confirm: the ability to restoring certain section of backup .xml valid and make sense only if I need restore on the SAME SUBVERSION, 2.6.4 -> 2.6.4, 2.6.0 - > 2.6.0, 2.6.0 -> 2.6.4, BUT NOT THE DIFFERENT VERSION (major or minor) 2.6.4 -> 2.7.0, 2.6.0 -> 2.7.0, 2.6.4 -> 2.7.2.

I understand You correctly?

If Your answer would be “yes”, so let’s to note that after I making already as You suggest (restoring all from .xml backup file and then restart):

previously installed packages disappear in Package manager pages, and from all main menus;
I find that some settings disappear (Interfaces Groups for example, but may be more, how to find that?);

So because of this, next try I decide making step-by-step: restoring from .xml backup file 1(one) category at a time , then hardware rebooting, to understanding on which step there are problem come in, to eliminate “damage” if possible to say that... ;)

stephenw10

The part that matters is the config version. So if you look in the config file you will see:

<?xml version="1.0"?>
<pfsense>
	<version>22.6</version>

When you import an older config into a newer pfSense version it runs it through a series of upgrade scripts for each version change so that the final imported config is compatible with the system. However the config version only appears once in the file so if you import only a section of it that won't contain the version and it will not be upgraded. You may end up with a config that cannot be loaded.
I personally never import sections of config, it's far safer to import only the complete config file.

See: https://docs.netgate.com/pfsense/en/latest/backup/restore.html#restore-options

Steve

Sergei_Shablovsky

@stephenw10 said in Unable to update and package install (ERROR: It was not possible to identify which pfSense kernel is installed):

The part that matters is the config version. So if you look in the config file you will see:
<?xml version="1.0"?>
<pfsense>
	<version>22.6</version>
When you import an older config into a newer pfSense version it runs it through a series of upgrade scripts for each version change so that the final imported config is compatible with the system. However the config version only appears once in the file so if you import only a section of it that won't contain the version and it will not be upgraded. You may end up with a config that cannot be loaded.
I personally never import sections of config, it's far safer to import only the complete config file.

See: https://docs.netgate.com/pfsense/en/latest/backup/restore.html#restore-options

Thank You for so detailed explanation. Of course, I re-read ALL Docs again carefully with a cup of tee.

Thank You again for patience and attention to my problem.

Sergei_Shablovsky

@stephenw10
So now the main question now are: how to manually ensure that settings from backup successfully restored ?, this mean step by step check each settings in backup config...

Because a lot of settings in each package, I do not remember exactly settings, of course. (May be needed to make PDF copy of each page of settings next time ;)

stephenw10

All the package settings are stored in the main config file. If you restore the complete config file all the package settings will come with it.
The settings shown in the GUI are only what is loaded from the config at boot.

Steve

Sergei_Shablovsky

@stephenw10 said in Unable to update and package install (ERROR: It was not possible to identify which pfSense kernel is installed):

The part that matters is the config version. So if you look in the config file you will see:
<?xml version="1.0"?>
<pfsense>
	<version>22.6</version>
When you import an older config into a newer pfSense version it runs it through a series of upgrade scripts for each version change so that the final imported config is compatible with the system. However the config version only appears once in the file so if you import only a section of it that won't contain the version and it will not be upgraded. You may end up with a config that cannot be loaded.
I personally never import sections of config, it's far safer to import only the complete config file.

See: https://docs.netgate.com/pfsense/en/latest/backup/restore.html#restore-options

Thank You for suggestions, Steve!

So, I check double twice: install pfSense on a bare metal, reboot, then install all needed packages (reboot after each package), then backup from .xml backup file, wait 2h, then reboot, the result are the same: packages disappear from installed, not possible to update/upgrade from menu or manually, ping from pfSense CLI on monitoring 1.1.1.1 / 8.8.8.8 are ok, but ping / traceroute on other sites - no, LAN are working (surfing web as test), but sometimes some images not loaded...

Your suggestion?

P.S.

$ host -t srv _https._tcp.packages.netgate.com
;;connection timed out; no servers could be reached;

$ host files01.netgate.com.
;;connection timed out; no servers could be reached;

$ host files00.netgate.com.
;;connection timed out; no servers could be reached;

# pkg-static update -f
Updating pfSense-core repository catalogue...
pkg-static: https://packages-beta.netgate.com/packages/pfSense_master_amd64-core/meta.txz: No address record
repository pfSense-core has no meta file, using default settings
pkg-static: https://packages-beta.netgate.com/packages/pfSense_master_amd64-core/packagesite.pkg: No address record
pkg-static: https://packages-beta.netgate.com/packages/pfSense_master_amd64-core/packagesite.txz: No address record
Unable to update repository pfSense-core
Updating pfSense repository catalogue...
pkg-static: https://packages-beta.netgate.com/packages/pfSense_master_amd64-pfSense_devel/meta.txz: No address record
repository pfSense has no meta file, using default settings
pkg-static: https://packages-beta.netgate.com/packages/pfSense_master_amd64-pfSense_devel/packagesite.pkg: No address record
pkg-static: https://packages-beta.netgate.com/packages/pfSense_master_amd64-pfSense_devel/packagesite.txz: No address record
Unable to update repository pfSense
Error updating repositories!

stephenw10

Are you installing 2.7?

The config you are restoring is setting the repo to next-dev-version so if you are installing 2.6 and then restoring that it will cause it to try to use the wrong repo and fail.

You don't need to install packages before restoring is will pull them in at the first boot after the install.

But it looks like you have some more general connectivity issue happening there. Those SRV records should resolve:

steve@steve-MMLP7AP-00 ~ $ host -t srv _https._tcp.packages.netgate.com
_https._tcp.packages.netgate.com has SRV record 10 10 443 pkg00-atx.netgate.com.
_https._tcp.packages.netgate.com has SRV record 10 10 443 pkg01-atx.netgate.com.
steve@steve-MMLP7AP-00 ~ $ host -t srv _https._tcp.packages-beta.netgate.com
_https._tcp.packages-beta.netgate.com has SRV record 10 10 443 pkg01-atx.netgate.com.
_https._tcp.packages-beta.netgate.com has SRV record 10 10 443 pkg00-atx.netgate.com.

Steve

NollipfSense

Please see Jim's note here below and why it's important to post in development section.

https://forum.netgate.com/topic/171891/update-failure-2-7-0-development-amd64-built-on-tue-apr-26-06-13-40-utc-2022-freebsd-12-3-stable

Sergei_Shablovsky

@stephenw10 said in Unable to update and package install (ERROR: It was not possible to identify which pfSense kernel is installed):

Are you installing 2.7?

Yes, pfSense 2.7 CE

The config you are restoring is setting the repo to next-dev-version so if you are installing 2.6 and then restoring that it will cause it to try to use the wrong repo and fail.

You don't need to install packages before restoring is will pull them in at the first boot after the install.

I trying several times with and without installing packages before restoring from .xml backup file.
Result are the same.

But it looks like you have some more general connectivity issue happening there. Those SRV records should resolve:

steve@steve-MMLP7AP-00 ~ $ host -t srv _https._tcp.packages.netgate.com
_https._tcp.packages.netgate.com has SRV record 10 10 443 pkg00-atx.netgate.com.
_https._tcp.packages.netgate.com has SRV record 10 10 443 pkg01-atx.netgate.com.
steve@steve-MMLP7AP-00 ~ $ host -t srv _https._tcp.packages-beta.netgate.com
_https._tcp.packages-beta.netgate.com has SRV record 10 10 443 pkg01-atx.netgate.com.
_https._tcp.packages-beta.netgate.com has SRV record 10 10 443 pkg00-atx.netgate.com.

Because ping/traceroute not working for anything excluding LANs, and ping are ok only for 1.1.1.1/8.8.8.8 I come with the same conclusion.

But where is the source of problem and how to find them?

Sergei_Shablovsky

@nollipfsense said in Unable to update and package install (ERROR: It was not possible to identify which pfSense kernel is installed):

Please Jim's note here below and why it's important to post in development section.

https://forum.netgate.com/topic/171891/update-failure-2-7-0-development-amd64-built-on-tue-apr-26-06-13-40-utc-2022-freebsd-12-3-stable

Thank You that point me.

Hm, I never imagine that Netgate was making so unstable upgrade...

stephenw10

@sergei_shablovsky said in Unable to update and package install (ERROR: It was not possible to identify which pfSense kernel is installed):

I never imagine that Netgate was making so unstable upgrade...

What?!

We specifically stopped updating the snapshot servers in order to shield end users from any instability these significant changes might introduce. The snapshot servers are still there and you can still pull pkgs from them. They are just not being updated currently. It will not prevent you accessing the servers.

You might be seeing an IPv6 issue. Try:

host -4t srv _https._tcp.packages-beta.netgate.com

Steve

Sergei_Shablovsky

@stephenw10 said in Unable to update and package install (ERROR: It was not possible to identify which pfSense kernel is installed):

@sergei_shablovsky said in Unable to update and package install (ERROR: It was not possible to identify which pfSense kernel is installed):

I never imagine that Netgate was making so unstable upgrade...

What?!

We specifically stopped updating the snapshot servers in order to shield end users from any instability these significant changes might introduce. The snapshot servers are still there and you can still pull pkgs from them. They are just not being updated currently. It will not prevent you accessing the servers.

According stopping to updating snapshots to end users - totally agree with you.

You might be seeing an IPv6 issue. Try:
host -4t srv _https._tcp.packages-beta.netgate.com

Thank You, Steve.

The result are:

;;connection timed out; no servers could be reached

stephenw10

Can it resolve anything?

Didn't you have an issue very similar to this previously?

Sergei_Shablovsky

@stephenw10 said in Unable to update and package install (ERROR: It was not possible to identify which pfSense kernel is installed):

Can it resolve anything?

Unfortunately, NO

Also I check ipv6 settings everywhere,- nothing used.

Didn't you have an issue very similar to this previously?

No any similar...

Sergei_Shablovsky

@stephenw10

The problem come right after update from .xml backup file. Because of this I try to determine which section of .xml backup file impact on this: as I describe before, installing one-by-one. Trying installing packages before, and after restoring... But results are the same: all looks like ok, but after System section restoring - going to wrong state, and connection to outside partially loss..

I more than sure that with .xml backup file are all ok, because I make 3 copy of them on 3 different media.

Sergei_Shablovsky

@stephenw10

Today I found that:

if disable DNS Resolver
reboot
when enabling DNS Resolver
After pressing Save button on DNS Resolver webGUI page, the follow error appear on top of the page (not on separate pop-up or info/alert part of page, just on upper background):

Warning: file_put_contents(/var/unbound/sslcert.crt): failed to open stream: No such file or directory in /etc/inc/unbound.inc on line 314
Warning: chmod(): No such file or directory in /etc/inc/unbound.inc on line 315
Warning: file_put_contents(/var/unbound/sslcert.key): failed to open stream: No such file or directory in /etc/inc/unbound.inc on line 316
Warning: chmod(): No such file or directory in /etc/inc/unbound.inc on line 317