Upgrade inconsistencies

Stewart

We are going through and upgrading all of our units to 2.6.0. The firmware versions of the units we are upgrading are kind of all over the place anywhere from 2.4.4-p3 to 2.5.2. So far, I think all of the 2.5.x units have upgraded fine. All of the 2.4.x units need to upgrade to 2.4.5 first and that's where my problems come in to play. First, here is my procedure:

Disable blocking for Suricata.
Disable blocks for pfBlocker-NG
Go to System -> Update -> Update Settings and point to "Previous stable version (2.4.5 DEPRECATED)"
Go to System -> Update -> System Update and point to "Previous stable version (2.4.5 DEPRECATED)"
Have it upgrade to 2.4.5
Up to this point everything proceeds as usual. Once the system reboots I normally give it about 20 minutes (because I move on to the next one) before I proceed.

My next set of procedures is:

Go to System -> Update -> Update Settings and point to "Latest stable version (2.5.x)"
Go to System -> Update -> System Update and point to "Latest stable version (2.5.x)"
At this step some will show that 2.6.0 is available. Some show that 2.5.2 is available. If it goes to 2.6.0, great. If it goes to 2.5.2 I go through the process once more to get it up to 2.6.0

But a couple of the units so far have gone wonky at this point. The first, I think, may be because of a bad SSD. I took the old config and put it into a new install of 2.6.0 and it's working fine now. The one I'm looking at right now give different messages.

If I set the branch to "Previous stable version (2.4.5 DEPRECATED)" then the Dashboard shows "The system is on a later version than official release. "
If I set the branch to "Latest stable version (2.5.x)" then the Dashboard shows "Version 2.5.2 is available but the System -> Update -> System Update page shows:
---Current Base System 2.4.5_1
---Latest Base System 2.4.5_1
--- Status Up to date.

Also of note, there are no logs in the GUI, anywhere. Every screen just shows:
No logs to display.

Here are some commands and results from CLI:

[2.4.5-RELEASE][root@pfSense]/root: pkg-static info -x pfSense-upgrade
pkg-static: Warning: Major OS version upgrade detected.  Running "pkg bootstrap -f" recommended
pfSense-upgrade-0.88

[2.4.5-RELEASE][root@pfSense]/root: pkg-static info -x pfSense-upgrade
pkg-static: Warning: Major OS version upgrade detected.  Running "pkg bootstrap -f" recommended
pfSense-upgrade-0.88

[2.4.5-RELEASE][root@GateKeeper.pfSense]/root: pkg bootstrap -f
The package management tool is not yet installed on your system.
Do you want to fetch and install it now? [y/N]: y
Bootstrapping pkg from pkg+https://pkg.pfsense.org/pfSense_v2_5_1_amd64-pfSense_v2_5_1, please wait...
Verifying signature with trusted certificate pkg.pfsense.org.20160406... done
process with pid 73696 still holds the lock
process with pid 73696 still holds the lock
process with pid 73696 still holds the lock
process with pid 73696 still holds the lock
process with pid 73696 still holds the lock
process with pid 73696 still holds the lock
pkg-static: Cannot get an exclusive lock on a database, it is locked by another process

Looks like there is a lock, even after an hour. So I reboot and try some more commands from researching.

[2.4.5-RELEASE][root@pfSense]/root: pkg-static bootstrap -f
pkg-static: Warning: Major OS version upgrade detected.  Running "pkg bootstrap -f" recommended
pkg(8) is already installed. Forcing reinstallation through pkg(7).
The package management tool is not yet installed on your system.
Do you want to fetch and install it now? [y/N]: y
Bootstrapping pkg from pkg+https://packages.netgate.com/pfSense_v2_4_5_amd64-pfSense_v2_4_5, please wait...
Verifying signature with trusted certificate pkg.pfsense.org.20160406... done
Installing pkg-1.15.6...
package pkg is already installed, forced install
Extracting pkg-1.15.6: 100%

[2.4.5-RELEASE][root@pfSense]/root: pkg bootstrap -f
The package management tool is not yet installed on your system.
Do you want to fetch and install it now? [y/N]: y
Bootstrapping pkg from pkg+https://packages.netgate.com/pfSense_v2_4_5_amd64-pfSense_v2_4_5, please wait...
Verifying signature with trusted certificate pkg.pfsense.org.20160406... done
Installing pkg-1.15.6...
package pkg is already installed, forced install
Extracting pkg-1.15.6: 100%

[2.4.5-RELEASE][root@pfSense]/root: pkg-static install -f pkg
Updating pfSense-core repository catalogue...
pfSense-core repository is up to date.
Updating pfSense repository catalogue...
pfSense repository is up to date.
All repositories are up to date.
Updating database digests format: 100%
The following 1 package(s) will be affected (of 0 checked):

Installed packages to be REINSTALLED:
        pkg-1.15.6 [pfSense]

Number of packages to be reinstalled: 1

3 MiB to be downloaded.

Proceed with this action? [y/N]: y
[1/1] Fetching pkg-1.15.6.txz: 100%    3 MiB   3.6MB/s    00:01
Checking integrity... done (0 conflicting)
[1/1] Reinstalling pkg-1.15.6...
[1/1] Extracting pkg-1.15.6: 100%
You may need to manually remove /usr/local/etc/pkg.conf if it is no longer needed.

At this point I'm going to try to upgrade again.

 0) Logout (SSH only)                  9) pfTop
 1) Assign Interfaces                 10) Filter Logs
 2) Set interface(s) IP address       11) Restart webConfigurator
 3) Reset webConfigurator password    12) PHP shell +  tools
 4) Reset to factory defaults         13) Update from console
 5) Reboot system                     14) Disable Secure Shell (sshd)
 6) Halt system                       15) Restore recent configuration
 7) Ping host                         16) Restart PHP-FPM
 8) Shell

Enter an option: 13

/bin/sh: cannot open /usr/local/sbin/-upgrade: No such file or directory
/usr/local/bin/php-cgi: Undefined symbol "alphasort@FBSD_1.5"
/usr/local/bin/php: Undefined symbol "alphasort@FBSD_1.5"

[2.4.5-RELEASE][root@pfSense]/root: pfSense-upgrade -d
/usr/local/bin/php: Undefined symbol "alphasort@FBSD_1.5"
/usr/local/bin/php: Undefined symbol "alphasort@FBSD_1.5"
>>> Updating repositories metadata...
Updating pfSense-core repository catalogue...
Fetching meta.conf: . done
Fetching packagesite.txz: . done
Processing entries: . done
pfSense-core repository update completed. 7 packages processed.
Updating pfSense repository catalogue...
Fetching meta.conf: . done
Fetching packagesite.txz: .......... done
Processing entries: .......... done
pfSense repository update completed. 525 packages processed.
All repositories are up to date.
ERROR: Unable to compare version of -repo

[2.4.5-RELEASE][root@pfSense]/root: pfSense-upgrade -c -d
/usr/local/bin/php: Undefined symbol "alphasort@FBSD_1.5"
/usr/local/bin/php: Undefined symbol "alphasort@FBSD_1.5"
>>> Updating repositories metadata...
pkg-static: Warning: Major OS version upgrade detected.  Running "pkg bootstrap -f" recommended
Updating pfSense-core repository catalogue...
pkg-static: Repository pfSense-core has a wrong packagesite, need to re-create database
Fetching meta.conf: . done
Fetching packagesite.txz: . done
Processing entries:
pkg-static: wrong architecture: FreeBSD:12:amd64 instead of FreeBSD:11:amd64
pkg-static: repository pfSense-core contains packages with wrong ABI: FreeBSD:12:amd64
Processing entries... done
Unable to update repository pfSense-core
Updating pfSense repository catalogue...
pfSense repository is up to date.
Error updating repositories!
>>> Upgrading pkg...
pkg-static: Warning: Major OS version upgrade detected.  Running "pkg bootstrap -f" recommended
Updating pfSense-core repository catalogue...
pkg-static: Repository pfSense-core has a wrong packagesite, need to re-create database
Fetching meta.conf: . done
Fetching packagesite.txz: . done
Processing entries:
pkg-static: wrong architecture: FreeBSD:12:amd64 instead of FreeBSD:11:amd64
pkg-static: repository pfSense-core contains packages with wrong ABI: FreeBSD:12:amd64
Processing entries... done
Unable to update repository pfSense-core
Updating pfSense repository catalogue...
pfSense repository is up to date.
Error updating repositories!
pkg-static: Repository pfSense-core has a wrong packagesite, need to re-create database
pkg-static: Repository pfSense-core cannot be opened. 'pkg update' required
Checking integrity... done (0 conflicting)
Your packages are up to date.
>>> Updating repositories metadata...
pkg-static: Warning: Major OS version upgrade detected.  Running "pkg bootstrap -f" recommended
Updating pfSense-core repository catalogue...
pkg-static: Repository pfSense-core has a wrong packagesite, need to re-create database
Fetching meta.conf: . done
Fetching packagesite.txz: . done
Processing entries:
pkg-static: wrong architecture: FreeBSD:12:amd64 instead of FreeBSD:11:amd64
pkg-static: repository pfSense-core contains packages with wrong ABI: FreeBSD:12:amd64
Processing entries... done
Unable to update repository pfSense-core
Updating pfSense repository catalogue...
Fetching meta.conf: . done
Fetching packagesite.txz: .......... done
Processing entries:
pkg-static: wrong architecture: FreeBSD:12:amd64 instead of FreeBSD:11:amd64
pkg-static: repository pfSense contains packages with wrong ABI: FreeBSD:12:amd64
Processing entries... done
Unable to update repository pfSense
Error updating repositories!
ERROR: Unable to compare version of -repo

As you can see, it isn't working and I'm out of ideas.