Can I rebuild the pkg database without installing updated packages?

MrPete

Can pfSense (or me, manually) rebuild pkg database from already-installed pkg info?

Somehow, my pkg database was completely wiped out (and replaced by a loop of links)

The troubleshooting guide suggests installing all packages from scratch.

That sounds painful:

At the least it's a version change from what I am already running.
It seems likely this would lose some or all pkg configuration information?

I am surprised that I don't find documentation on saving/restoring the pkg database, nor rebuilding it.

I attempted to copy the files from /var/db/pkg on my CARP mirror. That did exactly nothing to help make the installed packages show up. Perhaps I did something wrong?

Any help or insights MUCH appreciated!
Pete

bingo600

@mrpete

Maybe one of these tricks from here:
https://docs.netgate.com/pfsense/en/latest/troubleshooting/upgrades.html

Or i seem to remember a tip from the 2.4.x days.
Switch your System --> Update Branch , to the "Beta" , and switch it back ....
Dont do the update , just the switching.
Should supposedly download the "new" package DB , for each switch.

Ohh .. If the repos is now ok , and it's a package reinstall you need.

Diag --> Backup , then all the way down

MrPete

@bingo600 I removed the links, reinstated the folders, and did

pkg-static update -f

...which did something. Result: I can see available packages but nothing supposedly installed.
Tried the "reinstall" button in restore. It fails:

>>> Upgrading pkg... done.
>>> Updating repositories metadata... 
Updating pfSense-core repository catalogue...
Fetching meta.conf: . done
Fetching packagesite.pkg: . done
Processing entries: . done
pfSense-core repository update completed. 7 packages processed.
Updating pfSense repository catalogue...
Fetching meta.conf: . done
Fetching packagesite.pkg: .......... done
Processing entries: .......... done
pfSense repository update completed. 515 packages processed.
All repositories are up to date.
ERROR: Unable to compare version of pfSense-repo

That last line looks important. I don't see any good info on what it means. :(

ALSO: VERRRYYY interesting... If I do a backup, the XML file section on "InstalledPackages" shows the complete list of packages! Even though the package manager says none are installed. That sure seems like a bug to me.

MrPete

Made progress. Discovered /root/var/cache was also a link. Removed and remade as a folder. Now I could clean. Now I could

pkg-static clean -ay ; pkg-static install -fy pkg pfSense-repo pfSense-upgrade

But now, doing "Reinstall All Packages" in the gui results in:

>>> Setting vital flag on pfSense-upgrade... done.
ERROR: It was not possible to identify which pfSense meta package is installed
__RC=1
WARNING: Current pkg repository has a new PHP major
         version. pfSense should be upgraded before
         installing any new package.

So the install produces an inconsistency. :(

stephenw10

What pfSense version are you using? Did you restore a config into it? Are you using RAM disks?

Steve

MrPete

@stephenw10
I'm on 2.6.0-RELEASE (amd64) built on Mon Jan 31 19:57:53
Yes I have RAM disks enabled AND had reinstalled into 2.6 (a couple of months ago at least) with ZFS...and see that there's a new bug recorded saying this does not work. :(

What I observed that got this going:

my pkg database was gone. No packages installed, no packages available
Instead, there was a literal infinite loop of links where the "pkg" and "cache" folders ought to be. (Now I would guess due to the ZFS/RAM disk bug)

Have been trying to recover. (Perhaps I ought to turn off RAM disk and reboot... although not having RAM disk for /tmp is pretty serious when running on SSD's!)

OH... am hoping to avoid restoring an old config.

Strangely,

I can do a config backup right now, and
the backup contains ALL of my configured packages, and their config!
Yet pfSense thinks I have no packages installed.

Seems crazy that the backup system knows, but the package manager does not?!!

stephenw10

Yeah, remove the line that enables ram disks from the config before restoring it:

<use_mfs_tmpvar></use_mfs_tmpvar>

MrPete

@stephenw10 AWESOME. Worked great

SO:

Bug: zfs + ram disk for tmp and var
Result: wipes /var/db/pkg and replaces with links (and /root/var/db/pkg, and /var/db/cache)

Easy workaround for now:

Do a full backup including extra info. It will contain all of your packages AND settings
delete the link folders
recreate a base pkg database

# mkdir -p /var/db/pkg /root/var/db/pkg /root/var/cache
# pkg-static clean -ay; pkg-static install -fy pkg pfSense-repo pfSense-upgrade
# pkg-static upgrade -f

(from https://docs.netgate.com/pfsense/en/latest/troubleshooting/upgrades.html#forced-pkg-reinstall )

Edit the backup XML file as @stephenw10 described. One line to delete: <use_mfs_tmpvar></use_mfs_tmpvar>
Restore the backup. All packages will be restored including their configuration.