failed to fetch the repo data. Unable to perform update from 2.7.2 to 2.8.0 after restoring crashed 2.8.0 pfSense.

Wolfgangthegreat

Still cannot access the upgrade files, at the system > update I get "pfSense-repoc: si_get_packages: failed to run the pkg info command: /usr/local/sbin/pkg-static info -R --raw-format json-compact pfSense-pkg-* 2>&1 pfSense-repoc: no pfSense packages installed pfSense-repoc: failed to fetch the repo data"

Update from the console gives:
pfSense-repoc-static: si_get_packages: failed to run the pkg info command: /usr/local/sbin/pkg-static info -R --raw-format json-compact pfSense-pkg-* 2>&1
pfSense-repoc-static: no pfSense packages installed
pfSense-repoc-static: failed to fetch the repo data
failed to read the repo data.
failed to update the repository settings!!!
failed to update the repository settings!!!

comet424

@Wolfgangthegreat
ya i was getting those errors last night.. maybe servers down again?

i did try last night from the console shell

certctl rehash

and then things didnt work and i waited till this morning and i was able to update from gui.

Wolfgangthegreat

@comet424 OK, I was able now to fetch the upgrade files and upgraded successfully. FYI

GregBinSD

@Wolfgangthegreat
Thank you for the info. I won't be able to attempt the upgrade to 2.8.0 until tomorrow.
I'll let you know status then.

GregBinSD

@Wolfgangthegreat
...and to @comet424

I wasn't able to perform the 2.8.0 update this weekend, but when I got to the school this morning, it worked perfectly!

I appreciate the support from both of you, and from Netgate.

The backup/standby pfSense instance is back in place and ready in case I have a hardware failure, or a failure of the gray matter between my ears!

My best to all of you.

rcfa

@comet424 said in failed to fetch the repo data. Unable to perform update from 2.7.2 to 2.8.0 after restoring crashed 2.8.0 pfSense.:

certctl rehash

I just get a whole slew of errors when I run that command

root: certctl rehash
certctl: Skipping untrusted certificate c47d9980 (/etc/ssl/untrusted/c47d9980.0)
certctl: Skipping untrusted certificate 0b7c536a (/etc/ssl/untrusted/0b7c536a.0)
certctl: Skipping untrusted certificate 5e98733a (/etc/ssl/untrusted/5e98733a.0)
certctl: Skipping untrusted certificate 03179a64 (/etc/ssl/untrusted/03179a64.0)
certctl: Skipping untrusted certificate 5d3033c5 (/etc/ssl/untrusted/5d3033c5.0)
certctl: Skipping untrusted certificate 4304c5e5 (/etc/ssl/untrusted/4304c5e5.0)
certctl: Skipping untrusted certificate 4a6481c9 (/etc/ssl/untrusted/4a6481c9.0)
certctl: Skipping untrusted certificate d853d49e (/etc/ssl/untrusted/d853d49e.0)
certctl: Skipping untrusted certificate 26312675 (/etc/ssl/untrusted/26312675.0)
certctl: Skipping untrusted certificate ee1365c0 (/etc/ssl/untrusted/ee1365c0.0)
certctl: Skipping untrusted certificate 116bf586 (/etc/ssl/untrusted/116bf586.0)
certctl: Skipping untrusted certificate 76cb8f92 (/etc/ssl/untrusted/76cb8f92.0)
certctl: Skipping untrusted certificate 2e5ac55d (/etc/ssl/untrusted/2e5ac55d.0)
certctl: Skipping untrusted certificate 0c4c9b6c (/etc/ssl/untrusted/0c4c9b6c.0)
certctl: Skipping untrusted certificate 1320b215 (/etc/ssl/untrusted/1320b215.0)
certctl: Skipping untrusted certificate dc45b0bd (/etc/ssl/untrusted/dc45b0bd.0)
certctl: Skipping untrusted certificate 080911ac (/etc/ssl/untrusted/080911ac.0)
certctl: Skipping untrusted certificate 18856ac4 (/etc/ssl/untrusted/18856ac4.0)
certctl: Skipping untrusted certificate 349f2832 (/etc/ssl/untrusted/349f2832.0)
certctl: Skipping untrusted certificate 7aaf71c0 (/etc/ssl/untrusted/7aaf71c0.0)
certctl: Skipping untrusted certificate a8dee976 (/etc/ssl/untrusted/a8dee976.0)
certctl: Skipping untrusted certificate 57bcb2da (/etc/ssl/untrusted/57bcb2da.0)
certctl: Skipping untrusted certificate 9c2e7d30 (/etc/ssl/untrusted/9c2e7d30.0)
certctl: Skipping untrusted certificate 5a7722fb (/etc/ssl/untrusted/5a7722fb.0)
certctl: Skipping untrusted certificate f90208f7 (/etc/ssl/untrusted/f90208f7.0)
certctl: Skipping untrusted certificate 08063a00 (/etc/ssl/untrusted/08063a00.0)
certctl: Skipping untrusted certificate 3e44d2f7 (/etc/ssl/untrusted/3e44d2f7.0)
certctl: Skipping untrusted certificate 5a4d6896 (/etc/ssl/untrusted/5a4d6896.0)
certctl: Skipping untrusted certificate 442adcac (/etc/ssl/untrusted/442adcac.0)
certctl: Skipping untrusted certificate b1b8a7f3 (/etc/ssl/untrusted/b1b8a7f3.0)
certctl: Skipping untrusted certificate 1636090b (/etc/ssl/untrusted/1636090b.0)
certctl: Skipping untrusted certificate 66445960 (/etc/ssl/untrusted/66445960.0)
certctl: Skipping untrusted certificate c01cdfa2 (/etc/ssl/untrusted/c01cdfa2.0)
certctl: Skipping untrusted certificate cb59f961 (/etc/ssl/untrusted/cb59f961.0)
certctl: Skipping untrusted certificate 5e98733a (/etc/ssl/untrusted/5e98733a.0)
certctl: Skipping untrusted certificate 57bcb2da (/etc/ssl/untrusted/57bcb2da.0)
certctl: Skipping untrusted certificate 18856ac4 (/etc/ssl/untrusted/18856ac4.0)
certctl: Skipping untrusted certificate 08063a00 (/etc/ssl/untrusted/08063a00.0)

Any ideas on how to fix that?

stephenw10

Those are not errors, it's expected to skip untrusted certs.

But you also shouldn't need to run that manually from 2.7.2. It was only an issue in 2.7.0.

What problem are you seeing?

rcfa

@stephenw10 Pretty much the same things described here, but trying to go from 2.8.0 to the 2.8.1 beta.

Upgrading from the console:

Enter an option: 13

pfSense-repoc-static: failed to fetch the repo data
failed to read the repo data.
failed to update the repository settings!!!
failed to update the repository settings!!!
pfSense - Netgate Device ID: xyzblahblah

From the webUI:
1.PreUpgrade.png 2.PostUpgrade.png

Tried the fixes described in this thread and that one "Another instance of pfSense-upgrade is running. Try again later"

No change. Now while I can do w/o the beta, it seems something’s messed up, and that something will come to bite me once 2.8.1 is out of beta, so just shrugging the shoulders is just kicking the can down the road...

stephenw10

OK try at the command line: pfSense-repoc -ND

What error is shown?

Send me your NDI in chat and I can check what it should be seeing.

rcfa

@stephenw10 Still haven’t figured that one out.
BUT looking at the /cf/conf/config.xml of the two installations, I noticed an interesting discrepancy:

<pkg_repo_conf_path>/usr/local/etc/pfSense/pkg/repos/pfSense-repo-2_8_0.conf</pkg_repo_conf_path>

<pkg_repo_conf_path>2_8_1-preview</pkg_repo_conf_path>

Even though both systems refuse to see the 2.8.1 release, for former is stuck on 2.8.0 and the latter on 2.8.1.beta

Listing the (presumably) correct folder on the former yields:

root: ls /usr/local/etc/pfSense/pkg/repos/
pfSense-repo-2.7.2.abi                pfSense-repo-23_09_1_upgrade-cert.pem pfSense-repo-23_09_1_upgrade.name
pfSense-repo-2.7.2.altabi             pfSense-repo-23_09_1_upgrade-key.pem  pfSense-repo-2_8_0.abi
pfSense-repo-2.7.2.conf               pfSense-repo-23_09_1_upgrade.abi      pfSense-repo-2_8_0.altabi
pfSense-repo-2.7.2.default            pfSense-repo-23_09_1_upgrade.altabi   pfSense-repo-2_8_0.conf
pfSense-repo-2.7.2.descr              pfSense-repo-23_09_1_upgrade.conf     pfSense-repo-2_8_0.descr
pfSense-repo-2.7.2.name               pfSense-repo-23_09_1_upgrade.descr    pfSense-repo-2_8_0.name

while on the latter:

/root: ls /usr/local/etc/pfSense/pkg/repos/
pfSense-repo-0000.abi      pfSense-repo-0000.conf     pfSense-repo-0000.descr    pfSense-repo-0000.version
pfSense-repo-0000.altabi   pfSense-repo-0000.default  pfSense-repo-0000.name

Which looks rather different.

~~So I changed the line on the second one to a (somewhat) consistent~~

<pkg_repo_conf_path>/usr/local/etc/pfSense/pkg/repos/pfSense-repo-0000.conf</pkg_repo_conf_path>

~~Maybe I can fix this, when I know what this line really should be and what correctly should be in /usr/local/etc/pfSense/pkg/repos/~~

Well, I messed around with these files and settings, and now they are all consistent in the what I guess is the new format.
Then going twice through the troubleshooting section of the manual here finally got both systems upgraded to 2.8.1.

So far so good, but unfortunately, the trouble still isn’t over.

Going to the System > Update section in the WebUI, I still get:

pfSense-repoc: failed to fetch the repo data

and trying to install some package, I still, after hitting the green "confirm" button, get:

<package> installation failed!
Another instance of pfSense-upgrade is running.  Try again later

So close, yet….

stephenw10

Hmm, so pfSense-repoc -ND still fails?
But pkg -d update succeeds?

rcfa

@stephenw10

Yep, that’s what it looks like…
…execution.log.txt attached, just changed IP and the system ID, since this is a public forum.

Same old error for the former:

pfSense-repoc: failed to fetch the repo data
failed to read the repo data.

and same success for the latter:

pfSense repository is up to date.
All repositories are up to date.

The only thing in the latter that I don’t quite understand are some lines like:

* The requested document is not new enough
* Simulate an HTTP 304 response
* Closing connection

do they mean the file on the server is not newer than the local file, and thus no need to fetch it, or is that some sort of error that results in the file not getting fetched, and the success message at the end is misleading?

Selecting option 13 from the CLI, results in these:

pfSense-repoc-static: failed to fetch the repo data
failed to read the repo data.
failed to update the repository settings!!!
failed to update the repository settings!!!

which are mostly familiar, except for the "failed to update the repository settings!!!" addition.

And of course, any attempt of installing/updating a package from the web UI, still gives me this error:

Another instance of pfSense-upgrade is running.  Try again later

These seem to go hand in hand...

chrcoluk

There does still seem to be repo issues, 16 minutes to download 159MB of packages.

Maybe stick a cloudflare cache up or something?

The actual update was just couple of minutes so the effective window was mostly waiting for downloads.

stephenw10

There were some IPv6 connectivity issues last week but that should be resolved now.

rcfa

@stephenw10 Unfortunately, this isn’t it.
This very specific scenario has persisted for quite some time, and I’m sure it will eventually turn out to be something tiny, the question is just: what could cause these symptoms?
Had this very issue before, and now after, force upgrading to 2.8.1 (following the upgrading troubleshooting guide in the documentation), and I have it on two systems configured in a similar fashion (they are the end points of the same tunnel)
While it doesn’t interfere with regular operations, it makes upgrades/updates quite some hassle. And of course it’s bugging me, that something is off.

stephenw10

Hmm, what sort of tunnel? Is it somehow trying to pass the upgrade traffic across it?