6100 check_upgrade(1): unknown error

pfnewb2016 · Feb 6, 2025, 1:43 AM

Good day! I could use some help getting a 6100 running v24.03 to upgrade.
Let me know if can provide more details.

Thank you.

TL;DR:

Couldn't find host pfsense-plus-pkg.netgate.com in the .netrc file; using defaults
Could not resolve host: pfsense-plus-pkg.netgate.com

DNS resolution from Diagnostics works for some (all?) sites , but not for:
pfsense-plus-pkg.netgate.com.

www.netgate.com does resolve.

DNS Resolver is configured, multiple external DNS servers have been tried: 8.8.8.8, 8.8.4.4, 9.9.9.9.

# Details

At different times and in Dashboard or Update I see:

system is on the latest version
pfSense-repoc: failed to fetch the repo data AND status Up to date.

Changing branches doesn't help.

via ssh:

Upgrade from console fails, as does pkg -d update and pkg -d4 update

Error output:

24.03-RELEASE][admin@fw-fl2-rack-6100-23.client.lan]/root: pkg -d update
DBG(1)[84696]> pkg initialized
Updating pfSense-core repository catalogue...
DBG(1)[84696]> PkgRepo: verifying update for pfSense-core
DBG(1)[84696]> PkgRepo: need forced update of pfSense-core
DBG(1)[84696]> Pkgrepo, begin update of '/var/db/pkg/repo-pfSense-core.sqlite'
DBG(1)[84696]> Request to fetch pkg+https://pfsense-plus-pkg.netgate.com/pfSense_plus-v24_11_amd64-core/meta.conf
DBG(1)[84696]> curl_open
pkg: No SRV record found for the repo 'pfSense-core'
DBG(1)[84696]> Fetch: fetcher used: pkg+https
DBG(1)[84696]> curl> fetching https://pfsense-plus-pkg.netgate.com/pfSense_plus-v24_11_amd64-core/meta.conf
DBG(1)[84696]> CURL> No mirror set url to https://pfsense-plus-pkg.netgate.com/pfSense_plus-v24_11_amd64-core/meta.conf
DBG(1)[84696]> CURL> attempting to fetch from https://pfsense-plus-pkg.netgate.com/pfSense_plus-v24_11_amd64-core/meta.conf, left retry 3

Couldn't find host pfsense-plus-pkg.netgate.com in the .netrc file; using defaults
Could not resolve host: pfsense-plus-pkg.netgate.com
Closing connection

DBG(1)[84696]> CURL> No mirror set url to https://pfsense-plus-pkg.netgate.com/pfSense_plus-v24_11_amd64-core/meta.conf
DBG(1)[84696]> CURL> attempting to fetch from https://pfsense-plus-pkg.netgate.com/pfSense_plus-v24_11_amd64-core/meta.conf, left retry 2

Couldn't find host pfsense-plus-pkg.netgate.com in the .netrc file; using defaults
Could not resolve host: pfsense-plus-pkg.netgate.com
Closing connection
DBG(1)[84696]> CURL> No mirror set url to https://pfsense-plus-pkg.netgate.com/pfSense_plus-v24_11_amd64-core/meta.conf
DBG(1)[84696]> CURL> attempting to fetch from https://pfsense-plus-pkg.netgate.com/pfSense_plus-v24_11_amd64-core/meta.conf, left retry 1
Couldn't find host pfsense-plus-pkg.netgate.com in the .netrc file; using defaults
Could not resolve host: pfsense-plus-pkg.netgate.com
Closing connection

pkg: An error occured while fetching package
DBG(1)[84696]> Request to fetch pkg+https://pfsense-plus-pkg.netgate.com/pfSense_plus-v24_11_amd64-core/meta.txz
pkg: packagesite URL error for pkg+https://pfsense-plus-pkg.netgate.com/pfSense_plus-v24_11_amd64-core/meta.txz -- pkg+:// implies SRV mirror type
repository pfSense-core has no meta file, using default settings
DBG(1)[84696]> Request to fetch pkg+https://pfsense-plus-pkg.netgate.com/pfSense_plus-v24_11_amd64-core/packagesite.pkg
pkg: packagesite URL error for pkg+https://pfsense-plus-pkg.netgate.com/pfSense_plus-v24_11_amd64-core/packagesite.pkg -- pkg+:// implies SRV mirror type
DBG(1)[84696]> Request to fetch pkg+https://pfsense-plus-pkg.netgate.com/pfSense_plus-v24_11_amd64-core/packagesite.txz
pkg: packagesite URL error for pkg+https://pfsense-plus-pkg.netgate.com/pfSense_plus-v24_11_amd64-core/packagesite.txz -- pkg+:// implies SRV mirror type
Unable to update repository pfSense-core
Updating pfSense repository catalogue...
DBG(1)[84696]> PkgRepo: verifying update for pfSense
DBG(1)[84696]> PkgRepo: need forced update of pfSense
DBG(1)[84696]> Pkgrepo, begin update of '/var/db/pkg/repo-pfSense.sqlite'
DBG(1)[84696]> Request to fetch pkg+https://pfsense-plus-pkg.netgate.com/pfSense_plus-v24_11_amd64-pfSense_plus_v24_11/meta.conf
DBG(1)[84696]> curl_open
pkg: No SRV record found for the repo 'pfSense'
DBG(1)[84696]> Fetch: fetcher used: pkg+https
DBG(1)[84696]> curl> fetching https://pfsense-plus-pkg.netgate.com/pfSense_plus-v24_11_amd64-pfSense_plus_v24_11/meta.conf
DBG(1)[84696]> CURL> No mirror set url to https://pfsense-plus-pkg.netgate.com/pfSense_plus-v24_11_amd64-pfSense_plus_v24_11/meta.conf
DBG(1)[84696]> CURL> attempting to fetch from https://pfsense-plus-pkg.netgate.com/pfSense_plus-v24_11_amd64-pfSense_plus_v24_11/meta.conf, left retry 3

Couldn't find host pfsense-plus-pkg.netgate.com in the .netrc file; using defaults
Could not resolve host: pfsense-plus-pkg.netgate.com
Closing connection

pkg: An error occured while fetching package
DBG(1)[84696]> Request to fetch pkg+https://pfsense-plus-pkg.netgate.com/pfSense_plus-v24_11_amd64-pfSense_plus_v24_11/meta.txz
pkg: packagesite URL error for pkg+https://pfsense-plus-pkg.netgate.com/pfSense_plus-v24_11_amd64-pfSense_plus_v24_11/meta.txz -- pkg+:// implies SRV mirror type
repository pfSense has no meta file, using default settings
DBG(1)[84696]> Request to fetch pkg+https://pfsense-plus-pkg.netgate.com/pfSense_plus-v24_11_amd64-pfSense_plus_v24_11/packagesite.pkg
pkg: packagesite URL error for pkg+https://pfsense-plus-pkg.netgate.com/pfSense_plus-v24_11_amd64-pfSense_plus_v24_11/packagesite.pkg -- pkg+:// implies SRV mirror type
DBG(1)[84696]> Request to fetch pkg+https://pfsense-plus-pkg.netgate.com/pfSense_plus-v24_11_amd64-pfSense_plus_v24_11/packagesite.txz
pkg: packagesite URL error for pkg+https://pfsense-plus-pkg.netgate.com/pfSense_plus-v24_11_amd64-pfSense_plus_v24_11/packagesite.txz -- pkg+:// implies SRV mirror type

Unable to update repository pfSense
Error updating repositories!

================
Running these commands also fail with same error.
pkg-static clean -ay; pkg-static install -fy pkg pfSense-repo pfSense-upgrade
pkg-static update -f

================
Running : pkg-static -d update returns nothing, no errors, no output.

================
Running host -t srv _https._tcp.packages.netgate.com
Returns:
_https._tcp.packages.netgate.com has no SRV record

======
pfSense-upgrade -d -c
pfSense-repoc-static: failed to fetch the repo data
failed to read the repo data.
failed to update the repository settings!!!
failed to update the repository settings!!!

stephenw10 · Feb 6, 2025, 3:13 PM

@pfnewb2016 said in 6100 check_upgrade(1): unknown error:

pfsense-plus-pkg.netgate.com

That host uses an SRV record so it cannot be resolved directly. But it should show the two pkg hosts like:

[24.11-RELEASE][admin@4200.stevew.lan]/root: host -t srv _https._tcp.pfsense-plus-pkg.netgate.com
_https._tcp.pfsense-plus-pkg.netgate.com has SRV record 10 10 443 pfsense-plus-pkg01.atx.netgate.com.
_https._tcp.pfsense-plus-pkg.netgate.com has SRV record 10 10 443 pfsense-plus-pkg00.atx.netgate.com.

And they then resolve:

[24.11-RELEASE][admin@4200.stevew.lan]/root: host pfsense-plus-pkg00.atx.netgate.com
pfsense-plus-pkg00.atx.netgate.com has address 208.123.73.207
pfsense-plus-pkg00.atx.netgate.com has IPv6 address 2610:160:11:18::207
[24.11-RELEASE][admin@4200.stevew.lan]/root: host pfsense-plus-pkg01.atx.netgate.com
pfsense-plus-pkg01.atx.netgate.com has address 208.123.73.209
pfsense-plus-pkg01.atx.netgate.com has IPv6 address 2610:160:11:18::209

Your output never shows the individual pkg hosts though so it looks to be failing the SRV lookup. Does it work at the CLI as above?

pfnewb2016 · Feb 6, 2025, 6:46 PM

@pfnewb2016 said in 6100 check_upgrade(1): unknown error:
Both host cmds run and don't return any results or errors.

[24.03-RELEASE][admin@fw-fl2-rack-6100-23.client.lan]/root: host pfsense-plus-pkg00.atx.netgate.com
[24.03-RELEASE][admin@fw-fl2-rack-6100-23.client.lan]/root: host pfsense-plus-pkg01.atx.netgate.com
[24.03-RELEASE][admin@fw-fl2-rack-6100-23.client.lan]/root:

pfnewb2016 · Feb 6, 2025, 7:03 PM

@pfnewb2016
I'm going to switch to the 2ndary ISP and test later today but I'm interested in any other ideas.

stephenw10 · Feb 7, 2025, 1:11 AM

Does the SRV query return those hosts at least?

You should try using dig and/or dig @ to see what's happening there.

[25.03-BETA][admin@4200.stevew.lan]/root: dig pfsense-plus-pkg00.atx.netgate.com

; <<>> DiG 9.20.4 <<>> pfsense-plus-pkg00.atx.netgate.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 19888
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1432
;; QUESTION SECTION:
;pfsense-plus-pkg00.atx.netgate.com. IN	A

;; ANSWER SECTION:
pfsense-plus-pkg00.atx.netgate.com. 300	IN A	208.123.73.207

;; Query time: 130 msec
;; SERVER: 127.0.0.1#53(127.0.0.1) (UDP)
;; WHEN: Thu Feb 06 21:58:09 GMT 2025
;; MSG SIZE  rcvd: 79

[25.03-BETA][admin@4200.stevew.lan]/root: dig @8.8.8.8 pfsense-plus-pkg00.atx.netgate.com

; <<>> DiG 9.20.4 <<>> @8.8.8.8 pfsense-plus-pkg00.atx.netgate.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 21860
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;pfsense-plus-pkg00.atx.netgate.com. IN	A

;; ANSWER SECTION:
pfsense-plus-pkg00.atx.netgate.com. 300	IN A	208.123.73.207

;; Query time: 128 msec
;; SERVER: 8.8.8.8#53(8.8.8.8) (UDP)
;; WHEN: Thu Feb 06 22:10:22 GMT 2025
;; MSG SIZE  rcvd: 79

pfnewb2016 · Feb 7, 2025, 1:11 AM

@stephenw10

Thank you for your help, much appreciated.

I flipped to the backup ISP, the update worked immediately, now running 24.11.

This ISP has told me multiple times they aren't blocking / filtering, then I present evidence that it sure looks like they are, they've come back 2x and said we weren't onboarding properly and it's now fixed.
This is a bit different since DNS resolution is working for everything else, but I'm going to blame the ISP and move on.

Thanks again.

stephenw10 · Feb 7, 2025, 1:18 AM

Ah, good result. Hmm, I guess whatever DNS proxy/intercept they were doing doesn't like SRV queries. Fun!

pfnewb2016 · Feb 7, 2025, 1:29 AM

@stephenw10
Fun is more positive than my reaction!