pfsense available packages empty

[DaveWh]

@stephenw10 Seems to be back again:
Updating pfSense-core repository catalogue...
Certificate verification failed for /C=US/ST=Texas/L=Austin/O=Rubicon Communications, LLC (Netgate)/OU=pfSense Plus/CN=pfsense-plus-pkg00.atx.netgate.com
2202644480:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-Plus-snapshots-23_05-main/sources/FreeBSD-src-plus-RELENG_23_05/crypto/openssl/ssl/statem/statem_clnt.c:1921:

many, many more, all snipped out

pkg-static: https://pfsense-plus-pkg00.atx.netgate.com/pfSense_plus-v23_05_1_aarch64-pfSense_plus_v23_05_1/packagesite.txz: Authentication error
Unable to update repository pfSense
Error updating repositories!

[stephenw10]

Much has changed since Jan 2021! And that error is different.

Authentication error like that implies it's trying to use the wrong server CA cert.

The first thing to try here is to resave the update repo branch in System > Updates > Settings and retest.

Steve

[DaveWh]

@stephenw10 Excellent! That fixed it.

[bippus]

if you stumbled into this old thread, there is a new bug and a resolution in v2.70:

@SteveITS said in Package Manager wont load Available Packages:

https://docs.netgate.com/pfsense/en/latest/releases/2-7-1.html#troubleshooting

The update from 2.7.0 to 2.7.1 solved for me the "no Available Packages shown / empty list of Available Packages" Problem.

(the command: certctl rehash at 2.7.0 did not helped)

[TommyMoo]

Hello,
this avaible Package Problem, now appeared suddenly on my pfSense CE aswell :( Very disappointing!!!

I did:
sudo certctl rehash
Password:
Scanning /usr/share/certs/untrusted for certificates...
Scanning /usr/share/certs/trusted for certificates...
Scanning /usr/local/share/certs for certificates...

..... and helped nothing, Internet Connection is OK, using pfSense 2.7.2

What else can I do?? I have already searched here in the Forum, and did all, what people recommended to fix that! HELP please

[stephenw10]

If you're running 2.7.2 then rehashing the certs is done by the system anyway.

Try running: pkg-static -d update

What error is shown?

[TommyMoo]

@stephenw10 said in pfsense available packages empty:

pkg-static -d update

Hello Stephen, Thanks for your reply, I executet your command, result is very long ->

sudo pkg-static -d update
Password:
DBG(1)[78285]> pkg initialized
Updating Unofficial repository catalogue...
DBG(1)[78285]> PkgRepo: verifying update for Unofficial
pkg-static: repository meta has wrong version or wrong format
pkg-static: Repository Unofficial load error: meta file cannot be loaded
DBG(1)[78285]> PkgRepo: need forced update of Unofficial
DBG(1)[78285]> Pkgrepo, begin update of '/var/db/pkg/repo-Unofficial.sqlite'
DBG(1)[78285]> Request to fetch pkg+http://pkg.goxdrive.com.br/repo/FreeBSD:14:amd64/meta.conf
DBG(1)[78285]> curl_open
DBG(1)[78285]> Fetch: fetcher used: pkg+http
DBG(1)[78285]> curl> fetching http://pkg.goxdrive.com.br/repo/FreeBSD:14:amd64/meta.conf

DBG(1)[78285]> CURL> attempting to fetch from , left retry 3

• Couldn't find host e-sac.siteseguro.ws in the .netrc file; using defaults
• Trying 10.10.10.1:80...
• Connected to e-sac.siteseguro.ws (10.10.10.1) port 80

GET /repo/FreeBSD:14:amd64/meta.conf HTTP/1.1
Host: e-sac.siteseguro.ws
User-Agent: pkg/1.20.8
Accept: /
If-Modified-Since: Thu, 01 Jan 1970 00:00:00 GMT

< HTTP/1.1 200 OK
Fetching meta.conf: 0%< Cache-Control: private, no-store, no-cache, must-revalidate, max-age=0
< Cache-Control: post-check=0, pre-check=0
< Pragma: no-cache
< Expires: Sat, 26 Jul 2014 05:00:00 GMT
< Content-Type: image/gif
< Accept-Ranges: bytes
< Content-Length: 43
< Date: Wed, 09 Oct 2024 17:24:32 GMT
< Server: pfBlockerNG DNSBL
<
Fetching meta.conf: 100% 43 B 0.0kB/s 00:01

• Connection #0 to host e-sac.siteseguro.ws left intact
  pkg-static: Impossible to get the value from Last-Modified HTTP header
  pkg-static: cannot parse repository meta: error while parsing <unknown>: line: 1, column: 6 - 'invalid character in a key', character: '0x01'
  repository Unofficial has no meta file, using default settings
  DBG(1)[78285]> Request to fetch pkg+http://pkg.goxdrive.com.br/repo/FreeBSD:14:amd64/packagesite.pkg
  DBG(1)[78285]> curl_open
  DBG(1)[78285]> Fetch: fetcher used: pkg+http
  DBG(1)[78285]> curl> fetching http://pkg.goxdrive.com.br/repo/FreeBSD:14:amd64/packagesite.pkg

DBG(1)[78285]> CURL> attempting to fetch from , left retry 3

• Couldn't find host e-sac.siteseguro.ws in the .netrc file; using defaults
• Found bundle for host: 0x1cf00663c420 [serially]
• Re-using existing connection with host e-sac.siteseguro.ws

GET /repo/FreeBSD:14:amd64/packagesite.pkg HTTP/1.1
Host: e-sac.siteseguro.ws
User-Agent: pkg/1.20.8
Accept: /
If-Modified-Since: Thu, 01 Jan 1970 00:00:00 GMT

< HTTP/1.1 200 OK
Fetching packagesite.pkg: 0%< Cache-Control: private, no-store, no-cache, must-revalidate, max-age=0
< Cache-Control: post-check=0, pre-check=0
< Pragma: no-cache
< Expires: Sat, 26 Jul 2014 05:00:00 GMT
< Content-Type: image/gif
< Accept-Ranges: bytes
< Content-Length: 43
< Date: Wed, 09 Oct 2024 17:24:32 GMT
< Server: pfBlockerNG DNSBL
<
Fetching packagesite.pkg: 100% 43 B 0.0kB/s 00:01

• Connection #0 to host e-sac.siteseguro.ws left intact
  pkg-static: Impossible to get the value from Last-Modified HTTP header
  DBG(1)[78285]> PkgRepo: extracting packagesite.yaml of repo Unofficial
  DBG(1)[86680]> PkgRepo: extracting signature of repo in a sandbox
  pkg-static: Repo extraction failed
  Unable to update repository Unofficial
  Updating pfSense-core repository catalogue...
  DBG(1)[78285]> PkgRepo: verifying update for pfSense-core
  DBG(1)[78285]> Pkgrepo, begin update of '/var/db/pkg/repo-pfSense-core.sqlite'
  DBG(1)[78285]> Request to fetch pkg+https://pkg.pfsense.org/pfSense_v2_7_2_amd64-core/meta.conf
  DBG(1)[78285]> curl_open
  DBG(1)[78285]> Fetch: fetcher used: pkg+https
  DBG(1)[78285]> curl> fetching https://pkg.pfsense.org/pfSense_v2_7_2_amd64-core/meta.conf

DBG(1)[78285]> CURL> attempting to fetch from , left retry 3

• Couldn't find host pkg00-atx.netgate.com in the .netrc file; using defaults
• Trying 208.123.73.207:443...
• Connected to pkg00-atx.netgate.com (208.123.73.207) port 443
• ALPN: curl offers http/1.1
• CAfile: none
• CApath: /etc/ssl/certs/
• SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
• ALPN: server accepted http/1.1
• Server certificate:
• subject: CN=*.netgate.com
• start date: Mar 28 00:00:00 2024 GMT
• expire date: Apr 28 23:59:59 2025 GMT
• subjectAltName: host "pkg00-atx.netgate.com" matched cert's "*.netgate.com"
• issuer: C=GB; ST=Greater Manchester; L=Salford; O=Sectigo Limited; CN=Sectigo RSA Domain Validation Secure Server CA
• SSL certificate verify ok.
• using HTTP/1.1

GET /pfSense_v2_7_2_amd64-core/meta.conf HTTP/1.1
Host: pkg00-atx.netgate.com
User-Agent: pkg/1.20.8
Accept: /
If-Modified-Since: Wed, 06 Dec 2023 21:23:59 GMT

< HTTP/1.1 200 OK
Fetching meta.conf: 0%< Server: nginx
< Date: Wed, 09 Oct 2024 17:24:34 GMT
< Content-Type: application/octet-stream
< Content-Length: 163
< Last-Modified: Wed, 06 Dec 2023 21:23:58 GMT
< Connection: keep-alive
< ETag: "6570e66e-a3"
< Strict-Transport-Security: max-age=31536000; preload
< X-Content-Type-Options: nosniff
< X-XSS-Protection: 1; mode=block
< X-Robots-Tag: all
< X-Download-Options: noopen
< X-Permitted-Cross-Domain-Policies: none
< Accept-Ranges: bytes
<

• The requested document is not new enough
• Simulate an HTTP 304 response
• Closing connection

DBG(1)[78285]> Request to fetch pkg+https://pkg.pfsense.org/pfSense_v2_7_2_amd64-core/packagesite.pkg
DBG(1)[78285]> curl_open
DBG(1)[78285]> Fetch: fetcher used: pkg+https
DBG(1)[78285]> curl> fetching https://pkg.pfsense.org/pfSense_v2_7_2_amd64-core/packagesite.pkg

DBG(1)[78285]> CURL> attempting to fetch from , left retry 3

• Couldn't find host pkg00-atx.netgate.com in the .netrc file; using defaults
• Hostname pkg00-atx.netgate.com was found in DNS cache
• Trying 208.123.73.207:443...
• Connected to pkg00-atx.netgate.com (208.123.73.207) port 443
• ALPN: curl offers http/1.1
• CAfile: none
• CApath: /etc/ssl/certs/
• SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
• ALPN: server accepted http/1.1
• Server certificate:
• subject: CN=*.netgate.com
• start date: Mar 28 00:00:00 2024 GMT
• expire date: Apr 28 23:59:59 2025 GMT
• subjectAltName: host "pkg00-atx.netgate.com" matched cert's "*.netgate.com"
• issuer: C=GB; ST=Greater Manchester; L=Salford; O=Sectigo Limited; CN=Sectigo RSA Domain Validation Secure Server CA
• SSL certificate verify ok.
• using HTTP/1.1

GET /pfSense_v2_7_2_amd64-core/packagesite.pkg HTTP/1.1
Host: pkg00-atx.netgate.com
User-Agent: pkg/1.20.8
Accept: /
If-Modified-Since: Wed, 06 Dec 2023 21:23:59 GMT

< HTTP/1.1 200 OK
Fetching packagesite.pkg: 0%< Server: nginx
< Date: Wed, 09 Oct 2024 17:24:34 GMT
< Content-Type: application/octet-stream
< Content-Length: 1496
< Last-Modified: Wed, 06 Dec 2023 21:23:59 GMT
< Connection: keep-alive
< ETag: "6570e66f-5d8"
< Strict-Transport-Security: max-age=31536000; preload
< X-Content-Type-Options: nosniff
< X-XSS-Protection: 1; mode=block
< X-Robots-Tag: all
< X-Download-Options: noopen
< X-Permitted-Cross-Domain-Policies: none
< Accept-Ranges: bytes
<

• The requested document is not new enough
• Simulate an HTTP 304 response
• Closing connection

pfSense-core repository is up to date.
Updating pfSense repository catalogue...
DBG(1)[78285]> PkgRepo: verifying update for pfSense
DBG(1)[78285]> Pkgrepo, begin update of '/var/db/pkg/repo-pfSense.sqlite'
DBG(1)[78285]> Request to fetch pkg+https://pkg.pfsense.org/pfSense_v2_7_2_amd64-pfSense_v2_7_2/meta.conf
DBG(1)[78285]> curl_open
DBG(1)[78285]> Fetch: fetcher used: pkg+https
DBG(1)[78285]> curl> fetching https://pkg.pfsense.org/pfSense_v2_7_2_amd64-pfSense_v2_7_2/meta.conf

DBG(1)[78285]> CURL> attempting to fetch from , left retry 3

• Couldn't find host pkg00-atx.netgate.com in the .netrc file; using defaults
• Trying 208.123.73.207:443...
• Connected to pkg00-atx.netgate.com (208.123.73.207) port 443
• ALPN: curl offers http/1.1
• CAfile: none
• CApath: /etc/ssl/certs/
• SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
• ALPN: server accepted http/1.1
• Server certificate:
• subject: CN=*.netgate.com
• start date: Mar 28 00:00:00 2024 GMT
• expire date: Apr 28 23:59:59 2025 GMT
• subjectAltName: host "pkg00-atx.netgate.com" matched cert's "*.netgate.com"
• issuer: C=GB; ST=Greater Manchester; L=Salford; O=Sectigo Limited; CN=Sectigo RSA Domain Validation Secure Server CA
• SSL certificate verify ok.
• using HTTP/1.1

GET /pfSense_v2_7_2_amd64-pfSense_v2_7_2/meta.conf HTTP/1.1
Host: pkg00-atx.netgate.com
User-Agent: pkg/1.20.8
Accept: /
If-Modified-Since: Wed, 02 Oct 2024 19:28:18 GMT

< HTTP/1.1 200 OK
Fetching meta.conf: 0%< Server: nginx
< Date: Wed, 09 Oct 2024 17:24:35 GMT
< Content-Type: application/octet-stream
< Content-Length: 178
< Last-Modified: Wed, 02 Oct 2024 19:28:18 GMT
< Connection: keep-alive
< ETag: "66fd9ed2-b2"
< Strict-Transport-Security: max-age=31536000; preload
< X-Content-Type-Options: nosniff
< X-XSS-Protection: 1; mode=block
< X-Robots-Tag: all
< X-Download-Options: noopen
< X-Permitted-Cross-Domain-Policies: none
< Accept-Ranges: bytes
<

• The requested document is not new enough
• Simulate an HTTP 304 response
• Closing connection

DBG(1)[78285]> Request to fetch pkg+https://pkg.pfsense.org/pfSense_v2_7_2_amd64-pfSense_v2_7_2/packagesite.pkg
DBG(1)[78285]> curl_open
DBG(1)[78285]> Fetch: fetcher used: pkg+https
DBG(1)[78285]> curl> fetching https://pkg.pfsense.org/pfSense_v2_7_2_amd64-pfSense_v2_7_2/packagesite.pkg

DBG(1)[78285]> CURL> attempting to fetch from , left retry 3

• Couldn't find host pkg00-atx.netgate.com in the .netrc file; using defaults
• Hostname pkg00-atx.netgate.com was found in DNS cache
• Trying 208.123.73.207:443...
• Connected to pkg00-atx.netgate.com (208.123.73.207) port 443
• ALPN: curl offers http/1.1
• CAfile: none
• CApath: /etc/ssl/certs/
• SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
• ALPN: server accepted http/1.1
• Server certificate:
• subject: CN=*.netgate.com
• start date: Mar 28 00:00:00 2024 GMT
• expire date: Apr 28 23:59:59 2025 GMT
• subjectAltName: host "pkg00-atx.netgate.com" matched cert's "*.netgate.com"
• issuer: C=GB; ST=Greater Manchester; L=Salford; O=Sectigo Limited; CN=Sectigo RSA Domain Validation Secure Server CA
• SSL certificate verify ok.
• using HTTP/1.1

GET /pfSense_v2_7_2_amd64-pfSense_v2_7_2/packagesite.pkg HTTP/1.1
Host: pkg00-atx.netgate.com
User-Agent: pkg/1.20.8
Accept: /
If-Modified-Since: Wed, 02 Oct 2024 19:28:18 GMT

< HTTP/1.1 200 OK
Fetching packagesite.pkg: 0%< Server: nginx
< Date: Wed, 09 Oct 2024 17:24:36 GMT
< Content-Type: application/octet-stream
< Content-Length: 160456
< Last-Modified: Wed, 02 Oct 2024 19:28:18 GMT
< Connection: keep-alive
< ETag: "66fd9ed2-272c8"
< Strict-Transport-Security: max-age=31536000; preload
< X-Content-Type-Options: nosniff
< X-XSS-Protection: 1; mode=block
< X-Robots-Tag: all
< X-Download-Options: noopen
< X-Permitted-Cross-Domain-Policies: none
< Accept-Ranges: bytes
<

• The requested document is not new enough
• Simulate an HTTP 304 response
• Closing connection

pfSense repository is up to date.
Error updating repositories!

---

So strange, as few days ago, it still worked... will read now also the output and trying to understand what happened and how I can fix it...

[TommyMoo]

@stephenw10

Im sorry, looks like pfBlocker blocked someting, as I looked the protocoll result, I saw it went to 10.10.10.1 pf blocker...I switched off pfblocker, and now packages appear again!

Must find out now, why pfBlocker is causing this, at less, I now know, what to look for! Thank you very much!!

[stephenw10]

@TommyMoo said in pfsense available packages empty:

pkg+http://pkg.goxdrive.com.br/repo/FreeBSD:14:amd64/packagesite.pkg

That should not be there. It's some other pkg repo that could be pulling in conflicting pkgs.

That must have been added manually at some point. You should remove it or at least comment it out if you still see problems.

Steve

[Gertjan]

@TommyMoo said in pfsense available packages empty:

DBG(1)[78285]> PkgRepo: verifying update for Unofficial

Lol ... a PC, ok, a phone, why not. But now the firewall wants to take updates from "goxdrive.com.br".

pfSense was trying to take packages from an 'unknown' source.
As per admin instructions .....
Non-pfSense packages can, and often will, break pfSense.
Consider your system as 'non safe' from this moment.

Fastest solution : re install. This will bring you back to a save state and solve your issue.

[TommyMoo]

@Gertjan

Thanks for help, Ill fix that goxdrive.com.br, yes, I wanted to add additional pkg sources, but will never try or do it again. Thanks for bearing with me, sadly a selfmade problem, shame on me! I havent installed anything from that additional pkg source, just wanted to "checkout" what they have :( Problem is fixed, removed that PROBLEMMAKER pkg source, and now its working allf ine again, Learning never stops, will not try in future, to add other pkg sources aynmore.

[Gertjan]

@TommyMoo said in pfsense available packages empty:

I wanted to add additional pkg sources, but will never try or do it again

Oh, but you can
Just keep in mind what is said here.
Read the entire page, "agree with it" and you'll be fine.

For other, non official FreeBSD repositories : its like cleaning your gun with the safety off, and a bullet loaded : be ware of the potential consequences.

[stephenw10]

Yup there is a danger that any alternative repo may contain conflicting pkgs that don't have some customization that pfSense needs. Those can get pulled in and break stuff!

[TommyMoo]

@Gertjan
Ill avoid that, after that lesson, well, nothing bad has happened, luckily.

Im Newbie, on pfsense, been using OpenWRT for years, and wanted to learn new things, as I got now a cool x86 Box with 6 *2,5GB NICs. The only thing I miss, from OpenWRT is Smart Queue Management SQM, that has been performing much better then the bufferbloat fix on pfsense with CoDel.

But Im not complaining, maybe we get SQM one day in future, also on pfsense. With OWRT I had bufferbloat values < 1 second or 1-2 seconds ...with pfsense and codel...I couldnt reach these fantastic values...but overall, many things are better configurable via GUI in pfsense and Im OK with all myself, cheers and greetings from Berlin, Germany, Stay safe and well!

[TommyMoo]

@stephenw10 Thanks for your help, really brought me on the way, to find out the reason. Now all works fine, and as I didnt installed anything from that pkg source, there shouldnt be any problem. THe pkg GUI is working again without any problems. Wonderfull

[Gertjan]

@TommyMoo said in pfsense available packages empty:

maybe we get SQM one day in future, also on pfsense

Just did a test myself .... Totally forgot about that 'buffer bloat' thing.

https://www.waveform.com/tools/bufferbloat?test-id=99e551b1-b006-44ce-8760-33efcb1ff2ec

As I left 'ADSL' (VDSL) behind me last year, fiber now, I'm not looking for a Milli second here and another Milli second there.
better yet : my upload (about 850 Mbit) is as advertised, but I guess a college is downloading 'something', as my download should be the same.

Also, my pfSense is behind a 'ISP' router that handles the RJ45 to 'Led laser light' conversion.

I'm not using any shaper what so ever.

[TommyMoo]

@Gertjan

I from time to time, also love to game, and there, buffbloat fixing is important, congrats to you, to have fiber connection!! Me only on cable internet 1000 MBit (LOL, Vodafone promise is a lie, only keeps that speed in deep night, as cable internet is a shared connection, in evenings, often bad bufferbloat values and speeds below 1000 MBit) and my uplload as always in Germany, is sadly asynchron, limited to only 50 MBit upload sadly. But its all OK, I as a private person/user, can live with that. Enjoy your Fiber connection!! Hope one day, we will also be able to benefit Fiber Quality connections. SQM is just, fantastic on OpenWRT, best for fixing bufferbloat with SQM and piece_of_cake setup.

I do my Bufferbloat testing here -> https://speed.cloudflare.com/

Shows detailed info on nearly all parameters