pfSense Packages loading very slow

Gamienator 0

Hi there,
so the packages on the webinterface are really slow loading for two Days. Seeing in the Logs:

May 16 10:33:59	php-fpm	66149	/pkg_mgr.php: The command '/usr/local/sbin/pfSense-repo-setup' returned exit code '14', the output was ''

Running on CE 2.7.2, does anybody has to wait aswell 1 - 2 minutes on the Webinterface until you're able to choose packages?

Cheers,
Gamie

Gertjan

@Gamienator-0

In case of doubt, go console or SSH, and see what "13" can do for you.

stephenw10

Like trying to see the list of available packages in the package manager?

Try running at the command line: pkg-static -d update

Gamienator 0

Looks like there is something wrong for me:

@Gertjan said in pfSense Packages loading very slow:

In case of doubt, go console or SSH, and see what "13" can do for you.

Enter an option: 13

>>> Updating repositories metadata... 
Updating pfSense-core repository catalogue...
Fetching meta.conf: . done
Fetching packagesite.pkg: . done
Processing entries: . done
pfSense-core repository update completed. 4 packages processed.
Updating pfSense repository catalogue...
Fetching meta.conf: . done
Fetching packagesite.pkg: ......... done
Processing entries: .......... done
pfSense repository update completed. 550 packages processed.
All repositories are up to date.
Your packages are up to date

@stephenw10 said in pfSense Packages loading very slow:

Like trying to see the list of available packages in the package manager?

Try running at the command line: pkg-static -d update

DBG(1)[47209]> pkg initialized
Updating pfSense-core repository catalogue...
DBG(1)[47209]> PkgRepo: verifying update for pfSense-core
DBG(1)[47209]> Pkgrepo, begin update of '/var/db/pkg/repo-pfSense-core.sqlite'
DBG(1)[47209]> Request to fetch pkg+https://pkg.pfsense.org/pfSense_v2_7_2_amd64-core/meta.conf
DBG(1)[47209]> curl_open
DBG(1)[47209]> Fetch: fetcher used: pkg+https
DBG(1)[47209]> curl> fetching https://pkg.pfsense.org/pfSense_v2_7_2_amd64-core/meta.conf

DBG(1)[47209]> CURL> attempting to fetch from , left retry 3

* Couldn't find host pkg01-atx.netgate.com in the .netrc file; using defaults
*   Trying [2610:160:11:18::209]:443...
* Connected to pkg01-atx.netgate.com (2610:160:11:18::209) port 443
* ALPN: curl offers http/1.1
*  CAfile: none
*  CApath: /etc/ssl/certs/
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN: server accepted http/1.1
* Server certificate:
*  subject: CN=*.netgate.com
*  start date: Mar 28 00:00:00 2024 GMT
*  expire date: Apr 28 23:59:59 2025 GMT
*  subjectAltName: host "pkg01-atx.netgate.com" matched cert's "*.netgate.com"
*  issuer: C=GB; ST=Greater Manchester; L=Salford; O=Sectigo Limited; CN=Sectigo RSA Domain Validation Secure Server CA
*  SSL certificate verify ok.
* using HTTP/1.1
> GET /pfSense_v2_7_2_amd64-core/meta.conf HTTP/1.1
Host: pkg01-atx.netgate.com
User-Agent: pkg/1.20.8
Accept: */*
If-Modified-Since: Wed, 06 Dec 2023 21:23:59 GMT

< HTTP/1.1 200 OK
Fetching meta.conf:   0%< Server: nginx
< Date: Thu, 16 May 2024 14:30:21 GMT
< Content-Type: application/octet-stream
< Content-Length: 163
< Last-Modified: Wed, 06 Dec 2023 21:23:58 GMT
< Connection: keep-alive
< ETag: "6570e66e-a3"
< Strict-Transport-Security: max-age=31536000; preload
< X-Content-Type-Options: nosniff
< X-XSS-Protection: 1; mode=block
< X-Robots-Tag: all
< X-Download-Options: noopen
< X-Permitted-Cross-Domain-Policies: none
< Accept-Ranges: bytes
< 
* The requested document is not new enough
* Simulate an HTTP 304 response
* Closing connection

DBG(1)[47209]> Request to fetch pkg+https://pkg.pfsense.org/pfSense_v2_7_2_amd64-core/packagesite.pkg
DBG(1)[47209]> curl_open
DBG(1)[47209]> Fetch: fetcher used: pkg+https
DBG(1)[47209]> curl> fetching https://pkg.pfsense.org/pfSense_v2_7_2_amd64-core/packagesite.pkg

DBG(1)[47209]> CURL> attempting to fetch from , left retry 3

* Couldn't find host pkg01-atx.netgate.com in the .netrc file; using defaults
* Hostname pkg01-atx.netgate.com was found in DNS cache
*   Trying [2610:160:11:18::209]:443...
* Connected to pkg01-atx.netgate.com (2610:160:11:18::209) port 443
* ALPN: curl offers http/1.1
*  CAfile: none
*  CApath: /etc/ssl/certs/
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN: server accepted http/1.1
* Server certificate:
*  subject: CN=*.netgate.com
*  start date: Mar 28 00:00:00 2024 GMT
*  expire date: Apr 28 23:59:59 2025 GMT
*  subjectAltName: host "pkg01-atx.netgate.com" matched cert's "*.netgate.com"
*  issuer: C=GB; ST=Greater Manchester; L=Salford; O=Sectigo Limited; CN=Sectigo RSA Domain Validation Secure Server CA
*  SSL certificate verify ok.
* using HTTP/1.1
> GET /pfSense_v2_7_2_amd64-core/packagesite.pkg HTTP/1.1
Host: pkg01-atx.netgate.com
User-Agent: pkg/1.20.8
Accept: */*
If-Modified-Since: Wed, 06 Dec 2023 21:23:59 GMT

< HTTP/1.1 200 OK
Fetching packagesite.pkg:   0%< Server: nginx
< Date: Thu, 16 May 2024 14:30:22 GMT
< Content-Type: application/octet-stream
< Content-Length: 1496
< Last-Modified: Wed, 06 Dec 2023 21:23:59 GMT
< Connection: keep-alive
< ETag: "6570e66f-5d8"
< Strict-Transport-Security: max-age=31536000; preload
< X-Content-Type-Options: nosniff
< X-XSS-Protection: 1; mode=block
< X-Robots-Tag: all
< X-Download-Options: noopen
< X-Permitted-Cross-Domain-Policies: none
< Accept-Ranges: bytes
< 
* The requested document is not new enough
* Simulate an HTTP 304 response
* Closing connection

pfSense-core repository is up to date.
Updating pfSense repository catalogue...
DBG(1)[47209]> PkgRepo: verifying update for pfSense
DBG(1)[47209]> Pkgrepo, begin update of '/var/db/pkg/repo-pfSense.sqlite'
DBG(1)[47209]> Request to fetch pkg+https://pkg.pfsense.org/pfSense_v2_7_2_amd64-pfSense_v2_7_2/meta.conf
DBG(1)[47209]> curl_open
DBG(1)[47209]> Fetch: fetcher used: pkg+https
DBG(1)[47209]> curl> fetching https://pkg.pfsense.org/pfSense_v2_7_2_amd64-pfSense_v2_7_2/meta.conf

DBG(1)[47209]> CURL> attempting to fetch from , left retry 3

* Couldn't find host pkg01-atx.netgate.com in the .netrc file; using defaults
*   Trying [2610:160:11:18::209]:443...
* Connected to pkg01-atx.netgate.com (2610:160:11:18::209) port 443
* ALPN: curl offers http/1.1
*  CAfile: none
*  CApath: /etc/ssl/certs/
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN: server accepted http/1.1
* Server certificate:
*  subject: CN=*.netgate.com
*  start date: Mar 28 00:00:00 2024 GMT
*  expire date: Apr 28 23:59:59 2025 GMT
*  subjectAltName: host "pkg01-atx.netgate.com" matched cert's "*.netgate.com"
*  issuer: C=GB; ST=Greater Manchester; L=Salford; O=Sectigo Limited; CN=Sectigo RSA Domain Validation Secure Server CA
*  SSL certificate verify ok.
* using HTTP/1.1
> GET /pfSense_v2_7_2_amd64-pfSense_v2_7_2/meta.conf HTTP/1.1
Host: pkg01-atx.netgate.com
User-Agent: pkg/1.20.8
Accept: */*
If-Modified-Since: Mon, 13 May 2024 19:08:35 GMT

< HTTP/1.1 200 OK
Fetching meta.conf:   0%< Server: nginx
< Date: Thu, 16 May 2024 14:30:22 GMT
< Content-Type: application/octet-stream
< Content-Length: 178
< Last-Modified: Mon, 13 May 2024 19:08:35 GMT
< Connection: keep-alive
< ETag: "66426533-b2"
< Strict-Transport-Security: max-age=31536000; preload
< X-Content-Type-Options: nosniff
< X-XSS-Protection: 1; mode=block
< X-Robots-Tag: all
< X-Download-Options: noopen
< X-Permitted-Cross-Domain-Policies: none
< Accept-Ranges: bytes
< 
* The requested document is not new enough
* Simulate an HTTP 304 response
* Closing connection

DBG(1)[47209]> Request to fetch pkg+https://pkg.pfsense.org/pfSense_v2_7_2_amd64-pfSense_v2_7_2/packagesite.pkg
DBG(1)[47209]> curl_open
DBG(1)[47209]> Fetch: fetcher used: pkg+https
DBG(1)[47209]> curl> fetching https://pkg.pfsense.org/pfSense_v2_7_2_amd64-pfSense_v2_7_2/packagesite.pkg

DBG(1)[47209]> CURL> attempting to fetch from , left retry 3

* Couldn't find host pkg01-atx.netgate.com in the .netrc file; using defaults
* Hostname pkg01-atx.netgate.com was found in DNS cache
*   Trying [2610:160:11:18::209]:443...
* Connected to pkg01-atx.netgate.com (2610:160:11:18::209) port 443
* ALPN: curl offers http/1.1
*  CAfile: none
*  CApath: /etc/ssl/certs/
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN: server accepted http/1.1
* Server certificate:
*  subject: CN=*.netgate.com
*  start date: Mar 28 00:00:00 2024 GMT
*  expire date: Apr 28 23:59:59 2025 GMT
*  subjectAltName: host "pkg01-atx.netgate.com" matched cert's "*.netgate.com"
*  issuer: C=GB; ST=Greater Manchester; L=Salford; O=Sectigo Limited; CN=Sectigo RSA Domain Validation Secure Server CA
*  SSL certificate verify ok.
* using HTTP/1.1
> GET /pfSense_v2_7_2_amd64-pfSense_v2_7_2/packagesite.pkg HTTP/1.1
Host: pkg01-atx.netgate.com
User-Agent: pkg/1.20.8
Accept: */*
If-Modified-Since: Mon, 13 May 2024 19:08:35 GMT

< HTTP/1.1 200 OK
Fetching packagesite.pkg:   0%< Server: nginx
< Date: Thu, 16 May 2024 14:30:23 GMT
< Content-Type: application/octet-stream
< Content-Length: 160360
< Last-Modified: Mon, 13 May 2024 19:08:35 GMT
< Connection: keep-alive
< ETag: "66426533-27268"
< Strict-Transport-Security: max-age=31536000; preload
< X-Content-Type-Options: nosniff
< X-XSS-Protection: 1; mode=block
< X-Robots-Tag: all
< X-Download-Options: noopen
< X-Permitted-Cross-Domain-Policies: none
< Accept-Ranges: bytes
< 
* The requested document is not new enough
* Simulate an HTTP 304 response
* Closing connection

pfSense repository is up to date.
All repositories are up to date.

stephenw10

Those look good. Do they complete in a reasonable time?

Gertjan

@Gamienator-0 said in pfSense Packages loading very slow:

pkg-static -d update

It's trying, using the default IPv6 ... humm ...
Try also the good old IPv4 :

pkg-static -4 -d update

edit : oh, wait ... It was actually doing just fine ... sorry, never mind.

Gamienator 0

@stephenw10 Ah I See, 304 is not new enough. Year they do. Thats weird.

stephenw10

Yes that's expected. It should complete that in a few seconds.

Gamienator 0

@stephenw10 Sure thing. I‘m wondering why the command

/usr/local/sbin/pfSense-repo-setup

takes 1-2 minutes

stephenw10

Hmm, runs in <2s here:

[2.7.2-RELEASE][admin@t70.stevew.lan]/root: time pfSense-repo-setup
Messages:
Your TAC Subscription expired on 2023-07-08 05:00:00. Please renew your TAC Subscription or purchase a pfSense+ subscription by visiting the <a href="https://shop.netgate.com/products/pfsense-software-subscription" target="_blank" rel="noopener noreferrer">Netgate store</a>.
0.630u 0.217s 0:01.34 62.6%	3541+1121k 0+12io 12pf+0w

I assume you have the update branch set to 2.7.2?

Gamienator 0

@stephenw10 Exactly, it's 2.7.2

stephenw10

Hmm, no idea why that would take so long then. Try resaving the branch in update settings.

kesawi

I'm having a similar issue this morning as well.

Version information on the status page took a while to update and then returned the error message Unable to check for updates.

Going to the system update page is taking around a minute to load, and then the yellow retrieving cog sits there spinning for another minute before updating.

Package manager is also taking a long time to retrieve installed and available packages. When I updated from the status page it took around 30-60 seconds after clicking on the confirm button before the package update would start.

@stephenw10 said in pfSense Packages loading very slow:

Hmm, no idea why that would take so long then. Try resaving the branch in update settings.

I've tried resaving the branch to 2.7.2 in the update settings and that does not appear to have rectified it.

stephenw10

Hmm, it looks like there is an issue at the data center end currently that could be causing a problem here.

No noticable delays here right now though:

[24.03-RELEASE][admin@4200.stevew.lan]/root: pfSense-repoc -N
[24.03-RELEASE][admin@4200.stevew.lan]/root: time pkg update
Updating pfSense-core repository catalogue...
Fetching meta.conf: 100%    178 B   0.2kB/s    00:01    
Fetching packagesite.pkg: 100%    2 KiB   1.6kB/s    00:01    
Processing entries: 100%
pfSense-core repository update completed. 5 packages processed.
Updating pfSense repository catalogue...
pfSense repository is up to date.
All repositories are up to date.
0.125u 0.029s 0:01.47 9.5%	3223+296k 1+315io 3pf+0w

kesawi

Running that same command produces no noticeable delays for me, however accessing the system update and package manager through the webgui is very slow.

time pkg update
Updating pfSense-core repository catalogue...
Fetching meta.conf:   0%
Fetching packagesite.pkg:   0%
pfSense-core repository is up to date.
Updating pfSense repository catalogue...
Fetching meta.conf:   0%
Fetching packagesite.pkg:   0%
pfSense repository is up to date.
All repositories are up to date.
0.018u 0.013s 0:04.02 0.4%      7840+630k 85+0io 79pf+0w

Gertjan

@kesawi

Same thing for me, "time pkg update" on the console finished in a second or so.
The GUI, showing "Installed packages" or "Available packages" takes more then a minute to finish.

At first I thought it was a IPv6/IPv4 issue, so I forced to "-4" (modified script in /usr/local/libexec/pfSense-upgrade) but no dice.

stephenw10

Hmm, still slow now?

Does pfSense-repoc complete in reasonable time?

kesawi

@stephenw10 no, it's still taking a long time.

time pfSense-repoc
pfSense-repoc0.093u 0.028s 1:16.03 0.1% 2158+475k 0+40io 0pf+0w

WebGUI takes even longer.

stephenw10

Hmm, do you have IPv6 enabled?

stephenw10

Yeah seems like some IPv6 issue. Try setting 'Prefer IPv4' in Sys > Adv > Networking and retest.