Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Unable to upgrade due to HTTPS scanning (NOT USING A PROXY)

    Scheduled Pinned Locked Moved Problems Installing or Upgrading pfSense Software
    29 Posts 3 Posters 1.2k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • stephenw10S
      stephenw10 Netgate Administrator
      last edited by

      It may not use the word proxy but the only way to inspect https traffic is to proxy it. It's a transparent proxy but it probably still has a proxy port it can listen on directly.

      Can you use fetch at the pfSense CLI directly?

      [2.7.2-RELEASE][admin@t70.stevew.lan]/root: fetch https://www.pfsense.org
www.pfsense.org                                         25 kB   31 MBps    00s

      Or curl?:

      [2.7.2-RELEASE][admin@t70.stevew.lan]/root: curl https://www.pfsense.org
<!DOCTYPE html> <html lang="en"> <head> <script>(function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':
        new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],
        j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src=
        'https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);
....
      B 2 Replies Last reply Reply Quote 0
      • B
        brianjg @stephenw10
        last edited by brianjg

        @stephenw10 Both commands fail.

        curl https://www.pfsense.org

        curl: (60) SSL certificate problem: self signed certificate in certificate chain
More details here: https://curl.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

        fetch https://www.pfsense.org

        Certificate verification failed for /C=US/ST=Texas/L=Round Rock/O=Dell Technologies/OU=Cybersecurity/CN=Dell Technologies Root Certificate Authority 2018
34370682880:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-img-build/BUILD_NODE/amd64-ce/OS_MAJOR_VERSION/freebsd12/PLATFORM/aws/sources/FreeBSD-src-RELENG_2_6_0/crypto/openssl/ssl/statem/statem_clnt.c:1916:
fetch: https://www.pfsense.org: Authentication error
[2.6.0-RELEASE][admin@vxrail-pfsense-gateway.crk.lab.emc.com]/root:
        1 Reply Last reply Reply Quote 0
        • stephenw10S
          stephenw10 Netgate Administrator
          last edited by

          You may need to restart php or reboot if you have not done so since uploading the CA cert.

          What version is your other pfSense device that is working via the proxy?

          B 3 Replies Last reply Reply Quote 0
          • B
            brianjg @stephenw10
            last edited by

            @stephenw10 I've rebooted multiple times, same result.

            This is the only appliance I have.

            1 Reply Last reply Reply Quote 0
            • B
              brianjg @stephenw10
              last edited by

              @stephenw10 Also openssl is connecting fine,

              openssl s_client pkg01-atx.netgate.com:443
              CONNECTED(00000003)

              1 Reply Last reply Reply Quote 0
              • stephenw10S
                stephenw10 Netgate Administrator @brianjg
                last edited by

                @brianjg said in Unable to upgrade due to HTTPS scanning (NOT USING A PROXY):

                Every other device (including the current pfsense appliance) is routing traffic just fine.

                So by that you mean just routing? Not that the current pfSense appliance is able to connect?

                Is this something new that has just stopped? You recently added the proxy/filter?

                You can just install 2.7.2 clean and restore the config. The proxy setting is fixed there.

                B 1 Reply Last reply Reply Quote 0
                • B
                  brianjg @stephenw10
                  last edited by

                  @stephenw10 I've edited the /usr/local/share/pfSense/pkg/repos/pfSense-repo.conf from https to http only which allowed me to update to the latest 2.6 release.

                  I then changed the branch to 2.7 and exited the file again to try to upgrade but after running the command: # pkg-static clean -ay; pkg-static install -fy pkg pfSense-repo pfSense-upgrade It failed with,

                  pkg-static: Warning: Major OS version upgrade detected.  Running "pkg bootstrap -f" recommended
pkg-static: Repository pfSense-core missing. 'pkg update' required
pkg-static: No package database installed.  Nothing to do!
pkg-static: Warning: Major OS version upgrade detected.  Running "pkg bootstrap -f" recommended
Updating pfSense-core repository catalogue...
Fetching meta.conf: 100%    163 B   0.2kB/s    00:01
Fetching packagesite.pkg: 100%    2 KiB   1.8kB/s    00:01
Processing entries: 100%
pfSense-core repository update completed. 7 packages processed.
Updating pfSense repository catalogue...
Fetching meta.conf: 100%    163 B   0.2kB/s    00:01
Fetching packagesite.pkg: 100%  156 KiB 159.4kB/s    00:01
Processing entries:   0%
pkg-static: Newer FreeBSD version for package unzoo:
To ignore this error set IGNORE_OSVERSION=yes
- package: 1400085
- running kernel: 1203500

pkg-static: repository pfSense contains packages for wrong OS version: FreeBSD:14:amd64
Processing entries: 100%
Unable to update repository pfSense
Error updating repositories!

                  Is there a way to get it to proceed?

                  1 Reply Last reply Reply Quote 0
                  • B
                    brianjg @stephenw10
                    last edited by

                    @stephenw10 said in Unable to upgrade due to HTTPS scanning (NOT USING A PROXY):

                    @brianjg said in Unable to upgrade due to HTTPS scanning (NOT USING A PROXY):

                    Every other device (including the current pfsense appliance) is routing traffic just fine.

                    So by that you mean just routing? Not that the current pfSense appliance is able to connect?

                    Is this something new that has just stopped? You recently added the proxy/filter?

                    You can just install 2.7.2 clean and restore the config. The proxy setting is fixed there.

                    Yes, just routing is working. I've inherited this setup from the previous admin. It must have worked at some stage in the past since packages are installed and I can't install them now.

                    Yes I can deploy a new CE release but the issue persists, so since I can't install packages this isn't a solution.

                    1 Reply Last reply Reply Quote 0
                    • stephenw10S
                      stephenw10 Netgate Administrator
                      last edited by

                      What should work here is to install 2.7.2 clean then set the proxy rather than trying to use it transparently. The proxy setting works correctly in 2.7.2.

                      Can you test that in a VM?

                      The pkg repos are https only from 2.7.0.

                      I would have expected the imported CA cert to work for command line connections. 🤔

                      B 1 Reply Last reply Reply Quote 0
                      • B
                        brianjg @stephenw10
                        last edited by brianjg

                        @stephenw10
                        For any other VM I have no need to define any proxy settings.

                        I know it won't be popular but I deployed OPNsense, imported my CA cert and was able to grab updates/packages without an issue. No proxy settings just defined the upstream gateway.

                        Since I have no visibility beyond my initial upstream GW I've no idea about which device is doing the HTTPs inspection so setting that IP isn't really an option or solution for me.

                        In my mind once I have imported the cert the HTTPS should just work. As a test I deployed the 2.7.2 release and also had the same issue once the cert was imported.

                        1 Reply Last reply Reply Quote 0
                        • stephenw10S
                          stephenw10 Netgate Administrator
                          last edited by stephenw10

                          Ok how exactly did you import the CA cert?

                          Ah, you don't control the upstream proxy? Some other admin just gives you a CA cert to use and you're forced to trust they do the right thing with it?

                          B 1 Reply Last reply Reply Quote 0
                          • stephenw10S
                            stephenw10 Netgate Administrator
                            last edited by

                            Testing here an imported CA cert is used from the OS trust store by curl from the CLI in 2.7.2.

                            What error exactly do you see when you try that?

                            B 1 Reply Last reply Reply Quote 0
                            • stephenw10S
                              stephenw10 Netgate Administrator
                              last edited by

                              Or try running: pkg -d update

                              What error do you see there?

                              Part of that output should include:

                              * Couldn't find host pkg01-atx.netgate.com in the .netrc file; using defaults
* Hostname pkg01-atx.netgate.com was found in DNS cache
*   Trying 208.123.73.209:443...
* Connected to pkg01-atx.netgate.com (208.123.73.209) port 443
* ALPN: curl offers http/1.1
*  CAfile: none
*  CApath: /etc/ssl/certs/
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384

                              And the imported CA cert should be in /etc/ssl/certs if the trust store box was checked.

                              1 Reply Last reply Reply Quote 0
                              • B
                                brianjg @stephenw10
                                last edited by

                                @stephenw10 In short yes, but as I mentioned this pfsense is the only system having an issue with the cert.

                                We have an internal site which has the pem files available. I downloaded the file, edit in notepad, copy the cert and import it to the UI in pfsense as an existing CA and check the box to add to local trust store.

                                1 Reply Last reply Reply Quote 0
                                • B
                                  brianjg @stephenw10
                                  last edited by

                                  @stephenw10 Yes the cert is in the trust store, The command fails when using https.

                                  pkg -d update
                                  DBG(1)[1002]> pkg initialized
                                  Updating pfSense-core repository catalogue...
                                  DBG(1)[1002]> PkgRepo: verifying update for pfSense-core
                                  pkg: Repository pfSense-core has a wrong packagesite, need to re-create database
                                  DBG(1)[1002]> PkgRepo: need forced update of pfSense-core
                                  DBG(1)[1002]> Pkgrepo, begin update of '/var/db/pkg/repo-pfSense-core.sqlite'
                                  DBG(1)[1002]> Request to fetch pkg+https://pkg.pfsense.org/pfSense_v2_6_0_amd64-core/meta.conf
                                  DBG(1)[1002]> opening libfetch fetcher
                                  DBG(1)[1002]> Fetch > libfetch: connecting
                                  DBG(1)[1002]> Fetch: fetching from: https://pkg00-atx.netgate.com/pfSense_v2_6_0_amd64-core/meta.conf with opts "i"
                                  Certificate verification failed for /C=US/ST=Texas/L=Round Rock/O=Dell Technologies/OU=Cybersecurity/CN=Dell Technologies Root Certificate Authority 2018
                                  34375880704:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-img-build/BUILD_NODE/amd64-ce/OS_MAJOR_VERSION/freebsd12/PLATFORM/aws/sources/FreeBSD-src-RELENG_2_6_0/crypto/openssl/ssl/statem/statem_clnt.c:1916:
                                  DBG(1)[1002]> Fetch: fetching from: https://pkg00-atx.netgate.com/pfSense_v2_6_0_amd64-core/meta.conf with opts "i"

                                  1 Reply Last reply Reply Quote 0
                                  • B
                                    brianjg
                                    last edited by

                                    Also is there some reason the issuer shows as self-signed?

                                    6aed7d78-6d4c-4a1a-94c0-57144f6fb6a8-image.png

                                    1 Reply Last reply Reply Quote 0
                                    • B
                                      brianjg @stephenw10
                                      last edited by

                                      @stephenw10 said in Unable to upgrade due to HTTPS scanning (NOT USING A PROXY):

                                      It may not use the word proxy but the only way to inspect https traffic is to proxy it. It's a transparent proxy but it probably still has a proxy port it can listen on directly.

                                      Can you use fetch at the pfSense CLI directly?

                                      [2.7.2-RELEASE][admin@t70.stevew.lan]/root: fetch https://www.pfsense.org
www.pfsense.org                                         25 kB   31 MBps    00s

                                      Or curl?:

                                      [2.7.2-RELEASE][admin@t70.stevew.lan]/root: curl https://www.pfsense.org
<!DOCTYPE html> <html lang="en"> <head> <script>(function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':
        new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],
        j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src=
        'https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);
....

                                      Doing some additional testing led me to this thread. https://redmine.pfsense.org/issues/12737

                                      Using the command # env SSL_CERT_DIR=/etc/ssl/certs/ curl -v https://www.pfsense.org I'm able to connect to the site.

                                      1 Reply Last reply Reply Quote 0
                                      • stephenw10S
                                        stephenw10 Netgate Administrator
                                        last edited by

                                        Ah, OK. Yes in 2.6 that would be an issue. Just running env at the CLI will show you what it is set to.

                                        However that would not be an issue in 2.7.2. I thought you tested it there also and it still failed?

                                        B 1 Reply Last reply Reply Quote 0
                                        • B
                                          brianjg @stephenw10
                                          last edited by

                                          @stephenw10 said in Unable to upgrade due to HTTPS scanning (NOT USING A PROXY):

                                          Ah, OK. Yes in 2.6 that would be an issue. Just running env at the CLI will show you what it is set to.

                                          However that would not be an issue in 2.7.2. I thought you tested it there also and it still failed?

                                          Correct, I had the same experience in 2.6 and 2.7.2 env output in 2.7.2 does show the path however it still fails.

                                          4066663d-ce0d-47c6-803d-220745b4a484-image.png

                                          1 Reply Last reply Reply Quote 0
                                          • stephenw10S
                                            stephenw10 Netgate Administrator
                                            last edited by

                                            @brianjg said in Unable to upgrade due to HTTPS scanning (NOT USING A PROXY):

                                            env SSL_CERT_DIR=/etc/ssl/certs/ curl -v https://www.pfsense.org

                                            Yet that works there? And using the system env (which appears to be the same!) doesn't?

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.