Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Unable to upgrade due to HTTPS scanning (NOT USING A PROXY)

    Scheduled Pinned Locked Moved
    Problems Installing or Upgrading pfSense Software
    3
    29
    549
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • stephenw10S
      stephenw10 Netgate Administrator
      last edited by

      If you have something scanning inside https then that is proxying the traffic to do so. The pkg command used to update will only work if you tell it to use that proxy directly.

      However on 2.7.0 you might also hit this: https://redmine.pfsense.org/issues/14609

      B 1 Reply Last reply Reply Quote 0
      • B
        brianjg @stephenw10
        last edited by

        @stephenw10 OK lets me explain this better. The upstream GW inspects HTTPs traffic, it does not act as a proxy server. Entering the IP in the proxy section just breaks traffic. Every other device (including the current pfsense appliance) is routing traffic just fine. The only bit that is broken is the update option.

        And since I have imported the root CA cert why is pfSense still rejecting the cert as invalid as per the original error message.

        stephenw10S 1 Reply Last reply Reply Quote 0
        • stephenw10S
          stephenw10 Netgate Administrator
          last edited by

          It may not use the word proxy but the only way to inspect https traffic is to proxy it. It's a transparent proxy but it probably still has a proxy port it can listen on directly.

          Can you use fetch at the pfSense CLI directly?

          [2.7.2-RELEASE][admin@t70.stevew.lan]/root: fetch https://www.pfsense.org
www.pfsense.org                                         25 kB   31 MBps    00s

          Or curl?:

          [2.7.2-RELEASE][admin@t70.stevew.lan]/root: curl https://www.pfsense.org
<!DOCTYPE html> <html lang="en"> <head> <script>(function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':
        new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],
        j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src=
        'https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);
....
          B 2 Replies Last reply Reply Quote 0
          • B
            brianjg @stephenw10
            last edited by brianjg

            @stephenw10 Both commands fail.

            curl https://www.pfsense.org

            curl: (60) SSL certificate problem: self signed certificate in certificate chain
More details here: https://curl.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

            fetch https://www.pfsense.org

            Certificate verification failed for /C=US/ST=Texas/L=Round Rock/O=Dell Technologies/OU=Cybersecurity/CN=Dell Technologies Root Certificate Authority 2018
34370682880:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-img-build/BUILD_NODE/amd64-ce/OS_MAJOR_VERSION/freebsd12/PLATFORM/aws/sources/FreeBSD-src-RELENG_2_6_0/crypto/openssl/ssl/statem/statem_clnt.c:1916:
fetch: https://www.pfsense.org: Authentication error
[2.6.0-RELEASE][admin@vxrail-pfsense-gateway.crk.lab.emc.com]/root:
            1 Reply Last reply Reply Quote 0
            • stephenw10S
              stephenw10 Netgate Administrator
              last edited by

              You may need to restart php or reboot if you have not done so since uploading the CA cert.

              What version is your other pfSense device that is working via the proxy?

              B 3 Replies Last reply Reply Quote 0
              • B
                brianjg @stephenw10
                last edited by

                @stephenw10 I've rebooted multiple times, same result.

                This is the only appliance I have.

                1 Reply Last reply Reply Quote 0
                • B
                  brianjg @stephenw10
                  last edited by

                  @stephenw10 Also openssl is connecting fine,

                  openssl s_client pkg01-atx.netgate.com:443
                  CONNECTED(00000003)

                  1 Reply Last reply Reply Quote 0
                  • stephenw10S
                    stephenw10 Netgate Administrator @brianjg
                    last edited by

                    @brianjg said in Unable to upgrade due to HTTPS scanning (NOT USING A PROXY):

                    Every other device (including the current pfsense appliance) is routing traffic just fine.

                    So by that you mean just routing? Not that the current pfSense appliance is able to connect?

                    Is this something new that has just stopped? You recently added the proxy/filter?

                    You can just install 2.7.2 clean and restore the config. The proxy setting is fixed there.

                    B 1 Reply Last reply Reply Quote 0
                    • B
                      brianjg @stephenw10
                      last edited by

                      @stephenw10 I've edited the /usr/local/share/pfSense/pkg/repos/pfSense-repo.conf from https to http only which allowed me to update to the latest 2.6 release.

                      I then changed the branch to 2.7 and exited the file again to try to upgrade but after running the command: # pkg-static clean -ay; pkg-static install -fy pkg pfSense-repo pfSense-upgrade It failed with,

                      pkg-static: Warning: Major OS version upgrade detected.  Running "pkg bootstrap -f" recommended
pkg-static: Repository pfSense-core missing. 'pkg update' required
pkg-static: No package database installed.  Nothing to do!
pkg-static: Warning: Major OS version upgrade detected.  Running "pkg bootstrap -f" recommended
Updating pfSense-core repository catalogue...
Fetching meta.conf: 100%    163 B   0.2kB/s    00:01
Fetching packagesite.pkg: 100%    2 KiB   1.8kB/s    00:01
Processing entries: 100%
pfSense-core repository update completed. 7 packages processed.
Updating pfSense repository catalogue...
Fetching meta.conf: 100%    163 B   0.2kB/s    00:01
Fetching packagesite.pkg: 100%  156 KiB 159.4kB/s    00:01
Processing entries:   0%
pkg-static: Newer FreeBSD version for package unzoo:
To ignore this error set IGNORE_OSVERSION=yes
- package: 1400085
- running kernel: 1203500

pkg-static: repository pfSense contains packages for wrong OS version: FreeBSD:14:amd64
Processing entries: 100%
Unable to update repository pfSense
Error updating repositories!

                      Is there a way to get it to proceed?

                      1 Reply Last reply Reply Quote 0
                      • B
                        brianjg @stephenw10
                        last edited by

                        @stephenw10 said in Unable to upgrade due to HTTPS scanning (NOT USING A PROXY):

                        @brianjg said in Unable to upgrade due to HTTPS scanning (NOT USING A PROXY):

                        Every other device (including the current pfsense appliance) is routing traffic just fine.

                        So by that you mean just routing? Not that the current pfSense appliance is able to connect?

                        Is this something new that has just stopped? You recently added the proxy/filter?

                        You can just install 2.7.2 clean and restore the config. The proxy setting is fixed there.

                        Yes, just routing is working. I've inherited this setup from the previous admin. It must have worked at some stage in the past since packages are installed and I can't install them now.

                        Yes I can deploy a new CE release but the issue persists, so since I can't install packages this isn't a solution.

                        1 Reply Last reply Reply Quote 0
                        • stephenw10S
                          stephenw10 Netgate Administrator
                          last edited by

                          What should work here is to install 2.7.2 clean then set the proxy rather than trying to use it transparently. The proxy setting works correctly in 2.7.2.

                          Can you test that in a VM?

                          The pkg repos are https only from 2.7.0.

                          I would have expected the imported CA cert to work for command line connections. 🤔

                          B 1 Reply Last reply Reply Quote 0
                          • B
                            brianjg @stephenw10
                            last edited by brianjg

                            @stephenw10
                            For any other VM I have no need to define any proxy settings.

                            I know it won't be popular but I deployed OPNsense, imported my CA cert and was able to grab updates/packages without an issue. No proxy settings just defined the upstream gateway.

                            Since I have no visibility beyond my initial upstream GW I've no idea about which device is doing the HTTPs inspection so setting that IP isn't really an option or solution for me.

                            In my mind once I have imported the cert the HTTPS should just work. As a test I deployed the 2.7.2 release and also had the same issue once the cert was imported.

                            1 Reply Last reply Reply Quote 0
                            • stephenw10S
                              stephenw10 Netgate Administrator
                              last edited by stephenw10

                              Ok how exactly did you import the CA cert?

                              Ah, you don't control the upstream proxy? Some other admin just gives you a CA cert to use and you're forced to trust they do the right thing with it?

                              B 1 Reply Last reply Reply Quote 0
                              • stephenw10S
                                stephenw10 Netgate Administrator
                                last edited by

                                Testing here an imported CA cert is used from the OS trust store by curl from the CLI in 2.7.2.

                                What error exactly do you see when you try that?

                                B 1 Reply Last reply Reply Quote 0
                                • stephenw10S
                                  stephenw10 Netgate Administrator
                                  last edited by

                                  Or try running: pkg -d update

                                  What error do you see there?

                                  Part of that output should include:

                                  * Couldn't find host pkg01-atx.netgate.com in the .netrc file; using defaults
* Hostname pkg01-atx.netgate.com was found in DNS cache
*   Trying 208.123.73.209:443...
* Connected to pkg01-atx.netgate.com (208.123.73.209) port 443
* ALPN: curl offers http/1.1
*  CAfile: none
*  CApath: /etc/ssl/certs/
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384

                                  And the imported CA cert should be in /etc/ssl/certs if the trust store box was checked.

                                  1 Reply Last reply Reply Quote 0
                                  • B
                                    brianjg @stephenw10
                                    last edited by

                                    @stephenw10 In short yes, but as I mentioned this pfsense is the only system having an issue with the cert.

                                    We have an internal site which has the pem files available. I downloaded the file, edit in notepad, copy the cert and import it to the UI in pfsense as an existing CA and check the box to add to local trust store.

                                    1 Reply Last reply Reply Quote 0
                                    • B
                                      brianjg @stephenw10
                                      last edited by

                                      @stephenw10 Yes the cert is in the trust store, The command fails when using https.

                                      pkg -d update
                                      DBG(1)[1002]> pkg initialized
                                      Updating pfSense-core repository catalogue...
                                      DBG(1)[1002]> PkgRepo: verifying update for pfSense-core
                                      pkg: Repository pfSense-core has a wrong packagesite, need to re-create database
                                      DBG(1)[1002]> PkgRepo: need forced update of pfSense-core
                                      DBG(1)[1002]> Pkgrepo, begin update of '/var/db/pkg/repo-pfSense-core.sqlite'
                                      DBG(1)[1002]> Request to fetch pkg+https://pkg.pfsense.org/pfSense_v2_6_0_amd64-core/meta.conf
                                      DBG(1)[1002]> opening libfetch fetcher
                                      DBG(1)[1002]> Fetch > libfetch: connecting
                                      DBG(1)[1002]> Fetch: fetching from: https://pkg00-atx.netgate.com/pfSense_v2_6_0_amd64-core/meta.conf with opts "i"
                                      Certificate verification failed for /C=US/ST=Texas/L=Round Rock/O=Dell Technologies/OU=Cybersecurity/CN=Dell Technologies Root Certificate Authority 2018
                                      34375880704:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-img-build/BUILD_NODE/amd64-ce/OS_MAJOR_VERSION/freebsd12/PLATFORM/aws/sources/FreeBSD-src-RELENG_2_6_0/crypto/openssl/ssl/statem/statem_clnt.c:1916:
                                      DBG(1)[1002]> Fetch: fetching from: https://pkg00-atx.netgate.com/pfSense_v2_6_0_amd64-core/meta.conf with opts "i"

                                      1 Reply Last reply Reply Quote 0
                                      • B
                                        brianjg
                                        last edited by

                                        Also is there some reason the issuer shows as self-signed?

                                        6aed7d78-6d4c-4a1a-94c0-57144f6fb6a8-image.png

                                        1 Reply Last reply Reply Quote 0
                                        • B
                                          brianjg @stephenw10
                                          last edited by

                                          @stephenw10 said in Unable to upgrade due to HTTPS scanning (NOT USING A PROXY):

                                          It may not use the word proxy but the only way to inspect https traffic is to proxy it. It's a transparent proxy but it probably still has a proxy port it can listen on directly.

                                          Can you use fetch at the pfSense CLI directly?

                                          [2.7.2-RELEASE][admin@t70.stevew.lan]/root: fetch https://www.pfsense.org
www.pfsense.org                                         25 kB   31 MBps    00s

                                          Or curl?:

                                          [2.7.2-RELEASE][admin@t70.stevew.lan]/root: curl https://www.pfsense.org
<!DOCTYPE html> <html lang="en"> <head> <script>(function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':
        new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],
        j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src=
        'https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);
....

                                          Doing some additional testing led me to this thread. https://redmine.pfsense.org/issues/12737

                                          Using the command # env SSL_CERT_DIR=/etc/ssl/certs/ curl -v https://www.pfsense.org I'm able to connect to the site.

                                          1 Reply Last reply Reply Quote 0
                                          • stephenw10S
                                            stephenw10 Netgate Administrator
                                            last edited by

                                            Ah, OK. Yes in 2.6 that would be an issue. Just running env at the CLI will show you what it is set to.

                                            However that would not be an issue in 2.7.2. I thought you tested it there also and it still failed?

                                            B 1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post