Unable to upgrade due to HTTPS scanning (NOT USING A PROXY)

stephenw10

What should work here is to install 2.7.2 clean then set the proxy rather than trying to use it transparently. The proxy setting works correctly in 2.7.2.

Can you test that in a VM?

The pkg repos are https only from 2.7.0.

I would have expected the imported CA cert to work for command line connections.

brianjg

@stephenw10
For any other VM I have no need to define any proxy settings.

I know it won't be popular but I deployed OPNsense, imported my CA cert and was able to grab updates/packages without an issue. No proxy settings just defined the upstream gateway.

Since I have no visibility beyond my initial upstream GW I've no idea about which device is doing the HTTPs inspection so setting that IP isn't really an option or solution for me.

In my mind once I have imported the cert the HTTPS should just work. As a test I deployed the 2.7.2 release and also had the same issue once the cert was imported.

stephenw10

Ok how exactly did you import the CA cert?

Ah, you don't control the upstream proxy? Some other admin just gives you a CA cert to use and you're forced to trust they do the right thing with it?

stephenw10

Testing here an imported CA cert is used from the OS trust store by curl from the CLI in 2.7.2.

What error exactly do you see when you try that?

stephenw10

Or try running: pkg -d update

What error do you see there?

Part of that output should include:

* Couldn't find host pkg01-atx.netgate.com in the .netrc file; using defaults
* Hostname pkg01-atx.netgate.com was found in DNS cache
*   Trying 208.123.73.209:443...
* Connected to pkg01-atx.netgate.com (208.123.73.209) port 443
* ALPN: curl offers http/1.1
*  CAfile: none
*  CApath: /etc/ssl/certs/
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384

And the imported CA cert should be in /etc/ssl/certs if the trust store box was checked.

brianjg

@stephenw10 In short yes, but as I mentioned this pfsense is the only system having an issue with the cert.

We have an internal site which has the pem files available. I downloaded the file, edit in notepad, copy the cert and import it to the UI in pfsense as an existing CA and check the box to add to local trust store.

brianjg

@stephenw10 Yes the cert is in the trust store, The command fails when using https.

pkg -d update
DBG(1)[1002]> pkg initialized
Updating pfSense-core repository catalogue...
DBG(1)[1002]> PkgRepo: verifying update for pfSense-core
pkg: Repository pfSense-core has a wrong packagesite, need to re-create database
DBG(1)[1002]> PkgRepo: need forced update of pfSense-core
DBG(1)[1002]> Pkgrepo, begin update of '/var/db/pkg/repo-pfSense-core.sqlite'
DBG(1)[1002]> Request to fetch pkg+https://pkg.pfsense.org/pfSense_v2_6_0_amd64-core/meta.conf
DBG(1)[1002]> opening libfetch fetcher
DBG(1)[1002]> Fetch > libfetch: connecting
DBG(1)[1002]> Fetch: fetching from: https://pkg00-atx.netgate.com/pfSense_v2_6_0_amd64-core/meta.conf with opts "i"
Certificate verification failed for /C=US/ST=Texas/L=Round Rock/O=Dell Technologies/OU=Cybersecurity/CN=Dell Technologies Root Certificate Authority 2018
34375880704:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-img-build/BUILD_NODE/amd64-ce/OS_MAJOR_VERSION/freebsd12/PLATFORM/aws/sources/FreeBSD-src-RELENG_2_6_0/crypto/openssl/ssl/statem/statem_clnt.c:1916:
DBG(1)[1002]> Fetch: fetching from: https://pkg00-atx.netgate.com/pfSense_v2_6_0_amd64-core/meta.conf with opts "i"

brianjg

Also is there some reason the issuer shows as self-signed?

brianjg

@stephenw10 said in Unable to upgrade due to HTTPS scanning (NOT USING A PROXY):

It may not use the word proxy but the only way to inspect https traffic is to proxy it. It's a transparent proxy but it probably still has a proxy port it can listen on directly.

Can you use fetch at the pfSense CLI directly?

[2.7.2-RELEASE][admin@t70.stevew.lan]/root: fetch https://www.pfsense.org
www.pfsense.org                                         25 kB   31 MBps    00s

Or curl?:

[2.7.2-RELEASE][admin@t70.stevew.lan]/root: curl https://www.pfsense.org
<!DOCTYPE html> <html lang="en"> <head> <script>(function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':
        new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],
        j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src=
        'https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);
....

Doing some additional testing led me to this thread. https://redmine.pfsense.org/issues/12737

Using the command # env SSL_CERT_DIR=/etc/ssl/certs/ curl -v https://www.pfsense.org I'm able to connect to the site.

stephenw10

Ah, OK. Yes in 2.6 that would be an issue. Just running env at the CLI will show you what it is set to.

However that would not be an issue in 2.7.2. I thought you tested it there also and it still failed?

brianjg

@stephenw10 said in Unable to upgrade due to HTTPS scanning (NOT USING A PROXY):

Ah, OK. Yes in 2.6 that would be an issue. Just running env at the CLI will show you what it is set to.

However that would not be an issue in 2.7.2. I thought you tested it there also and it still failed?

Correct, I had the same experience in 2.6 and 2.7.2 env output in 2.7.2 does show the path however it still fails.

stephenw10

@brianjg said in Unable to upgrade due to HTTPS scanning (NOT USING A PROXY):

env SSL_CERT_DIR=/etc/ssl/certs/ curl -v https://www.pfsense.org

Yet that works there? And using the system env (which appears to be the same!) doesn't?