repo01.netgate.com TLS cert seems invalid

johnpoz

@seanmcb said in repo01.netgate.com TLS cert seems invalid:

don't see how it can be client side, as I said, ssllabs.com also reports an invalid cert

Not all things are used in the fashion that ssllabs tests for - the cert could be perfectly valid for how its used by pfsense. Such tests are not always valid ways to test functionality.

Maybe @stephenw10 could link to info or hint at how this specific server is used with pfsense and updates. To explain why ssllabs fails - even though working fine. I am not sure on the details, just know that not all things always work as ssllabs tests for..

stephenw10

Right, because it's never supposed to be accessed like that.

What do you see at the command line? For example:

[21.05.1-RELEASE][admin@1100-2.stevew.lan]/root: pkg -d update
DBG(1)[50476]> pkg initialized
Updating pfSense-core repository catalogue...
DBG(1)[50476]> PkgRepo: verifying update for pfSense-core
DBG(1)[50476]> Pkgrepo, begin update of '/var/db/pkg/repo-pfSense-core.sqlite'
DBG(1)[50476]> Request to fetch pkg+https://repo.netgate.com/pkg/pfSense_plus-v21_05_1_aarch64-core/meta.conf
DBG(1)[50476]> opening libfetch fetcher
DBG(1)[50476]> Fetch > libfetch: connecting
DBG(1)[50476]> Fetch: fetching from: https://repo00.netgate.com/pkg/pfSense_plus-v21_05_1_aarch64-core/meta.conf with opts "i"
DBG(1)[50476]> Fetch: fetcher chosen: https
DBG(1)[50476]> Request to fetch pkg+https://repo.netgate.com/pkg/pfSense_plus-v21_05_1_aarch64-core/packagesite.txz
DBG(1)[50476]> opening libfetch fetcher
DBG(1)[50476]> Fetch > libfetch: connecting
DBG(1)[50476]> Fetch: fetching from: https://repo00.netgate.com/pkg/pfSense_plus-v21_05_1_aarch64-core/packagesite.txz with opts "i"
pfSense-core repository is up to date.
Updating pfSense repository catalogue...
DBG(1)[50476]> PkgRepo: verifying update for pfSense
DBG(1)[50476]> Pkgrepo, begin update of '/var/db/pkg/repo-pfSense.sqlite'
DBG(1)[50476]> Request to fetch pkg+https://repo.netgate.com/pkg/pfSense_plus-v21_05_1_aarch64-pfSense_plus-v21_05_1/meta.conf
DBG(1)[50476]> opening libfetch fetcher
DBG(1)[50476]> Fetch > libfetch: connecting
DBG(1)[50476]> Fetch: fetching from: https://repo00.netgate.com/pkg/pfSense_plus-v21_05_1_aarch64-pfSense_plus-v21_05_1/meta.conf with opts "i"
DBG(1)[50476]> Fetch: fetcher chosen: https
DBG(1)[50476]> Request to fetch pkg+https://repo.netgate.com/pkg/pfSense_plus-v21_05_1_aarch64-pfSense_plus-v21_05_1/packagesite.txz
DBG(1)[50476]> opening libfetch fetcher
DBG(1)[50476]> Fetch > libfetch: connecting
DBG(1)[50476]> Fetch: fetching from: https://repo00.netgate.com/pkg/pfSense_plus-v21_05_1_aarch64-pfSense_plus-v21_05_1/packagesite.txz with opts "i"
pfSense repository is up to date.
All repositories are up to date.

Steve

seanmcb

@johnpoz said in repo01.netgate.com TLS cert seems invalid:

Not all things are used in the fashion that ssllabs tests for

Maybe so, but Firefox also shows "invalid cert" if you visit https://repo01.netgate.com/

johnpoz

And again that has zero to do with how pfsense uses it.. ssllab tests for how your browser would use a ssl cert. So yeah if ssllabs fails - its pretty much a given your browser would balk at it as well ;)

Relate it to how users say pkg.pfsense.org isn't working because they try and use A/AAAA record when its actually a SRV record, etc. Which is really _https._tcp.pkg.pfsense.org for example..

If you do not test functionally - on how the functionality was designed.. Then yeah you can see red herrings all over the place.

seanmcb

@johnpoz the certificate given by repo01.netgate.com has "Common name" of pfsense.org and "Alternative names" of *.pfsense.com *.pfsense.org pfsense.com pfsense.org. What is the advantage for the cert to omit repo01.netgate.com from its list of names?

stephenw10

It's because that repo is authenticated. It expects the client to send a cert and your browser does not. Neither does ssllabs. That said that's not the error I see if I try to visit it directly. I see the far more useful:

400 Bad Request
No required SSL certificate was sent

Steve

johnpoz

@seanmcb said in repo01.netgate.com TLS cert seems invalid:

cert to omit repo01.netgate.com from its list of names?

Sure looks like its there to me..

While I don't see a SAN entry - the CN is there that matches.. Again - trying to check a ssl cert without fully understanding how the cert is used in the specific applications design use of said ssl can and does lead to red herring rabbit holes ;)

seanmcb

@stephenw10 said in repo01.netgate.com TLS cert seems invalid:

It's because that repo is authenticated. It expects the client to send a cert and your browser does not. Neither does ssllabs

Ah, thanks for the actual explanation. Much appreciated.

I'll have access to the device again in about 10 hours and can try from the shell the steps you asked about.

stephenw10

Let us know if you're still seeing that.

seanmcb

@stephenw10 my output is quite like yours:

[21.05-RELEASE][admin@pfSense.localdomain]/root: pkg -d update
DBG(1)[42611]> pkg initialized
Updating pfSense-core repository catalogue...
DBG(1)[42611]> PkgRepo: verifying update for pfSense-core
DBG(1)[42611]> Pkgrepo, begin update of '/var/db/pkg/repo-pfSense-core.sqlite'
DBG(1)[42611]> Request to fetch pkg+https://repo.netgate.com/pkg/pfSense_plus-v21_05_1_aarch64-core/meta.conf
DBG(1)[42611]> opening libfetch fetcher
DBG(1)[42611]> Fetch > libfetch: connecting
DBG(1)[42611]> Fetch: fetching from: https://repo01.netgate.com/pkg/pfSense_plus-v21_05_1_aarch64-core/meta.conf with opts "i"
DBG(1)[42611]> Fetch: fetcher chosen: https
DBG(1)[42611]> Request to fetch pkg+https://repo.netgate.com/pkg/pfSense_plus-v21_05_1_aarch64-core/packagesite.txz
DBG(1)[42611]> opening libfetch fetcher
DBG(1)[42611]> Fetch > libfetch: connecting
DBG(1)[42611]> Fetch: fetching from: https://repo01.netgate.com/pkg/pfSense_plus-v21_05_1_aarch64-core/packagesite.txz with opts "i"
pfSense-core repository is up to date.
Updating pfSense repository catalogue...
DBG(1)[42611]> PkgRepo: verifying update for pfSense
DBG(1)[42611]> Pkgrepo, begin update of '/var/db/pkg/repo-pfSense.sqlite'
DBG(1)[42611]> Request to fetch pkg+https://repo.netgate.com/pkg/pfSense_plus-v21_05_1_aarch64-pfSense_plus-v21_05_1/meta.conf
DBG(1)[42611]> opening libfetch fetcher
DBG(1)[42611]> Fetch > libfetch: connecting
DBG(1)[42611]> Fetch: fetching from: https://repo01.netgate.com/pkg/pfSense_plus-v21_05_1_aarch64-pfSense_plus-v21_05_1/meta.conf with opts "i"
DBG(1)[42611]> Fetch: fetcher chosen: https
Fetching meta.conf: 100%    163 B   0.2kB/s    00:01    
DBG(1)[42611]> Request to fetch pkg+https://repo.netgate.com/pkg/pfSense_plus-v21_05_1_aarch64-pfSense_plus-v21_05_1/packagesite.txz
DBG(1)[42611]> opening libfetch fetcher
DBG(1)[42611]> Fetch > libfetch: connecting
DBG(1)[42611]> Fetch: fetching from: https://repo01.netgate.com/pkg/pfSense_plus-v21_05_1_aarch64-pfSense_plus-v21_05_1/packagesite.txz with opts "i"
DBG(1)[42611]> Fetch: fetcher chosen: https
Fetching packagesite.txz: 100%  129 KiB 131.8kB/s    00:01    
DBG(1)[42611]> PkgRepo: extracting packagesite.yaml of repo pfSense
DBG(1)[43042]> PkgRepo: extracting signature of repo in a sandbox
DBG(1)[42611]> Pkgrepo, reading new packagesite.yaml for '/var/db/pkg/repo-pfSense.sqlite'
Processing entries: 100%
pfSense repository update completed. 464 packages processed.
All repositories are up to date.

Biggest difference is repo00.netgate.com vs repo01.netgate.com and mine has additional output Fetch: fetcher chosen: https.

stephenw10

repo00 and repo01 should be identical there, that shouldn't matter.

seanmcb

@stephenw10 so do you think the part about Fetch: fetcher chosen: https is the difference that explains the failure I see?

I could always try a magic reboot, but I'm not in a huge rush to update. If there's something more we can troubleshoot to find this bug, I'm game.

stephenw10

@seanmcb said in repo01.netgate.com TLS cert seems invalid:

Processing entries: 100%
pfSense repository update completed. 464 packages processed.
All repositories are up to date.

It looks to be updating from the repo successfully. What failure are you seeing?

My output also shows it choosing https, I don't think that's an issue.

Steve

seanmcb

@stephenw10 the failure I'm seeing is as per my first message in this thread. The update fails with the error message text I pasted.

stephenw10

Hmm, OK try running at the command line: pfSense-upgrade -d

seanmcb

@stephenw10 said in repo01.netgate.com TLS cert seems invalid:

pfSense-upgrade -d

[21.05-RELEASE][admin@pfSense.localdomain]/root: pfSense-upgrade -d
>>> Updating repositories metadata... 
Updating pfSense-core repository catalogue...
Fetching meta.conf: . done
1082880000:error:141F0006:SSL routines:tls_construct_cert_verify:EVP lib:/usr/local/poudriere/jails/pfSense_plus-v21_05_aarch64/usr/src/crypto/openssl/ssl/statem/statem_lib.c:283:
Certificate verification failed for /C=US/ST=Texas/L=Austin/O=Rubicon Communications, LLC (Netgate)/CN=repo01.netgate.com
1082880000:error:0D0C5006:asn1 encoding routines:ASN1_item_verify:EVP lib:/usr/local/poudriere/jails/pfSense_plus-v21_05_aarch64/usr/src/crypto/openssl/crypto/asn1/a_verify.c:170:
1082880000:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/usr/local/poudriere/jails/pfSense_plus-v21_05_aarch64/usr/src/crypto/openssl/ssl/statem/statem_clnt.c:1915:
Child process pid=62247 terminated abnormally: Segmentation fault

mer

@seanmcb What hardware are you doing this on? My reason for asking is I had a very similar issue on a SG2100 (symptom of segfault was the same). Power cycling (not rebooting) cleared the issue. Literally, shutdown the system, removed power for at least 30 secs, then reapplied power. Issue went away. Suspect that openssl is using crypto hardware that can get wedged and the only cure is power cycle.

I have no opinion on the validity of the certificate.

seanmcb

@mer My hardware is a Netgate SG-1100.

I could try a power cycle, but I'm not in a big rush to update, and this bug is reproducible for the moment, so it's a chance to debug it, and maybe solve it.

mer

@seanmcb That's good, but my point is that if the root cause is the hardware itself getting wedged, there's not much debugging that can actually be done. Hopefully the netgate folks may have some commands that would say "yep hardware is wedged, can't get more info".

stephenw10

Hmm, so it hits that when you try to upgrade but 'pkg-static update' completes successfully?

That's odd. I would expect both to fail.

With that error on an SG-1100 though it's almost certainly the crypto hardware issue. If you power cycle it and it then succeeds that would confirm it.

Steve