Something screwed with packages.pfsense.org

[doktornotor]

Not really sure what's up there, guys.

• Getting SSL validation errors on several boxes, several others have no such problem:

The package server's SSL certificate could not be verified. The SSL certificate itself may be invalid, its chain of trust may have failed validation, or the server may have been impersonated. Downloaded packages may come from an untrusted source. Proceed with caution.

• Multiple people complaining lately about screwed downloads:
  https://forum.pfsense.org/index.php?topic=99406.0
  https://forum.pfsense.org/index.php?topic=99398.0
  https://forum.pfsense.org/index.php?topic=48347.msg553751#msg553751

[johnpoz]

where exactly are you getting this error?  in pfsense?

I just updated 2 packages, service watchdog and vnstat2 because there were some updates from my installed versions and went just fine.

I just tested packages.pfsense.org with ssl labs, and can hit it via my browser without any issues.
https://www.ssllabs.com/ssltest/analyze.html?d=packages.pfsense.org

Are you hitting it via ipv4 or ivp6?  Looks like only scores a C, but that seems to be due to this
This server is vulnerable to the POODLE attack. If possible, disable SSL 3 to mitigate. Grade capped to C.

It also reports and issue with crl in the trust path, but that seems to be outside pfsense control
RSA 4096 bits (e 65537) / SHA384withRSA
CRL ERROR: Request failed with HTTP status: 403 [http://crl.usertrust.com/AddTrustExternalCARoot.crl]

But I was able to access the crl.. Maybe they blocked ssl labs?

[cmb]

That message could be really misleading, as it just means curl exited with a non-0 return code. Doesn't necessarily mean a problem with the certificate, any failure to connect could result in that if it occurs during the cert check connection.

Is it reliably replicable on any of your systems doktornotor? If so I'd like to know an IP of an affected system, and if you could get a packet capture of the attempt that would be helpful as well.

[doktornotor]

Regarding my problem (not the failed downloads of various package parts others mentioned here) – yeah, it was very replicable with lots of HTTPS stuff. Was some of ~Sept. 4 2.2.5 snapshots where pretty much all HTTPS stopped working after a week. Had to reinstall with latest snapshot. (No idea what happened there, gitsync couldn't fix it either. E.g., the logs from Suricata/Snort rules downloads are here:)

Sep 14 00:45:46	php: suricata_check_for_rule_updates.php: [Suricata] Will retry in 15 seconds...
Sep 14 00:45:46	php: suricata_check_for_rule_updates.php: [Suricata] Rules download error: error setting certificate verify locations: CAfile: /usr/local/share/certs/ca-root-nss.crt CApath: none
Sep 14 00:45:31	php: suricata_check_for_rule_updates.php: [Suricata] Will retry in 15 seconds...
Sep 14 00:45:31	php: suricata_check_for_rule_updates.php: [Suricata] Rules download error: error setting certificate verify locations: CAfile: /usr/local/share/certs/ca-root-nss.crt CApath: none
Sep 14 00:45:16	php: suricata_check_for_rule_updates.php: [Suricata] Will retry in 15 seconds...
Sep 14 00:45:16	php: suricata_check_for_rule_updates.php: [Suricata] Rules download error: error setting certificate verify locations: CAfile: /usr/local/share/certs/ca-root-nss.crt CApath: none
Sep 14 00:45:01	php: suricata_check_for_rule_updates.php: [Suricata] Will retry in 15 seconds...
Sep 14 00:45:01	php: suricata_check_for_rule_updates.php: [Suricata] Rules download error: error setting certificate verify locations: CAfile: /usr/local/share/certs/ca-root-nss.crt CApath: none

[johnpoz]

Oh you were on a snapshot..

Maybe I should start a new thread in feedback to get their ssl labs score up.. C is pretty bad!!

edit:  Up to a B now..

Looks like they fixed the SSL 3 stuff.

TLS 1.2 Yes
TLS 1.1 Yes
TLS 1.0 Yes
SSL 3 No
SSL 2 No

[doktornotor]

FFS what's up with 2.2.5 certificates? It's been a week now since I updated to latest snapshot, and I'm back where I was. That's exactly what happened with the previous snapshot. What's expiring all those root certs after a week?!?! Can you revert whatever has been done there? Never seen such totally whacky issue.

NB: I have totally no issues with validating those certificates from any machine on local networks, so it's not like there'd be something blocked by firewall or whatever else. It's just pfSense box itself pretty much self-destructing SSL after a week. packages.pfsense.org, Snort/Suricata rule downloads, HTTPS lists downloads in pfBlockerNG -> FAIL.

:( >:( >:(

[doktornotor]

Right… Upgraded yet again to latest snapshots. Guess what - everything back to normal, with all packages reinstalled and exact same configuration. This is madness guys. (To be completely sure, I've rebooted twice before upgrade. Nothing could fix the suicidal SSL.)

[cmb]

@doktornotor:

FFS what's up with 2.2.5 certificates? It's been a week now since I updated to latest snapshot, and I'm back where I was. That's exactly what happened with the previous snapshot. What's expiring all those root certs after a week?!?! Can you revert whatever has been done there? Never seen such totally whacky issue.

I can't think of anything that's changed in that regard. What do you get trying to fetch something via HTTPS? Just 'fetch https://pfsense.org/ip.php' or something. fetch should spit out a more useful error.

@johnpoz:

edit:  Up to a B now..

Back to A+ again now, only change using a custom-generated dhparams.

[doktornotor]

@cmb:

I can't think of anything that's changed in that regard. What do you get trying to fetch something via HTTPS? Just 'fetch https://pfsense.org/ip.php' or something. fetch should spit out a more useful error.

That works. Anything using curl -> game over.

[doktornotor]

And - now I have /usr/local/share/certs/ca-root-nss.crt back. When it fucks itself up, the file is gone. I posted the suricata error above.

Cannot see anything there doing a weekly delete of root CA store either. And - sure like hell - I didn't delete it myself.

[johnpoz]

I see the A+ score - nice!!!  Much better than a C ;)