CRL has expired

ads76

I'm testing upgrades between 2.4.5p1 to 2.6.0. The upgrade itself seems to go fine, I'm just testing services post-upgrade and found an issue with OpenVPN refusing TLS auth saying the certificate revocation list expired:

Jun 17 17:57:25	openvpn	90723	1.2.3.4:42081 TLS Error: TLS handshake failed
Jun 17 17:57:25	openvpn	90723	1.2.3.4:42081 TLS Error: TLS object -> incoming plaintext read error
Jun 17 17:57:25	openvpn	90723	1.2.3.4:42081 TLS_ERROR: BIO read tls_read_plaintext error
Jun 17 17:57:25	openvpn	90723	1.2.3.4:42081 OpenSSL: error:1417C086:SSL routines:tls_process_client_certificate:certificate verify failed
Jun 17 17:57:25	openvpn	90723	1.2.3.4:42081 VERIFY ERROR: depth=0, error=CRL has expired: C=GB, ST=London, L=London, O=MyOrg, emailAddress=technical@myorg.com, CN=myuser, serial=2

If I disable the use of the CRL in the OpenVPN server instance, clients can connect fine.

The CRL was created on this very pfSense box and would have used the default lifetime of 9999 days. That may have been on 2.2.6 and carried up through reinstalls to 2.4.5p1 or it may have been created on 2.4.5p1, I can't remember now.

I think I can see the issue here:

me@myhost:~/$ openssl crl -in Revoked+Certs.crl -noout -text 
Certificate Revocation List (CRL):
<snip>
        Last Update: Jun 17 17:25:44 2022 GMT
        Next Update: Apr 25 17:25:44 1955 GMT

Does that next update field mean the expiry date? If so, it looks like it's off by about a century.

If I look at the one from the other firewall in the HA pair, which wasn't upgraded yet, I see this:

        Last Update: Jun 17 18:06:54 2022 GMT
        Next Update: Nov  1 18:06:54 2049 GMT

So the one that got upgraded got messed up, right?

Can anyone advise on how to either extend the date of the existing CRL, or recreate it without losing which certs were revoked? I revoked some certs and deleted the actual certs so I can't create a new CRL and then revoke the certs again. Googling I've seen suggestions to use easyrsa, but that doesn't appear to be on my pfSense box.

jimp

The CRL is rewritten with a fresh date every time it rewrites the OpenVPN configuration.

What hardware is this on? If it's a 32-bit ARM system it may be having a problem with the date going past 2038.

ads76

@jimp Thanks for your reply Jim.

This is AMD64. I'll make a tweak to the OpenVPN config in that case and see if the issue goes away. It didn't when I reconfigured the OpenVPN server instance to not use the CRL though, so we'll see. I also tried adding another entry to the CRL, but that didn't help. I'll come back to update on whether it helped or not.

ads76

Issue resolved. After I found that the CRL expiry was getting updated but was still for some reason in 1955 after making tweaks to the OpenVPN config, I looked at what was in config.xml for the CRL and found that the lifetime was 99999 days, rather than 9999 is recommended. Changed it to 9999 and re-imported and it's working properly now.

Thanks for pointing me in the right direction @jimp.

barryboden

Looks like we have a similar issue too. I've got a pair of Netgate 7100's which all of a sudden today are not letting users connect. When I look at the CRL it shows a date of Jan 1950 for the next issue.
I've tried recreating the CRL with 9900 days or setting the vpn server not to use a CRL but its still showing the error.
Is there a fix for the date roll over issue.
22.05-RELEASE (amd64)
built on Wed Jun 22 18:56:13 UTC 2022
FreeBSD 12.3-STABLE

ads76

@barryboden My issue may not be yours and I don't know about 22.05 (we're using CE), but I found if I exported the config (or just looked at it in /cf/conf/config.xml) in the <crl> section there was:

<lifetime>99999</lifetime>

I understand there was a change in OpenVPN between the versions used in 2.4.5p1 and 2.6.0 where the verification of the CRL was moved from being done by OpenVPN to being handled by OpenSSL which was stricter. It would be worth looking at what the CRL lifetime value is in your config.xml.

I found changing it in config.xml and rebooting didn't work, pfSense must write its config out before rebooting. I had to export the config through the GUI, update the lifetime field (to 9999) then reimport it. In a crisis, you can just disable use of the CRL until you figure it out but obviously that would allow users with revoked certs to log in again.

That's as far as I can help you. Hope it does.

mmulqueen

@barryboden See my posts here https://forum.netgate.com/topic/174167/no-clients-can-connect-to-openvpn-due-to-crl-expiry

I'd suggest recreating the CRL with a much shorter lifetime (I did 730 days). Be sure to edit the OpenVPN server settings to point to the new CRL and then restart the OpenVPN service.

barryboden

@ads76 thanks for your reply I did look in there any my config already says 9999, I've created new CRLs and if I set them to 9990 the dates look ok, but 9999 must roll over the year.
Adjusting this has got my clients connecting again, for 9 days.

jimp

I created a Redmine entry for this (https://redmine.pfsense.org/issues/13424) and I'll be working on a fix shortly. When I have one, I'll also create an entry in the System Patches package for it.

maverick_slo

@jimp Applied diff manually and restarted Openvpn server service.
It works after restart of service.

jimp

I merged the fix in yesterday evening.

You can install the System Patches package and then create an entry for a3c1589086ea67d25a28ec14ab95d7fd9ab25fa2 to apply the fix.

It will be added as a "Recommended Patch" in the System Patches package soon, but in the meantime it is safe to add a manual entry to obtain the fix now.

davetick

@jimp I've just applied that patch and restarted OpenVPN. CRL expiry error no longer in OpenVPN logs and clients now connecting again - thanks !

PFSense: 22.05-RELEASE (amd64)
KVM Guest
Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz
2 CPUs: 1 package(s) x 1 core(s) x 2 hardware threads
AES-NI CPU Crypto: Yes (active)
QAT Crypto: No

slu

@jimp said in CRL has expired:

You can install the System Patches package and then create an entry for a3c1589086ea67d25a28ec14ab95d7fd9ab25fa2 to apply the fix.

We run into the same issue, lost all VPN connections.
Can we apply this patch also in 2.6.0 CE?

jimp

The patch applies cleanly to 2.6.0, you can use it there.

slu

@jimp
thank you, the VPN working again.

Risfold

Thank you all for the comments and patch solution here. Many of my haproxy backends went down last week (ssl handshake errors) and diagnosing the issue was very difficult.

A lot of trial and error, I narrowed down the backend SSL verification and CRL, as the culprit. I stumbled upon this issue after searching errors related to a downed OpenVPN client. Applying the patch here (obviously) fixed both haproxy and OpenVPN issues I was having.

Just wanted to add my experience in case any others are having the same issues with haproxy, and are looking for a solution. Hopefully they will also find this thread.

Novacom

@jimp

Thank you ! Worked like a charm on 22.05-RELEASE (amd64)

ChrisHaa

Same issue here. Patch solved it within a minute. Thanks.

This has some additional information: https://blog.nuvotex.de/pfsense-crl-has-expired/

ccb056

Y2K all over again

had this same problem, applied the patch, fixed

Thanks Jim

khodorb

I had the same issue with version 2.5 and 22.05, i wonder if netgate has permanent fix for that