openVPN issue after yesterday update

Beerman

Today I tried the newest Build:

2.5.0-DEVELOPMENT (amd64)
built on Tue Dec 22 03:01:05 EST 2020

Unfortunately, the same problem. And there are only the 5 openvpn log events I mentioned in the first post.

Works fine in the 2.4.5-p1 version, and did worked fine in the 2.5.0 version before the 17th of December.

jimp

There is nothing wrong with OpenVPN on snapshots here.

Maybe your certificates are actually not valid? Maybe expired?

Beerman

@jimp Thx, for reply! :)

I have just tested the current snapshot, unfortunately without success.

At the moment, I cannot see any issues with my certificates. No expired cert.
Is the the possibility to turn on debug logs for openVPN?

It's funny that it already worked with the verfsion 2.5 , only unfortunately since the 17th of December it doesn´t anymore.

Beerman

Today I have tested a little bit again.

So I updated my system to version "2.5.0-DEVELOPMENT (amd64)
built on Sat Jan 02 03:00:38 EST 2021" from version "2.4.5-RELEASE-p1".

I also came across the following thread:
https://forum.netgate.com/topic/98244/pfsense-2-3-tls-handshake-failed-failed-running-command-tls-verify-script/7

There I found the post of "v0lZy" from Dec 28, 2017, 1:25 PM who had a similar problem.
https://forum.netgate.com/post/743055

I then changed "Certificate Depth" to "Do not Check" and then the connection worked again. Before that, the setting was on "One (Client+Server)".

When I change the setting back to "One (Client+Server)", I get the same error messages again:

WARNING: Failed running command (--tls-verify script): external program exited with error status: 1
OpenSSL: error:1417C086:SSL routines:tls_process_client_certificate:certificate verify failed
TLS_ERROR: BIO read tls_read_plaintext error
TLS Error: TLS object -> incoming plaintext read error
TLS Error: TLS handshake failed

(Unfortunately wirh no more information)

CA, Server Cert and User Cert were generated on the same system.

jimp

How long is the subject on that certificate?

The only similar failure I can recall like that which could be related is https://redmine.pfsense.org/issues/4521 but that has been broken a long time and wouldn't have only broken recently.

Beerman

@jimp

Server Cert:
C=XX,ST=Xxxxxxx,L=Xxxxxx,O=Xxxxxx,emailAddress=xxxxxxx@xxxxxx.xxxxx,CN=xxx-xxxxx.xxxxxx.xxxxx,OU=xxxxxxx.xxxxxx.xxxxx

Length: 118

Client Cert:
CN=xxx_xxxxxxx_xxx,C=XX,ST=Xxxxxxx,L=Xxxxxx,O=Xxxxxx,OU=xxxxxxx.xxxxxx.xxxxx

Length: 77

Seems not the problem...

Beerman

In the meantime, I created a new CA and new certificates, and with these it did work again.

And finally my blind eyes also found the option "Verbosity level" So i did test the connection again, with the original CA/certificates.

Here is the interesting part of the logs:

TLS Error: TLS handshake failed
TLS Error: TLS object -> incoming plaintext read error
TLS_ERROR: BIO read tls_read_plaintext error
OpenSSL: error:1417C086:SSL routines:tls_process_client_certificate:certificate verify failed
SSL alert (write): fatal: internal error
VERIFY SCRIPT ERROR: depth=1, C=XX, ST=Xxxxxxx, L=Xxxxxx, O=Yyyyy, emailAddress=openvpn@xxxxxx.yyyyy, CN=internal-ca.openvpn.xxxxxx.yyyyy, OU=openvpn.xxxxxx.yyyyy
WARNING: Failed running command (--tls-verify script): external program exited with error status: 1
TLS: executing verify command: /usr/local/sbin/ovpn_auth_verify tls srv-yyy99.xxxxxx.yyyyy 1 1 C=XX, ST=Xxxxxxx, L=Xxxxxx, O=Yyyyy, emailAddress=openvpn@xxxxxx.yyyyy, CN=internal-ca.openvpn.xxxxxx.yyyyy, OU=openvpn.xxxxxx.yyyyy
SSL state (accept): TLSv1.3 early data
Incoming Ciphertext -> TLS
BIO write tls_write_ciphertext 1250 bytes
ACK reliable_can_send active=0 current=0 : [8]
TLS: tls_process: chg=0 ks=S_START lame=S_UNDEF to_link->len=0 wakeup=604800
TLS: tls_multi_process: i=0 state=S_START, mysid=dc99e6a0 aa7269fa, stored-sid=36c952ab c7d44a4d, stored-ip=[AF_INET]80.xxx.yyy.zzz:28500

darcey

@beerman
I have certificate depth 1, a custom root CA and no intermediate CA.
I just upgraded 2.4.5-p3 (with a long-term working openvpn config) to 2.5.0. Now CA verification fails.
The log says /usr/local/sbin/ovpn_auth_verify returns 1. However, running the stated command from the terminal returns zero. Turning off CA verification allows client to connect successfully.
I wonder why openvpn process apparently sees a return value of 1 whilst I see zero from a root shell. I'd rather not regenerate certificates, though I did try just resaving them from web UI with no joy. So I have turned off tls depth verification for the time being.

reshadp

@darcey
I have the same issue.
Disabling certificate depth works, but I rather have a strict check.
OpenVPN adds additional environment variables and parameters at the end of the command so we need to test with those.

Also the shell script which does the TLS check has changed since the previous release (2.4.5-p1)

darcey

@reshadp
I am not so sure my statement re running the certificate check successfully via the command line was correct! I may have passed certifcate serial & config incorrectly to the script.
I have since removed the old CA and associated certificates, and generated a new PKI, this time via the pfsense web UI. The certificate depth check is now successful. So have re-enabled the check in the openvpn server config.
I'm not sure what changes between 2.4.5-p3 and 2.5.0 render a depth check of my previous PKI invalid.

DeaDSouL

I have the same issue since I've upgraded my pfSense to 2.5.0

darcey

I've not looked at the changes WRT to certificate depth check, but one way my previous certs (CA included) differ from those generated using the web UI, is an email field amongst the distinguished name fields. Apparently spaces should not cause any issues, but I've also avoided using them in the newly created PKI.

DeaDSouL

Here is the fix for the issue. (It worked for me)

qctech

Props to @Beerman

I was having the same issue at a site where fortunately the VPN is not mission critical. Your post pointed me in the right direction and it's now resolved.

New CA, server cert, user cert applied to the existing OpenVPN setup, new client config exported and it all works like it should.