OpenVPN through PFSense just stopped working - Help!

charry2014

Thank you for the suggestions - I ran the packet capture on the external gateway and see a small number of packets arriving. Running the capture on the OpenVPN interfaces shows nothing is arriving there. I imagine this means it is not our network provider or anything external - the problem is somewhere inside PFSense?

To check the other packages installed what sort of thing am I looking for? Only Snort, ntopng, and openvpn-client-export are installed and Snort's blocking is disabled.

kiokoman

if snort is not blocking ok
please copy here what you have in the packet capture
do you have anything logged in Status / System Logs / OpenVPN ?

charry2014

The logs in Status / System Logs / OpenVPN are interesting. They show that some time between 08:00 and 10:00 yesterday morning something changed and that from that point on only our one colleague can connect over the VPN. She still can.

This is starting to look like a certificate / Authentication / Handshaking error - I understand that if there is anything misconfigured with the certificates then the OpenVPN server will play dead as a security measure. I have exported new configuration details from our PFSense and now get lots of TLS errors in the server logs:

Jul 23 11:52:20	openvpn	22773	123.456.789.001:58706 Fatal TLS error (check_tls_errors_co), restarting
Jul 23 11:52:20	openvpn	22773	123.456.789.001:58706 TLS Error: TLS handshake failed
Jul 23 11:52:20	openvpn	22773	123.456.789.001:58706 TLS Error: TLS object -> incoming plaintext read error
Jul 23 11:52:20	openvpn	22773	123.456.789.001:58706 TLS_ERROR: BIO read tls_read_plaintext error
Jul 23 11:52:20	openvpn	22773	123.456.789.001:58706 OpenSSL: error:14089086:SSL routines:ssl3_get_client_certificate:certificate verify failed
Jul 23 11:52:20	openvpn	22773	123.456.789.001:58706 VERIFY ERROR: depth=0, error=unsupported certificate purpose: C=DE, ST=County, L=City, O=Company, emailAddress=my.email@company.com, CN=Company VPN Server
Jul 23 11:52:19	openvpn	22773	TCP connection established with [AF_INET]123.456.789.001:58706
Jul 23 11:52:14	openvpn	22773	123.456.789.001:63576 Fatal TLS error (check_tls_errors_co), restarting
Jul 23 11:52:14	openvpn	22773	123.456.789.001:63576 TLS Error: TLS handshake failed
Jul 23 11:52:14	openvpn	22773	123.456.789.001:63576 TLS Error: TLS object -> incoming plaintext read error
Jul 23 11:52:14	openvpn	22773	123.456.789.001:63576 TLS_ERROR: BIO read tls_read_plaintext error
Jul 23 11:52:14	openvpn	22773	123.456.789.001:63576 OpenSSL: error:14089086:SSL routines:ssl3_get_client_certificate:certificate verify failed
Jul 23 11:52:14	openvpn	22773	123.456.789.001:63576 VERIFY ERROR: depth=0, error=unsupported certificate purpose: C=DE, ST=County, L=City, O=Company, emailAddress=my.email@company.com, CN=Company VPN Server
Jul 23 11:52:13	openvpn	22773	TCP connection established with [AF_INET]123.456.789.001:63576
Jul 23 11:52:08	openvpn	22773	123.456.789.001:44894 Fatal TLS error (check_tls_errors_co), restarting
Jul 23 11:52:08	openvpn	22773	123.456.789.001:44894 TLS Error: TLS handshake failed
Jul 23 11:52:08	openvpn	22773	123.456.789.001:44894 TLS Error: TLS object -> incoming plaintext read error

I will dig a bit deeper and see what pops out...

kiokoman

@charry2014 said in OpenVPN through PFSense just stopped working - Help!:

error=unsupported certificate purpose

https://forum.netgate.com/topic/131903/unsupported-certificate-purpose/6

stephenw10

@charry2014 said in OpenVPN through PFSense just stopped working - Help!:

The server logs for OpenVPN show nothing,

What changed between that and now where you are seeing loads of errors in the logs.

The client certs could have expired. Is the config of the colleague in China much newer?

Steve

Rico

Check Diagnostics > Backup & Restore > Config History to clarify nothing changed.

-Rico

charry2014

Thank you for your help everyone. The Backup and Restore / Config History logs show there were no suspicious updates to the server - so everything seems to be going as planned there.

For better or worse our company has three OpenVPN servers running. These are mapped though different ports to separate tunnel networks to restrict certain classes of users (external contractors usually) from some of the server we run. If I try to establish a connection through one of these on port 10694 then I am able to see the TLS errors I quoted in the post above. Attempting the same thing on the 10691 admin server results in no OpenVPN server activity, but I do see packets in the packet logger diagnostic.

What I conclude so far is that our external connectivity is fine but something inside our OpenVPN configuration automagically went foobar all by itself.

charry2014

The packet capture from the admin VPN connection attempts (where nothing appears in the OpenVPN logs are)

11:13:18.516683 IP 192.168.10.18.33835 > 12.123.213.210.10691: UDP, length 54
11:13:20.552002 IP 192.168.10.18.33835 > 12.123.213.210.10691: UDP, length 54
11:13:24.622042 IP 192.168.10.18.33835 > 12.123.213.210.10691: UDP, length 54
11:13:32.149070 IP 192.168.10.18.33835 > 12.123.213.210.10691: UDP, length 54
11:13:48.803252 IP 192.168.10.18.33835 > 12.123.213.210.10691: UDP, length 54

stephenw10

The three OpenVPN servers are running on pfSense? If they are external then check packet captures to them.

If that traffic hits pfSense but seemignly does not reach the OpenVPN server check the firewall logs foir blocked traffic. Check the state table for any port 10691 states. If there are none something is blocking it.

Are you sure Snort is not blocking it? Nothing in the Snort blocked hosts table?

Steve

charry2014

Thank you for your replies. All OpenVPN servers are running in PFSense. I have tried a number of configuration changes and nothing has made a difference. I will schedule a network outage for lunchtime today and try to reboot the whole thing in desperation.

I conducted a review of how our PFSense is running and there are a few weird things - Firstly the OpenVPN as mentioned in this post, then the sync to the failover box is throwing hundreds of errors since around the time the VPN stopped working

A communications error occurred while attempting to call XMLRPC method host_firmware_version: Unable to connect to tls://172.16.0.20:8304. Error: Operation timed out	@ 2019-05-24 00:06:39

Finally one of our subnets is unaccessible even though I can see no changes in any configuration - I am still debugging exactly what the issue is here but no clients on the subnet are issued an IP address. Again the timing of this is interesting, the subnet became unavailable some hours after the above two problems occurred, perhaps when the DHCP leases expired.

All told, I am now completely puzzled what happened.

jecker

I realize this is an old issue, but I have experienced the same problem. I was able to fix the issue by accessing the WAN interface, press save, then press apply.