error: ssl handshake failed crypto error:0A000086:SSL routines::certificate verify failed
-
I'm using pfsense 2.7.2-RELEASE (amd64)
I'm getting the following error coming up repeatedly in my unbound logs -- almost every minute:Dec 8 07:27:56 unbound 99937 [99937:0] error: ssl handshake failed crypto error:0A000086:SSL routines::certificate verify failed Dec 8 07:27:56 unbound 99937 [99937:0] notice: ssl handshake failed 10.0.1.1 port 853The weird thing about this error is its complaining against an ssl handshake that is failing against the router itself -- The ip address of my pfsense installation is 10.0.1.1 and this is what seems to be generating the error -- or at least this is how I'm interpreting this.
The complete sequence of events where the unbound service stops and starts is here:
Dec 8 07:22:14 unbound 99937 [99937:0] info: start of service (unbound 1.18.0). Dec 8 07:22:14 unbound 99937 [99937:0] notice: init module 0: iterator Dec 8 07:22:14 unbound 99937 [99937:0] notice: Restart of unbound 1.18.0. Dec 8 07:22:14 unbound 99937 [99937:0] info: 8.000000 16.000000 5 Dec 8 07:22:14 unbound 99937 [99937:0] info: 4.000000 8.000000 13 Dec 8 07:22:14 unbound 99937 [99937:0] info: 2.000000 4.000000 4 Dec 8 07:22:14 unbound 99937 [99937:0] info: 1.000000 2.000000 5 Dec 8 07:22:14 unbound 99937 [99937:0] info: 0.262144 0.524288 14 Dec 8 07:22:14 unbound 99937 [99937:0] info: 0.131072 0.262144 20 Dec 8 07:22:14 unbound 99937 [99937:0] info: 0.065536 0.131072 36 Dec 8 07:22:14 unbound 99937 [99937:0] info: 0.032768 0.065536 36 Dec 8 07:22:14 unbound 99937 [99937:0] info: 0.016384 0.032768 24 Dec 8 07:22:14 unbound 99937 [99937:0] info: 0.008192 0.016384 11 Dec 8 07:22:14 unbound 99937 [99937:0] info: 0.004096 0.008192 8 Dec 8 07:22:14 unbound 99937 [99937:0] info: 0.002048 0.004096 1 Dec 8 07:22:14 unbound 99937 [99937:0] info: 0.000000 0.000001 4 Dec 8 07:22:14 unbound 99937 [99937:0] info: lower(secs) upper(secs) recursions Dec 8 07:22:14 unbound 99937 [99937:0] info: [25%]=0.0308907 median[50%]=0.0773689 [75%]=0.234291 Dec 8 07:22:14 unbound 99937 [99937:0] info: histogram of recursion processing times Dec 8 07:22:14 unbound 99937 [99937:0] info: average recursion processing time 0.844864 sec Dec 8 07:22:14 unbound 99937 [99937:0] info: server stats for thread 1: requestlist max 6 avg 0.80663 exceeded 0 jostled 0 Dec 8 07:22:14 unbound 99937 [99937:0] info: server stats for thread 1: 327 queries, 146 answers from cache, 181 recursions, 0 prefetch, 0 rejected by ip ratelimiting Dec 8 07:22:14 unbound 99937 [99937:0] info: 8.000000 16.000000 12 Dec 8 07:22:14 unbound 99937 [99937:0] info: 4.000000 8.000000 22 Dec 8 07:22:14 unbound 99937 [99937:0] info: 2.000000 4.000000 6 Dec 8 07:22:14 unbound 99937 [99937:0] info: 1.000000 2.000000 3 Dec 8 07:22:14 unbound 99937 [99937:0] info: 0.524288 1.000000 2 Dec 8 07:22:14 unbound 99937 [99937:0] info: 0.262144 0.524288 8 Dec 8 07:22:14 unbound 99937 [99937:0] info: 0.131072 0.262144 14 Dec 8 07:22:14 unbound 99937 [99937:0] info: 0.065536 0.131072 41 Dec 8 07:22:14 unbound 99937 [99937:0] info: 0.032768 0.065536 34 Dec 8 07:22:14 unbound 99937 [99937:0] info: 0.016384 0.032768 27 Dec 8 07:22:14 unbound 99937 [99937:0] info: 0.008192 0.016384 14 Dec 8 07:22:14 unbound 99937 [99937:0] info: 0.004096 0.008192 17 Dec 8 07:22:14 unbound 99937 [99937:0] info: 0.000000 0.000001 9 Dec 8 07:22:14 unbound 99937 [99937:0] info: lower(secs) upper(secs) recursions Dec 8 07:22:14 unbound 99937 [99937:0] info: [25%]=0.0238175 median[50%]=0.0711305 [75%]=0.28672 Dec 8 07:22:14 unbound 99937 [99937:0] info: histogram of recursion processing times Dec 8 07:22:14 unbound 99937 [99937:0] info: average recursion processing time 1.334821 sec Dec 8 07:22:14 unbound 99937 [99937:0] info: server stats for thread 0: requestlist max 10 avg 0.733032 exceeded 0 jostled 0 Dec 8 07:22:14 unbound 99937 [99937:0] info: server stats for thread 0: 424 queries, 203 answers from cache, 221 recursions, 0 prefetch, 0 rejected by ip ratelimiting Dec 8 07:22:14 unbound 99937 [99937:0] info: service stopped (unbound 1.18.0). Dec 8 07:20:21 unbound 99937 [99937:1] notice: ssl handshake failed 10.0.1.1 port 853 Dec 8 07:20:21 unbound 99937 [99937:1] error: ssl handshake failed crypto error:0A000086:SSL routines::certificate verify failed Dec 8 07:20:21 unbound 99937 [99937:1] notice: ssl handshake failed 10.0.1.1 port 853 Dec 8 07:20:21 unbound 99937 [99937:1] error: ssl handshake failed crypto error:0A000086:SSL routines::certificate verify failed Dec 8 07:19:38 unbound 99937 [99937:1] notice: ssl handshake failed 10.0.1.1 port 853 Dec 8 07:19:38 unbound 99937 [99937:1] error: ssl handshake failed crypto error:0A000086:SSL routines::certificate verify failed Dec 8 07:19:38 unbound 99937 [99937:1] notice: ssl handshake failed 10.0.1.1 port 853 Dec 8 07:19:38 unbound 99937 [99937:1] error: ssl handshake failed crypto error:0A000086:SSL routines::certificate verify failed Dec 8 07:15:06 unbound 99937 [99937:0] notice: ssl handshake failed 10.0.1.1 port 853 Dec 8 07:15:06 unbound 99937 [99937:0] error: ssl handshake failed crypto error:0A000086:SSL routines::certificate verify failed Dec 8 07:15:06 unbound 99937 [99937:0] notice: ssl handshake failed 10.0.1.1 port 853 Dec 8 07:15:06 unbound 99937 [99937:0] error: ssl handshake failed crypto error:0A000086:SSL routines::certificate verify failedI'm not exactly sure how to debug this error of the cause so I'm just going to post some screenshots of my various settings and certificates. I'm hoping someone can actually help with this:
-
I'm seeing something similar, but not quite the same.
Jan 1 08:10:06 unbound 56909 [56909:0] error: remote control failed ssl crypto error:0A000415:SSL routines::sslv3 alert certificate expired Jan 1 08:10:06 unbound 56909 [56909:0] notice: failed connection from 127.0.0.1 port 1236 Jan 1 08:10:05 unbound 56909 [56909:0] error: remote control failed ssl crypto error:0A000415:SSL routines::sslv3 alert certificate expired Jan 1 08:10:05 unbound 56909 [56909:0] notice: failed connection from 127.0.0.1 port 25279 Jan 1 08:10:04 unbound 56909 [56909:0] error: remote control failed ssl crypto error:0A000415:SSL routines::sslv3 alert certificate expired Jan 1 08:10:04 unbound 56909 [56909:0] notice: failed connection from 127.0.0.1 port 54171 Jan 1 08:10:03 unbound 56909 [56909:0] error: remote control failed ssl crypto error:0A000415:SSL routines::sslv3 alert certificate expired Jan 1 08:10:03 unbound 56909 [56909:0] notice: failed connection from 127.0.0.1 port 15107I've been troubleshooting, but can't seem to track it down yet. Have you had any luck? I'm running the pfSense 24.11-RELEASE.
-
@dmorda said in error: ssl handshake failed crypto error:0A000086:SSL routines::certificate verify failed:
alert certificate expired
Not really complicated. You have / are using a certificate that is expired.
I propose that you renew it
edit :
This one - here you have the name :
used by the Resolver (unbound).
-
@Gertjan Yep, that's the first thing I checked. I had unbound configured to use an active, non-expired, certificate. I then generated a new one specifically for it, deleted some older ones that were expired, but not in use. I then rebooted pfSense and the issue seems to have resolved itself. Hoping it stays resolved, thanks for chiming in!
-
@dmorda said in error: ssl handshake failed crypto error:0A000086:SSL routines::certificate verify failed:
Hoping it stays resolved
Certificates have an end-of-life date build in.
I "solved" this by using a cert for unbound that gets auto regenerated (by Encrypted) - as I use the same cert for the GUI, captive portal, my printers etc etc..
-
@Gertjan That cert to use is meaningless unless you have the enabled ssl/tls service enabled.
Now @kevdog does have it enabled - but why would be my question. Do you really have clients talking to unbound on pfsense via dot? Clients don't normally use dot, they would use doh.. And while you can get unbound to listen for doh, you would have to call it out in options.
If you don't have clients talking to unbound via dot, just uncheck that box..
-
@johnpoz said in error: ssl handshake failed crypto error:0A000086:SSL routines::certificate verify failed:
Now @kevdog does have it enabled - but why would be my question. Do you really have clients talking to unbound on pfsense via dot?
Has has DOT (DNS over TLS on port 853, TCP (only !), right?) enabled, because he has set up his LAN devices to use it - I guess ?
With, for example, Windows 10, external software was needed to make use of DOT, as some don't trust there own Wifi and don't trust their own network cables.
Recently, browsers ca be set up to make use of DOT, and Windows 11 (pro, at least) can also use it if needed.The bottom line : this must exist : it's pfSense, so why not, let's select and check every option in the GUI ^^
Btw : If "Enable SSL/TLS Service" is not selected, does unbound actually need a cert ?
-
@Gertjan said in error: ssl handshake failed crypto error:0A000086:SSL routines::certificate verify failed:
Btw : If "Enable SSL/TLS Service" is not selected, does unbound actually need a cert ?
no - that is my point.. If you don't actively have devices wanting to talk to unbound via dot there is little point in having that enabled. I find it odd that windows would default to using dot vs doh if wanting to use dns over tls.. Dot is almost exclusively for NS to use talking to other NSers.. I am not aware of clients that would default to dot vs doh. But sure its not out of the realm of possibility - MS does there own thing - I think they have servers using doh and clients using dot from some of the stuff I just read - which makes zero sense.. Clients would use doh and servers, that normally are dns servers etc and talk to other NSers which would normally be dot.
-
Thanks for this thread finally getting some traction since I've seemingly received this error for months.
So just to give some more information
I've checked all my certificates on pfSense.My unbound resolver is using a certificate known as pfsense.<domain>.com
and the actual certificate is issued through E5: Here is some of the actual certificate:Certificate: Data: Version: 3 (0x2) Serial Number: 03:67:40:6c:0a:52:f4:4f:95:6b:cf:4b:6c:2d:ca:42:a4:40 Signature Algorithm: ecdsa-with-SHA384 Issuer: C = US, O = Let's Encrypt, CN = E5 Validity Not Before: Nov 7 08:18:06 2024 GMT Not After : Feb 5 08:18:05 2025 GMT Subject: CN = pfsense.<domain>.com Subject Public Key Info: Public Key Algorithm: id-ecPublicKey Public-Key: (384 bit) pub: .... ASN1 OID: secp384r1 NIST CURVE: P-384 ...Certificate isn't expired so I don't think that's it.
So my latest log looks like the following - 10.0.1.1 is the actual address of the pfSense installation. In terms of what's causing the error I'm not sure:
Jan 1 12:37:26 unbound 1633 [1633:1] notice: ssl handshake failed 10.0.1.1 port 853 Jan 1 12:37:26 unbound 1633 [1633:1] error: ssl handshake failed crypto error:0A000086:SSL routines::certificate verify failed Jan 1 12:37:26 unbound 1633 [1633:1] notice: ssl handshake failed 10.0.1.1 port 853 Jan 1 12:37:26 unbound 1633 [1633:1] error: ssl handshake failed crypto error:0A000086:SSL routines::certificate verify failed Jan 1 12:35:33 unbound 1633 [1633:1] notice: ssl handshake failed 10.0.1.1 port 853 Jan 1 12:35:33 unbound 1633 [1633:1] error: ssl handshake failed crypto error:0A000086:SSL routines::certificate verify failed Jan 1 12:35:33 unbound 1633 [1633:1] notice: ssl handshake failed 10.0.1.1 port 853 Jan 1 12:35:33 unbound 1633 [1633:1] error: ssl handshake failed crypto error:0A000086:SSL routines::certificate verify failed Jan 1 12:32:27 unbound 1633 [1633:1] notice: ssl handshake failed 10.0.1.1 port 853 Jan 1 12:32:27 unbound 1633 [1633:1] error: ssl handshake failed crypto error:0A000086:SSL routines::certificate verify failed Jan 1 12:32:27 unbound 1633 [1633:1] notice: ssl handshake failed 10.0.1.1 port 853 Jan 1 12:32:27 unbound 1633 [1633:1] error: ssl handshake failed crypto error:0A000086:SSL routines::certificate verify failed Jan 1 12:32:10 unbound 1633 [1633:0] info: start of service (unbound 1.18.0).In terms of the question in regards to DOT/DOH.
I believe I'm using DOT. Although TLS DNS resolver queries aren't exactly necessary, I'm trying to observe best practices. In terms of who is looking up TLS encrypted queries -- my setup involves a local LAN with pfsense access. The pfsense is connected to another pfSense router through a site-to-site Wireguard VPN. I believe it's this site-to-site configuration that's causing the problem (more below). Why I don't believe it's the LAN clients causing this error is the following:My VM LAN clients using TLS Unbound resolution are VM's utilizing systemd-networkd and systemd-resolved. My resolve configuration file on the VM's look like:
[Resolve] DNSSEC=no Domains=~. <domain>.com DNS=10.0.1.1#pfsense.<domain>.com DNSOverTLS=opportunistic FallbackDNS=127.0.0.1 10.0.1.1#pfsense.<domain>.com 9.9.9.9#dns.quad9.net 149.112.112.112#dns.quad9.net 1.1.1.2#cloudflare-dns.com 1.1.1.3#cloudflare-dns.com 1.1.1.1#cloudflare-dns.com LLMNR=noAlthough this is the default configuration, you can see that since all these VM's get receive a reserved IP address through pfSense, the default configuration really isn't used since I believe Dynamically assigned IP addesses fall back on a "per-link" interface configuration that is sent through pfsense:
# resolvectl status Global Protocols: -LLMNR -mDNS DNSOverTLS=opportunistic DNSSEC=no/unsupported resolv.conf mode: stub Current DNS Server: 10.0.1.1#pfsense.<domain>.com DNS Servers: 10.0.1.1#pfsense.<domain>.com Fallback DNS Servers: 127.0.0.1 10.0.1.1 9.9.9.9 1.1.1.2 1.1.1.3 DNS Domain: <domain>.com ~. Link 2 (eth0) Current Scopes: DNS Protocols: +DefaultRoute -LLMNR -mDNS DNSOverTLS=opportunistic DNSSEC=no/unsupported Current DNS Server: 10.0.1.1 DNS Servers: 10.0.1.1 DNS Domain: <domain>.comBack to the site-to-site Wireguard VPN.
VPN Tunnel Network 10.99.210.0/30
Peer #1 - VPN - LAN IP address 10.0.1.1 with Tunnel address of 10.99.210.1
Peer #2 - VPN - LAN IP address of 10.1.0.1 with Tunnel address of 10.99.210.2Here is the actual Interfaces for the Wireguard VPN:
Peer #1 -
Peer #2 -
In terms of the error Log on Peer #2 - I'm getting this:
Dec 31 23:35:00 unbound 64101 [64101:2] notice: ssl handshake failed 10.99.210.1 port 38077 Dec 31 23:35:00 unbound 64101 [64101:2] error: ssl handshake failed crypto error:00000000:lib(0)::reason(0) Dec 30 23:16:52 unbound 64101 [64101:2] notice: ssl handshake failed 10.99.210.1 port 45160 Dec 30 23:16:52 unbound 64101 [64101:2] error: ssl handshake failed crypto error:00000000:lib(0)::reason(0) Dec 30 11:03:43 unbound 64101 [64101:2] notice: ssl handshake failed 10.99.210.1 port 19464 Dec 30 11:03:43 unbound 64101 [64101:2] error: ssl handshake failed crypto error:00000000:lib(0)::reason(0) Dec 30 05:27:21 unbound 64101 [64101:2] notice: ssl handshake failed 10.99.210.1 port 48125 Dec 30 05:27:21 unbound 64101 [64101:2] error: ssl handshake failed crypto error:00000000:lib(0)::reason(0) Dec 29 13:22:22 unbound 64101 [64101:2] notice: ssl handshake failed 10.99.210.1 port 25532 Dec 29 13:22:22 unbound 64101 [64101:2] error: ssl handshake failed crypto error:00000000:lib(0)::reason(0) Dec 29 09:41:12 unbound 64101 [64101:3] notice: ssl handshake failed 10.99.210.1 port 18575I have no idea if the two are related. In logs above, I don't know what the port numbers 48125, 25532, 18575, etc represent.
Hopefully that's more representative of the problem.
-
@kevdog said in error: ssl handshake failed crypto error:0A000086:SSL routines::certificate verify failed:
notice: ssl handshake failed 10.99.210.1 port 38077
why would you gateway be talking to unbound for dot, or why would you be talking to your gateway= for dot?
If you uncheck that box - do these errors go away.. Those ports are not dot port or doh ports - are those the source ports that 10.99.210.1 is talking to your 10.99.210.2 IP?
-
Ok hopefully I can explain my setup
I'm running two pfsense installations -- each running their own unbound server.
Each installation has domain overrides for servers running on their respective LAN.Although it's an unconventional setup since I don't have a bind setup, I want clients on say Network #1 when doing dot lookup or dns lookups to consult the local unbound server first, then if name resolution not found then use the pfsense unbound server on Network #2. The gateways represent the tunnel addresses of the pfsense installations at either end of the tunnel.
"If you uncheck that box - do the errors go away?" -- What box are you referring to?
In terms of ports for the wireguard for this site to site, the only wireguard port being employed is 51822 in the wireguard setup. I have no idea why those random ports are being used.
-
@kevdog no clue to how/why you would think such a setup would work - dns is not done in any sort of order - you cant say check this dns server first, etc.
You can not redirect dot or doh - not from a sane client, because the client should validate that its talking to the device it was told to talk to that validates via the cert be it the fqdn or the Ip in the cert.
If your already inside a tunnel, what is the point of dot or doh in the first place?
Have your clients ask unbound for whatever they are looking for - if you want unbound to then forward this to some dot server - then do that on unbound, which has zero to do with what unbound listening on dot or doh.
the ability to redirect dot or doh - invalidates one of its prime functions - validation that your actually talking the NS you told the client to talk too.
-
@johnpoz I admit the setup isn't ideal, however somehow despite the error messages the system seems to work -- clearly I don't really understand all the underpinnings of how things work.
How should I be constructing things??
Two pfsense installations running unbound with same domain. Each pfsense installation has domain overrides for subdomains running on their installation. Additionally each pfsense answering DNS over DOT on port 853.
domain.com------->>>Pfsense #1 (domain=domain.com) -> Overrides--->test.domain.com --->test2.domain.com --->test3.domain.com ------>>>Pfsense #2 (domain=domain.com) -> Overrides --->test4.domain.com --->test5.domain.com --->test6.domain.comEach installation can resolve locally, however if pfsense installations connected by vpn, I need name resolution for devices on Pfsense #1 network accessible to devices on Pfsense#2 network -- and vice versa. If VPN is broken or down, local domain overrides will still work.
I'm just making use right now of the unbound domain overrides section similar to this:
In terms of DOT -- no I don't need it between the pfsense nodes on either end of the tunnel -- however how do I have it only active for LAN clients but not for the tunnel?
I'm not looking actually to forward DNS requests, rather have unbound "resolve" them and then pass the answer back to the clients. In terms of resolving (not forwarding), how does each unbound server know what DNS server is definitive for a specific local domain that's split? I thought I was accomplishing this by listing the servers within the domain overrrides section.
