• Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login
Netgate Discussion Forum
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login

error: ssl handshake failed crypto error:0A000086:SSL routines::certificate verify failed

Scheduled Pinned Locked Moved DHCP and DNS
13 Posts 4 Posters 3.6k Views
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • D
    dmorda @Gertjan
    last edited by Jan 1, 2025, 1:26 PM

    @Gertjan Yep, that's the first thing I checked. I had unbound configured to use an active, non-expired, certificate. I then generated a new one specifically for it, deleted some older ones that were expired, but not in use. I then rebooted pfSense and the issue seems to have resolved itself. Hoping it stays resolved, thanks for chiming in!

    G 1 Reply Last reply Jan 1, 2025, 1:29 PM Reply Quote 0
    • G
      Gertjan @dmorda
      last edited by Gertjan Jan 1, 2025, 1:36 PM Jan 1, 2025, 1:29 PM

      @dmorda said in error: ssl handshake failed crypto error:0A000086:SSL routines::certificate verify failed:

      Hoping it stays resolved

      Certificates have an end-of-life date build in.

      I "solved" this by using a cert for unbound that gets auto regenerated (by Encrypted) - as I use the same cert for the GUI, captive portal, my printers etc etc..

      No "help me" PM's please. Use the forum, the community will thank you.
      Edit : and where are the logs ??

      J 1 Reply Last reply Jan 1, 2025, 3:31 PM Reply Quote 0
      • J
        johnpoz LAYER 8 Global Moderator @Gertjan
        last edited by Jan 1, 2025, 3:31 PM

        @Gertjan That cert to use is meaningless unless you have the enabled ssl/tls service enabled.

        Now @kevdog does have it enabled - but why would be my question. Do you really have clients talking to unbound on pfsense via dot? Clients don't normally use dot, they would use doh.. And while you can get unbound to listen for doh, you would have to call it out in options.

        If you don't have clients talking to unbound via dot, just uncheck that box..

        An intelligent man is sometimes forced to be drunk to spend time with his fools
        If you get confused: Listen to the Music Play
        Please don't Chat/PM me for help, unless mod related
        SG-4860 24.11 | Lab VMs 2.8, 24.11

        G 1 Reply Last reply Jan 1, 2025, 3:42 PM Reply Quote 0
        • G
          Gertjan @johnpoz
          last edited by Jan 1, 2025, 3:42 PM

          @johnpoz said in error: ssl handshake failed crypto error:0A000086:SSL routines::certificate verify failed:

          Now @kevdog does have it enabled - but why would be my question. Do you really have clients talking to unbound on pfsense via dot?

          Has has DOT (DNS over TLS on port 853, TCP (only !), right?) enabled, because he has set up his LAN devices to use it - I guess ?

          With, for example, Windows 10, external software was needed to make use of DOT, as some don't trust there own Wifi and don't trust their own network cables.
          Recently, browsers ca be set up to make use of DOT, and Windows 11 (pro, at least) can also use it if needed.

          The bottom line : this must exist : it's pfSense, so why not, let's select and check every option in the GUI ^^

          Btw : If "Enable SSL/TLS Service" is not selected, does unbound actually need a cert ?

          No "help me" PM's please. Use the forum, the community will thank you.
          Edit : and where are the logs ??

          J 1 Reply Last reply Jan 1, 2025, 4:44 PM Reply Quote 0
          • J
            johnpoz LAYER 8 Global Moderator @Gertjan
            last edited by Jan 1, 2025, 4:44 PM

            @Gertjan said in error: ssl handshake failed crypto error:0A000086:SSL routines::certificate verify failed:

            Btw : If "Enable SSL/TLS Service" is not selected, does unbound actually need a cert ?

            no - that is my point.. If you don't actively have devices wanting to talk to unbound via dot there is little point in having that enabled. I find it odd that windows would default to using dot vs doh if wanting to use dns over tls.. Dot is almost exclusively for NS to use talking to other NSers.. I am not aware of clients that would default to dot vs doh. But sure its not out of the realm of possibility - MS does there own thing - I think they have servers using doh and clients using dot from some of the stuff I just read - which makes zero sense.. Clients would use doh and servers, that normally are dns servers etc and talk to other NSers which would normally be dot.

            An intelligent man is sometimes forced to be drunk to spend time with his fools
            If you get confused: Listen to the Music Play
            Please don't Chat/PM me for help, unless mod related
            SG-4860 24.11 | Lab VMs 2.8, 24.11

            1 Reply Last reply Reply Quote 0
            • K
              kevdog
              last edited by Jan 1, 2025, 7:19 PM

              Thanks for this thread finally getting some traction since I've seemingly received this error for months.

              So just to give some more information
              I've checked all my certificates on pfSense.

              My unbound resolver is using a certificate known as pfsense.<domain>.com
              and the actual certificate is issued through E5: Here is some of the actual certificate:

              Certificate:
                  Data:
                      Version: 3 (0x2)
                      Serial Number:
                          03:67:40:6c:0a:52:f4:4f:95:6b:cf:4b:6c:2d:ca:42:a4:40
                      Signature Algorithm: ecdsa-with-SHA384
                      Issuer: C = US, O = Let's Encrypt, CN = E5
                      Validity
                          Not Before: Nov  7 08:18:06 2024 GMT
                          Not After : Feb  5 08:18:05 2025 GMT
                      Subject: CN = pfsense.<domain>.com
                      Subject Public Key Info:
                          Public Key Algorithm: id-ecPublicKey
                              Public-Key: (384 bit)
                              pub:
                                ....
                              ASN1 OID: secp384r1
                              NIST CURVE: P-384
              ...
              

              Certificate isn't expired so I don't think that's it.

              So my latest log looks like the following - 10.0.1.1 is the actual address of the pfSense installation. In terms of what's causing the error I'm not sure:

              Jan 1 12:37:26	unbound	1633	[1633:1] notice: ssl handshake failed 10.0.1.1 port 853
              Jan 1 12:37:26	unbound	1633	[1633:1] error: ssl handshake failed crypto error:0A000086:SSL routines::certificate verify failed
              Jan 1 12:37:26	unbound	1633	[1633:1] notice: ssl handshake failed 10.0.1.1 port 853
              Jan 1 12:37:26	unbound	1633	[1633:1] error: ssl handshake failed crypto error:0A000086:SSL routines::certificate verify failed
              Jan 1 12:35:33	unbound	1633	[1633:1] notice: ssl handshake failed 10.0.1.1 port 853
              Jan 1 12:35:33	unbound	1633	[1633:1] error: ssl handshake failed crypto error:0A000086:SSL routines::certificate verify failed
              Jan 1 12:35:33	unbound	1633	[1633:1] notice: ssl handshake failed 10.0.1.1 port 853
              Jan 1 12:35:33	unbound	1633	[1633:1] error: ssl handshake failed crypto error:0A000086:SSL routines::certificate verify failed
              Jan 1 12:32:27	unbound	1633	[1633:1] notice: ssl handshake failed 10.0.1.1 port 853
              Jan 1 12:32:27	unbound	1633	[1633:1] error: ssl handshake failed crypto error:0A000086:SSL routines::certificate verify failed
              Jan 1 12:32:27	unbound	1633	[1633:1] notice: ssl handshake failed 10.0.1.1 port 853
              Jan 1 12:32:27	unbound	1633	[1633:1] error: ssl handshake failed crypto error:0A000086:SSL routines::certificate verify failed
              Jan 1 12:32:10	unbound	1633	[1633:0] info: start of service (unbound 1.18.0).
              

              In terms of the question in regards to DOT/DOH.
              I believe I'm using DOT. Although TLS DNS resolver queries aren't exactly necessary, I'm trying to observe best practices. In terms of who is looking up TLS encrypted queries -- my setup involves a local LAN with pfsense access. The pfsense is connected to another pfSense router through a site-to-site Wireguard VPN. I believe it's this site-to-site configuration that's causing the problem (more below). Why I don't believe it's the LAN clients causing this error is the following:

              My VM LAN clients using TLS Unbound resolution are VM's utilizing systemd-networkd and systemd-resolved. My resolve configuration file on the VM's look like:

              [Resolve]
              DNSSEC=no
              Domains=~. <domain>.com
              DNS=10.0.1.1#pfsense.<domain>.com
              DNSOverTLS=opportunistic
              FallbackDNS=127.0.0.1 10.0.1.1#pfsense.<domain>.com 9.9.9.9#dns.quad9.net 149.112.112.112#dns.quad9.net 1.1.1.2#cloudflare-dns.com 1.1.1.3#cloudflare-dns.com 1.1.1.1#cloudflare-dns.com
              LLMNR=no
              

              Although this is the default configuration, you can see that since all these VM's get receive a reserved IP address through pfSense, the default configuration really isn't used since I believe Dynamically assigned IP addesses fall back on a "per-link" interface configuration that is sent through pfsense:

              # resolvectl status
              Global
                         Protocols: -LLMNR -mDNS DNSOverTLS=opportunistic DNSSEC=no/unsupported
                  resolv.conf mode: stub
                Current DNS Server: 10.0.1.1#pfsense.<domain>.com
                       DNS Servers: 10.0.1.1#pfsense.<domain>.com
              Fallback DNS Servers: 127.0.0.1 10.0.1.1 9.9.9.9 1.1.1.2 1.1.1.3
                        DNS Domain: <domain>.com ~.
              
              Link 2 (eth0)
                  Current Scopes: DNS
                       Protocols: +DefaultRoute -LLMNR -mDNS DNSOverTLS=opportunistic DNSSEC=no/unsupported
              Current DNS Server: 10.0.1.1
                     DNS Servers: 10.0.1.1
                      DNS Domain: <domain>.com
              

              Back to the site-to-site Wireguard VPN.

              VPN Tunnel Network 10.99.210.0/30
              Peer #1 - VPN - LAN IP address 10.0.1.1 with Tunnel address of 10.99.210.1
              Peer #2 - VPN - LAN IP address of 10.1.0.1 with Tunnel address of 10.99.210.2

              Here is the actual Interfaces for the Wireguard VPN:
              Peer #1 -
              Screenshot 2025-01-01 at 12.10.12 PM.png

              Peer #2 -
              Screenshot 2025-01-01 at 12.10.50 PM.png

              In terms of the error Log on Peer #2 - I'm getting this:

              Dec 31 23:35:00	unbound	64101	[64101:2] notice: ssl handshake failed 10.99.210.1 port 38077
              Dec 31 23:35:00	unbound	64101	[64101:2] error: ssl handshake failed crypto error:00000000:lib(0)::reason(0)
              Dec 30 23:16:52	unbound	64101	[64101:2] notice: ssl handshake failed 10.99.210.1 port 45160
              Dec 30 23:16:52	unbound	64101	[64101:2] error: ssl handshake failed crypto error:00000000:lib(0)::reason(0)
              Dec 30 11:03:43	unbound	64101	[64101:2] notice: ssl handshake failed 10.99.210.1 port 19464
              Dec 30 11:03:43	unbound	64101	[64101:2] error: ssl handshake failed crypto error:00000000:lib(0)::reason(0)
              Dec 30 05:27:21	unbound	64101	[64101:2] notice: ssl handshake failed 10.99.210.1 port 48125
              Dec 30 05:27:21	unbound	64101	[64101:2] error: ssl handshake failed crypto error:00000000:lib(0)::reason(0)
              Dec 29 13:22:22	unbound	64101	[64101:2] notice: ssl handshake failed 10.99.210.1 port 25532
              Dec 29 13:22:22	unbound	64101	[64101:2] error: ssl handshake failed crypto error:00000000:lib(0)::reason(0)
              Dec 29 09:41:12	unbound	64101	[64101:3] notice: ssl handshake failed 10.99.210.1 port 18575
              

              I have no idea if the two are related. In logs above, I don't know what the port numbers 48125, 25532, 18575, etc represent.

              Hopefully that's more representative of the problem.

              J 1 Reply Last reply Jan 1, 2025, 11:33 PM Reply Quote 0
              • J
                johnpoz LAYER 8 Global Moderator @kevdog
                last edited by Jan 1, 2025, 11:33 PM

                @kevdog said in error: ssl handshake failed crypto error:0A000086:SSL routines::certificate verify failed:

                notice: ssl handshake failed 10.99.210.1 port 38077

                why would you gateway be talking to unbound for dot, or why would you be talking to your gateway= for dot?

                If you uncheck that box - do these errors go away.. Those ports are not dot port or doh ports - are those the source ports that 10.99.210.1 is talking to your 10.99.210.2 IP?

                An intelligent man is sometimes forced to be drunk to spend time with his fools
                If you get confused: Listen to the Music Play
                Please don't Chat/PM me for help, unless mod related
                SG-4860 24.11 | Lab VMs 2.8, 24.11

                K 1 Reply Last reply Jan 2, 2025, 12:07 AM Reply Quote 0
                • K
                  kevdog @johnpoz
                  last edited by kevdog Jan 2, 2025, 12:08 AM Jan 2, 2025, 12:07 AM

                  @johnpoz

                  Ok hopefully I can explain my setup

                  I'm running two pfsense installations -- each running their own unbound server.
                  Each installation has domain overrides for servers running on their respective LAN.

                  Although it's an unconventional setup since I don't have a bind setup, I want clients on say Network #1 when doing dot lookup or dns lookups to consult the local unbound server first, then if name resolution not found then use the pfsense unbound server on Network #2. The gateways represent the tunnel addresses of the pfsense installations at either end of the tunnel.

                  "If you uncheck that box - do the errors go away?" -- What box are you referring to?

                  In terms of ports for the wireguard for this site to site, the only wireguard port being employed is 51822 in the wireguard setup. I have no idea why those random ports are being used.

                  J 1 Reply Last reply Jan 2, 2025, 12:24 AM Reply Quote 0
                  • J
                    johnpoz LAYER 8 Global Moderator @kevdog
                    last edited by johnpoz Jan 2, 2025, 12:26 AM Jan 2, 2025, 12:24 AM

                    @kevdog no clue to how/why you would think such a setup would work - dns is not done in any sort of order - you cant say check this dns server first, etc.

                    You can not redirect dot or doh - not from a sane client, because the client should validate that its talking to the device it was told to talk to that validates via the cert be it the fqdn or the Ip in the cert.

                    If your already inside a tunnel, what is the point of dot or doh in the first place?

                    Have your clients ask unbound for whatever they are looking for - if you want unbound to then forward this to some dot server - then do that on unbound, which has zero to do with what unbound listening on dot or doh.

                    the ability to redirect dot or doh - invalidates one of its prime functions - validation that your actually talking the NS you told the client to talk too.

                    An intelligent man is sometimes forced to be drunk to spend time with his fools
                    If you get confused: Listen to the Music Play
                    Please don't Chat/PM me for help, unless mod related
                    SG-4860 24.11 | Lab VMs 2.8, 24.11

                    K 1 Reply Last reply Jan 2, 2025, 12:47 AM Reply Quote 0
                    • K
                      kevdog @johnpoz
                      last edited by kevdog Jan 2, 2025, 12:49 AM Jan 2, 2025, 12:47 AM

                      @johnpoz I admit the setup isn't ideal, however somehow despite the error messages the system seems to work -- clearly I don't really understand all the underpinnings of how things work.

                      How should I be constructing things??

                      Two pfsense installations running unbound with same domain. Each pfsense installation has domain overrides for subdomains running on their installation. Additionally each pfsense answering DNS over DOT on port 853.

                      domain.com------->>>Pfsense  #1 (domain=domain.com) -> Overrides--->test.domain.com
                                                                                      --->test2.domain.com
                                                                                      --->test3.domain.com
                                     
                                 ------>>>Pfsense #2 (domain=domain.com) -> Overrides --->test4.domain.com
                                                                                      --->test5.domain.com
                                                                                      --->test6.domain.com
                      

                      Each installation can resolve locally, however if pfsense installations connected by vpn, I need name resolution for devices on Pfsense #1 network accessible to devices on Pfsense#2 network -- and vice versa. If VPN is broken or down, local domain overrides will still work.

                      I'm just making use right now of the unbound domain overrides section similar to this:
                      Screenshot 2025-01-01 at 5.40.14 PM.png

                      In terms of DOT -- no I don't need it between the pfsense nodes on either end of the tunnel -- however how do I have it only active for LAN clients but not for the tunnel?

                      I'm not looking actually to forward DNS requests, rather have unbound "resolve" them and then pass the answer back to the clients. In terms of resolving (not forwarding), how does each unbound server know what DNS server is definitive for a specific local domain that's split? I thought I was accomplishing this by listing the servers within the domain overrrides section.

                      1 Reply Last reply Reply Quote 0
                      13 out of 13
                      • First post
                        13/13
                        Last post
                      Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.
                        This community forum collects and processes your personal information.
                        consent.not_received