• Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login
Netgate Discussion Forum
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login

Unbound dns resolver stops resolving every few days after 22.05 upgrade

DHCP and DNS
3
14
1.6k
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • V
    vaidas
    last edited by Jul 5, 2022, 7:26 AM

    DNS service breaks every 1-2 days after upgrading from 22.01 to 22.05.
    It just stops resolving queries that are not cached (most recent sites are still resolved possible by client cache)
    Dashboard shows that unbound is running (status ok) but resolves just timing out.
    22.01 was fine for months no problems with unbound.
    After upgrading to 22.05 (did additional reboot) then ~2days and resolves just stopped.
    restart of unbound service (no need to reboot pfsense) solved problem but it returned again after few days.
    had a lots of problems with unbound totally crashing in CE2.5 then it was solved now problems again, this time unbound seems to report good status (running/ok) when it stops resolving, so watchdog service just to restart crashed service does not help.

    If there is any logs or something that would be helpful to troubleshoot please ask.
    I am attaching latest log file from unbound, had problem that day possible logs capture that.
    unboundlogs.txt

    G 1 Reply Last reply Jul 5, 2022, 7:50 AM Reply Quote 0
    • G
      Gertjan @vaidas
      last edited by Gertjan Jul 5, 2022, 7:55 AM Jul 5, 2022, 7:50 AM

      @vaidas said in Unbound dns resolver stops resolving every few days after 22.05 upgrade:

      so watchdog service just to restart

      The watchdog service is like dynamite.
      It never repairs stuff, it makes stuff even more messy, so you can't see the original mess.
      My advise : stop using it.

      unbound is big and can be slow to start.
      watchdog service is inpatient and stops a stating unbound to restart it : you've just created your own mess.

      See here :

      ....
      Jul  5 14:33:22 router unbound[55240]: [55240:0] notice: Restart of unbound 1.15.0.
      Jul  5 14:33:22 router unbound[55240]: [55240:0] notice: init module 0: validator
      Jul  5 14:33:22 router unbound[55240]: [55240:0] notice: init module 1: iterator
      Jul  5 14:33:22 router unbound[55240]: [55240:0] info: start of service (unbound 1.15.0).
      Jul  5 14:33:23 router unbound[55240]: [55240:0] info: generate keytag query _ta-4f66. NULL IN
      Jul  5 14:36:41 router unbound[55240]: [55240:0] info: service stopped (unbound 1.15.0).
      ...
      

      or some other signal stopped unbound 3 minutes after it started.
      A interface went down and up ?
      Look in the other logs around "Jul 5 14:36:41" to know what happened. unbound was asked to stop by some system event.

      Things get worse further along :

      Jul  5 14:44:15 router unbound[55240]: [55240:0] info: start of service (unbound 1.15.0).
      Jul  5 14:44:17 router unbound[55240]: [55240:0] info: service stopped (unbound 1.15.0).
      

      My unbound 22.05 restarts ones in a while, every 2 or 3 days or so, as I'm also using pfBlockerng-devel. Other reasons might exist.

      You've disabled this one :

      🔒 Log in to view

      right ?
      As it restarts unbound for every DHCP lease created or renewed.

      No "help me" PM's please. Use the forum, the community will thank you.
      Edit : and where are the logs ??

      V 1 Reply Last reply Jul 5, 2022, 7:58 AM Reply Quote 0
      • V
        vaidas @Gertjan
        last edited by vaidas Jul 5, 2022, 8:20 AM Jul 5, 2022, 7:58 AM

        @gertjan No I have register DHCP leases in DNS resolver as I need this service in my network. that's probably biggest reason why I use local DNS resolver.
        I have manually restarted unbound once in that log, because it was not resolving.
        other restarts are due to dhcp.
        this bug has been unresolved for long time https://redmine.pfsense.org/issues/5413
        but even with the restarts resolves worked in 22.01 and many previous versions (2.5 was mess)

        I don't use pfblockerng or other plugins related to DNS my only plugins I have is nut for my ups and openvpn export utility, also installed watchdog as of 2.5 version because unbound in that version was always crashing (it only monitors unbound but i can stop it just to test) whenever watchdog restarts service that crashed it should notify me (no notifications today about any restarts).
        same config worked fine in 22.01

        G 1 Reply Last reply Jul 5, 2022, 8:38 AM Reply Quote 0
        • G
          Gertjan @vaidas
          last edited by Jul 5, 2022, 8:38 AM

          @vaidas said in Unbound dns resolver stops resolving every few days after 22.05 upgrade:

          No I have register DHCP leases in DNS resolver as I need this service in my network. that's probably biggest reason why I use local DNS resolver.
          I have manually restarted unbound once in that log, because it was not resolving.
          other restarts are due to dhcp.

          This is a very old and known subject.
          Please understand : with any incoming DHCO lease, unbound gets restarted.
          If you have many devices on your LAN, the unbound restart frequency will get bigger and bigger.
          Majors issues will arrive as soon as a (big) swtch goes power down and power up : all attached devices will request a new lease at nearly the same time. Unbound will get hammered, and that is - I suspect - where it will get slammed to brain dead : you have your issue explained.

          Btw : I've myself about 40 devices on my LAN that I need to know by host name and IP. I've solved the issue by creating several static DHCP leases.

          I don't care about DHCP leases - the host names, or their IP's, on my captive portal networks.
          Only server type devices need to have a name and a fixed IPv4.

          This one :

          🔒 Log in to view

          stays checked, as these leases are imported by unboud upon startup and they never change.

          unbound restart issues are thus, for me, many years ago, solved.

          So : now you can change your opinion :

          @vaidas said in Unbound dns resolver stops resolving every few days after 22.05 upgrade:

          No I have register DHCP leases in DNS resolver as I need

          and solve the issue ;)

          No "help me" PM's please. Use the forum, the community will thank you.
          Edit : and where are the logs ??

          V 1 Reply Last reply Jul 5, 2022, 8:45 AM Reply Quote 0
          • V
            vaidas @Gertjan
            last edited by vaidas Jul 5, 2022, 8:54 AM Jul 5, 2022, 8:45 AM

            @gertjan then why even offer this option if it breaks everything. I spin up and down hosts/servers everyday that I need resolved by name, adding static records and then removing them just for that would be chore. But lets end this.
            Question is then why it has been working in 22.01 for months no problems, upgraded to 22.05 and it breaks every few days. (haven't changed any config or network size)
            if you only option that you suggest is disable that setting then I thank you I got you. Will wait maybe there another people that have other valid solutions.

            G 1 Reply Last reply Jul 5, 2022, 10:56 AM Reply Quote 0
            • G
              Gertjan @vaidas
              last edited by Gertjan Jul 5, 2022, 10:57 AM Jul 5, 2022, 10:56 AM

              @vaidas said in Unbound dns resolver stops resolving every few days after 22.05 upgrade:

              Will wait maybe

              Check the forum, you will find hundreds if not thousands of posts about this subject.
              Check pfSEnse redmine, proposals and bug reports have been made. Some are years old.
              Check this one : https://redmine.pfsense.org/issues/5413
              Yours : https://redmine.pfsense.org/issues/13337 fits right into first one, already 6 years old.
              edit : and I saw you found that one ^^

              For the last several ( ! ) years many people (like a lot) have asked about this issue.
              I'm not saying my proposal is 'the' solution. It's 'a' workaround.

              Btw : start thinking about what needs to be done when IPv6 isn't optional any more.

              @vaidas said in Unbound dns resolver stops resolving every few days after 22.05 upgrade:

              Question is then why it has been working in 22.01 for months no problems, upgraded to 22.05 and it breaks every few days.

              Probably pure luck ? Dono.
              Nothing changed in 22.05 - was different in 22.01.
              For me 22.01 and 22.05 are not showing any differences about DHCP/DNS. And I have the graphs to confirm this, you saw them.

              No "help me" PM's please. Use the forum, the community will thank you.
              Edit : and where are the logs ??

              V 1 Reply Last reply Jul 5, 2022, 11:30 AM Reply Quote 0
              • V
                vaidas @Gertjan
                last edited by vaidas Jul 5, 2022, 11:34 AM Jul 5, 2022, 11:30 AM

                @gertjan well thanks for trying to help, will need to change my workflows then and try the workaround or maybe migrate to windows server dns/dhcp stack as it would be nice to also have it with AD :) don't want to say this, but windows dns seems to be more stable nowadays :)
                Maybe there is a hope they seem to finally fixed multiple console problem on same network after many years, even thought I don't own any consoles, saw many people wanted it fixed.
                hey at least I got great bug number 13337

                I was hit by this
                https://redmine.pfsense.org/issues/11316
                hard that's why used watchdog at least in that case unbound would die so watchdog would be useful.

                1 Reply Last reply Reply Quote 0
                • V
                  vaidas
                  last edited by vaidas Jul 8, 2022, 12:37 PM Jul 8, 2022, 12:34 PM

                  Disabled register DHCP leases in DNS setting and outages still happening.
                  logs don't even show unbound restart.
                  the only 2 records for unbound log today when outage happened

                  Jul 8 20:56:06	unbound	28376	[28376:0] info: generate keytag query _ta-4f66. NULL IN
                  Jul 8 09:11:23	unbound	28376	[28376:0] info: generate keytag query _ta-4f66. NULL IN
                  

                  what is happening ?
                  Is there a way to download 22.05 image for reinstall maybe that would solve problem, I seriously considering rolling back to 22.01.

                  G 1 Reply Last reply Jul 8, 2022, 8:06 PM Reply Quote 0
                  • V
                    vaidas
                    last edited by Jul 8, 2022, 2:04 PM

                    possibly related to
                    https://forum.netgate.com/topic/173148/slow-dns-after-22-05/11

                    now at least DNS resolves recover after ~15 min (when disabled DNS registration of dhcp leases) but still 15min outages is annoying
                    Just getting dns time out or ip address not found in browser.

                    1 Reply Last reply Reply Quote 0
                    • G
                      Gertjan @vaidas
                      last edited by Gertjan Jul 8, 2022, 8:08 PM Jul 8, 2022, 8:06 PM

                      @vaidas said in Unbound dns resolver stops resolving every few days after 22.05 upgrade:

                      query _ta-4f66.

                      Or 20326 decimal.
                      Nothing special, '20326' is the mother of all DNSSEC keys at this moment. Its re fetched regularly. A correct fetch is the start of a good DNSSEC functionality.
                      So these messages :

                      🔒 Log in to view

                      are sometimes the only sign of life that unbound is giving while it's humming.

                      No "help me" PM's please. Use the forum, the community will thank you.
                      Edit : and where are the logs ??

                      1 Reply Last reply Reply Quote 0
                      • V
                        vaidas
                        last edited by vaidas Jul 10, 2022, 3:50 PM Jul 10, 2022, 3:48 PM

                        Strange but it seems that problem mostly went away after unchecking(disabling) DNSSEC setting. Still testing but for a more then a day I haven't seen any problems.

                        Even reenabling register DHCP leases did not cause any noticeable problems.

                        G 1 Reply Last reply Jul 11, 2022, 6:43 AM Reply Quote 0
                        • G
                          Gertjan @vaidas
                          last edited by Jul 11, 2022, 6:43 AM

                          @vaidas said in Unbound dns resolver stops resolving every few days after 22.05 upgrade:

                          unchecking(disabling) DNSSEC setting.

                          DNSSEC is an extension of DNS. It's a complicated thing, but 'on the wire' you'll find some more requests. There are the use A, AAAA, PTR, MX, CNAME. Added to that, thewe will be some DS, NSEC and DNSKEY.
                          These are just other UDP TCP packets addressed to the same DNS name servers unbound was already talking to.
                          If these are unknownon the domain name server, no issue, unbound proceeds without DNSSEC checking without any time lost.
                          Most, if not all TLDs (com org net etc etc etc) are DNNSEC signed.
                          The top level dot "." are signed , that's the 4f66 key you see in the unbound logs.
                          If a web site owner took the time to sign its domain name, like this one, a domain name I own/rent, the the entire chain will be ok, and dnssec will work.
                          DNS will work with DNSSEC as nearly fast as without DNSSEC.

                          It should not make DNS work slower or worse or something like that. If that's the case, there is an underlying access- or DNS problem.

                          No "help me" PM's please. Use the forum, the community will thank you.
                          Edit : and where are the logs ??

                          1 Reply Last reply Reply Quote 0
                          • P
                            pajinha
                            last edited by Jul 11, 2022, 11:25 AM

                            I own a Netgate 6100 and have been having the same issue. DNS resolving went to shit after 22.05 update. Until then it was working fine. Most of times wouldnt resolve until after a restart. have had to resort to 4G a few times :-(

                            I have been fiddling with a few settings but I think these last 2 have made it better for me:

                            Untick Enable DNSSEC Support

                            🔒 Log in to view

                            And on the outgoing interfaces I reduced it to only use WAN ( removed my VPN outgoing interfaces ).

                            I will be changing everything back to what I was using before the update but want to confirm slowly each option to try to single out the one that has broke it for me.

                            Will keep an eye here and in this other thread https://forum.netgate.com/topic/173148/slow-dns-after-22-05
                            to see if anyone has managed to single out the main issue...

                            Kind of big one for Netgate... not sure how they managed to screw this one.

                            G 1 Reply Last reply Jul 12, 2022, 8:41 AM Reply Quote 0
                            • G
                              Gertjan @pajinha
                              last edited by Jul 12, 2022, 8:41 AM

                              @pajinha said in Unbound dns resolver stops resolving every few days after 22.05 upgrade:

                              not sure how they managed to screw this one.

                              The forum mentions a couple of 'DNS' issues since 22.05.
                              But, what is a couple ?
                              22.05 has been downloaded and installed many thousands times (I can't tell, but I'm pretty sure).

                              @pajinha said in Unbound dns resolver stops resolving every few days after 22.05 upgrade:

                              ( removed my VPN outgoing interfaces )

                              If your DNS also goes over this VPN and the VPN is bad - as this can happen, they are not all equal and perfect - then, yeah, DNS looks bad.
                              Because your uplink is bad.
                              DNS is mostly UDP, these can get lost. unbound won't hammer away, and return a SERVFAIL.
                              TCP get renegotiated and is far more resilient.

                              For now, my DNS using 22.05 using default settings and no VPN is working as before. And don't tale my word for it, see for yourself.

                              No "help me" PM's please. Use the forum, the community will thank you.
                              Edit : and where are the logs ??

                              1 Reply Last reply Reply Quote 0
                              1 out of 14
                              • First post
                                1/14
                                Last post
                              Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.