• Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login
Netgate Discussion Forum
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login

DNS Resolver not caching correct?

Scheduled Pinned Locked Moved DHCP and DNS
56 Posts 5 Posters 9.1k Views
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • J
    johnpoz LAYER 8 Global Moderator @mrsunfire
    last edited by Aug 31, 2019, 1:07 PM

    @mrsunfire said in DNS Resolver not caching correct?:

    (ns6218.hostgator.com) in 184 ms

    So you see the authoritative ns are that far away from you - lot farther than 7ms ;)

    At any given time you might of talked to something in the path that took longer than normal, maybe something in path was congested, maybe the ns took longer to answer, etc.

    But that is how something is resolved - when none of it is cached.. now the once the ns for .com are cached no reason to ask roots for those, once the ns for whatever.com is cached no reason to talk to the gtld-servers.net ns - so you can just go ask the authoritative NS directly - but since its a lot farther than 7ms away from you - there can be delays... But in the big picture makes no matter. Since its only going to be once, and then its cached for the length of the TTL, and if you have prefetch setup and or zero answer - you will get an answer from unbound instantly and it will go refresh its stuff in the background..

    So what does it matter if took 1000ms - that is not even going to be really noticed since I doubt a site couple hundred ms away from you is going to load instantly anyway.. Worse case you added a whole second to the page load time (ONCE)

    There is little need of concerning your self if a site resolves in 30ms or 300ms, or even 1000ms.. Since once its cached - none of that matters any more. And in the big picture a fraction of a second in addition to the overall first load time of the page is meaningless.

    An intelligent man is sometimes forced to be drunk to spend time with his fools
    If you get confused: Listen to the Music Play
    Please don't Chat/PM me for help, unless mod related
    SG-4860 24.11 | Lab VMs 2.8, 24.11

    1 Reply Last reply Reply Quote 0
    • P
      perlenbacher
      last edited by Aug 31, 2019, 2:13 PM

      Hi johnpoz,

      could you post your DNS Resolver, General Settings and Advanced Settings please.

      It would be very handy for us all!

      Thanks, Perlen

      1 Reply Last reply Reply Quote 0
      • J
        johnpoz LAYER 8 Global Moderator
        last edited by johnpoz Aug 31, 2019, 3:11 PM Aug 31, 2019, 3:07 PM

        Sure here you go
        settings.png

        Notice that I have disabled automatic ACLs so you will have to create your own to allow queries.

        I have also changed from transparent to static for my zone.. Make sure you actually understand what settings do before changing them.. Any questions on what anything specifically does, just ask. Don't think this is some sort of guide to how you should set yours up.. These are my settings for my network and use case.. Most of them are just default.. Only a couple of changes really from out of the box settings. Which may or may not be good for your actual needs.

        Generally speaking - out of the box should be fine for pretty much everyone.

        As to the general settings - there are no dns set other than local... Here
        general.png

        An intelligent man is sometimes forced to be drunk to spend time with his fools
        If you get confused: Listen to the Music Play
        Please don't Chat/PM me for help, unless mod related
        SG-4860 24.11 | Lab VMs 2.8, 24.11

        1 Reply Last reply Reply Quote 1
        • P
          perlenbacher
          last edited by Aug 31, 2019, 4:15 PM

          Very interesting, much appreciated. Thanks!

          1 Reply Last reply Reply Quote 0
          • M
            mrsunfire
            last edited by Sep 1, 2019, 6:36 AM

            For info: this is the result of cacheoutput after two days and I'm happy with it:

            total.num.queries=21177
            total.num.queries_ip_ratelimited=0
            total.num.cachehits=15554
            total.num.cachemiss=5623
            total.num.prefetch=8627
            total.num.zero_ttl=9232
            total.num.recursivereplies=5623
            

            Thanks a lot @johnpoz !

            Netgate 6100 MAX

            1 Reply Last reply Reply Quote 0
            • J
              johnpoz LAYER 8 Global Moderator
              last edited by Sep 1, 2019, 8:13 AM

              @mrsunfire said in DNS Resolver not caching correct?:

              total.num.zero_ttl=9232

              that seems like a lot of low TTLs - you might want to play with uping the min vs letting them set like 60 seconds and 5 min ttls

              An intelligent man is sometimes forced to be drunk to spend time with his fools
              If you get confused: Listen to the Music Play
              Please don't Chat/PM me for help, unless mod related
              SG-4860 24.11 | Lab VMs 2.8, 24.11

              1 Reply Last reply Reply Quote 0
              • M
                mrsunfire
                last edited by Sep 1, 2019, 10:59 AM

                So with what should I start? And what does this change? Isn't this dangerous?

                Netgate 6100 MAX

                1 Reply Last reply Reply Quote 0
                • J
                  johnpoz LAYER 8 Global Moderator
                  last edited by johnpoz Sep 1, 2019, 11:18 AM Sep 1, 2019, 11:16 AM

                  In the advanced tab change the min ttl to say 1800 or 3600 vs letting them set such low ttls.. Could it cause issues - maybe. But I doubt it really..

                  Lets say you had something that was using a 60 second TTL, do the IPs even change - I had issue with some software where it checked a fqdn all the time, and the ttl was 60 seconds.. But upon check it multiple times the IP wasn't even changing - so what is the F'ing point of having dns query for it every 60 seconds.

                  Then you have them doing load balancing nonsense via dns

                  example
                  ;scribe.logs.roku.com. IN A

                  ;; ANSWER SECTION:
                  scribe.logs.roku.com. 60 IN A 52.21.150.70
                  scribe.logs.roku.com. 60 IN A 35.172.120.217
                  scribe.logs.roku.com. 60 IN A 34.233.159.203
                  scribe.logs.roku.com. 60 IN A 35.170.206.212
                  scribe.logs.roku.com. 60 IN A 52.200.248.59
                  scribe.logs.roku.com. 60 IN A 52.1.249.132
                  scribe.logs.roku.com. 60 IN A 34.226.55.236
                  scribe.logs.roku.com. 60 IN A 52.21.44.213

                  So while you might use 52.21.150.70 one time, next time is 25.172.120.217, etc. Only issue you could have is that 52.21.150.170 no longer works.

                  Have really never seen it be an issue.. But I wouldn't suggest you set the min to like 24 hours or anything.. If you find your having an issue getting somewhere, you can always just flush that specific entry from your cache.

                  This should lower the amount of both local queries and external queries.. Since now vs your local client asking for something every min with such a low ttl, it would only need to query 1 an hour for example.. Same goes for external queries. If your local device is not using a local cache - doesn't really matter what the ttl is for local queries - but would keep your dns from having to resolve it every freaking 60 seconds because some dumb iot device keeps asking for it every 30 freaking seconds, etc. because it has no local cache.

                  But hey if you got some local devices asking for whatever.something.tld every minute - you just got a 60x reduction in the number of external queries you have to do ;) if you change the min ttl to 1 hour. Multiply that by a few devices and few different fqdn being queried and in the course of 24 you could be doing 10s of 1000's of less queries.. Again prob not all that big of deal in the big picture.. But why do them if you don't really need to, etc.

                  An intelligent man is sometimes forced to be drunk to spend time with his fools
                  If you get confused: Listen to the Music Play
                  Please don't Chat/PM me for help, unless mod related
                  SG-4860 24.11 | Lab VMs 2.8, 24.11

                  1 Reply Last reply Reply Quote 0
                  • M
                    mrsunfire
                    last edited by Sep 1, 2019, 6:19 PM

                    I don't know if I should play around with that because everything runs fast and good for now. But I see if my WAN goes down unbound restarts. Any chance to disable this function and keep it running?

                    Netgate 6100 MAX

                    G 1 Reply Last reply Sep 2, 2019, 4:57 AM Reply Quote 0
                    • G
                      Gertjan @mrsunfire
                      last edited by Sep 2, 2019, 4:57 AM

                      @mrsunfire said in DNS Resolver not caching correct?:

                      But I see if my WAN goes down unbound restarts

                      It's possible that the restart of unbound wasn't needed. If the WAN IP stays the same, then maybe yes, it might not be needed.
                      I guess you should investigate why the WAN NIC goes down ? Missing an UPS ? Then add one.

                      There are many reasons that a WAN interface change has more effect, and then unbound should restart.
                      Loadbalancing. VPN usage (the tunnel is rebuild, and unbound should use the tunnel), etc.

                      However : there is no GUI setting that let you choose what to restart, or not.
                      Just stop ripping out the cable, or have people play with the power plug, and you'll be fine.

                      No "help me" PM's please. Use the forum, the community will thank you.
                      Edit : and where are the logs ??

                      1 Reply Last reply Reply Quote 0
                      • M
                        mrsunfire
                        last edited by Sep 2, 2019, 6:15 AM

                        The IP stays the same. I use cable and the inhouse coax is getting renewed. The old one sometimes has ingress and the connection goes down shortly. I have to say that pfSense then switches to my failover WAN2.

                        Any chance to disable unbound restart?

                        Netgate 6100 MAX

                        G 1 Reply Last reply Sep 2, 2019, 6:21 AM Reply Quote 0
                        • G
                          Gertjan @mrsunfire
                          last edited by Sep 2, 2019, 6:21 AM

                          Do you mean your upstream WAN Ethernet goes really down ? The upstream device resets ?
                          It's a cable modem - your mentioned 'coax' ?

                          @mrsunfire said in DNS Resolver not caching correct?:

                          Any chance to disable unbound restart?

                          Not without you actually changing the code.

                          Btw : the monitoring process will also reset the interface if the upstream gateway becomes unreachable. You could change these settings.

                          No "help me" PM's please. Use the forum, the community will thank you.
                          Edit : and where are the logs ??

                          1 Reply Last reply Reply Quote 0
                          • M
                            mrsunfire
                            last edited by Sep 2, 2019, 6:26 AM

                            Yes the cable modem loses connection to the upstream gateway of my ISP. Then a gateway alarm appears and it switches to my WAN2.

                            Netgate 6100 MAX

                            G 1 Reply Last reply Sep 2, 2019, 6:27 AM Reply Quote 0
                            • G
                              Gertjan @mrsunfire
                              last edited by Sep 2, 2019, 6:27 AM

                              @mrsunfire said in DNS Resolver not caching correct?:

                              Then a gateway alarm appears and it switches to my WAN2.

                              Ok. Pretty good reason to inform unbound about that event ^^

                              No "help me" PM's please. Use the forum, the community will thank you.
                              Edit : and where are the logs ??

                              1 Reply Last reply Reply Quote 0
                              • M
                                mrsunfire
                                last edited by Sep 2, 2019, 6:37 AM

                                Sure but why is it clearing the cache? An option to prevent that would be nice.

                                Netgate 6100 MAX

                                G 1 Reply Last reply Sep 2, 2019, 7:38 AM Reply Quote 0
                                • G
                                  Gertjan @mrsunfire
                                  last edited by Gertjan Sep 2, 2019, 7:43 AM Sep 2, 2019, 7:38 AM

                                  @mrsunfire said in DNS Resolver not caching correct?:

                                  Sure but why is it clearing the cache? An option to prevent that would be nice.

                                  Ah, now we are getting to the bottom of the subject.
                                  It's unbound that needs to restart when the state of one of it's interfaces change.
                                  I guess (my words) that unbound can't dynamic bind and unbind to IPs and/or ports.
                                  This means : shutting down, Start again, thus reading the config (which is probably is rebuild because some interface came up = a viable or is missing now).
                                  Side effect : cache is flushed / reset.

                                  As far as I know, the cache isn't written when unbound stops to some cache file and read back in when starting.
                                  Don't know if that is even possible - and I'm pretty sure this isn't done by pfSense.

                                  Other side effect : people that use packages like pfBlockerNG with huge DNSBL lists will see something else : it will take long time (several tens of seconds) for unbound to start up because it has to read throughout all these lists and loading them.
                                  If unbound restarts often - because, for example, it restarts when a DHCP lease comes in, they will experience DNS outages.

                                  No "help me" PM's please. Use the forum, the community will thank you.
                                  Edit : and where are the logs ??

                                  1 Reply Last reply Reply Quote 0
                                  • J
                                    johnpoz LAYER 8 Global Moderator
                                    last edited by johnpoz Sep 2, 2019, 10:09 AM Sep 2, 2019, 10:02 AM

                                    You could load the cache back

                                    You have to dump it first
                                    unbound-control dump_cache > unbound.dump

                                    Then reload it
                                    cat unbound.dump | unbound-control load_cache

                                    Here is where you could run into a problem with that - petty sure the ttls would be frozen at that moment in time. I have not done much playing with doing something like that - just don't really see the point.. So your stuff resolves again... So first query would be a couple extra ms.. But if your ttl's are frozen at the moment of the dump, then counters restart on reload then you could be serving up expired ttls - and depending on the how long it was down, etc. They could be expired by very long time.. And still have lots of time left on them, so might be getting bad info for whatever the time left on the ttl was?

                                    But sure - guess it could be coded to dump the cache every X minutes or something, and then on restart reload it. Would need easy way for users to flush other than restart.

                                    An intelligent man is sometimes forced to be drunk to spend time with his fools
                                    If you get confused: Listen to the Music Play
                                    Please don't Chat/PM me for help, unless mod related
                                    SG-4860 24.11 | Lab VMs 2.8, 24.11

                                    G 1 Reply Last reply Sep 2, 2019, 10:31 AM Reply Quote 0
                                    • G
                                      Gertjan @johnpoz
                                      last edited by Sep 2, 2019, 10:31 AM

                                      @johnpoz said in DNS Resolver not caching correct?:

                                      You could load the cache back
                                      You have to dump it first
                                      unbound-control dump_cache > unbound.dump
                                      Then reload it
                                      cat unbound.dump | unbound-control load_cache

                                      Me ?
                                      Ok.

                                      First, a snapshot from what the cache is doing now :

                                      unbound-control -c /var/unbound/unbound.conf stats_noreset | grep 'cache'
                                      

                                      Note : adding " -c /var/unbound/unbound.conf '" isn't optional.

                                      Now I dump to cache :

                                      unbound-control -c /var/unbound/unbound.conf dump_cache > unbound.dump
                                      

                                      For fun, look what's in this file : mine was nearly 3 Mega Bytes.

                                      Now, I restart unbound in the GUI. A command line command exists, I'm; lazy and googled already to much this morning.

                                      Another check :

                                      unbound-control -c /var/unbound/unbound.conf stats_noreset | grep 'cache'
                                      

                                      to see that unbound is running with an empty cache now.

                                      Show time :

                                      cat unbound.dump | unbound-control -c /var/unbound/unbound.conf load_cache
                                      

                                      and re check again to see that "things" have been loaded :

                                      unbound-control -c /var/unbound/unbound.conf stats_noreset | grep 'cache'
                                      

                                      That seems to work - indeed 👍

                                      You're right - I can ^^

                                      But pfSense can't / doesn't know how to do so.
                                      (I can of course edit the related source files where unbound is restarted - it's just plain PHP .... but hey, all is fine for me already )

                                      And yes, you're right. Mighty DNS gods would come down to earth when people start to load in old cache info and issues like that ....
                                      There are already to much "issues" with DNS/unbound - or, take note : it works perfectly well out of the box.

                                      No "help me" PM's please. Use the forum, the community will thank you.
                                      Edit : and where are the logs ??

                                      1 Reply Last reply Reply Quote 0
                                      • J
                                        johnpoz LAYER 8 Global Moderator
                                        last edited by johnpoz Sep 2, 2019, 11:15 AM Sep 2, 2019, 11:08 AM

                                        No didn't mean for you to do it ;) Just stating how it "could" be done hehehehe

                                        Guess it could be somewhat useful for someones who's unbound is restarting all the time.. I would look to why that is happening and stop it.. For example dhcp registration, or maybe pfblocker restarting every hour on a cron or something.

                                        Or maybe your connecting is flipping over to backup, or going down whatever.. The dhcp is easy, just don't have it register dhcp leases. Your connecting going down or flipping might be harder to fix - but that shouldn't be happening on a regular basis that is for sure.

                                        As to how often pfblocker restarts unbound - have to get with BBcan on that, off the top not sure when it might restart unbound.

                                        An intelligent man is sometimes forced to be drunk to spend time with his fools
                                        If you get confused: Listen to the Music Play
                                        Please don't Chat/PM me for help, unless mod related
                                        SG-4860 24.11 | Lab VMs 2.8, 24.11

                                        1 Reply Last reply Reply Quote 0
                                        56 out of 56
                                        • First post
                                          56/56
                                          Last post
                                        Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.
                                          This community forum collects and processes your personal information.
                                          consent.not_received