Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    23.01 Upgrade unbound Issue

    Scheduled Pinned Locked Moved General pfSense Questions
    66 Posts 11 Posters 27.2k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • johnpozJ
      johnpoz LAYER 8 Global Moderator @bingo600
      last edited by

      @bingo600 ok did a bit of looking at the neg cache time..

      So you can tell that neg cache is being used, if you looking up something you know will nx.. So for example I did a query for random.cnn.com

      If I look in the unbound cache, I can see that its counting down the ttl from 3600

      [23.01-RELEASE][admin@sg4860.local.lan]/var/unbound: unbound-control -c /var/unbound/unbound.conf dump_cache | grep lsjfdsld.cnn.com
      msg lsjfdsld.cnn.com. IN A 32899 1 3554 3 0 1 0
      [23.01-RELEASE][admin@sg4860.local.lan]/var/unbound: unbound-control -c /var/unbound/unbound.conf dump_cache | grep lsjfdsld.cnn.com
      msg lsjfdsld.cnn.com. IN A 32899 1 3546 3 0 1 0
      [23.01-RELEASE][admin@sg4860.local.lan]/var/unbound: 
      

      See where it goes down from 3554, to 3546.. You could try setting the min-neg cache setting to something lower and see if using that via similar test that I did.

      An intelligent man is sometimes forced to be drunk to spend time with his fools
      If you get confused: Listen to the Music Play
      Please don't Chat/PM me for help, unless mod related
      SG-4860 24.11 | Lab VMs 2.8, 24.11

      bingo600B 1 Reply Last reply Reply Quote 0
      • M
        moelassus @bingo600
        last edited by

        @bingo600 Which list are you using for DoH blocking? I'm currently using a list from "oneoffdallas" that appears to be maintained. Wondering if there is a better one.

        S bingo600B 2 Replies Last reply Reply Quote 0
        • S
          SteveITS Galactic Empire @moelassus
          last edited by

          @moelassus There's a TheGreatWall_DoH_IP list in pfBlocker, though the list itself says it was last updated in 2020.

          Pre-2.7.2/23.09: Only install packages for your version, or risk breaking it. Select your branch in System/Update/Update Settings.
          When upgrading, allow 10-15 minutes to restart, or more depending on packages and device speed.
          Upvote šŸ‘ helpful posts!

          M 1 Reply Last reply Reply Quote 0
          • M
            moelassus @SteveITS
            last edited by moelassus

            @steveits Yeah, TheGreatWall list doesn't appear to be maintained. For example, it doesn't include any of Apple's DoH servers. OneOffDallas's list was last updated in Dec 2022.

            Trying to block DoH is really a pointless exercise because bad actors aren't going to use a well-known DoH server anyway. DoH is a scourge. ā˜ŗļø

            johnpozJ 1 Reply Last reply Reply Quote 0
            • johnpozJ
              johnpoz LAYER 8 Global Moderator @moelassus
              last edited by

              @moelassus said in 23.01 Upgrade unbound Issue:

              going to use a well-known DoH server anyway

              While I agree with you - it would be quite possible for some bad software to use something on their own, and not a well known doh server. What it does do is stop stuff that is just trying to do you a "favor" and use doh without specifically asking you.. Those would normally point to a well known doh service..

              I block it - because I don't want my stuff using it, but sure if it just using some not well known doh server, not much I could do about that other than trying all of its https traffic, which is pretty difficult.

              An intelligent man is sometimes forced to be drunk to spend time with his fools
              If you get confused: Listen to the Music Play
              Please don't Chat/PM me for help, unless mod related
              SG-4860 24.11 | Lab VMs 2.8, 24.11

              1 Reply Last reply Reply Quote 1
              • bingo600B
                bingo600 @johnpoz
                last edited by bingo600

                @johnpoz said in 23.01 Upgrade unbound Issue:

                You could try setting the min-neg cache setting to something lower and see if using that via similar test that I did.

                Thanx .. I will try that šŸ‘

                And it works ....

                server:
                #log-queries: yes
                #log-replies: yes
                # Set max failed lookup cache time
                cache-max-negative-ttl: 10
                #
                
                [23.01-RELEASE][admin@fwall]/var/unbound: unbound-control -c /var/unbound/unbound.conf dump_cache | grep .cnn.com
                msg garbage.cnn.com. IN A 33155 1 6 3 0 1 0
                msg cnn.com. IN DS 33152 1 6 0 0 3 0
                [23.01-RELEASE][admin@fwall]/var/unbound: unbound-control -c /var/unbound/unbound.conf dump_cache | grep .cnn.com
                msg garbage.cnn.com. IN A 33155 1 4 3 0 1 0
                msg cnn.com. IN DS 33152 1 4 0 0 3 0
                [23.01-RELEASE][admin@fwall]/var/unbound: unbound-control -c /var/unbound/unbound.conf dump_cache | grep .cnn.com
                msg garbage.cnn.com. IN A 33155 1 3 3 0 1 0
                msg cnn.com. IN DS 33152 1 3 0 0 3 0
                [23.01-RELEASE][admin@fwall]/var/unbound: unbound-control -c /var/unbound/unbound.conf dump_cache | grep .cnn.com
                msg garbage.cnn.com. IN A 33155 1 2 3 0 1 0
                msg cnn.com. IN DS 33152 1 2 0 0 3 0
                [23.01-RELEASE][admin@fwall]/var/unbound: unbound-control -c /var/unbound/unbound.conf dump_cache | grep .cnn.com
                msg garbage.cnn.com. IN A 33155 1 1 3 0 1 0
                msg cnn.com. IN DS 33152 1 1 0 0 3 0
                [23.01-RELEASE][admin@fwall]/var/unbound: unbound-control -c /var/unbound/unbound.conf dump_cache | grep .cnn.com
                msg garbage.cnn.com. IN A 33155 1 0 3 0 1 0
                msg cnn.com. IN DS 33152 1 0 0 0 3 0
                [23.01-RELEASE][admin@fwall]/var/unbound: unbound-control -c /var/unbound/unbound.conf dump_cache | grep .cnn.com
                [23.01-RELEASE][admin@fwall]/var/unbound:
                

                If you find my answer useful - Please give the post a šŸ‘ - "thumbs up"

                pfSense+ 23.05.1 (ZFS)

                QOTOM-Q355G4 Quad Lan.
                CPUĀ  : Core i5 5250U, Ram : 8GB Kingston DDR3LV 1600
                LANĀ  : 4 x Intel 211, DiskĀ  : 240G SAMSUNG MZ7L3240HCHQ SSD

                1 Reply Last reply Reply Quote 0
                • bingo600B
                  bingo600 @moelassus
                  last edited by bingo600

                  @moelassus said in 23.01 Upgrade unbound Issue:

                  @bingo600 Which list are you using for DoH blocking? I'm currently using a list from "oneoffdallas" that appears to be maintained. Wondering if there is a better one.

                  I was basically following this guid
                  https://github.com/jpgpi250/piholemanual

                  URL List defined in pfS

                  ef87ed14-d70c-40c1-8eec-8f05e97ccbb0-image.png

                  And one of the lists got updates today
                  6055d9be-bfce-4272-bd0a-97f7fc2b1ecc-image.png

                  @moelassus - There's a pfSense specific guide in this doc
                  https://github.com/jpgpi250/piholemanual/tree/master/doc

                  I just skimmed the new guide ... He has made it very complicated.
                  I have attached the guide i followed, when i made it .... (a previous version)
                  Tip : USE FLOATING RULES ...
                  1-pfs-blockDOH-2021-simple.pdf.zip

                  /Bingo

                  If you find my answer useful - Please give the post a šŸ‘ - "thumbs up"

                  pfSense+ 23.05.1 (ZFS)

                  QOTOM-Q355G4 Quad Lan.
                  CPUĀ  : Core i5 5250U, Ram : 8GB Kingston DDR3LV 1600
                  LANĀ  : 4 x Intel 211, DiskĀ  : 240G SAMSUNG MZ7L3240HCHQ SSD

                  S 1 Reply Last reply Reply Quote 1
                  • I
                    inferno480
                    last edited by

                    Definitely still seeing some Unbound issues since upgrading to 23.01... sometimes it takes a couple days to happen, and you can "fix it" by restarting unbound but it appears to randomly crash/restart on its own then log messages about DNSKEYs being insecure.

                    I also had the issue of IPv6 interfaces not being auto-added to the ACL; I had to manually override the ACL and put in all of my v6 subnets / internal IPs which I thought was the only issue (error would be Query Refused) but I just had an incident where unbound crashed and gave "Server Fails" in nslookup w/the ACL fix in place. I did nothing but wait a few moments and it was working again.

                    Every server restart you see in this log was not caused by me and this self-fixed after several moments of trying various DNS lookups (suddenly they all worked). Not really sure what to do about this except revert back to 22.05.

                    Under DNS Resolver > Advanced Settings I have "Prefetch DNS Key support" and "Harden DNSSEC Data" both UNchecked (been unchecked since I had the Query Failed/ACL issues two days after I upgraded)

                    Feb 22 17:32:33 unbound 73817 [73817:5] info: failed to prime trust anchor -- DNSKEY rrset is not secure . DNSKEY IN
                    Feb 22 17:32:33 unbound 73817 [73817:5] info: generate keytag query _ta-4f66. NULL IN
                    Feb 22 17:31:25 unbound 73817 [73817:2] info: failed to prime trust anchor -- DNSKEY rrset is not secure . DNSKEY IN
                    Feb 22 17:31:25 unbound 73817 [73817:2] info: generate keytag query _ta-4f66. NULL IN
                    Feb 22 17:30:24 unbound 73817 [73817:2] info: failed to prime trust anchor -- DNSKEY rrset is not secure . DNSKEY IN
                    Feb 22 17:30:24 unbound 73817 [73817:2] info: generate keytag query _ta-4f66. NULL IN
                    Feb 22 17:29:21 unbound 73817 [73817:1] info: failed to prime trust anchor -- DNSKEY rrset is not secure . DNSKEY IN
                    Feb 22 17:29:21 unbound 73817 [73817:8] info: failed to prime trust anchor -- DNSKEY rrset is not secure . DNSKEY IN
                    Feb 22 17:29:21 unbound 73817 [73817:1] info: failed to prime trust anchor -- DNSKEY rrset is not secure . DNSKEY IN
                    Feb 22 17:29:21 unbound 73817 [73817:8] info: generate keytag query _ta-4f66. NULL IN
                    Feb 22 17:29:21 unbound 73817 [73817:1] info: generate keytag query _ta-4f66. NULL IN
                    Feb 22 17:29:21 unbound 73817 [73817:0] info: start of service (unbound 1.17.1).
                    Feb 22 17:29:21 unbound 73817 [73817:0] notice: init module 1: iterator
                    Feb 22 17:29:21 unbound 73817 [73817:0] notice: init module 0: validator
                    Feb 22 17:29:21 unbound 73817 [73817:0] notice: Restart of unbound 1.17.1.
                    (...)

                    I 1 Reply Last reply Reply Quote 0
                    • I
                      inferno480 @inferno480
                      last edited by

                      @inferno480 I had to truncate the log to fit my post in, but can attach it... I just don't think there's much of value to review in it unless I increase debugging somewhere. i.e. it shows the symptom more than the cause.

                      johnpozJ 1 Reply Last reply Reply Quote 0
                      • johnpozJ
                        johnpoz LAYER 8 Global Moderator @inferno480
                        last edited by

                        @inferno480 do you have dnssec checked, and your forwarding? If so to where?

                        Have you checked the date time on your box?

                        An intelligent man is sometimes forced to be drunk to spend time with his fools
                        If you get confused: Listen to the Music Play
                        Please don't Chat/PM me for help, unless mod related
                        SG-4860 24.11 | Lab VMs 2.8, 24.11

                        I 1 Reply Last reply Reply Quote 0
                        • I
                          inferno480 @johnpoz
                          last edited by

                          @johnpoz Time/Date seem accurate, they are NTP sync'd to 2.pfsense.pool.ntp.org - nothing noteworthy in the NTP logs

                          "DNSSEC" is checked under General Settings, I am using DNS Forwarding and my servers are Google DNS with the two V6 servers listed first, then the V4. DNS Resolution Behavior is "Use local DNS (127.0.0.1), fall back to remote DNS Servers (Default)". None of this was modified from 22.05 and I never had a problem until 23.01.

                          Any suggestions on additional logging I can enable (and how), for the next time it happens? I realize the randomness can make things difficult to troubleshoot.

                          S 1 Reply Last reply Reply Quote 0
                          • S
                            SteveITS Galactic Empire @inferno480
                            last edited by SteveITS

                            @inferno480 uncheck DNSSEC and I suspect your issues will disappear. Unbound seems more sensitive in this version, when using it and forwarding. As discussed in this and other threads and the pfSense troubleshooting doc, DNSSEC is irrelevant if you’re already trusting the other DNS servers to do the lookup for you.

                            Pre-2.7.2/23.09: Only install packages for your version, or risk breaking it. Select your branch in System/Update/Update Settings.
                            When upgrading, allow 10-15 minutes to restart, or more depending on packages and device speed.
                            Upvote šŸ‘ helpful posts!

                            1 Reply Last reply Reply Quote 0
                            • GertjanG
                              Gertjan
                              last edited by

                              @steveits

                              If 'DNSSEC' is enabled, pfSense, during preparation of the unbound start, gets a copy of the good, known DNSSEC root key.

                              /usr/bin/su -m unbound -c '/usr/local/sbin/unbound-anchor -a /var/unbound/root.key'
                              

                              If this fails, unbound will know it is using a not-good copy, and bail out.

                              So, you could check with :

                              /usr/bin/su -m unbound -4 -v -c '/usr/local/sbin/unbound-anchor -a /var/unbound/root.key'
                              

                              to know if your IPv4 works.
                              The same for IPv6 :

                              /usr/bin/su -m unbound -6 -v -c '/usr/local/sbin/unbound-anchor -a /var/unbound/root.key'
                              

                              Remember : no return message : all is well.
                              You could check content and time stamp of the /var/unbound/root.key file to see for yourself.

                              But : if you are forwarding, as said a million times time by now : disable DNSSEC.
                              Remember : forwarding means : you don't want certified DNS answers. You've decided to trust "some one else" 😊
                              That's why forwarding is not the default mode.

                              No "help me" PM's please. Use the forum, the community will thank you.
                              Edit : and where are the logs ??

                              1 Reply Last reply Reply Quote 0
                              • S
                                SteveITS Galactic Empire
                                last edited by

                                It seems there's also an IPv6 ACL bug, if set to listen on "all" interfaces, that now has a patch:
                                https://forum.netgate.com/topic/176989/problems-with-pfsense-ipv6-dns-function-does-it-exist/36

                                Pre-2.7.2/23.09: Only install packages for your version, or risk breaking it. Select your branch in System/Update/Update Settings.
                                When upgrading, allow 10-15 minutes to restart, or more depending on packages and device speed.
                                Upvote šŸ‘ helpful posts!

                                1 Reply Last reply Reply Quote 0
                                • S
                                  SteveITS Galactic Empire @bingo600
                                  last edited by

                                  @bingo600 said in 23.01 Upgrade unbound Issue:

                                  following this guid[e]
                                  https://github.com/jpgpi250/piholemanual

                                  I see that includes OneOffDallas. (@moelassus)

                                  It has an important typo though, the three local-zone lines have a leading space:

                                  local-zone: " use-application-dns.net."
                                  

                                  That doesn't work in my testing; needs to be:

                                  local-zone: "use-application-dns.net."
                                  

                                  Pre-2.7.2/23.09: Only install packages for your version, or risk breaking it. Select your branch in System/Update/Update Settings.
                                  When upgrading, allow 10-15 minutes to restart, or more depending on packages and device speed.
                                  Upvote šŸ‘ helpful posts!

                                  bingo600B 1 Reply Last reply Reply Quote 0
                                  • bingo600B
                                    bingo600 @SteveITS
                                    last edited by

                                    @steveits
                                    Nice catch šŸ‘

                                    Did you use the new giude , or the old one i posted as a"PDF/zip" here (bottom):
                                    https://forum.netgate.com/post/1089443

                                    I just briefly skimmed the new guide, and it seemed very "complicated" ..
                                    I have just implemented the "Old guide".

                                    /Bingo

                                    If you find my answer useful - Please give the post a šŸ‘ - "thumbs up"

                                    pfSense+ 23.05.1 (ZFS)

                                    QOTOM-Q355G4 Quad Lan.
                                    CPUĀ  : Core i5 5250U, Ram : 8GB Kingston DDR3LV 1600
                                    LANĀ  : 4 x Intel 211, DiskĀ  : 240G SAMSUNG MZ7L3240HCHQ SSD

                                    S 1 Reply Last reply Reply Quote 0
                                    • S
                                      SteveITS Galactic Empire @bingo600
                                      last edited by

                                      @bingo600 I just looked through what was on GitHub and set it up myself. I’d already done something similar using pfBlocker’s Greatwall list.

                                      Pre-2.7.2/23.09: Only install packages for your version, or risk breaking it. Select your branch in System/Update/Update Settings.
                                      When upgrading, allow 10-15 minutes to restart, or more depending on packages and device speed.
                                      Upvote šŸ‘ helpful posts!

                                      1 Reply Last reply Reply Quote 1
                                      • C
                                        Cylosoft @stephenw10
                                        last edited by

                                        @stephenw10 said in 23.01 Upgrade unbound Issue:

                                        Mmm, it's odd because if it's enabled and fails then whatever you're forwarding to doesn't support it. So I would expect it to be an all or nothing situatiuon.
                                        I wonder if it was previously disabled automatically in the old Unbound version. šŸ¤”

                                        I'm starting to think it's TLS forwarding. We have changed over about a dozen firewalls now and all are having DNS issues with 23. Disabling "Use SSL/TLS for outgoing DNS Queries to Forwarding Servers" seems to be the only way to keep DNS Resolver working. We just switched all of them this morning to see if that holds up.

                                        S 1 Reply Last reply Reply Quote 1
                                        • S
                                          SteveITS Galactic Empire @Cylosoft
                                          last edited by

                                          @cylosoft Interesting, let us know. I haven't noticed any DNS issues at home in a week after disabling DNSSEC while forwarding. Haven't upgraded others yet.

                                          Either way 23.01 does have different/problematic behavior than prior versions for people, since there are a lot of posts about DNS.

                                          To be fair I recall plenty of posts about DNS issues in 22.05, but I did not experience that in the routers we upgraded to 22.05.

                                          @stephenw10 said in 23.01 Upgrade unbound Issue:

                                          wonder if it was previously disabled automatically in the old Unbound version

                                          Maybe internally to Unbound? I suggested pfSense do that in a redmine and it was declined, in the context of not wanting to disable people's security choices unexpectedly.

                                          Pre-2.7.2/23.09: Only install packages for your version, or risk breaking it. Select your branch in System/Update/Update Settings.
                                          When upgrading, allow 10-15 minutes to restart, or more depending on packages and device speed.
                                          Upvote šŸ‘ helpful posts!

                                          johnpozJ 1 Reply Last reply Reply Quote 0
                                          • johnpozJ
                                            johnpoz LAYER 8 Global Moderator @SteveITS
                                            last edited by

                                            @steveits said in 23.01 Upgrade unbound Issue:

                                            in the context of not wanting to disable people's security choices unexpectedly.

                                            To be honest this is fair point of view to be honest.. Many Many years ago there was a pretty lively debate about this.. My point was there is zero point to asking for dnssec on your end if your forwarding. Where you forward is either doing it or their not - you asking for it is only going to be problematic at best.

                                            Maybe no issue, maybe issues like seems to be seeing more of currently. Might have to do with what domains your doing queries for and such.. Either way even if no problems - your doing extra queries for no good reason. Its not like you can ask for dnssec to where you forward and expect dnssec to actually function correctly. At least not with a service that doesn't explicitly support their clients to do such a thing, which I am not aware of any service running in such mode - I don't even think it would be possible to be honest.

                                            While its nice of pfsense to try and prevent you from shooting yourself in the foot when possible, to be fair the admin of the box/service/firewall/whatever needs or should take responsibility for proper configuration.. It is almost impossible to completely, lets use the term "idiot proof" something as complex as a firewall.

                                            I would like to see at least a clearer warning about - hey your prob going to have problems if you do this and forward..

                                            Something like the warning about strict qname use.

                                            warning.jpg

                                            The only caveat to keep in mind that pfsense is used by a lot of people that are not network engineers, that might have very limited if any understanding of how dns works at all, let alone with dnssec.. And many users don't even pay attention to the notes on check box items either ;)

                                            I have worked with many a fellow engineer over the years, that might be able to give you all the inner workings of bgp, or spanning tree, or igmp etc. etc.. But is pretty clueless to the inner workings of something as day to day as dns ;) So it might be a bit much to expect some home user trying to secure their network to understand all the ins and outs of how dns works, etc. See it all the time where users don't actually understand the difference between resolving and forwarding.

                                            It can be a difficult situation that is clear..

                                            An intelligent man is sometimes forced to be drunk to spend time with his fools
                                            If you get confused: Listen to the Music Play
                                            Please don't Chat/PM me for help, unless mod related
                                            SG-4860 24.11 | Lab VMs 2.8, 24.11

                                            1 Reply Last reply Reply Quote 3
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.