Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Unable to start Unbound after switch away from KEA

    Scheduled Pinned Locked Moved General pfSense Questions
    17 Posts 3 Posters 1.1k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • M
      michmoor LAYER 8 Rebel Alliance
      last edited by

      After an ugprade to 24.11 I decided to give KEA a try. I since then switched back to ISC and thought nothing of it.
      Checking my monitoring, I noticed that Unbound is not started - DNS failures. I tried to start Unbound and received the following error

      99191b6b-63d3-438f-bae8-b29be89d11ba-image.png

      This is a test SG-1100 unit.

      Firewall: NetGate,Palo Alto-VM,Juniper SRX
      Routing: Juniper, Arista, Cisco
      Switching: Juniper, Arista, Cisco
      Wireless: Unifi, Aruba IAP
      JNCIP,CCNP Enterprise

      1 Reply Last reply Reply Quote 0
      • stephenw10S
        stephenw10 Netgate Administrator
        last edited by

        Hmm, what do you see on line 17 of /var/unbound/unbound.conf ?

        I can't think of anything that would apply there but....

        M 1 Reply Last reply Reply Quote 0
        • M
          michmoor LAYER 8 Rebel Alliance @stephenw10
          last edited by

          @stephenw10 said in Unable to start Unbound after switch away from KEA:

          l

           cat /var/unbound/unbound.conf | nl
               1  ##########################
               2  # Unbound Configuration
               3  ##########################
          
               4  ##
               5  # Server configuration
               6  ##
               7  server:
          
               8  chroot: /var/unbound
               9  username: "unbound"
              10  directory: "/var/unbound"
              11  pidfile: "/var/run/unbound.pid"
              12  use-syslog: yes
              13  port: 53
              14  verbosity:
              15  hide-identity: yes
              16  hide-version: yes
              17  harden-glue: yes
              18  do-ip4: yes
              19  do-ip6: yes
              20  do-udp: yes
              21  do-tcp: yes
              22  do-daemonize: yes
              23  module-config: "validator iterator"
              24  unwanted-reply-threshold: 0
              25  num-queries-per-thread: 4096
              26  jostle-timeout: 200
              27  infra-keep-probing: yes
              28  infra-host-ttl: 900
              29  infra-cache-numhosts: 10000
              30  outgoing-num-tcp:
              31  incoming-num-tcp:
              32  edns-buffer-size: 1332
              33  cache-max-ttl: 86400
              34  cache-min-ttl: 0
              35  harden-dnssec-stripped: yes
              36  msg-cache-size: 4m
              37  rrset-cache-size: 8m
          
              38  num-threads: 2
              39  msg-cache-slabs: 2
              40  rrset-cache-slabs: 2
              41  infra-cache-slabs: 2
              42  key-cache-slabs: 2
              43  outgoing-range: 4096
              44  so-rcvbuf: 12m
              45  auto-trust-anchor-file: /var/unbound/root.key
              46  prefetch: no
              47  prefetch-key: no
              48  use-caps-for-id: no
              49  serve-expired: no
              50  sock-queue-timeout: 0
              51  aggressive-nsec: no
              52  # Statistics
              53  # Unbound Statistics
              54  statistics-interval: 0
              55  extended-statistics: yes
              56  statistics-cumulative: yes
          
              57  # TLS Configuration
              58  tls-cert-bundle: "/etc/ssl/cert.pem"
          
              59  # Interface IP addresses to bind to
              60  interface-automatic: yes
          
          
              61  # DNS Rebinding
          
          
          
          
              62  # Access lists
              63  include: /var/unbound/access_lists.conf
          
              64  # Static host entries
              65  include: /var/unbound/host_entries.conf
          
              66  # dhcp lease entries
              67  include: /var/unbound/dhcpleases_entries.conf
          
          
          
              68  # Domain overrides
              69  include: /var/unbound/domainoverrides.conf
              70  # Forwarding
              71  forward-zone:
              72          name: "."
              73          forward-addr: 9.9.9.9
              74          forward-addr: 149.112.112.112
          
          
          
          
              75  ###
              76  # Remote Control Config
              77  ###
              78  include: /var/unbound/remotecontrol.conf
          
          
          

          Firewall: NetGate,Palo Alto-VM,Juniper SRX
          Routing: Juniper, Arista, Cisco
          Switching: Juniper, Arista, Cisco
          Wireless: Unifi, Aruba IAP
          JNCIP,CCNP Enterprise

          M 1 Reply Last reply Reply Quote 0
          • M
            michmoor LAYER 8 Rebel Alliance @michmoor
            last edited by

            Took a hail mary and ran the command

            [24.11-RELEASE][admin@nyc-fw1-inet.moore.lan]/root: /usr/local/sbin/unbound
            [24.11-RELEASE][admin@nyc-fw1-inet.moore.lan]/root:

            Unbound started without issues.........

            No clue what is going on.

            Firewall: NetGate,Palo Alto-VM,Juniper SRX
            Routing: Juniper, Arista, Cisco
            Switching: Juniper, Arista, Cisco
            Wireless: Unifi, Aruba IAP
            JNCIP,CCNP Enterprise

            GertjanG 1 Reply Last reply Reply Quote 0
            • GertjanG
              Gertjan @michmoor
              last edited by

              @michmoor said in Unable to start Unbound after switch away from KEA:

              [24.11-RELEASE][admin@nyc-fw1-inet.moore.lan]/root: /usr/local/sbin/unbound

              That shouldn't work.
              To name one : unbound, by default, uses classic freebsd file paths like /usr/local/etc/unbound/unbound.conf to start up.

              [24.11-RELEASE][root@pfSense.bhf.tld]/root: unbound -h
              usage:  unbound [options]
                      start unbound daemon DNS resolver.
              -h      this help.
              -c file config file to read instead of /usr/local/etc/unbound/unbound.conf
                      file format is described in unbound.conf(5).
              -d      do not fork into the background.
              -p      do not create a pidfile.
              -v      verbose (more times to increase verbosity).
              -V      show version number and build options.
              
              Version 1.22.0
              BSD licensed, see LICENSE in source package for details.
              Report bugs to unbound-bugs@nlnetlabs.nl or https://github.com/NLnetLabs/unbound/issues
              

              There is a config file at /usr/local/etc/unbound/ but that one is a generic one, enumerating all the default config values, and it won't work in the pfSEnse environment.
              The actual unbound config file isn't there, it's here : /var/unbound/unbound.conf

              A manual unbound startup would look like

              unbound -c /var/unbound/unbound.conf
              

              That said, I"'m using 24.11 since yesterday, maybe things changed ?

              No "help me" PM's please. Use the forum, the community will thank you.
              Edit : and where are the logs ??

              1 Reply Last reply Reply Quote 0
              • stephenw10S
                stephenw10 Netgate Administrator
                last edited by

                Nope it's using that conf file from the custom location and failing for some reason.

                The first thing I would do is just resave the DNS Resolver settings to regenerate that conf file. Though nothing in that current file looks obviously like a problem to me.

                M 1 Reply Last reply Reply Quote 0
                • M
                  michmoor LAYER 8 Rebel Alliance @stephenw10
                  last edited by

                  @stephenw10

                  Starting unbound the way i did did allow the service to show green in the pfsense GUI but no DNS resolution was occurring

                   ps auxx | grep unbound
                  root    73481   0.0  0.2   12860   1936  -  Is   19:55      0:00.02 /usr/local/sbin/dhcpleases -l /var/dhcpd/var/db/dhcpd.leases -d moore.lan -p /var/run/unbound.pid -u /var/unbound/d
                  unbound 87474   0.0  0.7   40588   6956  -  Is   Thu23      0:18.40 /usr/local/sbin/unbound
                  root       27   0.0  0.2   12952   2064  0  S+   19:58      0:00.01 grep unbound
                  [24.11-RELEASE][admin@nyc-fw1-inet.moore.lan]/root: kill -9 87474
                  [24.11-RELEASE][admin@nyc-fw1-inet.moore.lan]/root: ps auxx | grep unbound
                  root   73481   0.0  0.2   12860   1936  -  Is   19:55      0:00.02 /usr/local/sbin/dhcpleases -l /var/dhcpd/var/db/dhcpd.leases -d moore.lan -p /var/run/unbound.pid -u /var/unbound/dh
                  root    2975   0.0  0.2   12952   2064  0  S+   19:59      0:00.01 grep unbound
                  [24.11-RELEASE][admin@nyc-fw1-inet.moore.lan]/root:
                  
                  

                  I have re-saved the unbound configuration again, apply changes , and still the service remains down with the following error seen in the log

                  dc726a88-bdbe-413d-9ebf-fc9b95f4ae5f-image.png

                  Firewall: NetGate,Palo Alto-VM,Juniper SRX
                  Routing: Juniper, Arista, Cisco
                  Switching: Juniper, Arista, Cisco
                  Wireless: Unifi, Aruba IAP
                  JNCIP,CCNP Enterprise

                  M 1 Reply Last reply Reply Quote 0
                  • M
                    michmoor LAYER 8 Rebel Alliance @michmoor
                    last edited by

                    Manually starting the service

                     unbound -c /var/unbound/unbound.conf
                    /var/unbound/unbound.conf:17: error: number expected
                    /var/unbound/unbound.conf:17: error: unknown keyword 'yes'
                    /var/unbound/unbound.conf:33: error: number expected
                    read /var/unbound/unbound.conf failed: 3 errors in configuration file
                    [1733101412] unbound[3798:0] fatal error: Could not read config file: /var/unbound/unbound.conf. Maybe try unbound -dd, it stays on the commandline to see more errors, or unbound-checkconf
                    
                    

                    Here is the unbound.conf again.

                     cat /var/unbound/unbound.conf
                    ##########################
                    # Unbound Configuration
                    ##########################
                    
                    ##
                    # Server configuration
                    ##
                    server:
                    
                    chroot: /var/unbound
                    username: "unbound"
                    directory: "/var/unbound"
                    pidfile: "/var/run/unbound.pid"
                    use-syslog: yes
                    port: 53
                    verbosity:
                    hide-identity: yes
                    hide-version: yes
                    harden-glue: yes
                    do-ip4: yes
                    do-ip6: yes
                    do-udp: yes
                    do-tcp: yes
                    do-daemonize: yes
                    module-config: "validator iterator"
                    unwanted-reply-threshold: 0
                    num-queries-per-thread: 4096
                    jostle-timeout: 200
                    infra-keep-probing: yes
                    infra-host-ttl: 900
                    infra-cache-numhosts: 10000
                    outgoing-num-tcp:
                    incoming-num-tcp:
                    edns-buffer-size: 1332
                    cache-max-ttl: 86400
                    cache-min-ttl: 0
                    harden-dnssec-stripped: yes
                    msg-cache-size: 4m
                    rrset-cache-size: 8m
                    
                    num-threads: 2
                    msg-cache-slabs: 2
                    rrset-cache-slabs: 2
                    infra-cache-slabs: 2
                    key-cache-slabs: 2
                    outgoing-range: 4096
                    so-rcvbuf: 12m
                    auto-trust-anchor-file: /var/unbound/root.key
                    prefetch: no
                    prefetch-key: no
                    use-caps-for-id: no
                    serve-expired: no
                    sock-queue-timeout: 0
                    aggressive-nsec: no
                    # Statistics
                    # Unbound Statistics
                    statistics-interval: 0
                    extended-statistics: yes
                    statistics-cumulative: yes
                    
                    # TLS Configuration
                    tls-cert-bundle: "/etc/ssl/cert.pem"
                    
                    # Interface IP addresses to bind to
                    interface-automatic: yes
                    
                    
                    # DNS Rebinding
                    
                    
                    
                    
                    # Access lists
                    include: /var/unbound/access_lists.conf
                    
                    # Static host entries
                    include: /var/unbound/host_entries.conf
                    
                    # dhcp lease entries
                    include: /var/unbound/dhcpleases_entries.conf
                    
                    
                    
                    # Domain overrides
                    include: /var/unbound/domainoverrides.conf
                    # Forwarding
                    forward-zone:
                            name: "."
                            forward-addr: 9.9.9.9
                            forward-addr: 149.112.112.112
                    
                    
                    
                    
                    ###
                    # Remote Control Config
                    ###
                    include: /var/unbound/remotecontrol.conf
                    
                    

                    Firewall: NetGate,Palo Alto-VM,Juniper SRX
                    Routing: Juniper, Arista, Cisco
                    Switching: Juniper, Arista, Cisco
                    Wireless: Unifi, Aruba IAP
                    JNCIP,CCNP Enterprise

                    M 1 Reply Last reply Reply Quote 0
                    • M
                      michmoor LAYER 8 Rebel Alliance @michmoor
                      last edited by

                      Counting the lines in the conf file, Line 17 is the following

                      hide-version: yes
                      
                      

                      I went into the GUI, DNS Resolver , Advanced Settings, and unchecked Hide Version.
                      Apply changes.
                      DNS Resolver starts without issues and name resolution finally works.

                      @stephenw10 what is that setting and why did having it checked break Unbound? Seems like a redmine issue but i don't know what the fault is.
                      Checking my 6100 HighAvailability pair, they have it enabled and no Unbound issues post upgrade. Even did a failover to test and it works.

                      Any guidance you can offer i would greatly appreciate it!

                      Firewall: NetGate,Palo Alto-VM,Juniper SRX
                      Routing: Juniper, Arista, Cisco
                      Switching: Juniper, Arista, Cisco
                      Wireless: Unifi, Aruba IAP
                      JNCIP,CCNP Enterprise

                      GertjanG 1 Reply Last reply Reply Quote 0
                      • GertjanG
                        Gertjan @michmoor
                        last edited by

                        @michmoor

                        This one :

                        a82c85f9-4bcf-42d1-a940-79213a4aeccf-image.png

                        runs because you've set :

                        22f836bd-0629-45cc-905d-a69f7320c418-image.png

                        If it is possible : disable that option.

                        dhcpdleases restarts unbound every time a new DHCP lease event is received.
                        This can give you the impression that 'unbound' wasn't running as it was 'restarting again (and again, and again etc etc).
                        Unbound restarts do take some time, several seconds, or more.

                        You see the command line option of that dhcpdleases process ?

                        -p /var/run/unbound.pid
                        

                        which means that when you restart manually unbound, the PID changes.
                        dhcpdleases also has to be restarted - not sure if you can do this manually.

                        No "help me" PM's please. Use the forum, the community will thank you.
                        Edit : and where are the logs ??

                        M 1 Reply Last reply Reply Quote 0
                        • M
                          michmoor LAYER 8 Rebel Alliance @Gertjan
                          last edited by

                          @Gertjan Thanks Gertjan but you are misunderstanding the problem.

                          DNS resolver wasn't running due to a parameter set it did not recognize or like for some reason. After negating that in the advanced options in Unbound, it started up successfully and resolved hostnames.

                          The issue here is why was it broken in the first place. The only connection i have to KEA is that it was the last change i made on the system before i reverted back.

                          Firewall: NetGate,Palo Alto-VM,Juniper SRX
                          Routing: Juniper, Arista, Cisco
                          Switching: Juniper, Arista, Cisco
                          Wireless: Unifi, Aruba IAP
                          JNCIP,CCNP Enterprise

                          1 Reply Last reply Reply Quote 0
                          • stephenw10S
                            stephenw10 Netgate Administrator
                            last edited by

                            Line 17 is actually hide-identity: yes. But that's not the issue that's a default setting. The problem is that you had no log verbosity value on the line before. It's expecting a number for the log verbosity and sees not-a-number at the start of line 17.

                            Resaving the Unbound Advanced page will have added that value back.

                            The same applies to the values for:

                            outgoing-num-tcp: 
                            incoming-num-tcp:
                            

                            However I have no idea what might have caused those to not be populated. Unbound should never have allowed those empty values.

                            M 1 Reply Last reply Reply Quote 0
                            • M
                              michmoor LAYER 8 Rebel Alliance @stephenw10
                              last edited by

                              @stephenw10 I gotcha.
                              As mentioned, the last change I made was the KEA switchover. I don't know if KEA is involved with Unbound in this case, but that was the last change I made. I don't understand what could suddenly cause Unbound to find a problem with the conf file. Outside of configuring Unbound via the GUI as far as i know there is no direct way to modify the configuration and have it saved. I could be wrong of course.

                              Firewall: NetGate,Palo Alto-VM,Juniper SRX
                              Routing: Juniper, Arista, Cisco
                              Switching: Juniper, Arista, Cisco
                              Wireless: Unifi, Aruba IAP
                              JNCIP,CCNP Enterprise

                              1 Reply Last reply Reply Quote 0
                              • stephenw10S
                                stephenw10 Netgate Administrator
                                last edited by

                                Indeed. I can see why it rejected that conf file because there are values missing. What I can't understand is how that was ever generated and passed the Unbound tests. 🤔

                                If you revert that change to the Unbound settings I imagine it will still start no problem.

                                M 1 Reply Last reply Reply Quote 0
                                • M
                                  michmoor LAYER 8 Rebel Alliance @stephenw10
                                  last edited by

                                  @stephenw10 I reverted my change in Unbound and here is the conf file now.

                                  ##########################
                                  # Unbound Configuration
                                  ##########################
                                  
                                  ##
                                  # Server configuration
                                  ##
                                  server:
                                  
                                  chroot: /var/unbound
                                  username: "unbound"
                                  directory: "/var/unbound"
                                  pidfile: "/var/run/unbound.pid"
                                  use-syslog: yes
                                  port: 53
                                  verbosity: 1
                                  hide-identity: yes
                                  hide-version: yes
                                  harden-glue: yes
                                  do-ip4: yes
                                  do-ip6: yes
                                  do-udp: yes
                                  do-tcp: yes
                                  do-daemonize: yes
                                  module-config: "validator iterator"
                                  unwanted-reply-threshold: 0
                                  num-queries-per-thread: 512
                                  jostle-timeout: 200
                                  infra-keep-probing: yes
                                  infra-host-ttl: 900
                                  infra-cache-numhosts: 10000
                                  outgoing-num-tcp: 10
                                  incoming-num-tcp: 10
                                  edns-buffer-size: 1332
                                  cache-max-ttl: 86400
                                  cache-min-ttl: 0
                                  harden-dnssec-stripped: yes
                                  msg-cache-size: 4m
                                  rrset-cache-size: 8m
                                  
                                  num-threads: 2
                                  msg-cache-slabs: 2
                                  rrset-cache-slabs: 2
                                  infra-cache-slabs: 2
                                  key-cache-slabs: 2
                                  outgoing-range: 4096
                                  so-rcvbuf: 12m
                                  auto-trust-anchor-file: /var/unbound/root.key
                                  prefetch: no
                                  prefetch-key: no
                                  use-caps-for-id: no
                                  serve-expired: no
                                  sock-queue-timeout: 0
                                  aggressive-nsec: no
                                  # Statistics
                                  # Unbound Statistics
                                  statistics-interval: 0
                                  extended-statistics: yes
                                  statistics-cumulative: yes
                                  
                                  # TLS Configuration
                                  tls-cert-bundle: "/etc/ssl/cert.pem"
                                  
                                  # Interface IP addresses to bind to
                                  interface-automatic: yes
                                  
                                  
                                  # DNS Rebinding
                                  
                                  
                                  
                                  
                                  # Access lists
                                  include: /var/unbound/access_lists.conf
                                  
                                  # Static host entries
                                  include: /var/unbound/host_entries.conf
                                  
                                  # dhcp lease entries
                                  include: /var/unbound/dhcpleases_entries.conf
                                  
                                  
                                  
                                  # Domain overrides
                                  include: /var/unbound/domainoverrides.conf
                                  # Forwarding
                                  forward-zone:
                                  	name: "."
                                  	forward-addr: 9.9.9.9
                                  	forward-addr: 149.112.112.112
                                  
                                  
                                  
                                  
                                  ###
                                  # Remote Control Config
                                  ###
                                  include: /var/unbound/remotecontrol.conf
                                  

                                  Firewall: NetGate,Palo Alto-VM,Juniper SRX
                                  Routing: Juniper, Arista, Cisco
                                  Switching: Juniper, Arista, Cisco
                                  Wireless: Unifi, Aruba IAP
                                  JNCIP,CCNP Enterprise

                                  1 Reply Last reply Reply Quote 0
                                  • stephenw10S
                                    stephenw10 Netgate Administrator
                                    last edited by

                                    Yup so now those 3 values are all populated. And I imagine it starts fine?

                                    M 1 Reply Last reply Reply Quote 0
                                    • M
                                      michmoor LAYER 8 Rebel Alliance @stephenw10
                                      last edited by

                                      @stephenw10

                                      Yep starts up fine, no issues.

                                      Unless i can reproduce it or others have a similar issue might have to throw this in the mystery box.

                                      Firewall: NetGate,Palo Alto-VM,Juniper SRX
                                      Routing: Juniper, Arista, Cisco
                                      Switching: Juniper, Arista, Cisco
                                      Wireless: Unifi, Aruba IAP
                                      JNCIP,CCNP Enterprise

                                      1 Reply Last reply Reply Quote 1
                                      • First post
                                        Last post
                                      Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.