Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Unbound crashes periodically with signal 11

    Scheduled Pinned Locked Moved 2.5 Development Snapshots (Retired)
    105 Posts 36 Posters 32.3k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • GertjanG
      Gertjan @Salander27 0
      last edited by

      @salander27-0 said in pfSense 2.50 snapshots have been dying for the past couple of days:

      Did you ever figure this out?

      Noop, sorry.
      Had other occupations.

      No "help me" PM's please. Use the forum, the community will thank you.
      Edit : and where are the logs ??

      1 Reply Last reply Reply Quote 0
      • F
        Fry-kun @Salander27 0
        last edited by Fry-kun

        @salander27-0 I just upgraded and I'm having the same problem
        I have both "Register DHCP leases in the DNS Resolver" and "Register DHCP static mappings in the DNS Resolver" enabled

        Edit: Enabled watchdog, too. This is ridiculous, hope it gets fixed for real very soon!

        S 1 Reply Last reply Reply Quote 0
        • S
          Salander27 0 @Fry-kun
          last edited by

          @fry-kun Ah, I have both of those settings enabled too. I was thinking "Enable registration of DHCP client names in DNS" was what the above posters were referring to but I was mistaken.

          The issue certainly seems to be DHCP-related. I had 4 crashes in a 10 minute span and then upped the DHCP lease time from 15 minutes to 6 hours and haven't seen any crashes yet (though I would expect a few in a few hours once the initial 6hr leases expire and get renewed). You may wish to increase your lease time as well just to help reduce the crash frequency.

          GertjanG 1 Reply Last reply Reply Quote 0
          • GertjanG
            Gertjan @Salander27 0
            last edited by

            @salander27-0 said in pfSense 2.50 snapshots have been dying for the past couple of days:

            though I would expect a few in a few hours once the initial 6hr leases expire and get renewed

            A lease gets renewed after half the duration of the lease.
            A 15 minutes lease will get renewed after 7 minutes.
            Known OS's like Windows, MAC etc are set up ask for leases lasting a day or two.

            Why 15 minutes ??
            Although, pfSense - the DHCP server - should handle that just fine.

            No "help me" PM's please. Use the forum, the community will thank you.
            Edit : and where are the logs ??

            1 Reply Last reply Reply Quote 0
            • jimpJ
              jimp Rebel Alliance Developer Netgate
              last edited by

              We need a lot more detail about configurations. Nobody here can reproduce this in the lab or on our edge systems.

              At a minimum we need:

              • List of installed and in-use packages (e.g. pfBlockerNG, DNSBL)
              • Contents of /var/unbound/unbound.conf
              • Whether or not DHCP lease registration is enabled or other similar features like "Register connected OpenVPN clients in the DNS Resolver"
              • If DHCP lease registration is enabled, we also need to know the lease time

              Remember: Upvote with the ๐Ÿ‘ button for any user/post you find to be helpful, informative, or deserving of recognition!

              Need help fast? Netgate Global Support!

              Do not Chat/PM for help!

              S F 2 Replies Last reply Reply Quote 0
              • E
                e1219
                last edited by

                @jimp

                I updated to the latest stable release 2.5.0 from 2.4.5 last night and have started experiencing this issue as well. I have made no changes to my config since updating.

                I see this error in my log which brought me to this thread.
                Feb 18 09:25:46 kernel pid 62259 (unbound), jid 0, uid 59: exited on signal 11

                Afterwards I see the following error as unbound pid 62259 died.
                Feb 18 09:27:06 dhcpleases 52367 Could not deliver signal HUP to process 62259: No such process.
                Feb 18 09:28:04 dhcpleases 52367 Could not deliver signal HUP to process 62259: No such process.
                Feb 18 09:30:40 dhcpleases 81970 Could not deliver signal HUP to process 62259: No such process.

                Here are some of the config details you mentioned, please let me know if there are any other details that might help.

                Installed packages:
                Avahi 2.1_1
                pfBlockerNG-devel 3.0.0_10
                Service_Watchdog 1.8.7_1

                Contents of /var/unbound/unbound.conf
                unbound.conf

                Enabled:
                Register DHCP leases in the DNS Resolver
                Register DHCP static mappings in the DNS Resolver
                Disabled:
                Register connected OpenVPN clients in the DNS Resolver

                My DHCP server is using the default default-lease-time (7200s) and default maximum lease time (86400s). Looking at my current lease table, my devices are respecting the 2hr lease duration, but register at different times.

                b94b16a4-9200-4eb9-8240-18de0bf94192-image.png

                1 Reply Last reply Reply Quote 0
                • S
                  Salander27 0 @jimp
                  last edited by

                  @jimp

                  Installed Packages:
                  acme 0.6.9_3
                  arping 1.2.2_2
                  iperf 3.0.2_5
                  nmap 1.4.4_2
                  openvpn-client-export 1.5_5
                  Service_Watchdog 1.8.7_1
                  softflowd 1.2.6_1
                  sudo 0.3_6

                  /var/unbound/unbound.conf
                  [0_1613672909675_unbound.conf](Uploading 100%)

                  Enabled:
                  Register DHCP leases in the DNS Resolver
                  Register DHCP static mappings in the DNS Resolver
                  Disabled:
                  Register connected OpenVPN clients in the DNS Resolver

                  Lease time is currently 6hrs (which is helping as I see there was only one crash in the last 12 hours) up from 15 minutes (which was crashing constantly).

                  1 Reply Last reply Reply Quote 0
                  • M
                    mkernalcon
                    last edited by

                    After updating just a few hours ago to the 2.5.0 release on our main router, I can confirm that I am having the same issue. I've temporarily fixed it by disabling the "Register DHCP leases in the DNS Resolver" option. I can confirm, looking through the logs, that several HUPs get sent properly, all to the same PID, before finally starting to fail with "No such process". The last HUP that doesn't immediately fail in DHCP logs, has exactly the same timestamp as "pid 55598 (unbound), jid 0, uid 59: exited on signal 11" in the general log. No information in the DNS resolver logs.

                    DHCP Leases are default (7200s) for all vlans except my "main" lan, which is 691200s. Looks like the HUPs that kill it come from the 7200s vlans, but this is probably just coincidence.

                    Installed Packages:
                    darkstat
                    iperf
                    nmap
                    nut
                    openvpn-client-export
                    Status_Traffic_Totals

                    unbound.conf (has not been edited manually):

                    ##########################
                    # Unbound Configuration
                    ##########################
                    
                    ##
                    # Server configuration
                    ##
                    server:
                    
                    chroot: /var/unbound
                    username: "unbound"
                    directory: "/var/unbound"
                    pidfile: "/var/run/unbound.pid"
                    use-syslog: yes
                    port: 53
                    verbosity: 2
                    hide-identity: yes
                    hide-version: yes
                    harden-glue: yes
                    do-ip4: yes
                    do-ip6: no
                    do-udp: yes
                    do-tcp: yes
                    do-daemonize: yes
                    module-config: "validator iterator"
                    unwanted-reply-threshold: 0
                    num-queries-per-thread: 512
                    jostle-timeout: 200
                    infra-host-ttl: 900
                    infra-cache-numhosts: 10000
                    outgoing-num-tcp: 10
                    incoming-num-tcp: 10
                    edns-buffer-size: 4096
                    cache-max-ttl: 86400
                    cache-min-ttl: 0
                    harden-dnssec-stripped: yes
                    msg-cache-size: 4m
                    rrset-cache-size: 8m
                    
                    num-threads: 12
                    msg-cache-slabs: 8
                    rrset-cache-slabs: 8
                    infra-cache-slabs: 8
                    key-cache-slabs: 8
                    outgoing-range: 4096
                    #so-rcvbuf: 4m
                    auto-trust-anchor-file: /var/unbound/root.key
                    prefetch: no
                    prefetch-key: no
                    use-caps-for-id: no
                    serve-expired: no
                    aggressive-nsec: no
                    # Statistics
                    # Unbound Statistics
                    statistics-interval: 0
                    extended-statistics: yes
                    statistics-cumulative: yes
                    
                    # TLS Configuration
                    tls-cert-bundle: "/etc/ssl/cert.pem"
                    
                    # Interface IP(s) to bind to
                    interface: 192.168.2.1
                    interface: 192.168.3.1
                    interface: 192.168.4.1
                    interface: 192.168.11.1
                    interface: 192.168.99.1
                    interface: 127.0.0.1
                    interface: ::1
                    
                    # Outgoing interfaces to be used
                    
                    # DNS Rebinding
                    # For DNS Rebinding prevention
                    private-address: 127.0.0.0/8
                    private-address: 10.0.0.0/8
                    private-address: ::ffff:a00:0/104
                    private-address: 172.16.0.0/12
                    private-address: ::ffff:ac10:0/108
                    private-address: 169.254.0.0/16
                    private-address: ::ffff:a9fe:0/112
                    private-address: 192.168.0.0/16
                    private-address: ::ffff:c0a8:0/112
                    private-address: fd00::/8
                    private-address: fe80::/10
                    # Set private domains in case authoritative name server returns a Private IP address
                    
                    
                    
                    # Access lists
                    include: /var/unbound/access_lists.conf
                    
                    # Static host entries
                    include: /var/unbound/host_entries.conf
                    
                    # dhcp lease entries
                    include: /var/unbound/dhcpleases_entries.conf
                    
                    
                    
                    # Domain overrides
                    include: /var/unbound/domainoverrides.conf
                    
                    
                    
                    
                    ###
                    # Remote Control Config
                    ###
                    include: /var/unbound/remotecontrol.conf
                    
                    1 Reply Last reply Reply Quote 0
                    • jimpJ
                      jimp Rebel Alliance Developer Netgate
                      last edited by

                      OK so nothing jumps out in those configs. I still can't make it crash here even hammering on it.

                      I see that Unbound 1.13.1 is out now, we might need to pull that in and test against it. I reopened https://redmine.pfsense.org/issues/11316 which was initially closed since we didn't have enough information.

                      Keep the details coming here on this forum post, we may still be able to spot a pattern.

                      Remember: Upvote with the ๐Ÿ‘ button for any user/post you find to be helpful, informative, or deserving of recognition!

                      Need help fast? Netgate Global Support!

                      Do not Chat/PM for help!

                      M 1 Reply Last reply Reply Quote 0
                      • M
                        mxw39 @jimp
                        last edited by mxw39

                        @jimp thanks for the support! Is there some debug command that helps collect logs? dumps syscalls before segfault? I hope to contribute my crashing unbound somehow.

                        1 Reply Last reply Reply Quote 0
                        • J
                          jkv
                          last edited by jkv

                          After upgrading two systems (one to CE 2.5.0 and the other, running negate hardware (SG-5100) , to + 21.02) I have also started seeing this issue.

                          Just in case it helps in spotting any common patterns - I note that:

                          • On both systems (between every 5 to 10 minutes) I see unbound stopping and restarting in the DNS Resolver log.
                          • Only on SG-5100 (pfsense + 21.02) I am also seeing (probably 8 times a day or so) in the General System log that unbound exited on signal 11 (for example "pid 73090 (unbound), jid 0, uid 59: exited on signal 11").
                          • The only packages installed in common on both systems are Cron (0.3.7_5), openvpn-client-export (1.5_5) and Service_Watchdog (1.8.7_1).
                          • The SG-5100 (pfsense + 21.02) also has installed arpwatch (0.2.0_4), freeradius3 (0.15.7_29), and pfBlockerNG-devel (3.0.0_10).
                          • Both systems have WAN connections with dynamic IPs (so ddns is in use on the WAN side).
                          • Both systems also have some Static DHCP entries set (with "Register DHCP static mappings in the DNS Resolver" enabled).
                          GertjanG 1 Reply Last reply Reply Quote 0
                          • GertjanG
                            Gertjan @jkv
                            last edited by

                            @jkv said in pfSense 2.50 snapshots have been dying for the past couple of days:

                            I see unbound stopping

                            &

                            @jkv said in pfSense 2.50 snapshots have been dying for the past couple of days:

                            General System log that unbound exited on signal 11

                            You see it dying.
                            You use Service_Watchdog to restart it - right ?

                            @jkv said in pfSense 2.50 snapshots have been dying for the past couple of days:

                            pfBlockerNG-devel (3.0.0_10).

                            How often is the pfBlockerNG-devel doing it's cron task ? This task is logged. Does it restart unbound ?
                            What happens when you stop "Service_Watchdog ", so it doesn't restart unbound ?

                            What I'm trying to find out : if Service_Watchdog detects that unbound stops, it launches another instance. But it was actually just stopping and restarting, ordered by pfBlockerNG-devel. So, two instances are started, one dies .....
                            This is just a theory, as I'm not using Service_Watchdog myself

                            Also, SG-5100 is an Intel based machine, so "You and I" are using the same executable / same binary. Only our "config" differs. I don't know nothing about ARM based binaries, but I tend to say the "Intel" ones are pretty solid.

                            These :

                            @jkv said in pfSense 2.50 snapshots have been dying for the past couple of days:

                            have some Static DHCP entries

                            do nothing to unbound. The "static DHCP settings" (host name IP relation) are copied in the /etc/hosts file during boot. this file is (also) read by unbound during it's initial start up. These 'static DHCP setting' rarely change, that is, only if you delete/modify/add one. Look at this file, you'll see what I mean.
                            (In the past) the "DHCP Registration / Register DHCP leases in the DNS Resolver" could be problematic. The ""static DHCP settings"" were never a source of issue.

                            No "help me" PM's please. Use the forum, the community will thank you.
                            Edit : and where are the logs ??

                            J 1 Reply Last reply Reply Quote 0
                            • J
                              jkv @Gertjan
                              last edited by

                              @gertjan

                              the cron job for pfBlockerNG-devel is hourly and there does not appear to be any correlation between this cron job and unbound exiting. I will do some testing with Service_Watchdog disabled to see what happens to unbound.

                              S 1 Reply Last reply Reply Quote 1
                              • S
                                Salander27 0
                                last edited by

                                I doubt that service watchdog is the cause of the issue. It wasn't even present on my installation until I installed it so I wouldn't have to manually restart unbound after the crashes.

                                1 Reply Last reply Reply Quote 0
                                • S
                                  Salander27 0
                                  last edited by

                                  If there was a way for me to get a testing version of pfSense with Unbound 1.13.1 I would be more than happy to install that promptly and give feedback as to whether or not it is helpful at dealing with the issue.

                                  Also, can we get the title of this forum post updated to something like "DNS Resolver/Unbound crashing on pfSense 2.5" so that we can attract the attention of anyone else searching for this issue?

                                  jimpJ 1 Reply Last reply Reply Quote 0
                                  • jimpJ
                                    jimp Rebel Alliance Developer Netgate @Salander27 0
                                    last edited by

                                    @salander27-0 said in pfSense 2.50 snapshots have been dying for the past couple of days:

                                    If there was a way for me to get a testing version of pfSense with Unbound 1.13.1 I would be more than happy to install that promptly and give feedback as to whether or not it is helpful at dealing with the issue.

                                    We brought it in for snapshots (2.6.0 in the branch choices) but a new one hasn't built yet which includes it. In theory the branches are close enough at the moment you may be able to manually install the pkg archive file from the snapshot repo without much harm.

                                    Remember: Upvote with the ๐Ÿ‘ button for any user/post you find to be helpful, informative, or deserving of recognition!

                                    Need help fast? Netgate Global Support!

                                    Do not Chat/PM for help!

                                    1 Reply Last reply Reply Quote 0
                                    • jimpJ
                                      jimp Rebel Alliance Developer Netgate
                                      last edited by

                                      I edited the title of the thread to more accurately describe the issue.

                                      It would also be helpful to know the hardware in the cases where this is happening (e.g. SG-3100, SG-5100, whitebox/custom hardware running CE, etc)

                                      Remember: Upvote with the ๐Ÿ‘ button for any user/post you find to be helpful, informative, or deserving of recognition!

                                      Need help fast? Netgate Global Support!

                                      Do not Chat/PM for help!

                                      F E 2 Replies Last reply Reply Quote 0
                                      • S
                                        Salander27 0
                                        last edited by

                                        @jimp What is the repo URL for the snapshot repo that I can find that updated package in? I checked pkg+https://packages-beta.netgate.com/packages/pfSense_master_amd64-core and pkg+https://packages-beta.netgate.com/packages/pfSense_master_amd64-pfSense_devel and both still had unbound-1.13.0_2.

                                        jimpJ 1 Reply Last reply Reply Quote 1
                                        • F
                                          fivetoedslothbear @jimp
                                          last edited by fivetoedslothbear

                                          @jimp In my case, it's a custom Mini-ITX box I made with a Gigabyte B-150N motherboard (dual gigabit Intel NIC), and this is what the dashboard says about it:

                                          CPU Type 	Intel(R) Celeron(R) CPU G3900 @ 2.80GHz
                                          2 CPUs: 1 package(s) x 2 core(s)
                                          AES-NI CPU Crypto: Yes (active)
                                          Hardware crypto 	AES-CBC,AES-CCM,AES-GCM,AES-ICM,AES-XTS
                                          Kernel PTI 	Enabled
                                          

                                          It's been running pfSense successfully for more years than I remember.

                                          I'm getting occasional unbound crashes, and turned on the watchdog to restart the service when it dies.

                                          ETA: I'm running the Community Edition.

                                          1 Reply Last reply Reply Quote 0
                                          • jimpJ
                                            jimp Rebel Alliance Developer Netgate @Salander27 0
                                            last edited by

                                            @salander27-0 said in Unbound crashes periodically with signal 11:

                                            @jimp What is the repo URL for the snapshot repo that I can find that updated package in? I checked pkg+https://packages-beta.netgate.com/packages/pfSense_master_amd64-core and pkg+https://packages-beta.netgate.com/packages/pfSense_master_amd64-pfSense_devel and both still had unbound-1.13.0_2.

                                            In my previous reply I said "but a new one hasn't built yet which includes it." -- check later tonight/tomorrow AM.

                                            Remember: Upvote with the ๐Ÿ‘ button for any user/post you find to be helpful, informative, or deserving of recognition!

                                            Need help fast? Netgate Global Support!

                                            Do not Chat/PM for help!

                                            S 1 Reply Last reply Reply Quote 1
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.