Unbound 1.8.1 only single thread processing DNS requests
-
I ran into the DNS over TLS memory leak in Unbound 1.7.3 which is included in pfSense. Per the instructions here I updated Unbound to 1.8.1
pkg update; pkg upgrade unbound
After the upgrade I found that Unbound appears to only be using a single thread. Note that only "thread 0" has any stats
>unbound-control -c /var/unbound/unbound.conf stats_noreset thread0.num.queries=1997 thread0.num.queries_ip_ratelimited=0 thread0.num.cachehits=21 thread0.num.cachemiss=1976 thread0.num.prefetch=3 thread0.num.zero_ttl=7 thread0.num.recursivereplies=1943 thread0.requestlist.avg=18.0273 thread0.requestlist.max=55 thread0.requestlist.overwritten=0 thread0.requestlist.exceeded=0 thread0.requestlist.current.all=25 thread0.requestlist.current.user=18 thread0.recursion.time.avg=0.360357 thread0.recursion.time.median=0.16633 thread0.tcpusage=0 thread1.num.queries=0 thread1.num.queries_ip_ratelimited=0 thread1.num.cachehits=0 thread1.num.cachemiss=0 thread1.num.prefetch=0 thread1.num.zero_ttl=0 thread1.num.recursivereplies=0 thread1.requestlist.avg=0 thread1.requestlist.max=0 thread1.requestlist.overwritten=0 thread1.requestlist.exceeded=0 thread1.requestlist.current.all=0 thread1.requestlist.current.user=0 thread1.recursion.time.avg=0.000000 thread1.recursion.time.median=0 thread1.tcpusage=0 thread2.num.queries=0 thread2.num.queries_ip_ratelimited=0 thread2.num.cachehits=0 thread2.num.cachemiss=0 thread2.num.prefetch=0 thread2.num.zero_ttl=0 thread2.num.recursivereplies=0 thread2.requestlist.avg=0 thread2.requestlist.max=0 thread2.requestlist.overwritten=0 thread2.requestlist.exceeded=0 thread2.requestlist.current.all=0 thread2.requestlist.current.user=0 thread2.recursion.time.avg=0.000000 thread2.recursion.time.median=0 thread2.tcpusage=0 thread3.num.queries=0 thread3.num.queries_ip_ratelimited=0 thread3.num.cachehits=0 thread3.num.cachemiss=0 thread3.num.prefetch=0 thread3.num.zero_ttl=0 thread3.num.recursivereplies=0 thread3.requestlist.avg=0 thread3.requestlist.max=0 thread3.requestlist.overwritten=0 thread3.requestlist.exceeded=0 thread3.requestlist.current.all=0 thread3.requestlist.current.user=0 thread3.recursion.time.avg=0.000000 thread3.recursion.time.median=0 thread3.tcpusage=0 total.num.queries=1997 total.num.queries_ip_ratelimited=0 total.num.cachehits=21 total.num.cachemiss=1976 total.num.prefetch=3 total.num.zero_ttl=7 total.num.recursivereplies=1943 total.requestlist.avg=18.0273 total.requestlist.max=55 total.requestlist.overwritten=0 total.requestlist.exceeded=0 total.requestlist.current.all=25 total.requestlist.current.user=18
Anyone have any ideas as to what may be going on here?
-
@imcdona Looks like this bug was fixed in Unbound 1.8.2 released today.
-
Yep : see https://www.netgate.com/blog/pfsense-2-4-4-release-p1-now-available.html => Notable Bug Fixes => Updated Unbound to 1.8.1 to address issues with memory leaks, especially in DNS over TLS support.
See also the identical ( !! ) thread just a little bit further down in the same forum "DHCP DNS" for an free bonus.
-
@imcdona On a side note, how do you access the Unbound stats? What command needs to be entered at the terminal?
-
@tibere86 said in Unbound 1.8.1 only single thread processing DNS requests:
Looks like this bug was fixed in Unbound 1.8.2 released today.
@gertjan said in Unbound 1.8.1 only single thread processing DNS requests:
Yep : see https://www.netgate.com/blog/pfsense-2-4-4-release-p1-now-available.html => Notable Bug Fixes => Updated Unbound to 1.8.1 to address issues with memory leaks, especially in DNS over TLS support.
See also the identical ( !! ) thread just a little bit further down in the same forum "DHCP DNS" for an free bonus.Wrong. They fixed the TLS memory issues. Unbound is still only processing requests from a single thread. The temporary fix is to add:
server: so-reuseport: no
-
@tibere86 said in Unbound 1.8.1 only single thread processing DNS requests:
@imcdona On a side note, how do you access the Unbound stats? What command needs to be entered at the terminal?
Run the following
unbound-control -c /var/unbound/unbound.conf stats_noreset
or for a list of other options run
unbound-control -c /var/unbound/unbound.conf
such as
unbound-control -c /var/unbound/unbound.conf dump_requestlist
-
Is there a bug open in redmine for this?
Is it considered a bug? Is the workaround required in 1.8.2?And also, is there a 1.8.2 we can manually install?
-
@imcdona Thanks!
-
@bbrendon said in Unbound 1.8.1 only single thread processing DNS requests:
Is there a bug open in redmine for this?
Not yet
Is it considered a bug?
Unbound considers it a problem with the default configuration on FreeBSD, so I'm not quite sure they consider it a bug.
Is the workaround required in 1.8.2?
No, since the default configuration changed. The workaround would still be safe to have, but it wouldn't make a difference since the default is 'no'.
And also, is there a 1.8.2 we can manually install?
No, it isn't even in FreeBSD ports yet, and since you can work around it with a simple config change, it probably wouldn't warrant an OOB update like the memory leak did.
-
@imcdona Add those lines where?
-
In the custom options box.
-
@imcdona From what I can tell, the default value of so-reuseport is no, and that it would need to be set to yes to use multiple threads? But also, I've found information suggesting that FreeBSD's setsockopt implementation didn't support the option at all until version 12 (https://nlnetlabs.nl/pipermail/unbound-users/2018-November/011134.html). Of course, I won't argue with experimental evidence to the contrary, I'm just a bit confused.
-
@johnpoz I tried that but didn't notice a change in behavior. I've seen other threads where unbound was crashing and restarting, could that be the cause of sluggish or timed-out DNS queries?
I've also noted that when PfBlockerNG-devel is enabled it places the line into the custom settings box but combines the lines. I was wondering if this is an issue? Are both formats allowed or is PfBlocerNG-devel not adding a line after "server:" where it should?
server: so-reuseport: no server:include: /var/unbound/pfb_dnsbl.*conf
-
You wont see change in behaviour, but when you reload/restart unbound, you will see different stats indicating that it's now multi-threaded .
Both
server: so-reuseport: no
server: so-reuseport: no
are fine
-
@ronpfs Is the missing white-space after "server:" also permitted?
server:include: /var/unbound/pfb_dnsbl.*conf
-
@lohphat said in Unbound 1.8.1 only single thread processing DNS requests:
Is the missing white-space after "server:" also permitted?
That's what I have in my settings :
server:qname-minimisation: yes server:so-reuseport: no server:include: /var/unbound/pfb_dnsbl.*conf
-
@ronpfs said in Unbound 1.8.1 only single thread processing DNS requests:
server:qname-minimisation: yes
This is now possible in the gui, no need for it custom just check the box in the gui.