Unbound DNS Over TLS Memory Leak
-
This post is deleted! -
Prior to upgrading to 2.4.4 I was using DNS over ssl with Cloudfare's IPv4 and IPv6 name servers.
Upgraded to 2.4.4 on Sept 25th, removed the custom unbound options and configured the same functionality using the gui. All good so far.
Upgraded pfBlockerNG package on Sept 26th and again on 27th Sept. (2.1.4_10 > 2.1.4_11 and then 2.1.4_12)
October 20th I installed two package updates, ntopng and pfBlockerNG (2.1.4_13)A couple of days later DNS stopped resolving. pfSense was out of memory and swap space.
After rebooting, i am monitoring the memory usage of unbound using top, which continues to climb quickly.
Have removed ntopng package completely and rebooted - no change
Disabled pfBlockerNG (including DNSBL) and again no change
Disabled "Use SSL/TLS for outgoing DNS Queries to Forwarding Servers" and although memory climbs a couple of Mb from the initial usage, it then stops.
Enable "Use SSL/TLS for outgoing DNS Queries to Forwarding Servers" and the memory starts to rapidly climb (10Mb or so a minute, dependent on DNS activity)From my change log I suspected something with ntopng and pfBlockerNG as that seemed to be what I'd changed. However removing or disabling those packages did not resolve the problem as I hoped.
I'm continuing to try and isolate the problem, but am running short of ideas, beyond rolling back to pre 2.4.4 upgrade and going step by step to see what causes the memory issue.
-
https://redmine.pfsense.org/issues/9059
-
Thanks for the information
-
Sorry guys, I thought I deleted my post because I made it in haste and then went to check redmine and found it was slated for 2.4.4-p1.
-
Is there an easy way to use the package manager to pull in the newer unbound? I see it mentioned in the bug but I'm not sure how to do that. I've just been restarting unbound every few days.
-
It is built on 2.4.5 snapshots but not something you can pull into 2.4.4 easily right this moment. You could play tricks with the pkg repo or install it directly but I wouldn't recommend doing that just yet.
I haven't seen any fallout from the upgrade on 2.4.5 snapshots so if other devs agree I may pick the change back so it will show up for 2.4.4 users. In that case it should then be possible to update with a simple
pkg upgrade unboundcommand. -
The new package is up. You can install it with
pkg update; pkg upgrade unboundfrom a shell prompt (NOT from Diag > Command).I'd test it out first on something non-production just in case, but I haven't had any problems here in my tests.
Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!
Need help fast? Netgate Global Support!
Do not Chat/PM for help!
-
Looks like it's working here OK. I'll post back if I see any issues. Thank you for your help.
-
@jimp it updated on the intel pfsense units but not arm. has it been sent out for both?
-
Should be up there, now. Check again.
-
@jimp Got it. Thanks.
-
@jimp said in Unbound DNS Over TLS Memory Leak:
I'd test it out first on something non-production just in case, but I haven't had any problems here in my tests.
Unbound is only serving requests from a single thread after I updated to Unbound to 1.8.1: https://forum.netgate.com/topic/138274/unbound-1-8-1-only-single-thread-processing-dns-requests
