Unbound seems to be restarting frequently
-
The issue is not in the package, it's in the scripts controlling it. The way the scripts are written now, they insert data into Unbound's config files, then restart the service for Unbound to reload those config files and pull in the changes. Dnsmasq is handled the same way, except it will hold its cache and re-read config files upon getting a SIGHUP.
The right way to handle local DNS changes, for Unbound at least, would basically be to do the opposite of what is being done now. Rather than write to the config files and bounce the service, you would use unbound-control to tell Unbound about the new local DNS changes and let it write the config files.
Whilst this sounds promising, have you code read unbound to check what unbound-control local_data and unbound-control local_data_remove actually do? If these commands force a cache flush, then there is no gain and potential losses in your proposed approach. If unbound uses a cache walk to discard affected RR(s), a different potential performance issue arises if this process is blocking, especially if unbound is started with no local RRs and they are added one at a time using unbound-control. If cache discard takes place via a non-blocking cache walk or no cached RR removal takes place, the possibility of race conditions exists, as stale RR data will be served for an indeterminate but potentially significant time.
Answers to all of your questions are in plain text here:
https://www.unbound.net/documentation/unbound-control.htmlNo, but the fact that local add / remove commands only process local data space makes it pretty clear.
No race condition exists. Simply process 1 unbound-control command at a time. unbound-control returns completion. At the rate they execute I would guess even with lots of local custom entries (say 200+), which even the documentation states not to use Unbound on its own as a full hosted DNS solution, this would happen very quickly. Since it would only happen on service start, the impact would be even more who gives a fuck sort of minimal. Ditto for local removals. Cap it off with the "well what if 500+ DHCP client releases happen at the end of the day" answer of "if your pfSense hardware can handle serving 500+ clients all day, it can easily handle removing 500+ local DNS entries quickly".
Rather than using unbound-control to insert all RRs on startup, a hybrid approach is possible - start unbound with the current state in the configuration files, then use unbound-control for subsequent changes.
Yup, I gave that as 1 of 2 possible ways to go.
One complexity in implementing your proposed approach is a frustrating non-orthogonality in the local_data and local_data_remove commands. local_data adds a single RR. local_data_remove removes all RRs for the given name - which might include both A and AAAA records for the same name (and, if local_data_remove for PTR records is called by the name rather than the IP address, this means IPv4 and/or IPv6 PTR records pointing to that name will also be removed). IPv4 and IPv6 address allocation use entirely separate processes that have no guarantees on temporal relationship. There are circumstances where you might want to remove just one RR, which means you have to remove all RRs for the name then re-add any wanted RRs.
Since we are only dealing with local FQDN and localHost.localDomain records, also a non issue. If there are multiple records pointing to the same FQDN / localHost.localDomain and a DHCP event causes them to need removal, now we only need to issue a single unbound-control remove command. If that breaks ANYTHING local DNS wise (it won't), than very very worst case scenario, we continue to bounce the service only on custom static override changes (please don't do this).
If there are advantages to using unbound-control, it might be better to add support for unbound control to pfSense by generating a new unbound configuration as at present, then diffing it with the old configuration. If there are changes solely in local data (i.e. in dhcpleases_entries.conf and/or host_entries.conf), pfSense could use the output of the diff to make appropriate calls to unbound-control, otherwise replace the old configuration with the new and send unbound a SIGHUP. This approach has the advantage of ensuring consistency (for example by ensuring explicitly configured local-data is not accidentally removed) and is easily removed if unbound gets saner SIGHUP behaviour in the future. One implementational complexity is the need to call unbound-control local_data for any explicit local-data and local-data-ptr in unbound.conf after you call unbound-control local_data_remove for the same name.
Yes, to the "somebody with the time needs to test this", I would agree with my own previous post.
No to the rest. You're over thinking it now. Forget SIGHUP for Unbound. Consistency is already being maintained in config files by current code. We need to dump the service restart bullshit, or dump Unbound. Love it or leave it. It will never work correctly if you aren't smart enough to maintain local records without bouncing the service or dumping the non-local cache needlessly.
Another possibility is to use an entirely separate DNS server for local zone(s), defining them in Unbound as stub zones. There is potential advantage here in terms of being able to DNSSEC sign these zones. However, when data changes, stale data would be cached in Unbound that would require carefully targeted calls to unbound-control flush_type or a costly (because of the cache walk) unbound-control flush_zone to remove.
I thought about that, so have others, and some have implemented it, it's not hard with current RELEASE stock packages. Use dnsmasq for local DNS, add domain overrides to Unbound to point to dnsmasq's local custom port. Domain overrides bypass Unbound's cache, so removing stale entries is a non-issue. The issue here is that it is not the right solution. It's an unnecessary workaround at best, and that's not what anyone wants to see time spent on. We would rather see time spent on correct implementation.
For the same reason, completely forget about commands issuing full cache walks too. Even on fast hardware, churning through a potential 1GB of tables, even in RAM, is just stupid when you have the option of not ever doing that.
Edit: Added considerations for pfSense hardware capacity & localHost.localDomain scenario. No methods changed.
-
Is it just me who's finding similar masturbation absurd? Why not fix the braindead SIGHUP handling instead?
-
Because if you had thought it through you would realize there's nothing to fix as far as Unbound itself goes. The design of Unbound is to not need absurd (old) things like custom SIGHUP handling because it has something far more versatile, unbound-control.
unbound-control is Unbound's SIGHUP replacement, just infinitely more useful, since you can feed it parameters and get success / failure returns. Far more securely and gracefully at that.
RTFM my friend.
To repeat again, and again, Unbound was picked for this reason. If the devs now regret that decision because they are stuck on using replaced for a good reason functions like throwing SIGHUP's all over the place, they should pick something older and worse than Unbound as the resolver.
And that would be sad, because Unbound is a GREAT caching DNS resolver when you use it correctly.
-
Is it just me who's finding similar masturbation absurd? Why not fix the braindead SIGHUP handling instead?
Because that would itself be absurd.
You are the master of solutions to problems that should never have existed in the first place. If I ever score a "paid by the line" coding position you are my first hire.
-
-
P.S. As for hires - not interested, thanks. ::)
Oh don't worry, I refuse pay by line type jobs. And all of my clients prefer dealing with people who know what they're doing. That was the point, but you missed it.
Don't forget, last time we did this, as soon as someone who knew what they were talking about finally chimed in, I believe it was something to the tune of "I think you are spot on here". Yup, it was.
As far as helping you, I think this is a good start:
http://www.goodtherapy.org/therapy-for-control-issues.html -
Perhaps, can you send me further personal shit, hire offers and shrink doctor recommendations via PM? It will get safely ignored there; plus it won't produce useless noise for other forum users here that noone's interested in reading.
Alternatively, just STFU and start coding.
-
Perhaps, can you send me further personal shit, hire offers and shrink doctor recommendations via PM? It will get safely ignored there; plus it won't produce useless noise for other forum users here that noone's interested in reading.
Alternatively, just STFU and start coding.
Because God forbid anyone else actually realize what a fucktard you are, right?
Ohhh, did you bump your head? Take your own advice there kiddo.
If you want to be a "I don't understand this so I'm going to troll and shit all over it" dick, on a non-technical and personal level, publicly. Then I will continue to hand you your ass, publicly. Perhaps your masturbation comment (really all of your comments) are more appropriate as a PM, since they do indeed fit your "useless noise for other forum users here that noone's interested in reading" category.
And again with the control issues. Seriously, get help. Your attitude is one only a mother on heroine could love.
-
Lick my swamp.
-
Snappy comeback.
-
Bug opened: https://redmine.pfsense.org/issues/5413
-
-
How many people having this issue are using a config that used to use the forwarder rather than the resolver? And how many of those people still have the top 3 DHCP options checked in the forwarder, even though the forwarder is disabled?
If this is the case, humor me here… go to the DNS forwarder page, check the top enable box to enable access to the 3 DHCP options boxes below it, uncheck all 3 DHCP options boxes, then uncheck the top enable box (so we don't unwantingly enable the forwarder), click the save button, verify that all 3 DHCP options boxes for the forwarder are now unchecked and disabled once the page reloads, and reboot pfSense for sanity.
Please post back as to what effect this has on Unbound reloads… either the same, less frequently, or eliminates them all together.
For me this stopped the repeated "unbound: service stopped", "unbound: start of service" messages 2-3 times per minute. Thanks - this was a longstanding issue.
It fit because this installation was previously dnsmasq, switched to unbound some time ago.
Specifically, the relevant part of the config export looked like this before:
<dnsmasq><regdhcpstatic><custom_options><domain_needed><no_private_reverse><interface></interface></no_private_reverse></domain_needed></custom_options></regdhcpstatic></dnsmasq>and like this after:
<dnsmasq><custom_options><domain_needed><no_private_reverse><interface></interface></no_private_reverse></domain_needed></custom_options></dnsmasq>It also took a reboot.
A more subtle issue for me is that machines seem to lose DNS resolution (maybe all connectivity?) for about 5 seconds every time their DHCP lease expires and is renewed. For now I've just lengthened DHCP leases significantly - they were short for testing. Separate issue I guess.
-
I had to go back to DNS Forwarder (dnsmasq) because of this. Thank God it's still inthere and only a checkbox away.
-
I know a bug report is not really the place for arguing about the merits of a solution, but I respectfully maintain some of my caution in https://forum.pfsense.org/index.php?topic=89589.msg568394#msg568394 , especially in relation to ky41083's assertion that there is no need ever to restart Unbound.
Changes in local-data can be handled via unbound-control as ky41083 says - though the inability to remove just the A or AAAA record will likely require some care, as both can exist for the same local host and there is no guaranteed temporal relationship between changes in A and AAAA. In particular, DHCP and DHCPv6 are entirely separate and not synchronous.
Unlike ky41083, I cannot see any alternative to restarting Unbound if there are configuration changes made to Unbound beyond changes to local data, as SIGHUP and unbound-control reload unfortunately amount to a reload at present (i.e. cache flush and re-read of the configuration files). unbound-control does not allow for on-the-fly reconfiguration of all aspects of Unbound. This is why I suggested a diff based approach on the forums as one possibility.
These are, of course, implementation points. I agree that Unbound should be reconfigured on-the-fly whenever possible. In time, I hope that Unbound will get saner SIGHUP handling, but this will likely be a lot of work.
Unfortunately, I have no time to work on this issue at present.
You are under thinking the problem and over thinking the solution.
Quickly now…
DHCPv4 & DHCPv6 are entirely separate and not synchronous, agreed. DNS records are only removed when a lease expires, right? How many milliseconds do you think will differ between v4 and v6 leases being obtained by the same host? So deleting just the A or just the AAAA record, vs removing them both at the same time, matters how much, in the real world? Milliseconds, that you can no longer resolve an internal host via DNS, which has been off for (DHCP lease period) anyhow.
If this causes issues for you, make your DHCPv4 & DHCPv6 lease expiration times identical, or fix your DHCP clients that aren't requesting v4 and v6 leases at the same time for who knows why.
If this is causing internal host overrides to be removed, you're putting persistent local host DNS records in the wrong place. DNS is where you put external DNS redirects. DHCP reservations is where you put persistent local host DNS records so they never get removed.
Please, I would love a correct list of things that cannot be changed without restarting Unbound, from someone that's not me.
-
This solved it for me:
https://forum.pfsense.org/index.php?topic=102470.msg578573#msg578573
- John -
Ring a ding ding same here.
Though I had track disabled, I had to just completely disable IPv6 in sys/adv/networking. -
This solved it for me:
https://forum.pfsense.org/index.php?topic=102470.msg578573#msg578573
- John -
Ring a ding ding same here.
Though I had track disabled, I had to just completely disable IPv6 in sys/adv/networking.There is an issue in Unbound, I will have to disable DNSBL so that there is no "server:include: /var/unbound/pfb_dnsbl.conf" in Unbound custom options window.
see my posts: https://forum.pfsense.org/index.php?topic=113193.0 and https://forum.pfsense.org/index.php?topic=114277.0
so far no response from pfSense Dev teams.
-
I also have found unbound is restarted whenever wan ipv6 is renewed on dhcp6.
so First this occurs
/rc.newwanipv6: rc.newwanipv6:then gateway etc. been set
After this occurscheck_reload_status Reloading filterAt same time in resolver log unbound is restarted.
Now with only small dnsbl lists loaded in pfblockerng this is not a major issue as I have found even uncached queries on my pfsense box are faster than when I used the asus ac68 as my router, however each unbound restart does also have to process the dnsbl lists which is a slow process if I have large lists loaded.
I will look at the pfsense code and see if I can submit a fix that will be accepted, its probably easy to hack this like someone did earlier in the thread but ideally a clean solution would be found that can be accepted as a improvement, since there is no need to restart unbound just for renewing the same ipv6 prefix.
using"SECOIT GmbH" code, is ok but I think will stop unbound been reloaded when dnsbl gets updated daily on my box, so instead I commented out a line in rc.newwanipv6 which forces a unbound restart. Later after sleep I will look into only skipping the unbound restart in rc.newwanipv6 when its not on bootup and when the actual ipv6 on the pfsense box has not changed, so in other words add a check to see if the new ipv6 matches the previous, if it does skip the reload.
-
I have WAN and LAN IPv6 both static and I am also affected by it.
-
For 2.3.x users without DNSBL active, this fixes everything. Someone could easily make a diff patch for DNSBL based on it as well:
https://forum.pfsense.org/index.php?topic=119467.msg661037#msg661037
