Unbound TCP buffer settings not sticky
-
./conf.default/config.xml or ./cf/conf/config.xml?
Same thing /conf is a link of /cf/conf
mark62 are you running pfsense 2.1 that has been upgraded? Likely you need to manually clean out unbound.
Uninstall unbound
Turn on rw
Find / -name *unbound*
Delete everything you can find
/conf/config.XML clear out all everything to do with unbound. Backup first maybe.
Once you are sure you have killed everything reinstall.
If you have problems with internet try adding a DNS to resolv.conf -
./conf.default/config.xml or ./cf/conf/config.xml?
Same thing /conf is a link of /cf/conf
Yeah but /conf.default is not the same thing thats used, as an example, for reverting to factory defaults.
-
./conf.default/config.xml or ./cf/conf/config.xml?
Same thing /conf is a link of /cf/conf
Yeah but /conf.default is not the same thing thats used for factory defaults.
Haha whoops Justin woke UP didn't notice default
-
Thanks guys. Where are the firmware upgrades and package upgrades hosted to do a DNS resolve on? Same for both? Do I just need pfsense.com. Suppose a netstat during a temp pkg install might answer for one of the two. Hope the host IP doesn't change often. Good suggestion to locally resolve should happen to scramble Unbound again.
I find one unbound install in the xml after I had uninstalled the pkg by GUI this morning. As well, two unbound files remain. I'll clean these up tonight, cron a reboot, and report back.
Really appreciate your help…
-
Thanks guys. Where are the firmware upgrades and package upgrades hosted to do a DNS resolve on? Same for both? Do I just need pfsense.com. Suppose a netstat during a temp pkg install might answer for one of the two. Hope the host IP doesn't change often. Good suggestion to locally resolve should happen to scramble Unbound again.
I find one unbound install in the xml after I had uninstalled the pkg by GUI this morning. As well, two unbound files remain. I'll clean these up tonight, cron a reboot, and report back.
Really appreciate your help…
files.pfsense.org - pbi
www.pfsense.com - config -
Cleaned the XML of remnant Unbound package info. The reinstall went well and seems to be working normally now. Odd though, the install reported 21_1 rather than 21_3. After install the "Installed Packages" reports 21_3. And the Unbound Status reports 21_3. Looks like outgoing-num-tcp: 0 & incoming-num-tcp: 0 are still making it from GUI to XML.
–-------------------
Checking for package installation...
Downloading http://files.pfsense.org/packages/amd64/8/All/unbound-1.4.21_1-amd64.pbi
version: 1.4.21
verbosity: 1
threads: 2
modules: 2 [ validator iterator ]
uptime: 51 seconds
unbound (pid 34067) is running…
Unbound Services 1.4.21_3
Unbound is a validating, recursive, and caching DNS resolver. This package is a drop in replacement for Services: DNS Forwarder and also supports DNSSEC extensions. Once installed please configure the Unbound service by visiting Services: Unbound DNS. -
Cleaned the XML of remnant Unbound package info. The reinstall went well and seems to be working normally now. Odd though, the install reported 21_1 rather than 21_3. After install the "Installed Packages" reports 21_3. And the Unbound Status reports 21_3. Looks like outgoing-num-tcp: 0 & incoming-num-tcp: 0 are still making it from GUI to XML.
–-------------------
Checking for package installation...
Downloading http://files.pfsense.org/packages/amd64/8/All/unbound-1.4.21_1-amd64.pbi
version: 1.4.21
verbosity: 1
threads: 2
modules: 2 [ validator iterator ]
uptime: 51 seconds
unbound (pid 34067) is running…
Unbound Services 1.4.21_3
Unbound is a validating, recursive, and caching DNS resolver. This package is a drop in replacement for Services: DNS Forwarder and also supports DNSSEC extensions. Once installed please configure the Unbound service by visiting Services: Unbound DNS.Great! Wagonza updated the file already so the configuration should be working properly as you see.
As far as I can see its just the packages list xml has the newer version number and the unbound package xml has the older version number still. Wagonza just needs to update that line.
-
Yup that's all. Package just needs a rebuild on the package servers as well. Glad to hear all ok markn62.
-
Not what I was hoping to report. Thought all was well after the reinstall. Watched for about an hour and everything appeared normal. Bout 30 minutes after I walked away Unbound died and would not restart. Pointed back to the stand-alone unbound service AGAIN. This is getting so frustrating!
-
Any logs (both system and package logs) giving an indication as to why?
-
Not what I was hoping to report. Thought all was well after the reinstall. Watched for about an hour and everything appeared normal. Bout 30 minutes after I walked away Unbound died and would not restart. Pointed back to the stand-alone unbound service AGAIN. This is getting so frustrating!
You will need to be much clearer and specific.
What do you mean it died? Still running but not responding? Crashed/No process?
What do you see in system.log? Or any other pertinent logs. Better yet pastebin the log so we can look.
Am I assuming correctly you mean a separate box running unbound when you say stand-alone unbound service?Is this a fresh 2.1 installation or a upgrade from prior versions to 2.1?
If an upgrade what packages do you have installed using pkg_info on the shell? -
By died I mean the unbound service stopped on it's own after about an hour of normal operation and the watchdog or manual restart wouldn't get it started again. Yes, stand-alone means a separate box. It is an upgrade from 2.0 to 2.1. Pkg_Info reports no packages installed even though there are. I'm not certain the name and location of the system and package log. I could binpaste the SNMP log if I knew how. Is that the insert code button? Bout 100 lines of text probably best not pasted in this post.
-
By died I mean the unbound service stopped on it's own after about an hour of normal operation and the watchdog or manual restart wouldn't get it started again. Yes, stand-alone means a separate box. It is an upgrade from 2.0 to 2.1. Pkg_Info reports no packages installed even though there are. I'm not certain the name and location of the system and package log. I could binpaste the SNMP log if I knew how. Is that the insert code button? Bout 100 lines of text probably best not pasted in this post.
/var/log
system.log
unbound.log? Is there one?
/tmp/php_errors.txtwhat happens when you try to start unbound manually?
-
Forum access is with a PC other than what has access to the router logs so it would take me a bit to transfer files around and post here. For now here is the SNMP; http://pastebin.com/embed_js.php?i=kTkuRnzX I hope I binpasted correctly using a hyperlink. As mentioned, a manual restart would do nothing. The service would remain unstarted.
A quick look via the GUI edit file I find php_errors.txt is empty even though the SNMP shows some. The router was rebooted early AM by cron. The unbound log only has CLOG and some odd symbol after the word. The system log has plenty and isn't cooperative by remote login. Large file I suppose. May have to post the system log this evening.
-
Forgot to mention installed packages. Installed and running fine for a few months other than Unbound are;
-
Cron
-
PhpSysInfo
-
Service Watchdog
-
SipProxD
-
Unbound
-
-
Just grab Winscp and use same credentials as ssh. Just like using a FTP server. Unbound should be complaining somewhere about something.
-
Oh it complained all right. I take it you didn't look at the SNMP file. First it installed, then ran normally for a bit, then started to lag on the median time, then started with PHP errors, warning about needing an increase in open files. Then finally that there was no such file or directory. So in short, the wheels eventually fell off. Never seen anything like this. I presume this slow failure occured as more demand hit the unbound package. I didn't get up at 4AM to see if Unbound would start again after the reboot. Cleaning up the XML required that I repopulate the Unbound settings via the GUI. I matched what was in the backup XML so it should have the same configuration as before the upgrade. The SNMP reports that version 1.4.21 installed, not 1.4.23. Presume the version reported isn't an issue.
-
check the unbound-control script and look where it is trying to start the binary from. Is the path correct? What is the path really?
I am going to be flying for some IT work tonight so I won't be around for a bit maybe! Darn this labor market.
-
What happens if you execute "/usr/pbi/unbound-amd64/sbin/unbound-control start" from the command line?
Do you get a warning on 'too many file descriptors requested'?The second issue is that there are two watchdog services running. The Unbound package has its own watchdog (log lines with Unbound_Alarm) called unbound_monitor.sh. So that starts it up Unbound and then so does the watchdog package. Hence why you seeing the "bind: address already in use" error message.
You should see unbound-1.4.21_1-amd64 using 'pbi_info'. Doesnt make a difference with _X bit, however I have bumped that on the package servers to avoid confusion in the future. Dont reinstall just yet as the package builders still need to build and create the updated version.
-
Thanks guys for all your help on this, much appreciated!
Bryan, have a safe flight. I find;
/usr/local/sbin/unbound-control
/usr/pbi/unbound-amd64/sbin/unbound-control
/usr/pbi/unbound-amd64/.sbin/unbound-controlPresume it's the …amd64/sbin...
Within this I have;
unbound
unbound-anchor
unbound-checkconf
unbound-control
unbound-control-setup
unbound-hostNot sure which file is the script. The unbound-control file is not readable. The unbound-control-setup has a line;
DESTDIR=/usr/pbi/unbound-amd64/etc/unboundWagonza,
I didn't know Unbound had its own watchdog, explains a lot. Perhaps certain package choices could be removed from the Wachdog service app list for those that will cause a conflict. If I execute "unbound-control start" I imagine Unbound will halt due to the fact the DNS forwarder is currently enabled and functioning in it's place. The GUI barks if fwd'r is enabled when unbound is enabled. This box is in active service so I have to be careful not to execute anything that will dump states or crash services. Can you assure nothing unexpected will happen except a possible warning?Thanks again,
Mark -
Ah ok so you got dnsmasq running again. No worries running that command will cause an error indicating it would not be able to bind to port 53 as its in use.
Other than that it wont cause any harm.Btw all those binaries you found are correct so nothing looks out of the ordinary.
As for the Unbound watchdog it should probably be removed and leave it up to the Watchdog service to handle. However since Unbound is going into 2.2 a different approach will be looked at.
-
What happens if you execute "/usr/pbi/unbound-amd64/sbin/unbound-control start" from the command line?
Do you get a warning on 'too many file descriptors requested'?Wagonza, I get this;
[1393946424] unbound[43135:0] warning: increased limit(open files) from 11095 to 16418
[1393946424] unbound[43135:0] error: bind: address already in use
[1393946424] unbound[43135:0] fatal error: could not open portsPresume the bind error is because the Dns Forwarder is in use. I can follow up on that by rechecking for open ports when I reattempt to start the Unbound pkg again. But what about the increase open files limit? Never seen this error message before.
-
The limit warning is because the normal process resource limits are being extended by unbound further than system has set them to as default. It has done so successfully.
This is driven by outgoing range and number queries. Though you're unbound is trying to take a ton of resources. 16000+ Try stripping out your custom outgoing range and number query options maybe. If you Google a bit your limit increase is oddball. Might have to do with multiple lines being the same.
Are these settings matching your other stand alone?
If so how much ram does it use? Do you see warnings?
What is your stand alone ulimit set to?You could try to remove the duplicate default settings in the script that generates the unbound configuration file. It may not handle duplicates well and is why we see strange issues.
-
Thanks Bryan for the explaination. I have 8GB ram serving ~200 PC users. An outgoing range of 900 on a single thread works fine on a stand-alone Unbound service in a 2G P4 box w/ 2G ram. The GUI doesn't allow setting the outgoing range but does so automatically at the 8192 number, as I recall. Setting a custom outgoing range in the Unbound Options causes the service to not start, or at least it did. This was my original concern when I first tried the Unbound package last year using my script from the P4, unbound has limited tuning as a package.
If it's automatic then perhaps I shouldn't be concerned. I'll plan an early AM restart of Unbound without a separate watchdog service active and see if the wheels stay on. The only other post-upgrade change made was incoming/outgoing buffers from 10 to 0 now that they reflect in the XML.
-
@Markn62 follow @bryan.paradis' advice. You can try reduce the number of buffers it uses and other resources.
The outgoing-range is a bit high and that value is automatically calculated based on Unbound docs.
So possibly the maths is wrong….ermm nope for some reason it has the value 8192 hard coded o_0.
Will fix that
-
Thanks Bryan for the explaination. I have 8GB ram serving ~200 PC users. An outgoing range of 900 on a single thread works fine on a stand-alone Unbound service in a 2G P4 box w/ 2G ram. The GUI doesn't allow setting the outgoing range but does so automatically at the 8192 number, as I recall. Setting a custom outgoing range in the Unbound Options causes the service to not start, or at least it did. This was my original concern when I first tried the Unbound package last year using my script from the P4, unbound has limited tuning as a package.
If it's automatic then perhaps I shouldn't be concerned. I'll plan an early AM restart of Unbound without a separate watchdog service active and see if the wheels stay on. The only other post-upgrade change made was incoming/outgoing buffers from 10 to 0 now that they reflect in the XML.
Unbound works well for others with 8192 default.
Unbound not working for you.
Try running without changing outgoing.If that works having dupliccate of that setting in the conf file may be the issue. Remedy d be to edit the. Inc
-
Mark - i need to go through a few things again and will update the package. So look for an update sometime to tomorrow.
There was a problem where Unbound wasnt compiled with Libevent and the value of 8192 would not work for those environments.
Afair it was fixed about 3 versions ago.Ill also add the outgoing-range to the advanced section - pop me a message with any other options you would want to see in the GUI.
-
Mark - i need to go through a few things again and will update the package. So look for an update sometime to tomorrow.
There was a problem where Unbound wasnt compiled with Libevent and the value of 8192 would not work for those environments.
Afair it was fixed about 3 versions ago.Ill also add the outgoing-range to the advanced section - pop me a message with any other options you would want to see in the GUI.
It might be the duplicate lines in the configuration from the hardcoded + custom ones.
Mark you should pastebin your resolver.log so we can have a look. It should have all the unbound error information.
-
Could you give us the output of
limits
-
Mark - i need to go through a few things again and will update the package. So look for an update sometime to tomorrow.
Ill also add the outgoing-range to the advanced section - pop me a message with any other options you would want to see in the GUI.
Wogonza,
The following are some of the customs I've needed in other non-pkg'd unbound services. Outgoing has been addressed. PfSense appears to calculate threads and slabs based on the number of cores detected, but some may want to only use one core in a multi-core. Not sure how Unbound pkg currently determines num-queries-per-thread. This may be a good candidate for options detuning. I don't see how ip6 can be set to no. Unbound doesn't seem to take it from elsewhere that IPv6 has been disabled. So either it should or needs to be added to custom. The do-tcp:no is now handled by a value of zero in outgoing-num-tcp and incoming-num-tcp. Private address I believe is handled in the GUI by the third tab "ACL", doing this from memory best I can. And forwards are already in custom. So-rcv/sndbuf is rem'd out in the script and no GUI entry for them. Not sure why they get put in the script and not used.So the asterisk items I offer for your consideration as the most needed ones.
*outgoing-range:
num-threads:
msg-cache-slabs:
rrset-cache-slabs:
infra-cache-slabs:
key-cache-slabs:
*num-queries-per-thread:*do-ip4:
*do-ip6:
*do-udp:
*do-tcp:private-address:
forward-zone:
name:
forward-addr:*so-rcvbuf:
*so-sndbuf: -
Mark you should pastebin your resolver.log so we can have a look. It should have all the unbound error information.
Bryan, The resolver.log only has a few lines of DnsForwarder outputs in it currently. From what I've seen the SNMP log, already pastebinned, has everything the resolver file had in it. Otherwise, I'll have to get this to you after I try restarting Unbound. This may be after Wagonza updates the package unless it's needed sooner.
-
Could you give us the output of limits
Bryan, were you asking this of me? I tried logging into PfSense with WinScp, no go. Tried logging on as root instead of admin using same password and also no go. Using SCP port 22. Ftp and Tftp didn't help. Finally copy/pasted from SSH session.
Limits reports;
Resource limits (current):
cputime infinity secs
filesize infinity kB
datasize 33554432 kB
stacksize 524288 kB
coredumpsize infinity kB
memoryuse infinity kB
memorylocked infinity kB
maxprocesses 5547
openfiles 11095
sbsize infinity bytes
vmemoryuse infinity kB
pseudo-terminals infinity
swapuse infinity kB -
Mark you should pastebin your resolver.log so we can have a look. It should have all the unbound error information.
Bryan, The resolver.log only has a few lines of DnsForwarder outputs in it currently. From what I've seen the SNMP log, already pastebinned, has everything the resolver file had in it. Otherwise, I'll have to get this to you after I try restarting Unbound. This may be after Wagonza updates the package unless it's needed sooner.
I get stuff like this in my resolver.log after unbound is installed. Looks about the same so I suppose you are right but I was getting parse errors and other things when my service wouldn't start.
Mar 4 22:12:59 unbound: [24172:0] info: service stopped (unbound 1.4.21). Mar 4 22:12:59 unbound: [24172:0] info: server stats for thread 0: 0 queries, 0 answers from cache, 0 recursions, 0 prefetch Mar 4 22:12:59 unbound: [24172:0] info: server stats for thread 0: requestlist max 0 avg 0 exceeded 0 jostled 0 Mar 4 22:16:22 unbound: [44092:0] notice: init module 0: iterator Mar 4 22:16:22 unbound: [44092:0] info: start of service (unbound 1.4.21). Mar 4 22:18:20 unbound: [44092:0] info: service stopped (unbound 1.4.21). Mar 4 22:18:20 unbound: [44092:0] info: server stats for thread 0: 0 queries, 0 answers from cache, 0 recursions, 0 prefetch Mar 4 22:18:20 unbound: [44092:0] info: server stats for thread 0: requestlist max 0 avg 0 exceeded 0 jostled 0 Mar 4 22:18:26 unbound: [88348:0] notice: init module 0: iterator Mar 4 22:18:26 unbound: [88348:0] info: start of service (unbound 1.4.21). Mar 4 22:19:10 unbound: [88348:0] info: service stopped (unbound 1.4.21). Mar 4 22:19:10 unbound: [88348:0] info: server stats for thread 0: 0 queries, 0 answers from cache, 0 recursions, 0 prefetch Mar 4 22:19:10 unbound: [88348:0] info: server stats for thread 0: requestlist max 0 avg 0 exceeded 0 jostled 0 Mar 4 22:19:13 unbound: [12458:0] notice: init module 0: iterator Mar 4 22:19:13 unbound: [12458:0] info: start of service (unbound 1.4.21). Mar 4 22:19:13 unbound: [12458:0] info: 192.168.55.3 pfsense.localdomain. A IN Mar 4 22:19:13 unbound: [12458:0] info: 192.168.55.3 pfsense.localdomain.dev.localdomain. A IN Mar 4 22:19:17 unbound: [12458:0] info: 192.168.55.3 0.pfsense.pool.ntp.org. AAAA IN Mar 4 22:19:54 unbound: [12458:0] info: service stopped (unbound 1.4.21). Mar 4 22:19:54 unbound: [12458:0] info: server stats for thread 0: 3 queries, 0 answers from cache, 3 recursions, 0 prefetch Mar 4 22:19:54 unbound: [12458:0] info: server stats for thread 0: requestlist max 0 avg 0 exceeded 0 jostled 0 Mar 4 22:19:54 unbound: [12458:0] info: average recursion processing time 0.241851 sec Mar 4 22:19:54 unbound: [12458:0] info: histogram of recursion processing times Mar 4 22:19:54 unbound: [12458:0] info: [25%]=0 median[50%]=0 [75%]=0 Mar 4 22:19:54 unbound: [12458:0] info: lower(secs) upper(secs) recursions Mar 4 22:19:54 unbound: [12458:0] info: 0.065536 0.131072 1 Mar 4 22:19:54 unbound: [12458:0] info: 0.131072 0.262144 1 Mar 4 22:19:54 unbound: [12458:0] info: 0.262144 0.524288 1 Mar 4 22:20:01 unbound: [44628:0] notice: init module 0: iterator Mar 4 22:20:01 unbound: [44628:0] info: start of service (unbound 1.4.21). Mar 4 22:24:02 unbound: [44628:0] info: service stopped (unbound 1.4.21). Mar 4 22:24:02 unbound: [44628:0] info: server stats for thread 0: 0 queries, 0 answers from cache, 0 recursions, 0 prefetch Mar 4 22:24:02 unbound: [44628:0] info: server stats for thread 0: requestlist max 0 avg 0 exceeded 0 jostled 0 Mar 4 22:24:08 unbound: [15496:0] notice: init module 0: iterator Mar 4 22:24:08 unbound: [15496:0] info: start of service (unbound 1.4.21).
Could you give us the output of limits
Bryan, were you asking this of me? I tried logging into PfSense with WinScp, no go. Tried logging on as root instead of admin using same password and also no go. Using SCP port 22. Ftp and Tftp didn't help.
Winscp should just work the same as SSH. Just use defaults and root then password.
Finally copy/pasted from SSH session.
Limits reports;
Resource limits (current):
cputime infinity secs
filesize infinity kB
datasize 33554432 kB
stacksize 524288 kB
coredumpsize infinity kB
memoryuse infinity kB
memorylocked infinity kB
maxprocesses 5547
openfiles 11095
sbsize infinity bytes
vmemoryuse infinity kB
pseudo-terminals infinity
swapuse infinity kBYes exactly just ssh in and run the command. As you can see unbound is adjusting your openfiles. What is the output of this
sysctl -a | grep file
I can't reproduce unbound wanting to increase openfiles. Either way it looks like it is normal if it thinks it is going to run out. Are you willing to paste your whole unbound.conf? Guess it is probably not there still?
Can you also give us an full output of:
ps aux
-
Yes exactly just ssh in and run the command. As you can see unbound is adjusting your openfiles. What is the output of this
sysctl -a | grep fileI can't reproduce unbound wanting to increase open files. Either way it looks like it is normal if it thinks it is going to run out. Are you willing to paste your whole unbound.conf? Guess it is probably not there still?
Can you also give us an full output of:
ps aux@Bryan
SysCtl
http://pastebin.com/embed_js.php?i=DczvR9EqPsAux
http://pastebin.com/embed_js.php?i=sumekXMa -
I uninstalled/reinstalled the Unbound pkg last night at 9PM PST. Not sure if Wagonza's changes were a part of the pkg download. The PfSense server downloaded ver _21 again. Launched this morning after disabling the forwarder and all seems well except I couldn't log into forum.pfsense.org. I have in the DNS forwarder host overrides;
pfsense.com=66.219.34.171
files.pfsense.org=66.111.2.167If you recall adding hostnames was recommended earlier in this thread as a way to recover Unbound from the PfSense server when Unbound won't start.
So I thought well maybe I need to add;
pfsense.org=192.207.126.26I entered into the forwarder then disabled unbound then enabled the forwarder to populate the changes so Unbound could pick it up from the revised hosts file. Then I reversed the services never having both services enabled at the same time. It's at this time Unbound refused to start again and I had to revert back to the forwarder pointing to another Win box running Unbound.
Is this strike 3?
-
I uninstalled/reinstalled the Unbound pkg last night at 9PM PST. Not sure if Wagonza's changes were a part of the pkg download. The PfSense server downloaded ver _21 again. Launched this morning after disabling the forwarder and all seems well except I couldn't log into forum.pfsense.org. I have in the DNS forwarder host overrides;
pfsense.com=66.219.34.171
files.pfsense.org=66.111.2.167If you recall adding hostnames was recommended earlier in this thread as a way to recover Unbound from the PfSense server when Unbound won't start.
So I thought well maybe I need to add;
pfsense.org=192.207.126.26I entered into the forwarder then disabled unbound then enabled the forwarder to populate the changes so Unbound could pick it up from the revised hosts file. Then I reversed the services never having both services enabled at the same time. It's at this time Unbound refused to start again and I had to revert back to the forwarder pointing to another Win box running Unbound.
Is this strike 3?
Why would you start the forwarder again? There has not be something telling us why unbound is not starting. You gave us zero information. ps aux while trying to start unbound? resolver.log? What does it say when you try to start unbound manually? Are you sure DNS Forwarder is off before trying to restart unbound?
-
As I mentioned, I started the Forwarder again to populate the hosts that Unbound recognizes. Restarting Unbound didn't pick up the host override entered in the Forwarder. The PfSense docs I read said that Unbound gets it's overrides from the Forwarder entries. Once Unbound quit I had to hurriedly get the system back online. No time to collect ps.aux or logs. I'll schedule this again for 4AM instead of 7AM when I can tolerate more downtime and get what you ask for and when I can reboot the router to perhaps clear port 53 in the event a disabled Forwarder doesn't release the port as it should. I underlined so you don't misunderstand that I know this was why Unbound didn't start. I don't know at this time.
I did say this was an online system right? I won't bother posting again unless I can fully document the results.
Sorry for the bother… -
As I mentioned, I started the Forwarder again to populate the hosts that Unbound recognizes. Restarting Unbound didn't pick up the host override entered in the Forwarder.
No such thing needed. Frankly, with all your issues noone's able to reproduce, I'd recommend to flatten and rebuild the box from scratch.
-
As I mentioned, I started the Forwarder again to populate the hosts that Unbound recognizes. Restarting Unbound didn't pick up the host override entered in the Forwarder. The PfSense docs I read said that Unbound gets it's overrides from the Forwarder entries.
You do not need to turn on the DNS Forwarder service to get the host overrides to populate. Unbound reads them from the config.xml everytime you cause the unbound.conf to be generated.
- Add overrides
- Press save & apply changes
- Go to services unbound -> save config and it will regenerate config and restart
- Check unbound status tab to see your host overrides as local data records
I mean maybe wagonza can make it regen config on service restart would seem clearer. It has nothing to do with starting dnsmasq it just it shares the page with DNS Forwarder and is read by unbound. Below what I saw following steps above.
# DNSMasq Host overrides local-data-ptr: "192.168.55.7 test.fudzilla.com" local-data: "test.fudzilla.com IN A 192.168.55.7"
Pertinent code: Everything in hosts file has a function here and unbound does it when it generates its conf. no need for dnsmasq to be running.
function read_hosts() { // Open /etc/hosts and extract the only dhcpleases info $etc_hosts = array(); foreach (file('/etc/hosts') as $line) { $d = preg_split('/\s/', $line, -1, PREG_SPLIT_NO_EMPTY); if (empty($d) || substr(reset($d), 0, 1) == "#") continue; if ($d[3] == "#") { $ip = array_shift($d); $fqdn = array_shift($d); $name = array_shift($d); if ($fqdn != "empty") { if ($name != "empty") array_push($etc_hosts, array(ipaddr => "$ip", fqdn => "$fqdn", name => "$name")); else array_push($etc_hosts, array(ipaddr => "$ip", fqdn => "$fqdn")); } } } return $etc_hosts; } /* * Setup /etc/hosts entries by overriding with local-data */ function unbound_add_host_entries() { global $config; /* XXX: break this out into a separate config file and make use of include */ $unboundcfg = $config['installedpackages']['unbound']['config'][0]; $syscfg = $config['system']; $dnsmasqcfg = $config['dnsmasq']; $unbound_entries = "local-zone: \"{$syscfg['domain']}\" transparent\n"; // IPv4 entries $unbound_entries .= "local-data-ptr: \"127.0.0.1 localhost\"\n"; $unbound_entries .= "local-data: \"localhost A 127.0.0.1\"\n"; $unbound_entries .= "local-data: \"localhost.{$syscfg['domain']} A 127.0.0.1\"\n"; // IPv6 entries if(function_exists("is_ipaddrv6")) { $unbound_entries .= "local-data-ptr: \"::1 localhost\"\n"; $unbound_entries .= "local-data: \"localhost AAAA ::1\"\n"; $unbound_entries .= "local-data: \"localhost.{$syscfg['domain']} AAAA ::1\"\n"; } $added_item_v4 = array(); $added_item_v6 = array(); if ($config['interfaces']['lan']) { $current_host = $syscfg['hostname'].".".$syscfg['domain']; $cfgip = get_interface_ip("lan"); if (is_ipaddr($cfgip)) { $unbound_entries .= "local-data-ptr: \"{$cfgip} {$current_host}\"\n"; $unbound_entries .= "local-data: \"{$current_host} A {$cfgip}\"\n"; $unbound_entries .= "local-data: \"{$syscfg['hostname']} A {$cfgip}\"\n"; $added_item_v4[$current_host] = true; } $cfgip6 = get_interface_ipv6("lan"); if (is_ipaddrv6($cfgip6)) { $unbound_entries .= "local-data-ptr: \"{$cfgip6} {$current_host}\"\n"; $unbound_entries .= "local-data: \"{$current_host} AAAA {$cfgip6}\"\n"; $unbound_entries .= "local-data: \"{$syscfg['hostname']} AAAA {$cfgip6}\"\n"; $added_item_v6[$current_host] = true; } } else { $sysiflist = get_configured_interface_list(); foreach ($sysiflist as $sysif) { if (!interface_has_gateway($sysif)) { $current_host = $syscfg['hostname'].".".$syscfg['domain']; $cfgip = get_interface_ip($sysif); if (is_ipaddr($cfgip)) { $unbound_entries .= "local-data-ptr: \"{$cfgip} {$current_host}\"\n"; $unbound_entries .= "local-data: \"{$current_host} A {$cfgip}\"\n"; $unbound_entries .= "local-data: \"{$syscfg['hostname']} A {$cfgip}\"\n"; $added_item_v4[$current_host] = true; } $cfgip6 = get_interface_ipv6($sysif); if (is_ipaddr($cfgip6)) { $unbound_entries .= "local-data-ptr: \"{$cfgip6} {$current_host}\"\n"; $unbound_entries .= "local-data: \"{$current_host} AAAA {$cfgip6}\"\n"; $unbound_entries .= "local-data: \"{$syscfg['hostname']} AAAA {$cfgip6}\"\n"; $added_item_v6[$current_host] = true; } if (is_ipaddr($cfgip) || is_ipaddr($cfgip6)) break; } } } // DNSMasq entries static host entries if (isset($dnsmasqcfg['hosts'])) { $hosts = $dnsmasqcfg['hosts']; $host_entries = "# DNSMasq Host overrides\n"; $added_item = array(); foreach ($hosts as $host) { $current_host = ($host['host'] != "") ? $host['host'].".".$host['domain'] : $host['domain']; if (function_exists("is_ipaddrv6") && is_ipaddrv6($host['ip'])) { if (!$added_item_v6[$current_host]) { $host_entries .= "local-data-ptr: \"{$host['ip']} {$current_host}\"\n"; $host_entries .= "local-data: \"{$current_host} IN AAAA {$host['ip']}\"\n"; $added_item_v6[$current_host] = true; } if ($host['aliases']['item'] && is_array($host['aliases']['item'])) foreach ($host['aliases']['item'] as $alias) { $current_alias = ($alias['host'] != "") ? $alias['host'].".".$alias['domain'] : $alias['domain']; if (!$added_item_v6[$current_alias]) { $host_entries .= "local-data: \"{$current_alias} IN AAAA {$host['ip']}\"\n"; $added_item_v6[$current_alias] = true; if ((!$added_item[$current_alias]) && (!empty($alias['description'])) && ($unboundcfg['txtsupport'] == 'on')) { $host_entries .= "local-data: '{$current_alias} TXT \"".addslashes($alias['description'])."\"'\n"; $added_item[$current_alias] = true; } } } } else { if (!$added_item_v4[$current_host]) { $host_entries .= "local-data-ptr: \"{$host['ip']} {$current_host}\"\n"; $host_entries .= "local-data: \"{$current_host} IN A {$host['ip']}\"\n"; $added_item_v4[$current_host] = true; } if ($host['aliases']['item'] && is_array($host['aliases']['item'])) foreach ($host['aliases']['item'] as $alias) { $current_alias = ($alias['host'] != "") ? $alias['host'].".".$alias['domain'] : $alias['domain']; if (!$added_item_v4[$current_alias]) { $host_entries .= "local-data: \"{$current_alias} IN A {$host['ip']}\"\n"; $added_item_v4[$current_alias] = true; if ((!$added_item[$current_alias]) && (!empty($alias['description'])) && ($unboundcfg['txtsupport'] == 'on')) { $host_entries .= "local-data: '{$current_alias} TXT \"".addslashes($alias['description'])."\"'\n"; $added_item[$current_alias] = true; } } } } if ((!$added_item[$current_host]) && (!empty($host['descr'])) && ($unboundcfg['txtsupport'] == 'on')) { $host_entries .= "local-data: '{$current_host} TXT \"".addslashes($host['descr'])."\"'\n"; $added_item[$current_host] = true; } } $unbound_entries .= $host_entries; } // Static DHCP entries $host_entries = "# DHCP Reservations\n"; if (isset($unboundcfg['regdhcpstatic']) && is_array($config['dhcpd'])) { foreach ($config['dhcpd'] as $dhcpif => $dhcpifconf) if(is_array($dhcpifconf['staticmap']) && isset($dhcpifconf['enable'])) foreach ($dhcpifconf['staticmap'] as $host) if ($host['ipaddr'] && $host['hostname']) { $current_host = $host['hostname'].".".$syscfg['domain']; if (!$added_item_v4[$current_host]) { $host_entries .= "local-data-ptr: \"{$host['ipaddr']} {$current_host}\"\n"; $host_entries .= "local-data: \"{$current_host} IN A {$host['ipaddr']}\"\n"; $added_item_v4[$current_host] = true; if ((!$added_item[$current_host]) && (!empty($host['descr'])) && ($unboundcfg['txtsupport'] == 'on')) { $host_entries .= "local-data: '{$host['hostname']}.{$syscfg['domain']} TXT \"".addslashes($host['descr'])."\"'\n"; $added_item[$current_host] = true; } } } $unbound_entries .= $host_entries; } // Static DHCPv6 entries $host_entries = "# DHCPv6 reservations\n"; if (isset($unboundcfg['regdhcpstatic']) && is_array($config['dhcpdv6'])) { foreach ($config['dhcpdv6'] as $dhcpif => $dhcpifconf) if(is_array($dhcpifconf['staticmap']) && isset($dhcpifconf['enable'])) foreach ($dhcpifconf['staticmap'] as $host) if ($host['ipaddrv6'] && $host['hostname']) { $current_host = $host['hostname'].".".$syscfg['domain']; if (!$added_item_v6[$current_host]) { $host_entries .= "local-data-ptr: \"{$host['ipaddrv6']} {$current_host}\"\n"; $host_entries .= "local-data: \"{$current_host} IN AAAA {$host['ipaddrv6']}\"\n"; $added_item_v6[$current_host] = true; if ((!$added_item[$current_host]) && (!empty($host['descr'])) && ($unboundcfg['txtsupport'] == 'on')) { $host_entries .= "local-data: '{$host['hostname']}.{$syscfg['domain']} TXT \"".addslashes($host['descr'])."\"'\n"; $added_item[$current_host] = true; } } } $unbound_entries .= $host_entries; } // Handle DHCPLeases added host entries $dhcplcfg = read_hosts(); $host_entries = "# /etc/hosts entries\n"; if(is_array($dhcplcfg)) { foreach($dhcplcfg as $key=>$host) { $current_host = $host['fqdn']; if (function_exists("is_ipaddrv6") && is_ipaddrv6($host['ipaddr'])) { if (!$added_item_v6[$current_host]) { $host_entries .= "local-data-ptr: \"{$host['ipaddr']} {$host['fqdn']}\"\n"; $host_entries .= "local-data: \"{$host['fqdn']} IN AAAA {$host['ipaddr']}\"\n"; $added_item_v6[$current_host] = true; if ((!empty($host['name'])) && (!$added_item_v6[$host['name']])) { $host_entries .= "local-data-ptr: \"{$host['ipaddr']} {$host['name']}\"\n"; $host_entries .= "local-data: \"{$host['name']} IN AAAA {$host['ipaddr']}\"\n"; $added_item_v6[$host['name']] = true; } } } else { if (!$added_item_v4[$current_host]) { $host_entries .= "local-data-ptr: \"{$host['ipaddr']} {$host['fqdn']}\"\n"; $host_entries .= "local-data: \"{$host['fqdn']} IN A {$host['ipaddr']}\"\n"; $added_item_v4[$current_host] = true; if ((!empty($host['name'])) && (!$added_item_v4[$host['name']])) { $host_entries .= "local-data-ptr: \"{$host['ipaddr']} {$host['name']}\"\n"; $host_entries .= "local-data: \"{$host['name']} IN A {$host['ipaddr']}\"\n"; $added_item_v4[$host['name']] = true; } } } } $unbound_entries .= $host_entries; } return $unbound_entries; } /* Setup any domain overrides that have been configured with stub-zone parameter */ function unbound_add_domain_overrides($pvt=false) { global $config; if (isset($config['dnsmasq']['domainoverrides'])) { $domains = $config['dnsmasq']['domainoverrides']; // Domain overrides that have multiple entries need multiple stub-addr: added $sorted_domains = msort($domains, "domain"); $result = array(); foreach($sorted_domains as $domain) { $domain_key = current($domain); if (!isset($result[$domain_key])) $result[$domain_key] = array(); $result[$domain_key][] = $domain['ip']; } $domain_entries = ""; foreach($result as $domain=>$ips) { if ($pvt == true) { if (strpos($domain, "in-addr.arpa") !== false) $domain_entries .= "local-zone: \"$domain\" transparent\n"; else $domain_entries .= "private-domain: \"$domain\"\n"; if (isset($config['installedpackages']['unbound']['config'][0]['dnssec_status'])) $domain_entries .= "domain-insecure: \"$domain\"\n"; } else { $domain_entries .= "stub-zone:\n"; $domain_entries .= "\tname: \"$domain\"\n"; foreach($ips as $ip) $domain_entries .= "\tstub-addr: $ip\n"; $domain_entries .= "\tstub-prime: no\n"; } } return $domain_entries; } }
Once Unbound quit I had to hurriedly get the system back online. No time to collect ps.aux or logs. I'll schedule this again for 4AM instead of 7AM when I can tolerate more downtime and get what you ask for and when I can reboot the router to perhaps clear port 53 in the event a disabled Forwarder doesn't release the port as it should. I underlined so you don't misunderstand that I know this was why Unbound didn't start. I don't know at this time.
Yes it is probably the reason. I understand circumstances but having a second SSH open or third to launch a couple commands wouldn't hurt in the future as you are the one that needs the information after all ;)
I did say this was an online system right? I won't bother posting again unless I can fully document the results.
Sorry for the bother…I understand production. No hostility meant. It isn't the clearest process or setup in the GUI ever. That is the problem with abstracting things into a GUI sometimes stuff gets meshed together and without looking at code you have no idea what is going on.
No such thing needed. Frankly, with all your issues noone's able to reproduce, I'd recommend to flatten and rebuild the box from scratch.
Don't think there is a need for this. I think it is just a misunderstanding. Think we have about everything covered now. Wagonza has fixed the conf generation clause order problem for custom options and probably this current problem as markn62 has noted that it was probably something to do with DNS Forwarder still running or something.
-
I uninstalled/reinstalled the Unbound pkg last night at 9PM PST. Not sure if Wagonza's changes were a part of the pkg download. The PfSense server downloaded ver _21 again.
Nope not yet.
Launched this morning after disabling the forwarder and all seems well except I couldn't log into forum.pfsense.org. I have in the DNS forwarder host overrides;
pfsense.com=66.219.34.171
files.pfsense.org=66.111.2.167If you recall adding hostnames was recommended earlier in this thread as a way to recover Unbound from the PfSense server when Unbound won't start.
So I thought well maybe I need to add;
pfsense.org=192.207.126.26I entered into the forwarder then disabled unbound then enabled the forwarder to populate the changes so Unbound could pick it up from the revised hosts file. Then I reversed the services never having both services enabled at the same time. It's at this time Unbound refused to start again and I had to revert back to the forwarder pointing to another Win box running Unbound.
Is this strike 3?
You shouldn't need to do any of the above.