Import host override list into forwarder

[markn62]

Is there a faster way than the GUI to populate host overrides into the forwarder?  I have about 170 entries making the GUI method rather tedious.  I tried editing the dnsmasq section of a current backup of config.xml, also tedious, then restored just the "DNS Forwarder section" of the xml which after a reboot resulted in unbound not starting due to the following error;

Unbound: error: error parsing local-data at 19 'host IN A x.x.x.x(unresolved)': Syntax error, could not parse the RR's type.

No syntax errors in the xml that I can see.  Any suggestions appreciated.

[johnpoz]

I am not sure about the resolver that is new to 2.2, but I just tested using the forwarder in 2.2 and this still works like it did in 2.1

create a file, I created /etc/extra, put your records in there.  Then in advanced section of the forwarder put

addn-hosts=/etc/extra

Restart the forwarder then those will resolve

[2.2-RC][root@pfSense.local.lan]/etc: cat /etc/extra
192.168.1.14 test.local.lan
192.168.1.15 test2.local.lan
192.168.1.16 test.other.lan
[2.2-RC][root@pfSense.local.lan]/etc: host test.other.lan
test.other.lan has address 192.168.1.16
[2.2-RC][root@pfSense.local.lan]/etc: host test.local.lan
test.local.lan has address 192.168.1.14
[2.2-RC][root@pfSense.local.lan]/etc: host test2.local.lan
test2.local.lan has address 192.168.1.15
[2.2-RC][root@pfSense.local.lan]/etc:

Along with anything that is also in the host over ride second of the forwarder.  I am not sure if there is command for unbound to do the same sort of thing?  Or if you can run the forwarder and resolver at the same time.  You mention forwarder but then you point to unbound error?  Unbound is used when resolver is used.

But this is a quick and dirty easy way to load as many host overides you would want to load

edit:  If your using the resolver, in the advanced box you can add lines like this

server:
local-data: "click01.aditic.net A 10.10.10.1"
local-data: "click02.aditic.net A 10.10.10.2"

For hosts you need vs putting in host override gui interface, that should allow you to post in lots of Hosts you want to resolve to local IPs.

[2.2-RC][root@pfSense.local.lan]/root: host click01.aditic.net
click01.aditic.net has address 10.10.10.1
[2.2-RC][root@pfSense.local.lan]/root: host click02.aditic.net
click02.aditic.net has address 10.10.10.2

[markn62]

Thanks John for your suggestion.  This looks like just what I need but can't make it work. I put my space(s) delimited file as /etc/hostscustom then gave this file the same properties as the hosts file 0755 if I recall.  Then I enabled the forwarder but could not ping by hostname the entries in the file.  This of course took down unbound that had to be restarted.  Do I need to take any extra spaces between the IP and the HOSTNAME?  The hostscustom file is formated below;

AP14SE 192.168.x.1
AP15NW 192.168.x.2
AP16 192.168.x.3

I'm fairly certain unbound takes it's hosts from the forwarder even if it's not enabled.  I thought by enabling/disabling the forwarder it would create a file that unbound uses.  Didn't seem too.  Do you know what location/file the forwarder puts it's host overrides so I can check that it's getting this far?

[johnpoz]

your format is backwards.

IP hostname

example.

192.168.1.14 test.local.lan
192.168.1.15 test2.local.lan
192.168.1.16 test.other.lan

Stuff that you put in the overrides is put into the actual /etc/hosts file

If your using the actual resolver, then you can put like this in the advanced box

server:
local-data: "click01.aditic.net A 10.10.10.1"
local-data: "click02.aditic.net A 10.10.10.2"

You can also store the those in .conf file here

/var/unbound/

[markn62]

Ok, I reversed the IPs/Hostnames.  Stopped Unbound service, started DNS Forwarder, and no additions were made to the /etc/hosts file.  I believe if I put them in the unbound configuration file it will get blown out on reboot will it not?  The Unbound Dns Advanced Settings has a Custom Settings box but I've found this to only accept a small command subset of a full unbound install so I'm not confident this area is where they should be dropped.

[doktornotor]

@markn62:

no additions were made to the /etc/hosts file

This is absolutely NOT how it works. Also, you are using completely wrong format if working with unbound (resolver) instead of dnsmasq (forwarder). Re-read the post right above yours.

[markn62]

Ya, for one, the custom hostnames are stored in /usr/pbi/unbound-amd64/etc/unbound/unbound.conf. Has overrides, host entries, dhcp reservations, etc.  Not a place to edit I'm fairly sure.  Unbound picks up whatever is in the dns forwarder host override section.  What is the easiest place to modify the host overrides where unbound will use them?

[doktornotor]

I give up. Kindly upgrade to 2.2 and only use the GUI for both forwarder and resolver overrides.

[markn62]

You jump in with one comment then give up?  The stamina…  ;-)
I'm on what http://updates.pfsense.org/_updaters/amd64 says is the latest PfSense upgrade 2.1.5-RELEASE (amd64)
built on Mon Aug 25 07:44:45 EDT 2014.  Where is 2.2?

[doktornotor]

You'd better wait till it's officially released if unable to find the snapshots.

[markn62]

I'll certainly wait for the official release but waiting for a solution to this is not in my DNA.  If I can't find the solution here I google til I find it.  Looks by my last post it was just about an hour.

I may have found the proper syntax for the dns forwarder advanced box.  Should be; address=/host.domain/x.x.x.x per http://thomasloughlin.com/pfsense-dnsmasq-advanced-setup/

The forwarder accepts the syntax. I'll see if unbound picks it up after a cron reboot.

[johnpoz]

Why would unbound pick up stuff you put in the forwarder section??  They are 2 different systems..

Notice how stuff you put in resolver overrides is not listed in the forwarder gui..

What are you using the forwarder or the resolver??  Are you trying to use both at the same time?  They can not run on the same port or IP..

What I can tell you for fact, is stuff you put in the host over rides is for sure placed into the /etc/hosts file - see attached.

What are you not understanding about the info given.. I have shown you how to import stuff be it the forwarder or the resolver.. What can I do to make it clearer for you - do you need more pictures?

"What is the easiest place to modify the host overrides where unbound will use them?"

As already stated you can paste them in the advanced box in the resolver section per my examples above, or you can put those entries in a .conf file in the folder where unbound picks up .conf files if you don't want to paste them into the advanced box.

If you want to use the resolver, then point to a file per my example and use the correct format IP name in your file.

[markn62]

John,
Are you serious?  What are you calling a resolver override?  Perhaps we have a terminology misunderstanding.  In the ver 2.1.5 GUI there is only forwarder host overrides, forwarder advanced, and unbound dsn advanced settings - custom options.

Dok says only use the GUI and you say otherwise yet you don't challenge Dok's suggestion.  And I'm finding your info repeatedly flawed.  Firstly the /etc/hosts file only has two entries, 127.x and my lan ip.  It is not picking up anything from the forwarder overrides or advanced.  Secondly, your suggestion to use addn-hosts=/etc/extra in the forwarder advanced produces an error at the top of this GUI page so referencing a custom hosts file is not do-able even though you insist it does.  It's not a file name or content issue, its a syntax issue.

So yes, I have tried all your suggestions with failure.  I get it and don't need pictures.  Anyone who has used PfSense for any time knows the unbound package uses the forwarder host overrides. You only need to cat /usr/pbi/unbound-amd64/etc/unbound/unbound.conf to understand this. And the forwarder doesn't have to be enabled for this to happen.  Certainly I'm not using the forwarder and unbound together.  Again if you've used PfSense for any time you would know enabling the forwarder will stop the unbound service so it's not possible to run concurrently, so why ask if I am?

I did determine after this AM's cron reboot that the forwarder advanced entry of "address=/host.domain/x.x.x.x doesn't get picked up by unbound even though the forwarder doesn't choke on it.  So still looking for a practical solution to entering a couple hundred private host overrides so nTopNG can DNS resolve the private IP's.  So far only the forwarder override entries and the DHCP static leases are making it into the unbound.conf file.

[doktornotor]

@markn62:

Anyone who has used PfSense for any time knows the unbound package uses the forwarder host overrides.

Not any more on 2.2 since it is not a package in the first place there. Stop sticking the overrides to obsolete places that were not intended for this anyway. If you want overrides for unbound, then kindly use the already suggested proper method above, or simply install 2.2 and use the GUI.

[johnpoz]

^ exactly.. What version are you on?  Are you still using 2.1.5?  With unbound package?  2.2 integrated unbound as the resolver.

Do you want me to remote in and set it up for you?  I can not help you if your trying to use unbound as a package and leverage the forwarder overrides, or how that used the forwarder stuff?  I never used the package other than some quick look at it.  But if your on 2.2 the info I have given is valid and tested on my own system and works as I have described.

If your using the unbound package, you should be able to put the entries as I have shown in a .conf file in the directory where unbound loads its .conf and have it load in whatever hosts you need.

I can always fire up my 2.1.5 vm and install the package and validate that.

[doktornotor]

The advanced config works the same with the 2.1.x package, except that you need trailing ; on each advanced config line, IIRC.

[johnpoz]

Yup and its in a different tab under the package gui page unbound advanced dns settings at the bottom there is custom box

So I just validated this works, you don't need server: in the package.  Not sure the package loads .conf files like the 2.2 integration does.

This is off 2.1.5 i386 vm.. Turned off forwarder, enabled unbound package.

[markn62]

Ahhh, so while I've been testing DNS Forwarder advanced entries you all have been talking Unbound Dns Advanced Settings.  And to not add a ; after address=/host.domain/x.x.x.x but after each line using syntax like so:

local-data: "click01.aditic.net A 10.10.10.1";
local-data: "click02.aditic.net A 10.10.10.2";

No wonder that above gave an error in the DNS Forwarder.  I've already been using this area with entries:
forward-zone:;name: ".";
forward-addr: 8.8.8.8;
forward-addr: 8.8.4.4;
forward-addr: 4.2.2.1;

And yes I'm still running version 2.1.5.  If I knew 2.2 was stable, and how to upgrade to it differently than using the firmware update GUI page, I might be inclined too.  Not sure why I'm getting dogged for not using a beta version.  This is a production box I can't just fiddle with anytime I wish.

Thanks for clarifying this.  Figured we had to be miscommunicating cause no suggestions seemed to work. I'll try Unbound Dns Advanced Settings - Custom Options entries later today, out of time this AM.

[johnpoz]

Here is the thing in your first post you stated forwarder, but then said there were unbound errors.  I asked for clarification right off the bat were you using the "forwarder" or unbound - with you saying unbound I also maybe in error assumed you were on 2.2.  Which is when unbound was integrated since you made no mention of using the "package"

IMHO 2.2 is stable - there are 0 bugs left that I am aware of, and been RC for quite some time.  It could drop final any day I would think.  To upgrade too it all you have to do is grab snap.  http://snapshots.pfsense.org/  There has not been a update since the 16, which also points to final being any day now ;)  Normally snaps are produced like twice a day.

[markn62]

Well I had a response but got snagged by the login timeout and forgot to copy/paste my post before submitting.  So I'll just apologize for assumptions made and the long thread.