Syntax for bulk adding Domain Overrides to DNS Resolver

[RickyBaker]

Hello, After solving one of my problems thanks to this fantastic community I'm ready to move on to the next.

I need to add a good number of Domain Overrides to the DNS Resolver and I found this thread pertaining to Host Overrides (https://forum.pfsense.org/index.php?topic=86986.30) so I think i know WHERE to do it (putting them line by line into the "Custom Options" text box after clicking "Show Custom Options"?), but I'm not 100% clear on the syntax.

This is the example I see for Host Overrides:

local-data: "click01.aditic.net A 10.10.10.1";
local-data: "click02.aditic.net A 10.10.10.2";

but I'm really not sure what it would be for Domain Overrides (or if it's even different).  To be perfectly honest, the "A" and the local-data aspects of this are mysteries to me.  Thanks in advance!

[KOM]

I'm not sure you can use the custom options in that way to add those hosts.  However, you could use the Diagnostics - Backup/Restore to create just a DNS Resolver backup XML file and then edit that and re-import it.  Here is an example if I add your first override:

 <unbound><enable></enable>

	<active_interface></active_interface>
	<outgoing_interface></outgoing_interface>

	 <hosts><host>click01</host>
		<domain>aditic.net</domain>
		<ip>10.10.10.1</ip>

		<aliases></aliases></hosts></unbound> 

To be perfectly honest, the "A" and the local-data aspects of this are mysteries to me.

The A denotes that this is a DNS A Record.

[RickyBaker]

Thanks for the tips.  Though I'm internally debating whether backing up and editing an xml file is really, all that much faster than just adding them through the GUI.  As a clarification, I'm looking to add DOMAIN overrides.  How would that change your example (if at all)?

[KOM]

How would that change your example (if at all)?

 <unbound>...

    <domainoverrides><domain>facebook.com</domain>
        <ip>10.0.0.1</ip></domainoverrides></unbound> 

[johnpoz]

"I need to add a good number of Domain Overrides to the DNS Resolver"

How many?  And why?  Are these actual domains that are resolved by some downstream local NS?  Or are you trying to block access to specific domains?

[doktornotor]

@johnpoz:

How many?  And why?  Are these actual domains that are resolved by some downstream local NS?  Or are you trying to block access to specific domains?

Reinventing pfBNG I guess :D

[johnpoz]

^ Yeah that is why I ask..

[RickyBaker]

@johnpoz:

"I need to add a good number of Domain Overrides to the DNS Resolver"

How many?  And why?  Are these actual domains that are resolved by some downstream local NS?  Or are you trying to block access to specific domains?

~180 for now. I have a program or 2 that I'd prefer not update automatically through conventional means.

Why? should I be checking out pfBlockerNG?

[doktornotor]

And you need 180 domain overrides to stop the shit from automatic updates? That program being W10, or what?

[johnpoz]

So you want to block them.. Then that is easy..

in your advanced box
server:
include: /etc/unbound_blockstuff

In in the file unbound_blockstuff
local-zone: "domain.com" redirect
local-data: "domain.com A 127.0.0.1"
local-zone: "otherdomain.org" redirect
local-data: "otherdomain.org A 127.0.0.1"

you could use 0.0.0.0 if you like that better, add as many domains as you want here..

You could put all the domains in the custom box directly.. But 180 is a lot!!

[KOM]

You could put all the domains in the custom box directly.

Thanks, I did not know that.

[RickyBaker]

@doktornotor:

And you need 180 domain overrides to stop the shit from automatic updates? That program being W10, or what?

I used to use Little Snitch, but that is annoying and stopped working with the last Mac OSX update.  180 is from a curated list (that admits it may be overkill) that I found on a reddit thread very specifically for this suite of programs…

@johnpoz:

So you want to block them.. Then that is easy..

in your advanced box
server:
include: /etc/unbound_blockstuff

In in the file unbound_blockstuff
local-zone: "domain.com" redirect
local-data: "domain.com A 127.0.0.1"
local-zone: "otherdomain.org" redirect
local-data: "otherdomain.org A 127.0.0.1"

you could use 0.0.0.0 if you like that better, add as many domains as you want here..

You could put all the domains in the custom box directly.. But 180 is a lot!!

Great thanks so much! I'll report back when I test it out!

[johnpoz]

so curious are these domains all actual domains, or subdomains of parent.  180 domains for phone home seems a bit much.. I am guessing they are just all hosts/subs off a parent or a couple of parents.

so you have

something.domain.tld, and somethingelse.domain.tld or something.something.domain.tld, etc..

If that is the case then you really only need 1 redirect that anything.domain.tld would return loopback or 0.0.0.0

That some device would use 180 actual different domains trying to update just seems nuts..

[RickyBaker]

@johnpoz:

you could use 0.0.0.0 if you like that better, add as many domains as you want here..

You could put all the domains in the custom box directly.. But 180 is a lot!!

I've read that 0.0.0.0 is better cause it avoids a windows issue with avoids the slowdown issue with the TCP loopback interface on later version of windows.  Is there anything special I need to add to the file to use 0.0.0.0?

edit: Sorry! didn't see your response!  As I was going through the list and adding it to a txt file like you described I noticed a few reduncancies.  Such as www.example.*, then 8 more entries like www.example.com, www.example.de, www.example.ntp etc.  but the vast majority of them are more in line with different numbered name servers, like ns1.example.com, ns2.example.com.  I think the list is actually meant to block the phone home but still allow the user to visit the core example.com.  That is pure speculation and yes it does seem excessive, but when I did run Little Snitch there were still a ton of rules that adding wildcards couldn't capture fully (fwiw)

[johnpoz]

yeah if they are using say country codes for the tld, you would need to put those all in, example.de, example.us, example.nl, etc.. or .org, .com, .net etc..

Sure there is debate which is better 127.x or 0.0.0.0 comes down to what is the client and what it actually does - its possible that 0.0.0.0 could be some ms faster.. So sure use that if not working on specific client or causing you problems on client then use the old 127.0.0.1 trick, etc.

[RickyBaker]

@johnpoz:

yeah if they are using say country codes for the tld, you would need to put those all in, example.de, example.us, example.nl, etc.. or .org, .com, .net etc..

Sure there is debate which is better 127.x or 0.0.0.0 comes down to what is the client and what it actually does - its possible that 0.0.0.0 could be some ms faster.. So sure use that if not working on specific client or causing you problems on client then use the old 127.0.0.1 trick, etc.

But nothing special to add to this or the client's host files?  I suppose its easy enough to replace all on the text file so I'll give it a spin tonight, just wasn't sure if 0.0.0.0 required some special finagling.

[johnpoz]

nope nothing to do on the clients, as long as they are using pfsense for their dns.

You can use 0.0.0.0 just like you would the loopback 127.0.0.1 in entry you put in the file.  Up to you which one you like better. I have used both in the past.  Been using the 0.0.0.0 as of late but really have noticed any sort of difference in either.

[RickyBaker]

haha well it worked! Cause I couldn't access a few of the domains that I purposely blocked….but somehow I also blocked the Google Name Servers and couldn't even do a normal search :o  I guess that's the risk of creating a text file of hundreds of domains from a curated list you didn't make :). Maybe I'll start smaller.  Or maybe I'll check out this PFBlockerNG...

[johnpoz]

google ns would be

;; QUESTION SECTION:
;google.com.                    IN      NS

;; ANSWER SECTION:
google.com.            345600  IN      NS      ns4.google.com.
google.com.            345600  IN      NS      ns2.google.com.
google.com.            345600  IN      NS      ns3.google.com.
google.com.            345600  IN      NS      ns1.google.com.

;; ADDITIONAL SECTION:
ns4.google.com.        345600  IN      A      216.239.38.10
ns2.google.com.        345600  IN      A      216.239.34.10
ns3.google.com.        345600  IN      A      216.239.36.10
ns1.google.com.        345600  IN      A      216.239.32.10

Yeah blocking those would block access to google ;)

[RickyBaker]

I can't say I totally understand what you posted there, but I didn't input any of those domains into the text file.  There are a number of IP address "domains" but none of them start with 216.x…and I (hope I) wouldn't be dumb enough to put a domain with the word google in there:)

[RickyBaker]

Soooo i pared down the list extensively and when I applied the changes, google's name servers were still blocked by pfsense.  Here's the list from the unbound_domains2block txt file, with names changed to protect the innocent:

local-zone: "*.licenses.example.com" redirect
local-data: "*.licenses.example.com A 0.0.0.0"
local-zone: "*.examplelogin.com" redirect
local-data: "*.examplelogin.com A 0.0.0.0"
local-zone: "ims-na1.examplelogin.com" redirect
local-data: "ims-na1.examplelogin.com A 0.0.0.0"
local-zone: "ims-prod06.examplelogin.com" redirect
local-data: "ims-prod06.examplelogin.com A 0.0.0.0"
local-zone: "ims-prod07.examplelogin.com" redirect
local-data: "ims-prod07.examplelogin.com A 0.0.0.0"
local-zone: "exampleid-na1.services.example.com" redirect
local-data: "exampleid-na1.services.example.com A 0.0.0.0"
local-zone: "na1e-acc.services.example.com" redirect
local-data: "na1e-acc.services.example.com A 0.0.0.0"
local-zone: "na1r.services.example.com" redirect
local-data: "na1r.services.example.com A 0.0.0.0"
local-zone: "ems.example.com" redirect
local-data: "ems.example.com A 0.0.0.0"

Any idea why any of these would block google's name servers?

[RickyBaker]

Realizing that I had a much more managable set of domains to enter into the default gui I decided to try these domains out one by one.  I was able to input all but the 2 at the top with the wildcard and the Google nameservers went unblocked.  The ones with the asterix gave this error:

The following input errors were detected:
A valid domain must be specified.

Now maybe the issue with this new paired down list was the asterix, and it wasn't just Google's name servers that were being blocked but maybe EVERYTHING?  However, looking at my old 180 domain long list, there are no entries that begin with a wild card, but a few that end it one.  However, Testing this pared down version without asterixes still blocks google's name servers.

Finally, I also tested with the redirect to 127.0.0.1 with the same results as 0.0.0.0

So I'm still curious why using the domains in a text file like you suggested @johnpoz is blocking Google nameservers (and possibly everything?) and also curious if I can use a wildcard symbol for domains in the default UI to accomplish "*.licenses.example.com"?