Slow DNS Lookup with internal domain name appended to DNS Lookup! [SOLVED]



  • I have a very odd DNS Lookup problem making for failed lookups.
    I have done extensive searching on the pfsense forum and internet.
    My setup is as follows:
    A new install of Pfsense 2.4.2-RELEASE-p1 in a Netgate SG-1000 behind a Cisco EPC3925 Modem/Router with two Bind DNS Servers as Authority for my local network and Caching Forward Server too.

    My DNS lookup from browsers on my home network is intermitently slow. One moment it gives a 'Server Not Found', and then the next it is nice and snappy.

    I have noticed whilst looking at the dialog between the DNS Server running Ubuntu and my Desktop machine running Gentoo an intermittent NXDomain* error and ServFail error. My desktop is xxx.xxx.xxx.3 and the server is xxx.xxx.xxx.220 and my local network mynetwork.local
    The names have been changed to protect the innocent!

    Here is an extract:

    tcpdump -n host xxx.xxx.xxx.220
    
    17:06:55.009278 IP xxx.xxx.xxx.3.33832 > xxx.xxx.xxx.220.53: 35756+ A? forum.pfsense.org. (35)
    17:07:00.014388 IP xxx.xxx.xxx.3.37626 > xxx.xxx.xxx.220.53: 17406+ A? forum.pfsense.org.mynetwork.local. (46)
    17:07:00.014777 IP xxx.xxx.xxx.220.53 > xxx.xxx.xxx.3.37626: 17406 NXDomain* 0/1/0 (96)
    17:07:00.014975 IP xxx.xxx.xxx.3.58573 > xxx.xxx.xxx.220.53: 37792+ A? forum.pfsense.org. (35)
    17:07:03.355614 IP xxx.xxx.xxx.220.53 > xxx.xxx.xxx.3.40010: 39367 ServFail 0/0/0 (42)
    
    

    Well Well Well….
    Why is my local domain address added to the DNS lookup?

    My pfsense is a new install on a Netgate SG-1000 behind a Cisco EPC3925 Modem/Router.
    I have the latest db.root file.

    My named.conf.options file is as follows:

    acl "trusted" {
    	xxx.xxx.xxx.0/24;
    	127.0.0.1;
    };
    
    options {
    	directory "/var/cache/bind";
    	version "Not disclosed";
    	recursion yes;
    	allow-query { trusted; };
    	allow-recursion { trusted; };
    	allow-query-cache { trusted; };
    	allow-transfer { trusted; };
    	forwarders {
    		// same as those on the WAN on the Cisco Router
                    8.8.8.8; 212.113.0.3;
            };
    	dnssec-enable no;
    	auth-nxdomain no;    # conform to RFC1035
    	listen-on port 53 { 127.0.0.1; xxx.xxx.xxx.220; } ;
    	listen-on-v6 		{ none; };
    };
    

    and the start of my foreward zone file

    $TTL 604800	; 1 week
    $ORIGIN mynetwork.local.
    @ 		IN SOA	dnsserver.mynetwork.local. admin.mynetwork.local. (
    				2018030901 ; serial
    				604800     ; refresh (1 week)
    				86400      ; retry (1 day)
    				2419200    ; expire (4 weeks)
    				604800     ; minimum (1 week)
    				)
    			NS	ns1.mynetwork.local.
    			NS	ns2.mynetwork.local.
    			A	xxx.xxx.xxx.220
    			MX	10 mail.mynetwork.local.
    

    Is there something in the router adding the mynetwork.local to the query or an error of some sort in my Bind files?
    I have even setup the pfsense router with DNS Resolver and DNS Forwarder but I still have the same problem?

    Many thanks


  • Rebel Alliance Global Moderator

    Search suffix is the client.. This is really common and standard..

    Notice the client waiting like 5 seconds before it sends the query with the added search suffix… I wouldn't worry about a search suffix being added by your client, what I would look info is why your unable to resolve forum.pfsense.org



  • OK, that is helpful, perhaps it is something to do with the Firewall on the pfsense internal router.
    Here are my settings

    Firewall / NAT / Port Forward

    Interface 	Protocol 	Source Address 	Source Ports 	Dest. Address 	Dest. Ports NAT IP 	NAT Ports 	Description
    WAN		TCP		*		*		WAN address	xxxxx	    xx.xx.xx.xx	(SSH)
    WAN		TCP		*		*		WAN address	xxxxx	    xx.xx.xx.xx	(HTTPS)
    WAN		TCP		*		*		WAN address	xxxxx	    xx.xx.xx.xx	(IMAP/S)
    WAN		TCP		*		*		WAN address	xxxxx	    xx.xx.xx.xx	(SMTP/S)
    
    

    Firewall / NAT / 1:1

    Nothing
    

    Firewall / NAT / Outbound

    Outbound NAT Mode	- Automatic outbount NAT
    Mappings		- none
    Automatic Rules		
    Interface Source 			 	Source Port Destination	Dest. Port  NAT Address NAT Port Static Port	Description
    WAN 	127.0.0.0/8 xx.xx.xx.0/24 xx.xx.xx.0/24 	* 	* 	500 	    WAN address * 	 V		Auto created rule for ISAKMP
    WAN 	127.0.0.0/8 xx.xx.xx.0/24 xx.xx.xx.0/24 	* 	* 	* 	    WAN address	* 	 V		Auto created rule 
    

    Firewall / NAT / NPt

    Nothing
    

    Firewall / Rules / WAN

    States     Protocol 	Source 		Port 	Destination 	Port 	Gateway	Queue 	Schedule 	Description 	Actions
    0 /114 KiB	* 	Reserved
    		Not assigned by IANA 	* 	* 		* 	* 	* 			Block bogon networks 	
    0 /0 B	IPv4 TCP 	* 		* 	xx.xx.xx.210 	(SSH)	* 	none 	  		NAT ssh to xendom3 	
    1 /22.07 MiB IPv4 TCP 	* 		* 	xx.xx.xx.211 	(HTTPS) * 	none 	  		NAT https to xenzimbra3 
    0 /0 B	IPv4 TCP 	* 		* 	xx.xx.xx.211 	(IMAP/S) * 	none 	  		NAT IMAP to xenzimbra3 	
    0 /0 B	IPv4 TCP 	* 		* 	xx.xx.xx.211 	(SMTP/S) * 	none 	  		NAT IMAP SSMTP to xenzimbra3 0 /0 B	IPv4 UDP 	* 		* 	WAN address 	(OpenVPN)*	none 	  		OpenVPN Remote Connection wizard 
    

    Firewall / Rules / LAN

    States 	    Protocol 	Source 	Port 	                Destination 	Port 	    Gateway 	Queue 	Schedule 	Description 	Actions
    2 /43.48 MiB	* 	    * 	    * 	                    LAN Address 	443,80,22 	* 	        * 		            Anti-Lockout Rule 	
    0 /85 KiB	    * 	Reserved    Not assigned by IANA 	* 	            * 	        * 	        * 	    * 		    Block bogon networks 	
    266 /2.76 GiB	IPv4    * 	    LAN net 	            * 	            * 	        * 	        * 	    none 	  	Default allow LAN to any rule 	
    0 /0 B	        IPv6    * 	    LAN net 	            * 	            * 	        * 	        * 	    none 	  	Default allow LAN IPv6 to any rule 
    
    

    I do not have OpenVPN working at the moment.
    Should I have something to allow the DNS queries through, surely this is covered by letting everything from the LAN through as it is.


  • Rebel Alliance Global Moderator

    Dude your hiding your rfc1918 address space???

    And your blocking bogon on lan?  Really - how do you think that ever going to come into play?

    What does any of that have to do with pfsense resolving something… What happens when you diag, dns lookup?

    "with two Bind DNS Servers as Authority for my local network and Caching Forward Server too."  Your Cisco is router so its also natting?  Where do you clients point for dns if you have local dns..

    Where does pfsense point for dns?  It out of the box will resolve from roots.. Did you point it to some bind your running?  How do they resolve?

    None of that stuff you posted has to do with how dns resolves or doesn't... Other than you seem to think you need to hide rfc1918 space??  Really dude - think for 2 seconds.. How would I knowing you use 192.168.1/24 be an issue... I use 192.168.2 through 192.168.10 and also have tunnel networks of 10.0.8 and 10.0.200/24 Are you going to now hack me? ;)

    Oh and I use 172.16.0/30 as transit between my edge pfsense and a lab downstream pfsense vm...

    If you obfuscate for all I know you pulled some public space out of thin air and using that which would cause its own problems.. For all I know from that posting your using pfsense.org NS address space locally which is why you can not resolve pfsense.org



  • Thank you, johnpoz, for taking the time to have a look for me. Clearly I am doing something wrong here, and am hoping to get it right as soon as poss.
    I take your point about the full info regarding ip addresses and names.

    When I go to Diagnostics / DNS Lookup google.com I get:

    Result 	Record type
    89.16.167.134	A
    Timings
    8.8.8.8	5029 msec
    212.113.0.3	30 msec
    
    

    My clients point to my two DNS internal servers for the local addressing and they forward everything else to the same DNS Servers as my IP Provider (see below)

    pfsense gets it's WAN address and DNS Servers from the Cisco Router outside. Here are my Interface settings

    pfsense/General Setup
    DNS Servers		- blank
    DNS Server Override 	- checked
    
    interfaces/WAN
    
    IPv4 Configuration Type		- DHCP	(the Cisco deviec sets a static IP address so the port forwarding works)
    IPv6 Configuration Type		- DHCP
    Block private networks and loopback addresses - Not Checked
    Block bogon networks		- Checked
    
    interfaces/LAN
    
    IPv4 Configuration Type		- Static IPv4
    IPv6 Configuration Type		- none
    IPv4 Address			- 10.6.77.1	/24
    IPv4 Upstream gateway		- none
    Block private networks and loopback addresses - Not Checked
    Block bogon networks		- Checked
    
    Services/DHCP Server/LAN
    
    Subnet				- 10.6.77.0
    Subnet Mask			- 255.255.255.0
    Range				- 10.6.77.100 to 10.6.77.109
    
    WINS Servers			- 10.6.77.220
    
    DNS Servers			- 10.6.77.45
    				- 10.6.77.220
    
    Nothing else checked 
    

    Here are my Bind DNS Setting

    named.conf.options

    acl "trusted" {
    	10.6.77.0/24;
    	127.0.0.1;
    };
    
    options {
    	directory "/var/cache/bind";
    	version "Not disclosed";
    	recursion yes;
            allow-query { trusted; };
            allow-recursion { trusted; };
            allow-query-cache { trusted; };
            allow-transfer { trusted; };
    	forwarders {
                    8.8.8.8; 212.113.0.3;
            };
            dnssec-enable no;
            auth-nxdomain no;    # conform to RFC1035
            listen-on port 53 { 127.0.0.1; 10.6.77.220; } ;
            listen-on-v6            { none; };
    
    };
    

    named.conf.local

    include "/etc/bind/rndc.key";
    include "/etc/bind/zones.rfc1918";
    
    zone"fsoft.nnet" IN {
    type master;
    file "fsoft.nnet.fwd.zone";
    allow-transfer { 10.6.77.45; };
    also-notify { 10.6.77.45; };
    };
    
    zone"77.6.10.in-addr.arpa" IN {
    type master;
    file "fsoft.nnet.rev.zone";
    allow-transfer { 10.6.77.45; };
    also-notify { 10.6.77.45; };
    };
    

    and the start of my zone file

    $TTL 604800	; 1 week
    $ORIGIN fsoft.nnet.
    @ 		IN SOA	antares.fsoft.nnet. admin.fsoft.nnet. (
    				2018030901 ; serial
    				604800     ; refresh (1 week)
    				86400      ; retry (1 day)
    				2419200    ; expire (4 weeks)
    				604800     ; minimum (1 week)
    				)
    			NS	ns1.fsoft.nnet.
    			NS	ns2.fsoft.nnet.
    			MX	10 mail.fsoft.nnet.
    ns1			A	10.6.77.220
    antares		A	10.6.77.220
    ns2			A	10.6.77.45
    pytroll64		A	10.6.77.45
    
    ....
    

    here is my rfc1918 which is the default when I installed Bind.

    zone "10.in-addr.arpa"      { type master; file "/etc/bind/db.empty"; };
    
    zone "16.172.in-addr.arpa"  { type master; file "/etc/bind/db.empty"; };
    zone "17.172.in-addr.arpa"  { type master; file "/etc/bind/db.empty"; };
    zone "18.172.in-addr.arpa"  { type master; file "/etc/bind/db.empty"; };
    zone "19.172.in-addr.arpa"  { type master; file "/etc/bind/db.empty"; };
    zone "20.172.in-addr.arpa"  { type master; file "/etc/bind/db.empty"; };
    zone "21.172.in-addr.arpa"  { type master; file "/etc/bind/db.empty"; };
    zone "22.172.in-addr.arpa"  { type master; file "/etc/bind/db.empty"; };
    zone "23.172.in-addr.arpa"  { type master; file "/etc/bind/db.empty"; };
    zone "24.172.in-addr.arpa"  { type master; file "/etc/bind/db.empty"; };
    zone "25.172.in-addr.arpa"  { type master; file "/etc/bind/db.empty"; };
    zone "26.172.in-addr.arpa"  { type master; file "/etc/bind/db.empty"; };
    zone "27.172.in-addr.arpa"  { type master; file "/etc/bind/db.empty"; };
    zone "28.172.in-addr.arpa"  { type master; file "/etc/bind/db.empty"; };
    zone "29.172.in-addr.arpa"  { type master; file "/etc/bind/db.empty"; };
    zone "30.172.in-addr.arpa"  { type master; file "/etc/bind/db.empty"; };
    zone "31.172.in-addr.arpa"  { type master; file "/etc/bind/db.empty"; };
    
    zone "168.192.in-addr.arpa" { type master; file "/etc/bind/db.empty"; };
    

    The pfsense router I presume NATs to the Cicso router which NATs to the internet. The pfsense router is connected on a 192.168.0.0 net as 192.168.0.22 to the Cisco router 192.168.0.1

    What settings should I have then on the pfsense router?
    What am I doing wrong with the Bind DNS settings?
    Is my rfc1918 correct?

    Regards


  • Rebel Alliance Global Moderator

    "Timings
    8.8.8.8 5029 msec
    212.113.0.3 30 msec:

    Dude how do you want this to work… Why do you have pfsense pointing to isp and googledns if your running your own resolving bind servers?

    If your pointing all your clients your bind NS... Then just point pfsense to them as well!!!  Just use the forwarder in pfsense, set it to use your bind servers local IPs, have pfsense point to loopback.. Now pfsense will be able to resolve all your local stuff, and if needs to lookup up something public will get it from your bind forwarders.

    forwarders {
                    8.8.8.8; 212.113.0.3;

    Why would you not just let bind resolve and use dnssec?

    Your answer from 8.8.8.8 seems CRAZY SLOW
    8.8.8.8 5029 msec

    From a client query 8.8.8.8 directly for a few things, how long does it take to resolve???

    example

    dig @8.8.8.8 www.google.com

    ; <<>> DiG 9.11.2-P1 <<>> @8.8.8.8 www.google.com
    ; (1 server found)
    ;; global options: +cmd
    ;; Got answer:
    ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 28762
    ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

    ;; OPT PSEUDOSECTION:
    ; EDNS: version: 0, flags:; udp: 512
    ;; QUESTION SECTION:
    ;www.google.com.                        IN      A

    ;; ANSWER SECTION:
    www.google.com.        168    IN      A      172.217.9.68

    ;; Query time: 12 msec
    ;; SERVER: 8.8.8.8#53(8.8.8.8)
    ;; WHEN: Sat Mar 10 07:14:18 Central Standard Time 2018
    ;; MSG SIZE  rcvd: 59

    12 msec... You see 5000 ms?? (5 seconds)  That is just nuts, are you on satellite internet or something?



  • Right, for some reason I am not understanding what I do to get this correct. I think I must be slow like the DNS.
    My Internet is via a dish pointing to a mountain top, anything from 5 to 15Mb/s down and 1Mb/s up. Not great but could be worse.

    Here is dig from my PC

    dig @8.8.8.8 www.google.com
    
    ; <<>> DiG 9.11.2-P1 <<>> @8.8.8.8 www.google.com
    ; (1 server found)
    ;; global options: +cmd
    ;; Got answer:
    ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 40106
    ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
    
    ;; OPT PSEUDOSECTION:
    ; EDNS: version: 0, flags:; udp: 512
    ;; QUESTION SECTION:
    ;www.google.com.			IN	A
    
    ;; ANSWER SECTION:
    www.google.com.		266	IN	A	216.58.198.68
    
    ;; Query time: 39 msec
    ;; SERVER: 8.8.8.8#53(8.8.8.8)
    ;; WHEN: Sat Mar 10 13:28:08 UTC 2018
    ;; MSG SIZE  rcvd: 59
    

    Much faster than from pfsense was, although it has speeded up somewhat now.

    I have my own DNS servers in the DHCP for the clients to pick up.

    The WAN Interface that has the address & DNS servers supplied by the Cisco Router.

    I cannot see anywhere to put a DNS address in the Services / DNS Forwarder

    As for the WAN & LAN 'Block private networks and loopback addresses' and 'Block bogon networks' should they both be unchecked for both interfaces?

    I am trying my best to get my head round this, but not doing very well.

    Regards


  • Rebel Alliance Global Moderator

    you can leave bogon on your wan, and rfc1918 as well.. Those are default unless pfsense was downstream from your internal networks using rfc1918 blocking that on wan is fine.  While you should never really see bogon, blocking it on wan is also fine.

    On lan side bogon is going to cause you problem possible since it include all zero space.  And its just really impossible that you would ever see bogon address space into your wan as source.

    If your using the forwarder on pfsense it will use what you set in system for nameservers, which should be your internal bind servers.  Nothing else!!!  Do not




  • OK, johnpoz, thanks so much for some detail. I will give that a go and see what happens.
    Your post finishes with a 'Do not'
    Do no what?



  • Unfortunately I still have lots of slow DNS lookups.
    I have had to put my old Netgear Router on pro temp, which works fine, no DNS lookup problems, so not a problem with my DNS Servers then.
    I am going to do a fresh install and then work my way through bit by bit when I get back home next week.
    Thanks very much for trying to help.
    Regards


  • Rebel Alliance Global Moderator

    Sorry should of finished with do not check those boxes..

    Where are you getting slow dns - your clients?  If so that has NOTHING to do with pfsense.. If you correctly setup pfsense to ask your bind servers, and pfsense dns is slow - then again that is your bind servers talking to the interent..  Other than pfsense handling the connection, it has nothing to do with dns..

    So lets see a slow dns from a client doing a dig to your bind servers.  And or the internet directly… Lets see query from pfsense..

    How is it someone runs 2 bind servers but knows zero about how to troubleshoot dns?

    Your not running snort or any ips on pfsense are you, something that could be blocking the traffic?  5000ms for a dns query is going to cause time outs for sure..



  • I have at last managed to sort it out, yes I do know how to troubleshoot DNS even if it does not look as if I do.
    I am not running any Snort or the like.

    With a fresh install of pfSense:-
    Interfaces/WAN and Interfaces/LAN I have the Block private networks and loopback addresses unchecked and Block bogon networks checked.
    I put my DNS servers into System/General Setup/DNS Server Settings and checked the DNS Server Override.
    Then in Services/DNS Resolver/General Settings I checked DNSSEC and DNS Query Forwarding
    I changed my DNS Servers to remove one which seemed to be the cause of the problem and now doing a lookup on pfsense.org I get this:
    127.0.0.143 msec
    8.8.8.8 22 msec
    212.113.0.3 5066 msec
    10.6.77.45 2 msec
    10.6.77.13 4 msec
    10.6.77.17 11 msec
    That's morel like it!
    Thanks again for your ibnput


  • Rebel Alliance Global Moderator

    "I do know how to troubleshoot DNS even if it does not look as if I do."
    "Then in Services/DNS Resolver/General Settings I checked DNSSEC and DNS Query Forwarding"

    From your 2nd statement, sorry but I tend to disagree with the 1st statement.  Form where your forwarding in your output also makes that 1st statement suspect - sorry.  But I do not believe you actually understand how this stuff is suppose to function.  If you want pfsense to be able to resolve internal, then point it to internal and let those internal resolve the public stuff pfsense might need.  If they can not resolve external, then setup pfsense to use those when only wanting to resolve those specific domains they are authoritative for or can resolve via forwarding.  And setup pfsense to be able to resolve public stuff for you, etc.

    "212.113.0.3  5066 msec"

    Where is that ns in respect to you?  The Space Station? ;)

    So does 10.6.x.x support dnssec?  Do they resolve internal and external?  Does this 212.113 box support dnssec?

    resolver1.eu.level3.net

    I show they do not
    ;; QUESTION SECTION:
    ;sigfail.verteiltesysteme.net.  IN      A

    ;; ANSWER SECTION:
    sigfail.verteiltesysteme.net. 60 IN    A      134.91.78.139

    ;; Query time: 133 msec
    ;; SERVER: 212.113.0.3#53(212.113.0.3)

    They for sure will not be able to resolve internal stuff.

    You can not and hope to not have problems point a system to different NS that can not resolve the same stuff.  Pointing to public dns like google and level3 thinking they will resolve your internal stuff not going to work.  You can not be sure which order NS will be queried..  And if they return a NX for something you query even if you go down the line in order you dns client after getting back an NX would not ask another NS for the same thing until the neg cache set when getting that expires, etc.

    Using NS that can not all resolve the same stuff is asking for nothing but problems.

    I would assume those 10.6.x.x are your internal NS, which can resolve your internal stuff.  Can they also resolve public - where they forward, do they resolve - do they support dnssec.

    While google does…l that level 3 resolver sure doesn't

    ;; Got answer:
    ;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 34373
    ;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

    ;; OPT PSEUDOSECTION:
    ; EDNS: version: 0, flags:; udp: 512
    ;; QUESTION SECTION:
    ;sigfail.verteiltesysteme.net.  IN      A

    ;; Query time: 285 msec
    ;; SERVER: 8.8.8.8#53(8.8.8.8)
    ;; WHEN: Tue Mar 20 10:37:38 Central Daylight Time 2018
    ;; MSG SIZE  rcvd: 57

    See how level 3 answers even those the sig is bad..

    ;; QUESTION SECTION:
    ;sigfail.verteiltesysteme.net.  IN      A

    ;; ANSWER SECTION:
    sigfail.verteiltesysteme.net. 60 IN    A      134.91.78.139

    ;; Query time: 133 msec
    ;; SERVER: 212.113.0.3#53(212.113.0.3)
    ;; WHEN: Tue Mar 20 10:34:50 Central Daylight Time 2018
    ;; MSG SIZE  rcvd: 73



  • Finally, and I am pretty sure it is Finally, I have found the problem.
    Nothing to do with my internal DNS or pfSense on my SG-1000 but the external Cisco Router/Modem provided by my ISP.
    That was blocking Pings, so no wonder there were lots of ICMP UDP Port not found errors instead of a reply with the address of the servers on the tcpdump output. This seemed to be intermittent so very difficult to track down.
    Hooray!!!
    And another thank you to jonpoz for helping me.
    Regards