DNS server behind pf/dnsmasq gets timeout for one specific domain w/strange SOA

iamzam · Oct 18, 2013, 10:57 PM

I have a strange problem that I am trying to figure out.

We have two Windows 2003 DNS servers, and a RHEL6 BIND 9.8.2 named server that both sit in RFC 1918 IP space behind the pfSense router. These all provide DNS resolution for various client systems.

A week ago someone mentioned that they had run across one domain that would not resolve, and when I tried it, sure enough I couldn't get any of our three local DNS servers to give me anything on the domain.

Any other DNS servers including the pfSense 1.2.3 DNS forwarder works fine with this domain.

I do see a potential problem with the domain in the SOA record, the authoritative server is just a single dot "." like this:

rapportive.com.		3378	IN	SOA	. hostmaster.rapportive.com. 2013041614 3600 3600 3600 3600

When I do a query from one of our local DNS servers to the authoritative server for the domain it times out:

# dig @ns1.worldwidedns.net soa rapportive.com

; <<>> DiG 9.8.2rc1-RedHat-9.8.2-0.17.rc1.el6_4.6 <<>> @ns1.worldwidedns.net soa rapportive.com
; (1 server found)
;; global options: +cmd
;; connection timed out; no servers could be reached

and the states for the IP address of that authoritative server are like this:


udp 	207.97.208.112:53 <- 172.20.20.81:46767 	NO_TRAFFIC:SINGLE 	
udp 	172.20.20.81:46767 -> 108.170.120.203:46767 -> 207.97.208.112:53 	SINGLE:NO_TRAFFIC 	
udp 	207.97.208.112:53 <- 172.20.20.81:9710 	NO_TRAFFIC:SINGLE 	
udp 	172.20.20.81:9710 -> 108.170.120.203:9710 -> 207.97.208.112:53 	SINGLE:NO_TRAFFIC 	
udp 	207.97.208.112:53 <- 172.20.20.81:54516 	NO_TRAFFIC:SINGLE 	
udp 	172.20.20.81:54516 -> 108.170.120.203:54516 -> 207.97.208.112:53 	SINGLE:NO_TRAFFIC

From the same server I can query @8.8.8.8 and i get the response:

# dig @8.8.8.8 soa rapportive.com

; <<>> DiG 9.8.2rc1-RedHat-9.8.2-0.17.rc1.el6_4.6 <<>> @8.8.8.8 soa rapportive.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 55886
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;rapportive.com.			IN	SOA

;; ANSWER SECTION:
rapportive.com.		3378	IN	SOA	. hostmaster.rapportive.com. 2013041614 3600 3600 3600 3600

;; Query time: 57 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Fri Oct 18 18:45:01 2013
;; MSG SIZE  rcvd: 78

or the local pfsense interface:

# dig @172.20.20.254 soa rapportive.com

; <<>> DiG 9.8.2rc1-RedHat-9.8.2-0.17.rc1.el6_4.6 <<>> @172.20.20.254 soa rapportive.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 43581
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;rapportive.com.			IN	SOA

;; ANSWER SECTION:
rapportive.com.		2721	IN	SOA	. hostmaster.rapportive.com. 2013041614 3600 3600 3600 3600

;; Query time: 26 msec
;; SERVER: 172.20.20.254#53(172.20.20.254)
;; WHEN: Fri Oct 18 18:55:57 2013
;; MSG SIZE  rcvd: 78

I am wondering if pf or dnsmasq or something else in pfSense inspects DNS traffic and ensures that responses to queries are valid, and bails out if there is some anomaly. That is the only thing I can think of to explain this.

Does anyone else have any way to explain this?

johnpoz · Oct 18, 2013, 11:12 PM

"I am wondering if pf or dnsmasq or something else in pfSense inspects DNS traffic and ensures that responses to queries are valid, "

NO..

So that your domain - which points to
Name Servers:
ns1.worldwidedns.net
ns2.worldwidedns.net
ns3.worldwidedns.net

I seem to get SOA info just fine
; <<>> DiG 9.9.2-P1 <<>> @ns1.worldwidedns.net rapportive.com soa
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 43652
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 3, ADDITIONAL: 4
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1280
;; QUESTION SECTION:
;rapportive.com. IN SOA

;; ANSWER SECTION:
rapportive.com. 3600 IN SOA . hostmaster.rapportive.com. 201 3041614 3600 3600 3600 3600

;; AUTHORITY SECTION:
rapportive.com. 3600 IN NS ns3.worldwidedns.net.
rapportive.com. 3600 IN NS ns1.worldwidedns.net.
rapportive.com. 3600 IN NS ns2.worldwidedns.net.

;; ADDITIONAL SECTION:
ns3.worldwidedns.net. 86400 IN A 174.143.111.161
ns1.worldwidedns.net. 86400 IN A 207.97.208.112
ns2.worldwidedns.net. 86400 IN A 207.97.208.176

;; Query time: 45 msec
;; SERVER: 207.97.208.112#53(207.97.208.112)
;; WHEN: Fri Oct 18 18:10:28 2013
;; MSG SIZE rcvd: 207

From your first query your asking a server SOA for rapportive.com not the SOA of rapportive.com

dig shows it query format is
Usage: dig [@global-server] [domain] [q-type] [q-class] {q-opt}
{global-d-opt} host [@local-server] {local-d-opt}
[ host [@local-server] {local-d-opt} […]]

iamzam · Oct 19, 2013, 12:20 AM

@johnpoz:

"I am wondering if pf or dnsmasq or something else in pfSense inspects DNS traffic and ensures that responses to queries are valid, "

NO..

So that your domain - which points to
Name Servers:
ns1.worldwidedns.net
ns2.worldwidedns.net
ns3.worldwidedns.net

I seem to get SOA info just fine

NO..

I am not sure what you are responding to but I was not asking if my domain is working correctly, it is not my domain. I was saying that this domain on the internet (which I have nothing to do with) does not resolve although it does for everyone else. AndI think it is because DNS traffic from my name servers is being blocked by pfsense, because of it's invalid SOA record, which contains an illegal single period for the authoritative server.

and you can see that pfsense is blocking this traffic by the states from my server to the server that I queried that contains the invalid record:

udp 	207.97.208.112:53 <- 172.20.20.81:46767 	NO_TRAFFIC:SINGLE 	
udp 	172.20.20.81:46767 -> 108.170.120.203:46767 -> 207.97.208.112:53 	SINGLE:NO_TRAFFIC 	
udp 	207.97.208.112:53 <- 172.20.20.81:9710 	NO_TRAFFIC:SINGLE 	
udp 	172.20.20.81:9710 -> 108.170.120.203:9710 -> 207.97.208.112:53 	SINGLE:NO_TRAFFIC 	
udp 	207.97.208.112:53 <- 172.20.20.81:54516 	NO_TRAFFIC:SINGLE 	
udp 	172.20.20.81:54516 -> 108.170.120.203:54516 -> 207.97.208.112:53 	SINGLE:NO_TRAFFIC

If you go here, this tool thinks the SOA record is invalid as well:

http://dnscheck.iis.se/

Host name is illegal (syntax error at Top all-numeric)

Error while checking SOA MNAME for rapportive.com ().

Or this one if you like:

http://dnscheck.pingdom.com/?domain=rapportive.com

@johnpoz:

From your first query your asking a server SOA for rapportive.com not the SOA of rapportive.com

dig shows it query format is
Usage: dig [@global-server] [domain] [q-type] [q-class] {q-opt}
{global-d-opt} host [@local-server] {local-d-opt}
[ host [@local-server] {local-d-opt} […]]

NO..

you can have the query type after the question or before it, it does not matter:

before:


 dig @172.20.20.254 soa rapportive.com

; <<>> DiG 9.8.2rc1-RedHat-9.8.2-0.17.rc1.el6_4.6 <<>> @172.20.20.254 soa rapportive.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 1932
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;rapportive.com.			IN	SOA

;; ANSWER SECTION:
rapportive.com.		3600	IN	SOA	. hostmaster.rapportive.com. 2013041614 3600 3600 3600 3600

;; Query time: 42 msec
;; SERVER: 172.20.20.254#53(172.20.20.254)
;; WHEN: Fri Oct 18 20:06:45 2013
;; MSG SIZE  rcvd: 78

after:


# dig @172.20.20.254 rapportive.com soa

; <<>> DiG 9.8.2rc1-RedHat-9.8.2-0.17.rc1.el6_4.6 <<>> @172.20.20.254 rapportive.com soa
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 62463
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;rapportive.com.			IN	SOA

;; ANSWER SECTION:
rapportive.com.		3589	IN	SOA	. hostmaster.rapportive.com. 2013041614 3600 3600 3600 3600

;; Query time: 27 msec
;; SERVER: 172.20.20.254#53(172.20.20.254)
;; WHEN: Fri Oct 18 20:06:56 2013
;; MSG SIZE  rcvd: 78

johnpoz · Oct 19, 2013, 1:20 PM

Dude my point was I am behind pfsense - and its not blocking anything.

Pfsense is not going to block something in a dns packet - its is not doing layer 7. So what does it care is in the query response.

Where in your state tables are you seeing anything blocked?

iamzam · Oct 19, 2013, 3:41 PM

Thanks for your help, I was a bit aggravated last night.

In order for you to replicate, you would need another DNS server behind pfsense (version 1.2.3 to be sure you have got the same exact stuff) and then try to resolve rapportive.com through that name server from your client, which is also behind pfsense (DNS forwarder should probably be on but I get the same thing when I turn it off).

As far as the states, the ones that I posted are from my bind server (172.20.20.81) after getting the failure when attempting to " dig @ns1.worldwidedns.net soa rapportive.com ".

I then filtered through the states for the IP address of ns1.worldwidedns.net and that is what I saw.

I admit this is very strange but I am assuming that pf does some sort of DNS fixup that does validation on query responses…