DNS Resolver fails with SERVFAIL but 1.1.1.1 resolves host just fine



  • I have a strange problem. When I try to resolve host c.na80.content.force.com using DNS Resolver (on 127.0.0.1) I get a SERVFAIL:

    [2.4.4-RELEASE][root@fw-pwn]/root: dig @127.0.0.1 c.na80.content.force.com
    
    ; <<>> DiG 9.12.2-P1 <<>> @127.0.0.1 c.na80.content.force.com
    ; (1 server found)
    ;; global options: +cmd
    ;; Got answer:
    ;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 26553
    ;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1
    
    ;; OPT PSEUDOSECTION:
    ; EDNS: version: 0, flags:; udp: 4096
    ;; QUESTION SECTION:
    ;c.na80.content.force.com.	IN	A
    
    ;; Query time: 1363 msec
    ;; SERVER: 127.0.0.1#53(127.0.0.1)
    ;; WHEN: Tue Feb 04 09:40:28 PST 2020
    ;; MSG SIZE  rcvd: 53
    

    When I try using 1.1.1.1 it works:

    [2.4.4-RELEASE][root@fw-pwn]/root: dig @1.1.1.1 c.na80.content.force.com
    
    ; <<>> DiG 9.12.2-P1 <<>> @1.1.1.1 c.na80.content.force.com
    ; (1 server found)
    ;; global options: +cmd
    ;; Got answer:
    ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 49942
    ;; flags: qr rd ra; QUERY: 1, ANSWER: 6, AUTHORITY: 0, ADDITIONAL: 1
    
    ;; OPT PSEUDOSECTION:
    ; EDNS: version: 0, flags:; udp: 1452
    ;; QUESTION SECTION:
    ;c.na80.content.force.com.	IN	A
    
    ;; ANSWER SECTION:
    c.na80.content.force.com. 300	IN	CNAME	na80.force.com.
    na80.force.com.		300	IN	CNAME	na80-ph2.force.com.
    na80-ph2.force.com.	300	IN	CNAME	na80-ph2.ph2.r.force.com.
    na80-ph2.ph2.r.force.com. 30	IN	A	13.110.0.210
    na80-ph2.ph2.r.force.com. 30	IN	A	13.110.1.82
    na80-ph2.ph2.r.force.com. 30	IN	A	13.110.2.210
    
    ;; Query time: 127 msec
    ;; SERVER: 1.1.1.1#53(1.1.1.1)
    ;; WHEN: Tue Feb 04 09:43:23 PST 2020
    ;; MSG SIZE  rcvd: 196
    

    If I try using +trace and 127.0.0.1 it seems to be working - not sure from the output:

    [2.4.4-RELEASE][root@fw-pwn]/root: dig @127.0.0.1 c.na80.content.force.com +trace
    
    ; <<>> DiG 9.12.2-P1 <<>> @127.0.0.1 c.na80.content.force.com +trace
    ; (1 server found)
    ;; global options: +cmd
    .			84876	IN	NS	a.root-servers.net.
    .			84876	IN	NS	b.root-servers.net.
    .			84876	IN	NS	c.root-servers.net.
    .			84876	IN	NS	d.root-servers.net.
    .			84876	IN	NS	e.root-servers.net.
    .			84876	IN	NS	f.root-servers.net.
    .			84876	IN	NS	g.root-servers.net.
    .			84876	IN	NS	h.root-servers.net.
    .			84876	IN	NS	i.root-servers.net.
    .			84876	IN	NS	j.root-servers.net.
    .			84876	IN	NS	k.root-servers.net.
    .			84876	IN	NS	l.root-servers.net.
    .			84876	IN	NS	m.root-servers.net.
    .			84876	IN	RRSIG	NS 8 0 518400 20200217050000 20200204040000 33853 . s5M6eDGRBr0xL2fPwEflYM2WLsxDe77bjou4hVoBJ6LG/VVomHznsQcW 0z5N0OWavEKv0MrtRSal7qYwjHcB8Cw/dMK8b7aTtdPQ4BAIpUQQ0Vpv cR5FYetEjEn/vfGojwRCgoZ0+JUrAQRaHY8Q8Z1WLdINRJtpywxPJcFS j/TC2+P91Fnnt+0rsGQnAflXhV5cxlwR49rdBRu1/qd7dmzf5ByCpNnn h2yQ8I3q3KS9o5KXr8d/u7N4gLFyguaWDdsGSxxx9YB7AWAETjJ+rpvH j8Fdp3pE+vY4uXNSR9/71xad/AAzak2LX8001Py3TH+rjcgjXgbFpGlo 9A4oFw==
    ;; Received 525 bytes from 127.0.0.1#53(127.0.0.1) in 0 ms
    
    com.			172800	IN	NS	e.gtld-servers.net.
    com.			172800	IN	NS	b.gtld-servers.net.
    com.			172800	IN	NS	j.gtld-servers.net.
    com.			172800	IN	NS	m.gtld-servers.net.
    com.			172800	IN	NS	i.gtld-servers.net.
    com.			172800	IN	NS	f.gtld-servers.net.
    com.			172800	IN	NS	a.gtld-servers.net.
    com.			172800	IN	NS	g.gtld-servers.net.
    com.			172800	IN	NS	h.gtld-servers.net.
    com.			172800	IN	NS	l.gtld-servers.net.
    com.			172800	IN	NS	k.gtld-servers.net.
    com.			172800	IN	NS	c.gtld-servers.net.
    com.			172800	IN	NS	d.gtld-servers.net.
    com.			86400	IN	DS	30909 8 2 E2D3C916F6DEEAC73294E8268FB5885044A833FC5459588F4A9184CF C41A5766
    com.			86400	IN	RRSIG	DS 8 1 86400 20200217050000 20200204040000 33853 . 0+xmLMOG2+2RE7jYn5aa7fotwt9aj8ugWOLKlcqZAYZVJJXxd8QyqR5s N1S6DwChDv9vNiHl5wJYnHK9tZkgCfW5wVly8wYZnINSCczUh6tG86Ex g5ScsvCIK28Btdy7LAQRdBlBowqClI73OB5EVysmhO2JNyEs5gKNIt+i StmxmiS3aXGH8tIek97WSKLIlZbL7lCe5ODxPPCkURJ6va+TD6/0Bw9Z Np4cdFG1q1uoblZy85VMRq/MT8jvla9emJM8zHY6P+TfapP8GNlMRbCt 70WCkSazlWq6qMmLR7gQF6in64I5d2vksP39133DRYSd5y1+R38wpQQl GJ2g3g==
    ;; Received 1184 bytes from 198.41.0.4#53(a.root-servers.net) in 31 ms
    
    force.com.		172800	IN	NS	udns1.salesforce.com.
    force.com.		172800	IN	NS	udns2.salesforce.com.
    force.com.		172800	IN	NS	udns3.salesforce.com.
    force.com.		172800	IN	NS	udns4.salesforce.com.
    force.com.		172800	IN	NS	pch1.salesforce-dns.com.
    force.com.		86400	IN	DS	21188 13 2 0B6E0B5F08880171FE1C25F510B8A5F71C62BFC82585B1F546278768 A7EF5F1F
    force.com.		86400	IN	RRSIG	DS 8 2 86400 20200211063741 20200204052741 56311 com. mSxWKqZcR4zJna25KYC/Q7Bj7iBQoIWNQ8AXwXLgw2eXh2RPfxJTdkMG 87eA8yFvSJS1Zaao5rv8OpXwgwrTO0B1bFR2+51vWkN9T/We6zb9YQMg Spd8gZJaJyKIDvg/ZdKMbvcjzcYltBmWQvMkpqLYWwdimyQi18UP67BS bL0FUdD6xeY83PvoPCoajrxtST6SrNP37ITjq1cNzTXRTw==
    ;; Received 641 bytes from 192.43.172.30#53(i.gtld-servers.net) in 74 ms
    
    c.na80.content.force.com. 300	IN	CNAME	na80.force.com.
    c.na80.content.force.com. 300	IN	RRSIG	CNAME 13 4 300 20200217235356 20191219235042 23873 force.com. AgVT8dcBvb2cEEzXrxgRlurOYftXlj/kq6xLGL0POVP0VE+QdEjNCzHh EsCubYW+kX9gjJ/dWHDJjLg7Mhh5kQ==
    na80.force.com.		300	IN	CNAME	na80-ph2.force.com.
    na80.force.com.		300	IN	RRSIG	CNAME 13 3 300 20200319111245 20200119101245 23873 force.com. 7+SpoZmYK2rocesHWbE3kdT4LHgJiMmD/OPKXv/Yqsb/YvAcW7V4/OYU 36DcevJvqvdg3QIYrTnaTRdinVD5Tg==
    na80-ph2.force.com.	300	IN	CNAME	na80-ph2.ph2.r.force.com.
    na80-ph2.force.com.	300	IN	RRSIG	CNAME 13 3 300 20200309133842 20200109125202 23873 force.com. GWU6XpuwbYHQckHqNMypoEAeDS53JXpnK3UBmwQXYh4rkJi1LEtPOclM JcpvTAzBQ/nziESd7s5/twN5eTnX6Q==
    37iviu5phujhjeqaipp04v46a8qh51gk.force.com. 10 IN NSEC3	1 0 0 C7EB9E0E 37J16MAAQS76KJ8UN52UPRP6IR45IOGG CNAME RRSIG
    37iviu5phujhjeqaipp04v46a8qh51gk.force.com. 10 IN RRSIG	NSEC3 13 3 10 20200219140510 20191221133335 23873 force.com. qLD/fsyKK0U1rGKmbWMcDpa+THjrNhljf/n0LGSth5Cc1ZYarwBahYdd 4wCUPUQuWNpQI09AP9wM18UrZP6Bsg==
    ph2.r.force.com.	86400	IN	NS	ns1-ph2.salesforce.com.
    413tte56j9719p9p1gub3ov754p4llgl.force.com. 10 IN NSEC3	1 0 0 C7EB9E0E 4140A0K4J07QMM6SCR90DVC32LBN0S7F NS
    413tte56j9719p9p1gub3ov754p4llgl.force.com. 10 IN RRSIG	NSEC3 13 3 10 20200329020408 20200129010408 23873 force.com. WUqBRQoYl5vpJlsXkbaamD5O8sMWaOoyNRoAKAxIhChpvEG9pZ7/aNPe wtO1iwjmEWaQ5kpj+bEs0A39I5TMIA==
    ;; Received 843 bytes from 206.223.122.1#53(pch1.salesforce-dns.com) in 5 ms
    

    I can ping and query each nameserver correctly:

    force.com.		172800	IN	NS	udns1.salesforce.com.
    force.com.		172800	IN	NS	udns2.salesforce.com.
    force.com.		172800	IN	NS	udns3.salesforce.com.
    force.com.		172800	IN	NS	udns4.salesforce.com.
    force.com.		172800	IN	NS	pch1.salesforce-dns.com.
    

    Eventually I found that turning OFF Enable DNSSEC Support in DNS Resolver allows the query to work. What does this mean? That their DNSSEC has an invalid signature? Why does 1.1.1.1 and all other DNS servers I tried work except 127.0.0.1?



  • @pwnell I'm having the same problem with manage.kmail-lists.com.

    The last time this happened it was a temporary issue with a domains DNSSEC settings which resolved itself, but this particular domain has been broken for months and I don't understand why as its not using DNSSEC. :(

    As you noticed, turning off DNSSEC fixes it, but its a pretty crappy fix as its removing a layer of DNS security for ALL lookups then.



  • @pwnell said in DNS Resolver fails with SERVFAIL but 1.1.1.1 resolves host just fine:

    Eventually I found that turning OFF Enable DNSSEC Support in DNS Resolver allows the query to work. What does this mean? That their DNSSEC has an invalid signature?

    The sub domain "c.na80.content.force.com" isn't using DNSSEC - see https://dnsviz.net/d/c.na80.content.force.com/dnssec/

    ( the graph is pure art btw. This can't be a good sign )

    It resolves just fine using the resolver using the default settings :

    [2.4.5-RC][root@pfsense.brit-hotel-fumel.net]/root: dig @127.0.0.1 c.na80.content.force.com +short
    na80.force.com.
    na80-ph2.force.com.
    na80-ph2.ph2.r.force.com.
    13.110.1.82
    13.110.2.210
    13.110.0.210
    

    @pwnell said in DNS Resolver fails with SERVFAIL but 1.1.1.1 resolves host just fine:

    Why does 1.1.1.1 and all other DNS servers I tried work except 127.0.0.1?

    As you can see, 127.0.0.1 (which is the shortcut to 127.0.0.1:53 or the entrance of the local Resolver on pfSense) does its job.
    It returns a bunch of CNAME'S
    Each of them can be resolved to A's.



  • @Gertjan I feel I should add, I'm using TLS forwarding to Cloudflare. I wonder if @pwnell is too?



  • @Alex-Atkin-UK Nope - just had DNSSEC on in DNS Resolver and my DNS in setup points to 1.1.1.1



  • Forwarding and DNSSEC doesn't make sense.
    You can safely shut down DNSSEC. See https://forum.netgate.com/topic/139771/setup-dns-over-tls-on-pfsense-2-4-4-p2-guide for more info. You just have to trust Cloudfare that they give you the correct info.

    @Alex-Atkin-UK @pwnell Cloudfare is 1.1.1.1. It's considered better to use the domain name 'cloudflare-dns.com" although the "1.1.1.1" ALT domain is included in the list.



  • @Gertjan Thanks - I have switched to TLS and disabled DNSSEC.


Log in to reply