Periodic since 2.2 pages load blank, certs invalid
-
Start listening to these guys. There's something strange and it looks like it's in unbound.
$ dig @192.168.223.1 www.google.com ; <<>> DiG 9.8.3-P1 <<>> @192.168.223.1 www.google.com ; (1 server found) ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 53699 ;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0 ;; QUESTION SECTION: ;www.google.com. IN A ;; Query time: 81 msec ;; SERVER: 192.168.223.1#53(192.168.223.1) ;; WHEN: Mon Feb 9 12:23:46 2015 ;; MSG SIZE rcvd: 32
-
After bouncing unbound:
$ dig @192.168.223.1 www.google.com ; <<>> DiG 9.8.3-P1 <<>> @192.168.223.1 www.google.com ; (1 server found) ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 54480 ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0 ;; QUESTION SECTION: ;www.google.com. IN A ;; ANSWER SECTION: www.google.com. 300 IN A 216.58.216.36 ;; Query time: 1333 msec ;; SERVER: 192.168.223.1#53(192.168.223.1) ;; WHEN: Mon Feb 9 12:26:36 2015 ;; MSG SIZE rcvd: 48
-
When only unbound can be used and DNS Sec is set to ON, and port 53 is blocked except to pfsense
-A DNS lookup from any computer to one of the domains causes unbound to stop resolving anything, all lookups fail
(persists until unbound service is restarted)I can not confirm this, worked fine for me in this setup (with some service interruptions, 5-7times a day)
When you say interruptions, could those have been unbound not responding?
Someone did mention that one of the times I was unable to restart the service manually (as I was not available) it began working again after 45-50 minutes.….
My thread is this here:
https://forum.pfsense.org/index.php?topic=88272
:-)
-
Start listening to these guys. There's something strange and it looks like it's in unbound.
I'm guessing it happened a few moments after doing a lookup on one of the domains I mentioned?
-
Yup.
Unbound starts returning SERVFAIL for random domains after querying at least one or both of those hostnames. I'm doing it one more time at unbound log level 5.
ETA: Ironically, I had to kill unbound to post this because:
$ dig forum.pfsense.org
; <<>> DiG 9.8.3-P1 <<>> forum.pfsense.org
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 30471
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0;; QUESTION SECTION:
;forum.pfsense.org. IN A;; Query time: 1781 msec
;; SERVER: 192.168.223.1#53(192.168.223.1)
;; WHEN: Mon Feb 9 17:46:41 2015
;; MSG SIZE rcvd: 35 -
Have you guys considered setting the advanced settings that prevent excessive replies and purge things when excessive replies and poisoning is possibly happening?
-
https://forum.pfsense.org/index.php?topic=88466.msg488411#msg488411
-
Yup.
Unbound starts returning SERVFAIL for random domains after querying at least one or both of those hostnames. I'm doing it one more time at unbound log level 5.
This sounds like the manifestation of the issue with DNSSEC enabled. At some point, you may want to try it again with DNSSEC disabled; you should then see all domains being resolved to a hostile IP, bad certs for https, etc. like reported in the beginning of this thread. Not sure if this would help with the diagnostics.
-
Yup.
Unbound starts returning SERVFAIL for random domains after querying at least one or both of those hostnames. I'm doing it one more time at unbound log level 5.
This sounds like the manifestation of the issue with DNSSEC enabled. At some point, you may want to try it again with DNSSEC disabled; you should then see all domains being resolved to a hostile IP, bad certs for https, etc. like reported in the beginning of this thread. Not sure if this would help with the diagnostics.
Yes, DNSSEC was the change I made that made it go from the bad domain resolutions to the failure to resolve at all.
I've had both issues and DNSSEC was the difference between which one I got.
-
Uncheck DNSSEC, Save, Apply…. There is most certainly something here....
$ dig ns3.csof.net ; <<>> DiG 9.8.3-P1 <<>> ns3.csof.net ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 1411 ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 5, ADDITIONAL: 3 ;; QUESTION SECTION: ;ns3.csof.net. IN A ;; ANSWER SECTION: ns3.csof.net. 600 IN A 195.22.26.199 ;; AUTHORITY SECTION: csof.net. 1792 IN NS ns62.domaincontrol.com. csof.net. 1792 IN NS ns61.domaincontrol.com. csof.net. 1792 IN NS ns4.csof.net. csof.net. 1792 IN NS ns2.csof.net. csof.net. 1792 IN NS ns1.csof.net. ;; ADDITIONAL SECTION: ns4.csof.net. 1792 IN A 54.72.8.183 ns2.csof.net. 1793 IN A 212.6.183.201 ns1.csof.net. 1792 IN A 54.77.72.254 ;; Query time: 29 msec ;; SERVER: 192.168.223.1#53(192.168.223.1) ;; WHEN: Mon Feb 9 18:48:25 2015 ;; MSG SIZE rcvd: 203 $ dig api-nyc01.exip.org ; <<>> DiG 9.8.3-P1 <<>> api-nyc01.exip.org ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 14360 ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0 ;; QUESTION SECTION: ;api-nyc01.exip.org. IN A ;; ANSWER SECTION: api-nyc01.exip.org. 10 IN A 195.22.26.248 ;; Query time: 206 msec ;; SERVER: 192.168.223.1#53(192.168.223.1) ;; WHEN: Mon Feb 9 18:48:35 2015 ;; MSG SIZE rcvd: 52 $ dig www.pfsense.org ; <<>> DiG 9.8.3-P1 <<>> www.pfsense.org ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 51593 ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 4, ADDITIONAL: 0 ;; QUESTION SECTION: ;www.pfsense.org. IN A ;; ANSWER SECTION: www.pfsense.org. 10 IN A 195.22.26.248 ;; AUTHORITY SECTION: org. 172779 IN NS ns1.csof.net. org. 172779 IN NS ns2.csof.net. org. 172779 IN NS ns3.csof.net. org. 172779 IN NS ns4.csof.net. ;; Query time: 159 msec ;; SERVER: 192.168.223.1#53(192.168.223.1) ;; WHEN: Mon Feb 9 18:48:39 2015 ;; MSG SIZE rcvd: 129
-
looks like unbound is allowing itself to get polluted by this nonsense:
gridbug:etc cjl$ dig @ns1.csof.net. com. ns ; <<>> DiG 9.8.3-P1 <<>> @ns1.csof.net. com. ns ; (1 server found) ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 31834 ;; flags: qr aa rd; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 4 ;; WARNING: recursion requested but not available ;; QUESTION SECTION: ;com. IN NS ;; ANSWER SECTION: com. 172800 IN NS ns1.csof.net. com. 172800 IN NS ns2.csof.net. com. 172800 IN NS ns3.csof.net. com. 172800 IN NS ns4.csof.net. ;; ADDITIONAL SECTION: ns1.csof.net. 100 IN A 54.77.72.254 ns2.csof.net. 100 IN A 212.6.183.201 ns3.csof.net. 100 IN A 195.22.26.199 ns4.csof.net. 100 IN A 54.72.8.183 ;; Query time: 163 msec ;; SERVER: 54.77.72.254#53(54.77.72.254) ;; WHEN: Mon Feb 9 18:58:46 2015 ;; MSG SIZE rcvd: 165 gridbug:etc cjl$ dig @ns1.csof.net. . ns ; <<>> DiG 9.8.3-P1 <<>> @ns1.csof.net. . ns ; (1 server found) ;; global options: +cmd ;; connection timed out; no servers could be reached gridbug:etc cjl$ dig @ns1.csof.net. net. ns ; <<>> DiG 9.8.3-P1 <<>> @ns1.csof.net. net. ns ; (1 server found) ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 23620 ;; flags: qr aa rd; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 4 ;; WARNING: recursion requested but not available ;; QUESTION SECTION: ;net. IN NS ;; ANSWER SECTION: net. 172800 IN NS ns1.csof.net. net. 172800 IN NS ns2.csof.net. net. 172800 IN NS ns3.csof.net. net. 172800 IN NS ns4.csof.net. ;; ADDITIONAL SECTION: ns1.csof.net. 100 IN A 54.77.72.254 ns2.csof.net. 100 IN A 212.6.183.201 ns3.csof.net. 100 IN A 195.22.26.199 ns4.csof.net. 100 IN A 54.72.8.183 ;; Query time: 161 msec ;; SERVER: 54.77.72.254#53(54.77.72.254) ;; WHEN: Mon Feb 9 18:59:19 2015 ;; MSG SIZE rcvd: 162 gridbug:etc cjl$ dig @ns1.csof.net. com. ns ; <<>> DiG 9.8.3-P1 <<>> @ns1.csof.net. com. ns ; (1 server found) ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 57675 ;; flags: qr aa rd; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 4 ;; WARNING: recursion requested but not available ;; QUESTION SECTION: ;com. IN NS ;; ANSWER SECTION: com. 172800 IN NS ns1.csof.net. com. 172800 IN NS ns2.csof.net. com. 172800 IN NS ns3.csof.net. com. 172800 IN NS ns4.csof.net. ;; ADDITIONAL SECTION: ns1.csof.net. 100 IN A 54.77.72.254 ns2.csof.net. 100 IN A 212.6.183.201 ns3.csof.net. 100 IN A 195.22.26.199 ns4.csof.net. 100 IN A 54.72.8.183 ;; Query time: 164 msec ;; SERVER: 54.77.72.254#53(54.77.72.254) ;; WHEN: Mon Feb 9 18:59:23 2015 ;; MSG SIZE rcvd: 165 gridbug:etc cjl$ dig @ns1.csof.net. org. ns ; <<>> DiG 9.8.3-P1 <<>> @ns1.csof.net. org. ns ; (1 server found) ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 20295 ;; flags: qr aa rd; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 4 ;; WARNING: recursion requested but not available ;; QUESTION SECTION: ;org. IN NS ;; ANSWER SECTION: org. 172800 IN NS ns1.csof.net. org. 172800 IN NS ns2.csof.net. org. 172800 IN NS ns3.csof.net. org. 172800 IN NS ns4.csof.net. ;; ADDITIONAL SECTION: ns1.csof.net. 100 IN A 54.77.72.254 ns2.csof.net. 100 IN A 212.6.183.201 ns3.csof.net. 100 IN A 195.22.26.199 ns4.csof.net. 100 IN A 54.72.8.183 ;; Query time: 162 msec ;; SERVER: 54.77.72.254#53(54.77.72.254) ;; WHEN: Mon Feb 9 18:59:27 2015 ;; MSG SIZE rcvd: 165 gridbug:etc cjl$ dig @ns1.csof.net. edu. ns ; <<>> DiG 9.8.3-P1 <<>> @ns1.csof.net. edu. ns ; (1 server found) ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 28770 ;; flags: qr aa rd; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 4 ;; WARNING: recursion requested but not available ;; QUESTION SECTION: ;edu. IN NS ;; ANSWER SECTION: edu. 172800 IN NS ns1.csof.net. edu. 172800 IN NS ns2.csof.net. edu. 172800 IN NS ns3.csof.net. edu. 172800 IN NS ns4.csof.net. ;; ADDITIONAL SECTION: ns1.csof.net. 100 IN A 54.77.72.254 ns2.csof.net. 100 IN A 212.6.183.201 ns3.csof.net. 100 IN A 195.22.26.199 ns4.csof.net. 100 IN A 54.72.8.183 ;; Query time: 161 msec ;; SERVER: 54.77.72.254#53(54.77.72.254) ;; WHEN: Mon Feb 9 18:59:31 2015 ;; MSG SIZE rcvd: 165 gridbug:etc cjl$ dig @ns1.csof.net. gov. ns ; <<>> DiG 9.8.3-P1 <<>> @ns1.csof.net. gov. ns ; (1 server found) ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 46880 ;; flags: qr aa rd; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 4 ;; WARNING: recursion requested but not available ;; QUESTION SECTION: ;gov. IN NS ;; ANSWER SECTION: gov. 172800 IN NS ns1.csof.net. gov. 172800 IN NS ns2.csof.net. gov. 172800 IN NS ns3.csof.net. gov. 172800 IN NS ns4.csof.net. ;; ADDITIONAL SECTION: ns1.csof.net. 100 IN A 54.77.72.254 ns2.csof.net. 100 IN A 212.6.183.201 ns3.csof.net. 100 IN A 195.22.26.199 ns4.csof.net. 100 IN A 54.72.8.183 ;; Query time: 162 msec ;; SERVER: 54.77.72.254#53(54.77.72.254) ;; WHEN: Mon Feb 9 18:59:34 2015 ;; MSG SIZE rcvd: 165 gridbug:etc cjl$ dig @ns1.csof.net. mil. ns ; <<>> DiG 9.8.3-P1 <<>> @ns1.csof.net. mil. ns ; (1 server found) ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 21791 ;; flags: qr aa rd; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 4 ;; WARNING: recursion requested but not available ;; QUESTION SECTION: ;mil. IN NS ;; ANSWER SECTION: mil. 172800 IN NS ns1.csof.net. mil. 172800 IN NS ns2.csof.net. mil. 172800 IN NS ns3.csof.net. mil. 172800 IN NS ns4.csof.net. ;; ADDITIONAL SECTION: ns1.csof.net. 100 IN A 54.77.72.254 ns2.csof.net. 100 IN A 212.6.183.201 ns3.csof.net. 100 IN A 195.22.26.199 ns4.csof.net. 100 IN A 54.72.8.183 ;; Query time: 160 msec ;; SERVER: 54.77.72.254#53(54.77.72.254) ;; WHEN: Mon Feb 9 18:59:40 2015 ;; MSG SIZE rcvd: 165 gridbug:etc cjl$ dig @ns1.csof.net. us. ns ; <<>> DiG 9.8.3-P1 <<>> @ns1.csof.net. us. ns ; (1 server found) ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 5930 ;; flags: qr aa rd; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 4 ;; WARNING: recursion requested but not available ;; QUESTION SECTION: ;us. IN NS ;; ANSWER SECTION: us. 172800 IN NS ns1.csof.net. us. 172800 IN NS ns2.csof.net. us. 172800 IN NS ns3.csof.net. us. 172800 IN NS ns4.csof.net. ;; ADDITIONAL SECTION: ns1.csof.net. 100 IN A 54.77.72.254 ns2.csof.net. 100 IN A 212.6.183.201 ns3.csof.net. 100 IN A 195.22.26.199 ns4.csof.net. 100 IN A 54.72.8.183 ;; Query time: 161 msec ;; SERVER: 54.77.72.254#53(54.77.72.254) ;; WHEN: Mon Feb 9 18:59:43 2015 ;; MSG SIZE rcvd: 164
-
Back to dnsmasq for me until someone says something.
-
I don't know which I should be.
Upset that unbound is doing this
Relieved that everything I was seeing can be reproduced -
You should be thankful you found something that made it easily-reproducible. No need to be mad. Things happen - Look at BIND's history. And you can always just run dnsmasq in the meantime even though that's the exact opposite advice you got at first. :/
-
I'm really out of my league trying to diagnose this, but here is the output of 'unbound-control -c /var/unbound/unbound.conf lookup slashdot.org', when the issue is occuring with DNSSEC enabled:
unbound-control -c /var/unbound/unbound.conf lookup slashdot.org The following name servers are used for lookup of slashdot.org. ;rrset 85785 4 0 7 0 org. 172185 IN NS ns1.csof.net. org. 172185 IN NS ns2.csof.net. org. 172185 IN NS ns3.csof.net. org. 172185 IN NS ns4.csof.net. ;rrset 83879 2 1 11 4 org. 83879 IN DS 21366 7 1 E6C1716CFB6BDC84E84CE1AB5510DAC69173B5B2 org. 83879 IN DS 21366 7 2 96EEB2FFD9B00CD4694E78278B5EFDAB0A80446567B69F634DA078F0D90F01BA org. 83879 IN RRSIG DS 8 1 86400 20150219170000 20150209160000 16665 . RxYpI0BzYpGTE/PjRQdR4SZaxlvXCja3SJyx10JagTfz20gnltl4ar94GOwp8bA/ktY/7JxMoJvzCTAtcsGaTGRv04yDHr7WaydMxZuPCP9YT9Ixc+fX9IAZlSfwLCkBQgiC0mVeRiq+LmbIJhI2grJbTtvy96O9mipAqkFR42g= ;{id = 16665} ;rrset 1185 1 0 3 0 ns4.csof.net. 1185 IN A 54.72.8.183 ;rrset 197 1 0 8 0 ns3.csof.net. 197 IN A 195.22.26.199 ;rrset 1185 1 0 8 0 ns2.csof.net. 1185 IN A 212.6.183.201 ;rrset 1185 1 0 8 0 ns1.csof.net. 1185 IN A 54.77.72.254 Delegation with 4 names, of which 4 can be examined to query further addresses. It provides 4 IP addresses. 54.77.72.254 NoDNSSEC rto 344 msec, ttl 286, ping 84 var 65 rtt 344, tA 0, tAAAA 0, tother 0, EDNS 0 probed. 212.6.183.201 NoDNSSEC rto 365 msec, ttl 286, ping 69 var 74 rtt 365, tA 0, tAAAA 0, tother 0, EDNS 0 probed. 195.22.26.199 rto 96256 msec, ttl 286, ping 0 var 94 rtt 376, tA 3, tAAAA 3, tother 3, EDNS 0 assumed. 54.72.8.183 NoDNSSEC rto 315 msec, ttl 286, ping 99 var 54 rtt 315, tA 0, tAAAA 0, tother 0, EDNS 0 probed.
And then after restarting unbound:
unbound-control -c /var/unbound/unbound.conf lookup slashdot.org The following name servers are used for lookup of slashdot.org. ;rrset 86374 6 0 2 0 org. 86374 IN NS a0.org.afilias-nst.info. org. 86374 IN NS a2.org.afilias-nst.info. org. 86374 IN NS b0.org.afilias-nst.org. org. 86374 IN NS b2.org.afilias-nst.org. org. 86374 IN NS c0.org.afilias-nst.info. org. 86374 IN NS d0.org.afilias-nst.org. ;rrset 86374 2 1 2 0 org. 86374 IN DS 21366 7 1 E6C1716CFB6BDC84E84CE1AB5510DAC69173B5B2 org. 86374 IN DS 21366 7 2 96EEB2FFD9B00CD4694E78278B5EFDAB0A80446567B69F634DA078F0D90F01BA org. 86374 IN RRSIG DS 8 1 86400 20150219170000 20150209160000 16665 . RxYpI0BzYpGTE/PjRQdR4SZaxlvXCja3SJyx10JagTfz20gnltl4ar94GOwp8bA/ktY/7JxMoJvzCTAtcsGaTGRv04yDHr7WaydMxZuPCP9YT9Ixc+fX9IAZlSfwLCkBQgiC0mVeRiq+LmbIJhI2grJbTtvy96O9mipAqkFR42g= ;{id = 16665} ;rrset 86374 1 0 1 0 d0.org.afilias-nst.org. 172774 IN A 199.19.57.1 ;rrset 86374 1 0 1 0 d0.org.afilias-nst.org. 172774 IN AAAA 2001:500:f::1 ;rrset 86374 1 0 1 0 c0.org.afilias-nst.info. 172774 IN A 199.19.53.1 ;rrset 86374 1 0 1 0 c0.org.afilias-nst.info. 172774 IN AAAA 2001:500:b::1 ;rrset 86374 1 0 1 0 b2.org.afilias-nst.org. 172774 IN A 199.249.120.1 ;rrset 86374 1 0 1 0 b2.org.afilias-nst.org. 172774 IN AAAA 2001:500:48::1 ;rrset 86374 1 0 1 0 b0.org.afilias-nst.org. 172774 IN A 199.19.54.1 ;rrset 86374 1 0 1 0 b0.org.afilias-nst.org. 172774 IN AAAA 2001:500:c::1 ;rrset 86374 1 0 1 0 a2.org.afilias-nst.info. 172774 IN A 199.249.112.1 ;rrset 86374 1 0 1 0 a2.org.afilias-nst.info. 172774 IN AAAA 2001:500:40::1 ;rrset 86374 1 0 1 0 a0.org.afilias-nst.info. 172774 IN A 199.19.56.1 ;rrset 86374 1 0 1 0 a0.org.afilias-nst.info. 172774 IN AAAA 2001:500:e::1 Delegation with 6 names, of which 0 can be examined to query further addresses. It provides 12 IP addresses. 2001:500:e::1 not in infra cache. 199.19.56.1 not in infra cache. 2001:500:40::1 not in infra cache. 199.249.112.1 not in infra cache. 2001:500:c::1 not in infra cache. 199.19.54.1 not in infra cache. 2001:500:48::1 not in infra cache. 199.249.120.1 rto 356 msec, ttl 874, ping 8 var 87 rtt 356, tA 0, tAAAA 0, tother 0, EDNS 0 probed. 2001:500:b::1 not in infra cache. 199.19.53.1 rto 482 msec, ttl 874, ping 22 var 115 rtt 482, tA 0, tAAAA 0, tother 0, EDNS 0 probed. 2001:500:f::1 not in infra cache. 199.19.57.1 not in infra cache.
-
I'll ask again…
In Services: DNS Resolver: Advanced
Have you tried checking:
Harden Glue
Harden DNSSEC data
Unwanted Reply Threshold (10 million)
etc... ???
-
Have you tried checking:
Harden Glue
Harden DNSSEC data
These two in particular, if you don't have them enabled, enable them. I changed things to enable both those by default, and we'll add config upgrade code to turn those on for anyone who doesn't have them enabled upon upgrade to 2.2.1.
-
Yeah - I went on a voyage of discovery to find a combo that worked…
Less a matter of being smart. More a matter of random experimentation.Even reading and searching online yielded little results.
But yeah - Fixes alot.
I'm glad you guys are making it defaults.
-
Couple more of the hardened stuff here: https://forum.pfsense.org/index.php?topic=88466.msg488511#msg488511
- harden-referral-path is hardcoded to no in unbound.inc ATM :(
- harden-below-nxdomain can be set via the advanced config
(created a feature request for the two above)
- there's also this use-caps-for-id thing (patch to expose in GUI in 4205 or stick use-caps-for-id: yes to advanced config.
Another thing that makes me wonder… dnsmasq AFAICT is not compiled with DNSSEC support at all on pfSense even though it's apparently supported now. Hmmm. And of course, no matter what you are completely screwed with unsigned domains, and the DNS protocol is a piece of insecure crap.
A couple of suggestions for your own servers:
- sign your domains (ahem… why's pfsense.org not signed?!)
- if your registrar does not allow you to do so, switch registrar
- if the TLD you are using is not signed, choose a different one
- use DANE/TLSA for your critical servers at least.
Some useful tools:
-
DNSSEC Test - check your resolvers
-
DNSSEC Analyzer - check your domain for DNSSEC problems
-
Update for our situation (only dns active, harden glue settings not (yet) updated).
The situation happened again a few minutes ago, and here it is how it looked in the resolver.log (first time "csof" is visible today) :Feb 10 10:03:18 pf unbound: [24807:1] info: query response was ANSWER Feb 10 10:03:18 pf unbound: [24807:1] info: response for 239.75.246.46.in-addr.arpa. PTR IN Feb 10 10:03:18 pf unbound: [24807:1] info: reply from <75.246.46.in-addr.arpa.> 91.213.246.6#53 Feb 10 10:03:18 pf unbound: [24807:1] info: query response was REFERRAL Feb 10 10:03:18 pf unbound: [24807:1] info: resolving ns1.transitionalprotocol.net. AAAA IN Feb 10 10:03:18 pf unbound: [24807:1] info: resolving ns2.transitionalprotocol.net. A IN Feb 10 10:03:18 pf unbound: [24807:1] info: resolving f.gtld-servers.net. AAAA IN Feb 10 10:03:18 pf unbound: [24807:1] info: resolving g.gtld-servers.net. AAAA IN Feb 10 10:03:18 pf unbound: [24807:1] info: resolving k.gtld-servers.net. AAAA IN Feb 10 10:03:18 pf unbound: [24807:1] info: resolving l.gtld-servers.net. AAAA IN Feb 10 10:03:18 pf unbound: [24807:1] info: resolving ns2.transitionalprotocol.net. AAAA IN Feb 10 10:03:18 pf unbound: [24807:1] info: resolving h.gtld-servers.net. AAAA IN Feb 10 10:03:18 pf unbound: [24807:1] info: resolving i.gtld-servers.net. AAAA IN Feb 10 10:03:18 pf unbound: [24807:1] info: resolving ns1.transitionalprotocol.net. A IN Feb 10 10:03:18 pf unbound: [24807:1] info: resolving m.gtld-servers.net. AAAA IN Feb 10 10:03:18 pf unbound: [24807:1] info: resolving d.gtld-servers.net. AAAA IN Feb 10 10:03:18 pf unbound: [24807:1] info: response for a.ns.portlane.net. AAAA IN Feb 10 10:03:18 pf unbound: [24807:1] info: reply from <portlane.net.>80.67.0.6#53 Feb 10 10:03:18 pf unbound: [24807:1] info: query response was ANSWER Feb 10 10:03:18 pf unbound: [24807:1] info: response for b.ns.portlane.net. AAAA IN Feb 10 10:03:18 pf unbound: [24807:1] info: reply from <portlane.net.>80.67.0.6#53 Feb 10 10:03:18 pf unbound: [24807:1] info: query response was ANSWER Feb 10 10:03:18 pf unbound: [24807:1] info: response for 125.10.113.188.in-addr.arpa. PTR IN Feb 10 10:03:18 pf unbound: [24807:1] info: reply from <10.113.188.in-addr.arpa.> 80.240.240.2#53 Feb 10 10:03:18 pf unbound: [24807:1] info: query response was ANSWER Feb 10 10:03:18 pf unbound: [24807:1] info: resolving 113.188.in-addr.arpa. DS IN Feb 10 10:03:18 pf unbound: [24807:1] notice: sendto failed: Operation not permitted Feb 10 10:03:18 pf unbound: [24807:1] notice: remote address is 2001:500:13::c7d4:35 port 53 Feb 10 10:03:18 pf unbound: [24807:1] info: error sending query to auth server 2001:500:13::c7d4:35 port 53 Feb 10 10:03:18 pf unbound: [24807:1] info: response for ns1.transitionalprotocol.net. AAAA IN Feb 10 10:03:18 pf unbound: [24807:1] info: reply from <net.>192.52.178.30#53 Feb 10 10:03:18 pf unbound: [24807:1] info: query response was REFERRAL Feb 10 10:03:18 pf unbound: [24807:1] info: resolving ns2.csof.net. AAAA IN Feb 10 10:03:18 pf unbound: [24807:1] info: resolving ns1.csof.net. AAAA IN Feb 10 10:03:18 pf unbound: [24807:1] info: resolving e.gtld-servers.net. AAAA IN Feb 10 10:03:18 pf unbound: [24807:1] info: resolving d.gtld-servers.net. AAAA IN Feb 10 10:03:18 pf unbound: [24807:1] info: response for ns2.transitionalprotocol.net. A IN Feb 10 10:03:18 pf unbound: [24807:1] info: reply from <net.>192.52.178.30#53 Feb 10 10:03:18 pf unbound: [24807:1] info: query response was REFERRAL Feb 10 10:03:18 pf unbound: [24807:0] info: response for 125.74.1.116.in-addr.arpa. PTR IN Feb 10 10:03:18 pf unbound: [24807:0] info: reply from <1.116.in-addr.arpa.> 202.103.224.69#53 Feb 10 10:03:18 pf unbound: [24807:0] info: query response was NXDOMAIN ANSWER Feb 10 10:03:18 pf unbound: [24807:1] info: response for ns2.transitionalprotocol.net. AAAA IN Feb 10 10:03:18 pf unbound: [24807:1] info: reply from <net.>192.26.92.30#53 Feb 10 10:03:18 pf unbound: [24807:1] info: query response was REFERRAL Feb 10 10:03:18 pf unbound: [24807:1] info: response for ns2.transitionalprotocol.net. A IN Feb 10 10:03:18 pf unbound: [24807:1] info: reply from <transitionalprotocol.net.>212.6.183.201#53 Feb 10 10:03:18 pf unbound: [24807:1] info: query response was ANSWER Feb 10 10:03:18 pf unbound: [24807:1] info: response for ns1.transitionalprotocol.net. A IN Feb 10 10:03:18 pf unbound: [24807:1] info: reply from <net.>192.31.80.30#53 Feb 10 10:03:18 pf unbound: [24807:1] info: query response was REFERRAL Feb 10 10:03:18 pf unbound: [24807:1] info: response for ns2.csof.net. AAAA IN Feb 10 10:03:18 pf unbound: [24807:1] info: reply from <net.>192.43.172.30#53 Feb 10 10:03:18 pf unbound: [24807:1] info: query response was REFERRAL Feb 10 10:03:18 pf unbound: [24807:1] info: response for ns2.transitionalprotocol.net. AAAA IN Feb 10 10:03:18 pf unbound: [24807:1] info: reply from <transitionalprotocol.net.>212.6.183.201#53 Feb 10 10:03:18 pf unbound: [24807:1] info: query response was nodata ANSWER Feb 10 10:03:18 pf unbound: [24807:1] info: response for 113.188.in-addr.arpa. DS IN Feb 10 10:03:18 pf unbound: [24807:1] info: reply from <188.in-addr.arpa.> 199.212.0.53#53 Feb 10 10:03:18 pf unbound: [24807:1] info: query response was nodata ANSWER Feb 10 10:03:18 pf unbound: [24807:1] info: NSEC RRset for the referral proved not a delegation point Feb 10 10:03:18 pf unbound: [24807:1] info: NSEC RRset for the referral proved no DS. Feb 10 10:03:18 pf unbound: [24807:1] info: Verified that unsigned response is INSECURE Feb 10 10:03:18 pf unbound: [24807:1] info: response for ns1.transitionalprotocol.net. A IN Feb 10 10:03:18 pf unbound: [24807:1] info: reply from <transitionalprotocol.net.>54.77.72.254#53 Feb 10 10:03:18 pf unbound: [24807:1] info: query response was ANSWER Feb 10 10:03:18 pf unbound: [24807:1] info: response for ns1.csof.net. AAAA IN Feb 10 10:03:18 pf unbound: [24807:1] info: reply from <net.>192.26.92.30#53 Feb 10 10:03:18 pf unbound: [24807:1] info: query response was REFERRAL Feb 10 10:03:18 pf unbound: [24807:1] info: response for 239.75.246.46.in-addr.arpa. PTR IN Feb 10 10:03:18 pf unbound: [24807:1] info: reply from <239.75.246.46.in-addr.arpa.> 195.22.26.248#53 Feb 10 10:03:18 pf unbound: [24807:1] info: query response was ANSWER Feb 10 10:03:18 pf unbound: [24807:1] info: resolving 246.46.in-addr.arpa. DS IN Feb 10 10:03:18 pf unbound: [24807:1] info: NSEC RRset for the referral proved not a delegation point Feb 10 10:03:18 pf unbound: [24807:1] info: NSEC RRset for the referral proved no DS. Feb 10 10:03:18 pf unbound: [24807:1] info: Verified that unsigned response is INSECURE Feb 10 10:03:18 pf unbound: [24807:1] info: response for ns2.csof.net. AAAA IN Feb 10 10:03:18 pf unbound: [24807:1] info: reply from <csof.net.>208.109.255.32#53 Feb 10 10:03:18 pf unbound: [24807:1] info: query response was nodata ANSWER Feb 10 10:03:18 pf unbound: [24807:0] info: response for 166.31.215.188.in-addr.arpa. PTR IN Feb 10 10:03:18 pf unbound: [24807:0] info: reply from <31.215.188.in-addr.arpa.> 89.39.166.2#53 Feb 10 10:03:18 pf unbound: [24807:0] info: query response was THROWAWAY Feb 10 10:03:19 pf unbound: [24807:1] info: response for ns1.csof.net. AAAA IN Feb 10 10:03:19 pf unbound: [24807:1] info: reply from <csof.net.>208.109.255.32#53 Feb 10 10:03:19 pf unbound: [24807:1] info: query response was nodata ANSWER Feb 10 10:03:19 pf unbound: [24807:1] info: resolving 205.112.194.173.in-addr.arpa. PTR IN Feb 10 10:03:19 pf unbound: [24807:1] info: resolving ns2.google.com. AAAA IN Feb 10 10:03:19 pf unbound: [24807:1] info: resolving ns4.google.com. AAAA IN Feb 10 10:03:19 pf unbound: [24807:1] info: resolving ns1.google.com. AAAA IN Feb 10 10:03:19 pf unbound: [24807:1] info: response for ns1.transitionalprotocol.net. AAAA IN Feb 10 10:03:19 pf unbound: [24807:1] info: reply from <transitionalprotocol.net.>212.6.183.201#53 Feb 10 10:03:19 pf unbound: [24807:1] info: query response was nodata ANSWER Feb 10 10:03:19 pf unbound: [24807:1] info: response for 205.112.194.173.in-addr.arpa. PTR IN Feb 10 10:03:19 pf unbound: [24807:1] info: reply from <194.173.in-addr.arpa.> 216.239.36.10#53 Feb 10 10:03:19 pf unbound: [24807:1] info: query response was ANSWER Feb 10 10:03:20 pf unbound: [24807:1] info: resolving www.insign.ch. A IN Feb 10 10:03:20 pf unbound: [24807:1] info: resolving dns2.insign.ch. AAAA IN Feb 10 10:03:20 pf unbound: [24807:1] info: resolving dns1.insign.ch. AAAA IN Feb 10 10:03:20 pf unbound: [24807:1] info: response for www.insign.ch. A IN Feb 10 10:03:20 pf unbound: [24807:1] info: reply from <insign.ch.>46.175.9.120#53 Feb 10 10:03:20 pf unbound: [24807:1] info: query response was ANSWER Feb 10 10:03:20 pf unbound: [24807:1] info: NSEC3s for the referral proved no DS. Feb 10 10:03:20 pf unbound: [24807:1] info: Verified that unsigned response is INSECURE Feb 10 10:03:20 pf unbound: [24807:1] info: resolving 64.94.172.95.in-addr.arpa. PTR IN Feb 10 10:03:20 pf unbound: [24807:1] info: resolving ns-b.pnap.net. AAAA IN Feb 10 10:03:20 pf unbound: [24807:1] info: resolving ns-c.pnap.net. AAAA IN Feb 10 10:03:20 pf unbound: [24807:1] info: resolving ns-a.pnap.net. AAAA IN Feb 10 10:03:20 pf unbound: [24807:1] info: resolving 185.112.194.173.in-addr.arpa. PTR IN Feb 10 10:03:20 pf unbound: [24807:1] info: resolving ns3.google.com. AAAA IN Feb 10 10:03:20 pf unbound: [24807:1] info: resolving ns4.google.com. AAAA IN Feb 10 10:03:20 pf unbound: [24807:1] info: resolving ns1.google.com. AAAA IN Feb 10 10:03:20 pf unbound: [24807:1] info: response for 185.112.194.173.in-addr.arpa. PTR IN Feb 10 10:03:20 pf unbound: [24807:1] info: reply from <194.173.in-addr.arpa.> 216.239.32.10#53 Feb 10 10:03:20 pf unbound: [24807:1] info: query response was ANSWER Feb 10 10:03:20 pf unbound: [24807:1] info: response for 64.94.172.95.in-addr.arpa. PTR IN Feb 10 10:03:20 pf unbound: [24807:1] info: reply from <94.172.95.in-addr.arpa.> 64.95.61.4#53 Feb 10 10:03:41 pf unbound: [24807:0] info: query response was REFERRAL Feb 10 10:03:41 pf unbound: [24807:0] info: resolving dns4.bigrock.in. AAAA IN Feb 10 10:03:41 pf unbound: [24807:0] info: resolving dns2.bigrock.in. AAAA IN Feb 10 10:03:41 pf unbound: [24807:0] info: resolving asia1.akam.net. AAAA IN Feb 10 10:03:41 pf unbound: [24807:0] notice: sendto failed: Operation not permitted Feb 10 10:03:41 pf unbound: [24807:0] notice: remote address is 2600:1401:2::43 port 53 Feb 10 10:03:41 pf unbound: [24807:0] info: error sending query to auth server 2600:1401:2::43 port 53 Feb 10 10:03:41 pf unbound: [24807:1] info: response for ns0.mirasystem.net. A IN Feb 10 10:03:41 pf unbound: [24807:1] info: reply from <net.>54.72.8.183#53 Feb 10 10:03:41 pf unbound: [24807:1] info: query response was ANSWER Feb 10 10:03:42 pf unbound: [24807:0] info: response for asia1.akam.net. AAAA IN Feb 10 10:03:42 pf unbound: [24807:0] info: reply from <akam.net.>184.85.248.67#53 Feb 10 10:03:42 pf unbound: [24807:0] info: query response was nodata ANSWER Feb 10 10:03:42 pf unbound: [24807:0] info: response for dns2.bigrock.in. AAAA IN Feb 10 10:03:42 pf unbound: [24807:0] info: reply from <bigrock.in.>2.22.230.64#53 Feb 10 10:03:42 pf unbound: [24807:0] info: query response was nodata ANSWER Feb 10 10:03:42 pf unbound: [24807:1] info: response for ns1.epn.ru. A IN Feb 10 10:03:42 pf unbound: [24807:1] info: reply from <epn.ru.>195.22.26.248#53 Feb 10 10:03:42 pf unbound: [24807:1] info: query response was ANSWER Feb 10 10:03:42 pf unbound: [24807:1] info: response for ns0.epn.ru. AAAA IN Feb 10 10:03:42 pf unbound: [24807:1] info: reply from <epn.ru.>195.22.26.248#53 Feb 10 10:03:42 pf unbound: [24807:1] info: query response was nodata ANSWER Feb 10 10:03:42 pf unbound: [24807:1] info: response for ns0.epn.ru. A IN Feb 10 10:03:42 pf unbound: [24807:1] info: reply from <epn.ru.>195.22.26.248#53 Feb 10 10:03:42 pf unbound: [24807:1] info: query response was ANSWER Feb 10 10:03:42 pf unbound: [24807:1] info: response for ns1.epn.ru. AAAA IN Feb 10 10:03:42 pf unbound: [24807:1] info: reply from <epn.ru.>195.22.26.248#53 Feb 10 10:03:42 pf unbound: [24807:1] info: query response was nodata ANSWER Feb 10 10:03:42 pf unbound: [24807:1] info: response for ns1.hn.cnc.cn. AAAA IN Feb 10 10:03:42 pf unbound: [24807:1] info: reply from <cnc.cn.>210.52.207.2#53 Feb 10 10:03:42 pf unbound: [24807:1] info: query response was REFERRAL Feb 10 10:03:42 pf unbound: [24807:1] info: response for ns1.hn.cnc.cn. A IN Feb 10 10:03:42 pf unbound: [24807:1] info: reply from <cnc.cn.>210.52.207.2#53 Feb 10 10:03:42 pf unbound: [24807:1] info: query response was REFERRAL Feb 10 10:03:42 pf unbound: [24807:1] info: response for ns2.hn.cnc.cn. A IN Feb 10 10:03:42 pf unbound: [24807:1] info: reply from <cnc.cn.>210.52.207.2#53 Feb 10 10:03:42 pf unbound: [24807:1] info: query response was REFERRAL Feb 10 10:03:42 pf unbound: [24807:1] info: response for ns2.hn.cnc.cn. AAAA IN</cnc.cn.></cnc.cn.></cnc.cn.></epn.ru.></epn.ru.></epn.ru.></epn.ru.></bigrock.in.></akam.net.></net.></insign.ch.></transitionalprotocol.net.></csof.net.></csof.net.></net.></transitionalprotocol.net.></transitionalprotocol.net.></net.></net.></transitionalprotocol.net.></net.></net.></net.></portlane.net.></portlane.net.>
At the end, all requests get resolved as "195.22.26.248".
Trigger request couldn't yet be found (the log rule on port 53 for requests not going over the router returned nothing at this moment).So when it starts, every single request then go over one of these ns1.csof.net name server, and get answered as if csof.net was a root server, but with wrong data. For example:
Normal case when querying the pfsense resolver (when everything is ok) :
om@ompc:~> dig -t ns @192.168.1.100 ch. ; <<>> DiG 9.9.5-rpz2+rl.14038.05-P1 <<>> -t ns @192.168.1.100 ch. ; (1 server found) ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 62805 ;; flags: qr rd ra ad; QUERY: 1, ANSWER: 7, AUTHORITY: 0, ADDITIONAL: 14 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 4096 ;; QUESTION SECTION: ;ch. IN NS ;; ANSWER SECTION: ch. 85292 IN NS c.nic.ch. ch. 85292 IN NS b.nic.ch. ch. 85292 IN NS e.nic.ch. ch. 85292 IN NS d.nic.ch. ch. 85292 IN NS a.nic.ch. ch. 85292 IN NS h.nic.ch. ch. 85292 IN NS f.nic.ch. ;; ADDITIONAL SECTION: a.nic.ch. 171692 IN A 130.59.1.80 a.nic.ch. 171692 IN AAAA 2001:620::4 b.nic.ch. 171692 IN A 130.59.211.10 b.nic.ch. 171692 IN AAAA 2001:620::5 c.nic.ch. 171692 IN A 147.28.0.39 c.nic.ch. 171692 IN AAAA 2001:418:1::39 d.nic.ch. 171692 IN A 200.160.0.5 d.nic.ch. 171692 IN AAAA 2001:12ff:0:a20::5 e.nic.ch. 171692 IN A 194.0.17.1 e.nic.ch. 171692 IN AAAA 2001:678:3::1 f.nic.ch. 171692 IN A 194.146.106.10 f.nic.ch. 171692 IN AAAA 2001:67c:1010:2::53 h.nic.ch. 171692 IN A 194.42.48.120 ;; Query time: 0 msec ;; SERVER: 192.168.1.100#53(192.168.1.100) ;; WHEN: Tue Feb 10 10:55:52 CET 2015 ;; MSG SIZE rcvd: 427
When querying this ns1.csof.net server :
om@ompc:~> dig -t ns @ns1.csof.net ch. ; <<>> DiG 9.9.5-rpz2+rl.14038.05-P1 <<>> -t ns @ns1.csof.net ch. ; (1 server found) ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 45232 ;; flags: qr aa rd; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 4 ;; WARNING: recursion requested but not available ;; QUESTION SECTION: ;ch. IN NS ;; ANSWER SECTION: ch. 172800 IN NS ns1.csof.net. ch. 172800 IN NS ns2.csof.net. ch. 172800 IN NS ns3.csof.net. ch. 172800 IN NS ns4.csof.net. ;; ADDITIONAL SECTION: ns1.csof.net. 100 IN A 54.77.72.254 ns2.csof.net. 100 IN A 212.6.183.201 ns3.csof.net. 100 IN A 195.22.26.199 ns4.csof.net. 100 IN A 54.72.8.183 ;; Query time: 54 msec ;; SERVER: 54.77.72.254#53(54.77.72.254) ;; WHEN: Tue Feb 10 10:45:26 CET 2015 ;; MSG SIZE rcvd: 164
I now increased the verbosity of unbound with "unbound-control -c /var/unbound/unbound.conf verbosity 4" and waiting for another hit… If you have other suggestions in the mean time, please feel free.