Periodic since 2.2 pages load blank, certs invalid

cmb

@Derelict:

If you were affected by this, what part of the world are you in? Just curious. Interesting incident.

North East USA.

Seems to be geographically diverse. At least two in the US in this thread, one in the UK.

kejianshi

The pfsense in question is in Maryland, for me.

Trel

By the way, I can confirm for sure that pfsense was seeing the bad DNS.

I have an alias for facebook.com
I saw this in the resolver log

filterdns: adding entry 195.22.26.248 to table Social_Test on host facebook.com

195.22.26.248 is the bad IP. PFSense itself saw that when it did its update on the alias resolution.

kejianshi

https://www.virustotal.com/en/ip-address/195.22.26.248/information/

https://www.robtex.net/en/advisory/ip/195/22/26/248/

Seems like there is an associated IP block thats pretty much into everything bad.

Trel

@kejianshi:

https://www.virustotal.com/en/ip-address/195.22.26.248/information/

https://www.robtex.net/en/advisory/ip/195/22/26/248/

Seems like there is an associated IP block thats pretty much into everything bad.

I don't doubt that, but that doesn't answer the question as to how when using google dns and level3 dns with unbound, that legitimate sites started resolving to this IP range, unless I'm missing something.

kejianshi

No idea

agreenfield1

Same thing happened to me this morning: https certs signed by lolcat, all dns inquiries not handled by pfsense directly give 195.22.26.248, and using the Google DNS and Level 3 dns servers. I was able to resolve the issue for the time being by checking the 'Allow DNS server list to be overridden by DHCP/PPP on WAN' box, which presumably switched pfsense from using the compromised/poisoned DNS server to my ISPs DNS server.

I originally thought this issue was unrelated to pfsense, and posted the issue here:https://forum.pfsense.org/index.php?topic=88238.0. But after seeing this thread, it seems like pfsense 2.2 / DNS Resolver / Unbound may be a factor?

Configuration: PFSense 2.2, DNS Resolver, GoogleDNS and Level3 as primary and secondary DNS servers respectively.

kejianshi

Nope - Because the same thing was happening to me using dnsmasq…

Actually switching to unbound + DNSSEC cured it.

Derelict

I originally thought this issue was unrelated to pfsense, and posted the issue here:https://forum.pfsense.org/index.php?topic=88238.0. But after seeing this thread, it seems like pfsense 2.2 / DNS Resolver / Unbound may be a factor?

This has nothing to do with pfSense. It has to do with you relying solely on google/level3 for all your DNS and someone is playing with it.

kejianshi

Yep… Now who could do that on a broad basis?

Derelict

It's intriguing.

Trel

Actually once I switched fully to unbound + DNSSEC only, I had a new issue. At the same times, unbound would stop working. The service would be running, but it wouldn't resolve anything until I restarted the service.

I finally found a common thread for that happening. It almost always directly followed someone doing a lookup of

api-nyc01.exip.org

or

ns3.csof.net

The IP for those are in the 195.22.x range that was mentioned earlier.

Almost without fail, trying to access one of those, causes unbound to stop working until I restart the service.

If someone is willing to look at that, because of how it lines up, it looks like trying to access/doing a lookup on those domains will either cause the blank pages and lolcat certs, or will cause unbound to stop resolving until the service is restarted.

It's too coincidental to ignore in this case.

doktornotor

@Trel:

It almost always directly followed someone doing a lookup of
api-nyc01.exip.org
or
ns3.csof.net
The IP for those are in the 195.22.x range that was mentioned earlier.
Almost without fail, trying to access one of those, causes unbound to stop working until I restart the service.

Tried both, unbound still working. :) Apparently no NSA love here. :'( ;D

Trel

Now, I also had block rules in place for that range of IP.
I wonder if that could interact in some way.

Additionally, if you have Snort/Suricata installed, do you now have alerts mentioning the Anubis DNS Sinkhole?

doktornotor

@Trel:

if you have Snort/Suricata installed

Noooooooooooooo!!!

kejianshi

!!!

heper

(unbound has been enabled for more then a month without issues … until an hour ago)
i've suddenly been experiencing the blank pages + dns redirects to buydomains.com for lots of valid domains.

i tried to fix it by enabling dnssec ... didn't help

for now i've enabled "forwarding mode" on unbound ... this seems to fix the issue.

Derelict

Unbound in resolver mode? That makes no sense. The deal is is that makes it pretty much impossible to affect everything. They have to target specific name servers for specific domains (or .com, or . (root) etc..

What DNS servers are you handing out to your clients? Running unbound means nothing if your clients are going to 8.8.8.8 / 8.8.4.4 for DNS.

Want an easy way to find out? Block TCP/UDP 53 on LAN to everything but your unbound and see what breaks. :) Or pass with logging and see what's logged….

kejianshi

After you properly set up DNS and DNSSEC, you still have to clear DNS cache on each client and also have to make sure your clients are not infected with something or running some stupid browser add-on that hijacks things.

2chemlud

@Derelict:

…. Block TCP/UDP 53 on LAN to everything but your unbound and see what breaks. :) Or pass with logging and see what's logged....

Port 53 is not allowed at my network for more than a year. Doing fine with the DNS servers in the General setup and keep awful devices such as Buffallo Linkstations etc from phoning home…