DNS Servers being blocked
-
Off and on I see DNS servers being blocked. Like now, I see 8.8.8.8 being blocked for "SURICATA DNS Unsolicited response". I've seen Google, OpenDNS, and Quad9 DNS IPs be blocked for various reasons. Some, it's because of the response generated with an IP of a malicious domain. I assume it's because an infected machine queried for a honeypot or other known bad IP but it blocks the DNS server for the response. Ultimately I suppose I'd prefer the DNS servers not respond to those requests but there isn't really a way to stop them. Do I need to whitelist the IPs for the DNS servers? Is that normal?
-
@stewart said in DNS Servers being blocked:
Off and on I see DNS servers being blocked. Like now, I see 8.8.8.8 being blocked for "SURICATA DNS Unsolicited response". I've seen Google, OpenDNS, and Quad9 DNS IPs be blocked for various reasons. Some, it's because of the response generated with an IP of a malicious domain. I assume it's because an infected machine queried for a honeypot or other known bad IP but it blocks the DNS server for the response. Ultimately I suppose I'd prefer the DNS servers not respond to those requests but there isn't really a way to stop them. Do I need to whitelist the IPs for the DNS servers? Is that normal?
Yes, it is customary to whitelist the DNS servers used by the firewall and your clients (or put them in a Pass List) when using Legacy Mode.
-
I never realized that. We'll make the changes. Thanks!
-
@stewart said in DNS Servers being blocked:
I never realized that. We'll make the changes. Thanks!
The default Pass List will contain the DNS servers configured on the firewall, but if your clients are using something else (or if you are using forwarders) then you would need to manually add the other DNS servers to a Pass List.
The easiest and most flexible way to do this is to create an Alias on the firewall to contain all of your trusted DNS server IP addresses. Then go to the PASS LISTS tab and create a custom pass list. Leave the default-checked options in place, and in the bottom text box start typing the name of the alias you configured. It should auto-populate. Save the the new list.
Now go to the INTERFACE SETTINGS tab for the Suricata interface and scroll down until you see the section for specifying a Pass List. Select the name of your custom Pass List in the drop-down box and save the change. Restart Suricata on the interface and that should do it. Your trusted DNS hosts will no longer be blocked.
-
We have 9 different aliases that we implement that go into our Suricata alias that goes into the passlist. Email filtering services, hosted voip providers, compliance scanners, etc. Once it's set up, it's really easy to make the needed changes. Just need to make one for DNS. I appreciate the assistance!
-
@stewart said in DNS Servers being blocked:
"SURICATA DNS Unsolicited response"
Still, the DNS replies are being marked by Suricata (?) as "SURICATA DNS Unsolicited response".
Ok to whitelist them upfront, but it keep me wondering : what else is dropped ?edit : anyway : never mind.
-
@gertjan said in DNS Servers being blocked:
@stewart said in DNS Servers being blocked:
"SURICATA DNS Unsolicited response"
Still, the DNS replies are being marked by Suricata (?) as "SURICATA DNS Unsolicited response".
Ok to whitelist them upfront, but it keep me wondering : what else is dropped ?edit : anyway : never mind.
Here is a link I just found discussing this particular rule. Could be due to some asymmetric routing issue, or it could be malicious.
-
@stewart said in DNS Servers being blocked:
We have 9 different aliases that we implement that go into our Suricata alias that goes into the passlist. Email filtering services, hosted voip providers, compliance scanners, etc. Once it's set up, it's really easy to make the needed changes. Just need to make one for DNS. I appreciate the assistance!
And just to be clear -- putting a host in a Pass List will prevent a block but will not suppress the alert on the ALERTS tab. So you will still see alerts on this rule. If you want to suppress a particular rule entirely, then you can add the rule to a Suppress List. Doing that will prevent the alert and the corresponding block. Finally, you can also completely disable that particular rule and accomplish the same thing as a Suppress List entry. A Suppress List allows you more options for tailoring the alert suppression by host IP. Disabling a rule will disable it for all hosts. Disabling is more CPU efficient, though, since with a Suppress List the rule is still processed.
-
But if I whitelist just those legitimate IPs, I'm only accepting the potential DoS attacks from the likes of Google, OpenDNS, and QuadDNS specific DNS servers. I would think that would still leave me pretty safe.
-
@stewart said in DNS Servers being blocked:
But if I whitelist just those legitimate IPs, I'm only accepting the potential DoS attacks from the likes of Google, OpenDNS, and QuadDNS specific DNS servers. I would think that would still leave me pretty safe.
I agree. We have to trust some hosts out there, and the big DNS providers are probably OK.
-
@stewart said in DNS Servers being blocked:
We have 9 different aliases that we implement that go into our Suricata alias that goes into the passlist. Email filtering services, hosted voip providers, compliance scanners, etc. Once it's set up, it's really easy to make the needed changes. Just need to make one for DNS. I appreciate the assistance!
Good setup! Using the nested aliases feature is a great way to manage a firewall. Makes future maintenance changes easy to accomplish and much less prone to error. For example, removing a retired host or updating the IP address for an active host. You change it in one place (the affected alias) and that change is automatically propagated to all impacted firewall rules (and packages such as Suricata or Snort). No chance that you forget to change the IP in some rule if it's used in many places.
-
And while I know pfSense fairly well, it makes it easy to train and instruct other techs as they come onboard who aren't as knowledgeable.
-
Commentary and question.
-
If you want heightened DNS security and you properly enable DNSSEC validation (client side of DNSSEC), you cannot block any destination port 53. Additionally, EDNS0 (4K response) needs to be supported - eg: no limiting to 512 bytes. This also means that you cannot "forward" to a third party - this is inherently advantageous for a couple reasons: A) using the root servers [directly] means lowest possibly latency and the ability to ensure application of policy and B) you aren't disclosing whom you [may] communicate with, to a potentially interested 3rd party. Using company 'X' DNS servers to forward your DNS lookups adds latency and presents a "picture" of where you may go, whom you [may] communicate with, etc. - excellent marketing material and paints an interesting picture of an organization's activities, just through lookup requests. Lastly, when you forward, you are subject to whatever controls/protections that the upstream may (or may not) apply - so you could actually end up having a "reduced" security disposition vs. handling internally and ensuring application of policy.
-
Given the fact that one might inadvertently have something incorrectly blocked (outbound), may be good to have something automated that does periodic lookup and updates an alias table. Thinking may be something as "DNS_root_servers", "DNS_com_servers", "DNS_net_servers" and so on. The lists would be relatively short, but immensely critical to preventing inappropriate action(s).
As to the 'hit' on the alert for a DNS lookup, it begs a couple questions:
- If root server - why? The root servers are tightly controlled and heavily monitored for attacks, etc. The addresses are Anycast'd, defacto (have been for many years) to provide resilience and capacity..
- What caused the rule to come into existence in the first place? (use case). Was the rule vetted against traffic streams to known 'good' sites, such the roots or TLD servers? Have seen a few spurious instances of this (between internal DNS servers and root servers) and leads one to believe that possibly a request (or second request) was missed in transit, therefore Suricata thought it was a response without a request? eg: PF knew it was valid and passed it, but Suricata didn't catch it in the stream. Would like to move to "inline", but alas IXL/IXLV interfaces aren't supported by netmap. :( The real test would be to enable query logging and determine that when an error occurs (in Suricata) - was there any issue according to the DNS Server's logs?
Looked around pfBlockerNG and didn't happen to see anything that simply creates or enables creation of such alias lists for inclusion/use elsewhere - is that correct?
Thanks!
-
-
I'll admit I've never touched DNSSEC. Sometimes the computers go to the domain controller for DNS. If it doesn't know it checks whatever the forwards are (usually OpenDNS, then QuadDNS, then Google and almost never the actual ISP). Sometimes they go straight to those remote DNS servers. Sure it could paint a picture of where the employees of the organization may be going to but what alternative is there?
You talk about enabling DNSSEC client side but as far as I know that's something that embedded into the DNS of a specific domain so I'm confused as to how you would implement what you are talking about. EDNS0 is just the extension to allow longer DNS entries. I don't see how pfSense would be in a position to affect it unless it is an intermediary DNS forwarder or relay. I don't see how you would enable/disable it on anything, really. Your DNS client either knows how to handle it or it needs to be updated from what I gather. Are you saying we should create our own internal root servers and download all DNS into them and use them for forwarded queries? I would agree that limits latency and information leaks but SMBs wouldn't have a viable way of doing that. Your first two bullet points read like Chapter 7 of a technical textbook for a class I just started. They are all terms I understand but don't know how you are applying them and it doesn't make sense to me.
As to the "hit", here are a couple of examples:
Example 1:
A machine gets infected with ZeroAccess. It hits the DNS server to query where to find its CnC server. Suricata sees that query response from the DNS server as malicious because it knows the payload of the DNS response has the offending data so it blocks the DNS server. Suddenly, nobody is able to query DNS and an entire company is affected because of 1 machine. Obviously this needs to be rectified but Suricata is also blocking all communication with the ZeroAccess servers anyway so blocking DNS doesn't really help unless those CnC servers change (which they sometimes do).Example 2:
[1:2023883:1] ET DNS Query to a *.top domain - Likely Hostile [**] [Classification: Potentially Bad Traffic] [Priority: 2] {UDP} x.x.x.x:6215 -> 8.8.8.8:53
Someone's machine triggered the block by querying a .top domain. Why the rules consider all of .top as malicious I don't know.Example 3:
[1:2240001:2] SURICATA DNS Unsolicited response [**] [Classification: Generic Protocol Command Decode] [Priority: 3] {UDP} 8.8.8.8:53 -> x.x.x.x:24603
A possible DNS amplification attack. Not sure how to rectify it, but it blocks unsolicited responses which is probably a good thing.pfBlocker has the ipV4 and ipV6 tabs to add whitelist entries that go into the firewall rules. Nothing is automated, though. It's all manual.
-
@stewart said in DNS Servers being blocked:
If it doesn't know it checks whatever the forwards are (usually OpenDNS, then QuadDNS, then Google and almost never the actual ISP).
How exactly are you accomplishing that? Your clients are using all of these? Sounds like your AD dns forwards to all of these.. How are you setting it up to use those sequentially and in order? You can never be sure what NS is going to be asked when you list them as forwarders.. Even when you list them as 1st, 2nd, etc
Listing servers to forward too that do not answer the same is asking for problems... Quad and Open both filter.. But they don't always filter the same... So using both of them is going to be problematic... If you want to use a filtering NS to forward to.. then pick a company and list their multiple IPs you can use if you really want to put in more than one... But all of them are normally going to be anycast so putting in more than 1 is pointless.
I am not aware of any way in Microsoft you can have it subsequently ask different forwarders.. Even if you could when would it do this? Only on timeouts since if you get back a NX, there is zero reason to go ask the next NS.. On a time out before you ask the next one, how long do you wait before asking the next guy? By time you maybe get an answer from the 4th one your client for sure has given up..
Resolving is always going to be better choice - sure if you need to use some conditional forwarders for specific domains that is fine.. If your on a high latency connection where resolving might be problematic - say sat connection or something. Then sure forward... If you want to leverage some company filtering your dns responses then pick "1" company and only forward to their NSs..
if you have your clients pointing to anything other than your local NS, ie your AD DNS then your going to have problems as well... You do not point clients to NS that do not resolve the same stuff... Since again you can never be sure which NS going to be asked... Listing local NS and public NS pointless!!! And going to be nothing but problematic. Pointing to NS that filter and others that don't filter again its just plain borked!
-
@stewart said in DNS Servers being blocked:
Example 2:
[1:2023883:1] ET DNS Query to a .top domain - Likely Hostile [*] [Classification: Potentially Bad Traffic] [Priority: 2] {UDP} x.x.x.x:6215 -> 8.8.8.8:53
Someone's machine triggered the block by querying a .top domain. Why the rules consider all of .top as malicious I don't know.Example 2:
[1:2023883:1] ET DNS Query to a .top domain - Likely Hostile [*] [Classification: Potentially Bad Traffic] [Priority: 2] {UDP} x.x.x.x:6215 -> 8.8.8.8:53
Someone's machine triggered the block by querying a .top domain. Why the rules consider all of .top as malicious I don't know.Could be a buggy rule or it could be that the .top domains are dirt cheap and are used for suspect uses, can you do a packet capture from SURICATA ?
Do you run SURICATA just on the WAN interface ?
I use snort and see some iphone apps querying .top domains, makemkv does it also for the licensing.
-
Blocking all queries for anything**.top** is going to be a bad idea..
What is the rule number that is? They are not even listed in the top 20 shady TLDs
https://www.spamhaus.org/statistics/tlds/
https://www.symantec.com/blogs/feature-stories/top-20-shady-top-level-domainsAs with any IPS - having it block out of the box is going to cause you PAIN... You need to run it in monitor mode for quite some time looking at the hits on the rules to weed out the noise, and what is common for your network..
If because they are cheap - then man you need to be blocking a lot of tlds -- You can get sales of many .tlds for less than a buck the first year, etc. Namecheap almost always has a list of tlds for 0.88 .site, .pw, .host, etc.
-
below/inline
I'll admit I've never touched DNSSEC. Sometimes the computers go to the domain controller for DNS. If it doesn't know it checks whatever the forwards are (usually OpenDNS, then QuadDNS, then Google and almost never the actual ISP). Sometimes they go straight to those remote DNS servers. Sure it could paint a picture of where the employees of the organization may be going to but what alternative is there?
It should be that the DNS servers always go straight to the root servers. However, when you have "default forwarders" (in MS terms) - those specified IP Addresses will have [effectively] 100% visibility into all queries. Think of it this way, if you're already seeing instances where a 3rd party is queried, but the query eventually makes it to the roots (as last attempt via config) - that means that queries are already being dropped (overloaded DNS Server?) and all that's happening is that latency is being injected. In the case of MS, DNS fixates on each configured DNS for seven (7) seconds. Reducing this can create a self DOS'ing effect. BIND has a better behavior, but that aside. In contrast, if you use root hints - the 'knowledge' is spread across so that no singular entity has "the picture". Most organizations use a dedicated recursion tier for this purpose. eg: internal DNS Servers have default forwarders to one (or more) DNS Servers that handle all Internet lookups ("recursion").
You talk about enabling DNSSEC client side but as far as I know that's something that embedded into the DNS of a specific domain so I'm confused as to how you would implement what you are talking about. EDNS0 is just the extension to allow longer DNS entries. I don't see how pfSense would be in a position to affect it unless it is an intermediary DNS forwarder or relay. I don't see how you would enable/disable it on anything, really. Your DNS client either knows how to handle it or it needs to be updated from what I gather. Are you saying we should create our own internal root servers and download all DNS into them and use them for forwarded queries? I would agree that limits latency and information leaks but SMBs wouldn't have a viable way of doing that. Your first two bullet points read like Chapter 7 of a technical textbook for a class I just started. They are all terms I understand but don't know how you are applying them and it doesn't make sense to me.
Making no assertions on background, some quick reference info on DNSSEC:
- DNSSEC has two 'halves: 1) server: signing of the zone (the embedded component) and 2) consumer: validation
- the zone signing enables two chains: 1) authenticity of namespace (zones/records of the namespace) and 2) authenticity of infrastructure (devices serving DNS)
- the validation is simply that, validation of those two chains during resolution to determine authenticity.
- validation is handled in the recursion tier, not the consumer. Standard consumer has "no clue" about DNSSEC or validation - this is by intention. Protection is provided 'from above' which also increases useful of caching. eg: if you just validated the com TLD, then for duration of TTL - no need to re-validate COM TLD server(s). To put overhead of DNS lookups into a "cost" perspective for contrast: an authoritative query is $0.01, recursive query is $0.25, recursive DNSSEC query is $1.00, DDNS update is $1.50, DHCP lease is $5.00 and an average web page is about $350.00.
The 'trust anchor' provides an initial entry point for "trust" to begin. This is normally the root servers as a known 'good' entry point. A quick note on caching and DNS. Caching after a certain duration (TTL) is a liability. eg: nothing should have a TTL greater than about 7-8 hours. The recursion tier (or wherever you are handling recursion) should [optimally] reduce any TTL over 8 hours to 8 hours. There's still a lot of DNS infrastructure that have egregiously long TTLs in existence, which is diametrically opposing fluidity (change). Internally, TTLs should be the lessor of: 1 hour or shortest DHCP lease duration. Think of it as: DHCP consumer gets 1 hour lease, moves to another location (new lease/IP), now cached record is invalid - the liability side of caching.
In short, simply "signing" one's external DNS zone(s) doesn't provide any real value unless something on the consumer side validates - using those records (starting from the roots [logical trailing "."] on down). In simplistic terms:
Using "www.google.com." as example. Note that the trailing period is rarely entered, but is inferred - defacto.
Recursion becomes: . com google www
. is the query to the roots, which point the requesting DNS Server to the com TLD servers which then point the requesting DNS Server to google's nameservers and so on. The linkage derived through the NS records of parent to child namespace.If you wanted to see it in action, use dig. eg: "dig @<dns server ip> <fqdn to lookup> +trace". Additional flags that may be of interest "+dnssec", "+cdflag".
One should never create a "false root" - except for extremely rare and very unique cases. False root environments can be excessively challenging to manage, often being far more trouble than any value (suspect ROI on false roots is a grotesquely negative value).
The point being use standard recursion (eg: use the root servers and don't forward to any 3rd party). The point about the ACLs is around a protective measure against what appears to be some rule(sets) in IDS/IPS that incorrectly block "known good" destinations - especially traumatic in terms of root servers and/or TLD servers.
The EDNS0 comment is that all too often, there is something in the path that squelches DNS packets that exceed 512 bytes. For example: Cisco's "fixup" does a wonderful job of this. Packet shapers (aka "packet-shavers") also commonly create trauma in terms of DNS packets. Just be on the lookout/aware that there could be something else in-path that can create an issue. One of the historical means to help illuminate whether there was an issue around EDNS0 packets getting through the path was: "dig rs.dns-oarc.net txt +short", performed on the DNS server 'against the DNS server [itself]'. This would validate if the path supported 4k packets. More recent resolver variants may try to use smaller reply size [initially] to help avoid potential path issues.
As to the "hit", here are a couple of examples:
Example 1:
A machine gets infected with ZeroAccess. It hits the DNS server to query where to find its CnC server. Suricata sees that query response from the DNS server as malicious because it knows the payload of the DNS response has the offending data so it blocks the DNS server. Suddenly, nobody is able to query DNS and an entire company is affected because of 1 machine. Obviously this needs to be rectified but Suricata is also blocking all communication with the ZeroAccess servers anyway so blocking DNS doesn't really help unless those CnC servers change (which they sometimes do).That's why for DNS driven issues, it needs to be handled -IN- the DNS infrastructure. IDS/IPS solutions will invariably target the wrong 'consumer'. DNS 'RPZ' feeds "are your friend" for such things, alternatively something like FireEye which has direct DNS integration(s) to create/cause change within the DNS Server. The DNS server needs the intelligence to stop/address the issue. Remember, that from a communications stand point, DNS servers will logically appear as akin to a NAT'ing firewall - the actual "source" consumer is not known, except by the DNS Server.
While it may not be "elegant" and if RPZ feed(s) aren't an option - another avenue would be to get the IDS/IPS either inline (or port mirror for DNS Server port/s) and have it "block" the consumer's source IP on the firewall's inside interface to prevent that consumer from being able to go outbound. This would be as result of "consumer made CNC query to DNS Server or received CNC response from DNS Server", therefore we block all traffic destined anywhere from that source IP.
It wasn't all that long ago when where was malware that remained dormant if it -could- resolve a DNS entry and then only a couple weeks later a newer version went out that remained dormant if it -couldn't- resolve a DNS entry. eg: folks scrambled to add a DNS entry, but the "v2" strain worked in opposition. Never dull....
Example 2:
[1:2023883:1] ET DNS Query to a *.top domain - Likely Hostile [**] [Classification: Potentially Bad Traffic] [Priority: 2] {UDP} x.x.x.x:6215 -> 8.8.8.8:53
Someone's machine triggered the block by querying a .top domain. Why the rules consider all of .top as malicious I don't know.Example 3:
[1:2240001:2] SURICATA DNS Unsolicited response [**] [Classification: Generic Protocol Command Decode] [Priority: 3] {UDP} 8.8.8.8:53 -> x.x.x.x:24603
A possible DNS amplification attack. Not sure how to rectify it, but it blocks unsolicited responses which is probably a good thing.pfBlocker has the ipV4 and ipV6 tabs to add whitelist entries that go into the firewall rules. Nothing is automated, though. It's all manual.
Wonder how much of a pain it would be to be able to add a feature/function that would enable being able to get the nameserver IP Addresses for various items, such as the roots and TLD name servers for com/net/org to help reduce such behaviors. Not perfect, but may help in some cases.
-
Sorry, I was generalizing various companies into a single response. Normally if I'm using OpenDNS I will use 208.67.222.123 and 208.67.220.123 as both should filter the same. I normally used OpenDNS filtering primarily but some customers I'll use QuadDNS instead. Some providers have devices, like VoIP phones, that they require to use Google DNS 8.8.8.8, and 8.8.4.4.
If there is an AD then all clients point to it and it has forwarders that go out to OpenDNS. If there isn't a DC then I'll generally just use OpenDNS as forwarders in the router. I almost never use the actual ISP DNS servers if I can help it. Sometimes when there is an issue, though, an ISP will require it to test with.
I've often wondered about DNS sequential vs serial requests. They may be queried sequentially if a response doesn't come in time but not wait for a timeout from the first source before querying the second. If you put a secondary DNS server into a PC in addition to the Domain Controller they will fail randomly but it usually does work, sometimes for years. We often find that to be the case when we pick up a new client (we are an MSP) where the previous companies or in house tech put in 8.8.8.8 as a secondary DNS server and they never really knew when some PCs would start and stop connecting to the domain. Take out the Google DNS and the issues resolve but I've never determined a rhyme or reason to it.
By resolving, is the reference to having something like pfSense essentially be a DNS caching server? It would still need forwards and would cache the lookup. I believe MS DNS servers also cache the lookups but I've never looked into seeing just how long it holds the cache. Would the best policy for pfSense, then, to be a resolver instead of a forwarder? Seems like it would be. I haven't really paid much attention to it. DCs would use it as the forwarder and it would be the resolver and cache the responses.
-
Your caching NS should never cache anything longer than the TTL.. Be it MS, Bind, Unbound, Dnsmasq, etc. etc.. Atleast it shouldn't be.
Do you have the AD NS resolve, or do you have it forward.. If you had it forward then you could have it forward to pfsense and pfsense would resolve. Then you could also leverage dnssec, dnssec on a forwarder is pretty freaking pointless. Where dnssec matters is on the resolver.
So have your AD forward to unbound on pfsense, and pretty much out of the box your golden. I am NOT a fan of these services like quad and opendns... While they do provide a service for billy bob the user, no thanks I can do that myself.. I don't need to send you every query I make so you can filter it for me ;) I can filter out what I want to filter on it on my own so that clients don't get an answer to somebadsite.org
Pfblocker on pfsense can make it easy for you to do as well... Or something as simple as pihole can make it no brainer for local filtering vs sending every single query to some company that says sure we are good, and don't do anything with your info, etc. etc..
