(SOLVED) Snort detecting INDICATOR-COMPROMISE suspicious .null DNS query
-
Hi.
I found about 4 of these entries in the snort alerts, and they are sourced from the WAN IP.
"INDICATOR-COMPROMISE suspicious .null dns query"
I have snort's IPS policy set to security without blocking offenders, running on the WAN and DMZ interfaces. My pfsense version is 2.4.4-RELEASE-p2 (amd64)
The server in the DMZ is accessible from the Internet.
-
@tman904 said in Snort detecting INDICATOR-COMPROMISE suspicious .null dns query on my WAN:
INDICATOR-COMPROMISE suspicious .null dns query
And what was the specific query for? .null is a valid OpenNIC tld - just because snort flags/blocks it doesn't mean its actually bad.. Something asks pfsense for host.domain.tld, then yeah its going to try and resolve or forward that query (based on your settings in pfsense) out its wan IP.
btw - moved your thread to IDS/IPS section.
-
I don't know the contents of the query, but all of the queries are not going to dns servers on have set on the computers or pfsense. I looked some of them up and some go to japan others to the netherlands.
-
And what are some of these IPs... Resolving which is pfsense default setting, then yeah the queries would go all over the globe as it "resolves" and actually talks to the authoritative NS for domain its resolving.
OpenNIC is not normal public domains from ICANN, etc. pfsense would not be able to actually resolve those, unless setup for OpenNIC.
-
202.12.27.33
193.0.14.129
198.41.0.4
199.9.14.201
192.36.148.17also I didn't setup anything with opennic. I wasn't even aware of that.
-
@tman904 said in Snort detecting INDICATOR-COMPROMISE suspicious .null dns query on my WAN:
202.12.27.33
Those are all roots..
;; QUESTION SECTION:
;33.27.12.202.in-addr.arpa. IN PTR;; ANSWER SECTION:
33.27.12.202.in-addr.arpa. 86400 IN PTR M.ROOT-SERVERS.NET.;; ANSWER SECTION:
17.148.36.192.in-addr.arpa. 86400 IN PTR i.root-servers.net.;; ANSWER SECTION:
129.14.0.193.in-addr.arpa. 21599 IN PTR k.root-servers.net.So yeah - when unbound on pfsense is asked for host.something.NULL as the tld, it would ask roots - hey roots who is the NS for this tld .null...
You need to track down on your lan side network what is asking for whatever.NULL to see if legit or not... Could be billy bob user trying to access some p0rn site ;)
-
ok I'll start looking.
-
but those are never going to resolve unless you setup your DNS to understand how to talk to OpenNIC
You can turn on full query logging in unbound, but has to be done in the custom options box
server:
log-queries: yesedit, example I just did a query for
Apr 21 05:36:10 unbound 65356:0 info: 192.168.9.100 test.domain.null. A IN NXDOMAIN 0.013277 0 120
So you can see that my pc at 192.168.9.100 asked for that .null domain. If you turn on full query logging, you should be able to figure out who is asking for it, and exactly what real quick.
-
quick tcpdump for udp port 53 shows odd things. The server is resolving stackoverflow.com and some yandex stuff zen.spamhaus.org
-
Those don't seem too odd to me... Those are very legit domains.
See my edit above - you prob have easier time just turning on logging in unbound and then looking through the log for .null
-
I turned that on, but it isn't sending queries at the moment. I'll let it settle for a while and let you know. Thank you for the help btw.
-
Do you run pfBlocker?
It could be some dubious hosts hitting your WAN interface and pfBlocker doing a lookup for the host.
Is the alert just in the WAN interface or on the WAN and DMZ.
Also have a poke about in the /var/log/snort directory, there may be a file that you can do a u2spew or u2boat command.
-
And how would pfblocker know to look for something.null? It could maybe do a PTR for an IP..
Oh you mean in one of its block lists? Having a null tld in a block list could be very problematic, since there is no way to resolve those without setting up use of OpenNIC
-
@tman904 As @bmeeks has recommended several times in this forum, you should turn off monitoring WAN in Snort and instead run it on the internal interfaces such as LAN and in this case DMZ. This will make it easy to see which device on your network is causing these alerts by looking at your internal IP addresses. Since everything is blocked by default on pFsense, running Snort on the WAN just duplicates what pFsense already does.
-
NogBadTheBad
The alert is only on my WAN but sourced from the WAN's IP.Johnpoz I turned log level up to 5 on the dns resolver, but I still havn't found anything.
-
@jdeloach said in Snort detecting INDICATOR-COMPROMISE suspicious .null dns query on my WAN:
@tman904 As @bmeeks has recommended several times in this forum, you should turn off monitoring WAN in Snort and instead run it on the internal interfaces such as LAN and in this case DMZ. This will make it easy to see which device on your network is causing these alerts by looking at your internal IP addresses. Since everything is blocked by default on pFsense, running Snort on the WAN just duplicates what pFsense already does.
@bmeeks also states that if you have open ports on the WAN interface to enable Snort on the WAN interface.
-
@tman904 said in Snort detecting INDICATOR-COMPROMISE suspicious .null dns query on my WAN:
NogBadTheBad
The alert is only on my WAN but sourced from the WAN's IP.Johnpoz I turned log level up to 5 on the dns resolver, but I still havn't found anything.
Is there anything in the Snort log directory?
-
I thought you want to run snort on WAN if you have open ports which I do.
Yes there is a directory for each interface. As well as an update log file. The gui did state the most recent update was successful.
-
@tman904 said in Snort detecting INDICATOR-COMPROMISE suspicious .null dns query on my WAN:
Johnpoz I turned log level up to 5 on the dns resolver, but I still havn't found anything.
Doesn't actually log queries... You have to set the query as stated in the option box..
-
at the moment I have two ssh connections to the pfsense and I'm running.
tcpdump -ni em0 udp port 53 |grep .null -LAN
tcpdump -ni em1 udp port 53 |grep .null -DMZHopefully that will grab it at some point.
-
That not going to catch anything.. Test it!!! Do a query with that in the fqdn..
-
-LAN and -DMZ aren't in the command.
-
And again TEST it...
where do you think in that tcpdump the info of the query would be listed? So how could you grep for it.
-
well can you explain that to me?
There they are
07:33:13.204333 IP 192.168.0.141.57062 > 192.168.0.2.53: 57715+ [1au] A? test.null. (38)
07:33:13.241310 IP 192.168.0.141.57062 > 192.168.0.2.53: 57715+ A? test.null. (27)
07:33:54.634610 IP 192.168.0.141.59863 > 4.2.2.2.53: 5400+ [1au] A? test.null. (50)guess I didn't need the -A
This is what I used
"tcpdump -ni em0 udp port 53 |grep .null"
"tcpdump -ni em1 udp port 53 |grep .null" -
Ok so now that you have validate your method can find an example - yeah lets see if you find anything at the time you see the snort log.
When I try that grep doesn't find my test
But without grep I see it
[2.4.4-RELEASE][admin@sg4860.local.lan]/root: tcpdump -ni igb0 udp port 53
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on igb0, link-type EN10MB (Ethernet), capture size 262144 bytes
06:39:44.111865 IP 192.168.9.100.61488 > 192.168.9.253.53: 11023+ [1au] A? test.domain.null. (57) -
The same thing that the gui says lines up with the logs. But it says it's from the WAN IP. with grep .null on my dmz interface log file in /var/log/snort doesn't seem to have .null in it.
It took a few seconds after I ran the command. Also if you change the snap length it should get to grep quicker.
-
I just enabled snort on my LAN interface. If it's sourced from there it should line up in the logs for LAN and WAN.
-
@tman904 said in Snort detecting INDICATOR-COMPROMISE suspicious .null dns query on my WAN:
I just enabled snort on my LAN interface. If it's sourced from there it should line up in the logs for LAN and WAN.
Are your LAN and DMZ VLANS, if they are and both share the same parent interface you can just run Snort on the parent interface as it puts the interface into promiscuous mode.
-
In my case LAN and DMZ are two physical interfaces. Hopefully by tonight the queries will be sent again. Maybe it's nothing but it doesn't seem normal to me.
Thanks again for everyone's help with this issue. I'll report back if it's finds anything by tonight or tomorrow.
-
I would be curious as well to what they are looking up.. but domain.null could be almost anything..
Allowing for unbound to resolve opennic tlds is a simple as adding a few stub zones pointing to the opennic NSs..
Here I added stub zones for geek and null so could get to search engine and fine a null site, etc..
That snort marks them as suspicious could be seen a few different ways... For starters anyone trying to do something bad, writing code so it could resolve opennic tlds would be "extra" work ;)
But maybe its someone trying to circumvent something??
It could be just honest sort of mistake where a search suffix is adding .null to something and causing the query to roots for whatever.com.null for example..
You have my curiosity cat meowing ;) So please report back what you find...
-
Under /var/log/snort/snort_interfaceRANDOMNUMBER there should be a file with u2 in the file name, do a u2spewfoo FILENAME or a u2uboat FILENAME > output.pcap.
The first command will dump the output to screen the second will create a file that can be read by Wireshark.
Also I'd be tempted to use Balanced rather than Security.
-
I'll run those commands and see if I spot anything in wireshark.
As of today/tonight I haven't seen incidents on DMZ or LAN that line up with the WAN incidents. I wonder if it's simply because it hasn't happened again since I starting logging on LAN? That would make since since I was only logging on WAN and DMZ when it happened yesterday.
What do you guys think? Could it have came from pfsense directly or is it more likely something on the LAN caused it when I wasn't logging there?
-
pfsense would not have been trying to to some domain.null it only checks for updates, the rss feed if you have that setup and package updates. all would be pfsense/netgate domains.
Did you install any unofficial packages? Here is the thing trying to access any of the opennic tlds from some application would just be stupid - UNLESS!!! the documentation of said software gave you info on how to make sure you can resolve opennic tlds.. Its a pretty small use base in the big picture to be honest.. Other than say the browser addon users..
Guess the software could be hard coded to query an opennic NS... But you stated the queries were to root servers looking for whatever.null - which would never work.. Your just going to get back NX from roots for any of the opennic tlds
-
Here is the course of action I took, note this is after I enabled snort on the LAN interface.
Here's my thought process step by step. Hopefully it's understandable and will help someone in the future facing something similar.
##########################################################################################################
cause found LAN snort compromise report in snort alerts - coming from 192.168.0.132 on the LAN interfacefirst course of action was to look at status->arp table - didn't yeld anything so client hasn't sent in roughly five minutes or so
second course of action was to look at status->dhcp leases - didn't have an active lease but found the IP in the expired leases
third course of action was to set the mac address from that lease for 192.168.0.132 to a static dhcp mapping to IP 192.168.0.250
fourth course of action was to create a rule under firewall->rules->LAN that passes any traffic sourced from 192.168.0.250. I also added a pass rule for 192.168.0.132 just in case the host doesn't request a lease for some reason. Finally I added a description saying this is the compromised host, and set all packets matching the rules to be logged for either of those two hosts. The reason I'm passing the traffic is to see a working example of what they are doing.
note - 192.168.0.250 finally sent packets see below.
LAN tcp 192.168.0.250:53028 -> 209.85.200.189:443 ESTABLISHED:ESTABLISHED 101 / 146 15 KiB / 25 KiB
WAN tcp X.X.X.X:XXXX (192.168.0.250:53028) -> 209.85.200.189:443 ESTABLISHED:ESTABLISHED 101 / 146 15 KiB / 25 KiB
##########################################################################################################To be honest I'm still not sure exactly what's going on yet. I think that it's just an https connection, when I do a whois on the destination IP it returns google.
Anyway for now that is how the situation stands.
-
Yeah that is google, and where is doing dns queries for opennic?
-
Well I definitely found the culprit. Yesterday a snort alert showed up from the existing server for .win. I'm 99.9% sure it originated from the server and somehow got to the other segment. In any case I need to rebuild the server. Hopefully I can get by with some hot standby/swap kind of thing. Can't have much downtime. Wish me luck I'll need it lol.
Thanks for everyone's help I really appreciate it.
-
@tman904 said in Snort detecting INDICATOR-COMPROMISE suspicious .null dns query on my WAN:
I'm 99.9% sure it originated from the server and somehow got to the other segment. In any case I need to rebuild the server.
What?
So you have a packet capture of this query? "somehow" is a not good RCA for what is happening in your network.. Your saying its now doing queries for something.win, that is not a opennic tld? That is a valid normal new tld that you can register anywhere..
Blocking queries to domains because you don't like the tld and its not .com or .org is sure fire way to break the internet for your users..
Maybe its some legit software installed on the server checking to see if there update available.. And that company just happen to use .win for their tld because its cheap and hip ;)
-
I don't plan on blocking them. I found them in the snort alerts again, just .win this time. After this cropped up I discovered the Web mail panel for the server was exposed to the internet. Unfortunately it runs a product that also has a management ui for the mail server part on 443 also exposed. I blocked that port along with the Web mail. smtp/Imap is ok atm.
I dug deeper and the server hasn't been patched for a few years. In the servers admin logs, it shows the firewalls lan ip successfully logged in at a time and date I didn't login. Thats when i got the permission to build a new firewall with pfsense. Also when we started using snort. This is why I think it's hacked. It was installed before I came on board. So I don't know the history, and almost no documentation of anything.
Tldr version
These dns queries, along with the web mail and admin panel facing the Internet. make me think the server may have been compromised. -
@tman904 said in Snort detecting INDICATOR-COMPROMISE suspicious .null dns query on my WAN:
These dns queries,
What was the query for? Can you not get this info from snort? just saying something.win is not very useful logging.. I can see the alert could be called that, but there should be a log of the actual query done.
-
Is there a way to filter under status dns resolver? Also if the dns request wasn't answered will it not show up in the log?
