Finding internal IP causing block

andrewhancock91 · Feb 15, 2019, 6:34 PM

Hi all,
I'm looking for some help or advise again, my DNS servers keep getting blocked with this alert:

The problem I'm running in to is finding what device on the network is causing the alert, when I look in the pfSense states table immediately after the alert appears (within 30 seconds probably) for whatever device has a state on port 54902 there is nothing. I've also monitored my internal DNS server and can't find any calls to an address that resolves in the 195.22.26.192/26 network. I'm extremely new to Snort and IDS/IPS so this is my first time trying to track down an alert that has to be coming from inside, there is probably something really simple that I am missing to track this down but I can't seem to figure it out!
Thanks in advance

bmeeks · Feb 16, 2019, 3:52 PM

@andrewhancock91 said in Finding internal IP causing block:

Hi all,
I'm looking for some help or advise again, my DNS servers keep getting blocked with this alert:

The problem I'm running in to is finding what device on the network is causing the alert, when I look in the pfSense states table immediately after the alert appears (within 30 seconds probably) for whatever device has a state on port 54902 there is nothing. I've also monitored my internal DNS server and can't find any calls to an address that resolves in the 195.22.26.192/26 network. I'm extremely new to Snort and IDS/IPS so this is my first time trying to track down an alert that has to be coming from inside, there is probably something really simple that I am missing to track this down but I can't seem to figure it out!
Thanks in advance

Are you running Snort on your WAN interface and using NAT? If so, you are seeing one of the pitfalls of that setup. Run Snort on your LAN interface instead so that all local IP addresses show without NAT. When you run Snort on the WAN it sees traffic after NAT has been applied for outbound traffic and before NAT is undone for inbound traffic. Thus any LAN host's address will always show up as having the WAN public IP address. This makes tracking down a local host quite hard. With blocking enabled, the default setting for Snort is for it to clear all states associated with a blocked IP. That's why the state table is empty of entries for the blocked IP address when you look.

Move your Snort setup to your LAN. You won't sacrifice any security by doing this. Your firewall is dropping all unsolicited inbound traffic anyway unless you have added your own rules to override that action.

NogBadTheBad · Feb 16, 2019, 4:21 PM

I occasionally see this alert only on my WAN interface, never ever see it on my snort enabled LAN interfaces.

In your screenshot it shows 8.8.8.8 using port 53 ( DNS ) talking to the WAN interface, very odd.

bmeeks · Feb 16, 2019, 4:32 PM

@nogbadthebad said in Finding internal IP causing block:

I occasionally see this alert only on my WAN interface, never ever see it on my snort enabled LAN interfaces.

In your screenshot it shows 8.8.8.8 using port 53 ( DNS ) talking to the WAN interface, very odd.

Here is a very long thread on this rule going all the way back to 2014 at least. I have not had the chance to read it all yet.

andrewhancock91 · Feb 16, 2019, 4:32 PM

@bmeeks
I definitely see what you're saying but I have disabled blocking and set Snort to alert only while trying to troubleshoot this but the states are still not there. I also have Snort running on my LAN interface in alert only with the same ruleset that is applied to the WAN but it doesn't trigger any of these DNS errors, it actually only triggers alerts for internal traffic (I have multiple LAN interfaces)

@NogBadTheBad
Sounds like you've experienced what I'm seeing, it only triggers on the WAN interface. I've tried changing my DNS to different things like 9.9.9.9 or OpenDNS and the alert follows whatever DNS servers I'm currently using so the requests are definitely being triggered from something on the LAN.

bmeeks · Feb 16, 2019, 4:41 PM

@andrewhancock91

Here is the final topic from the thread I linked: It explains why the source address is 8.8.8.8 and port 53. Read the reply from the Anubis guy and then the question from "Leonard" below it.

You can also back up one to a previous message to get a better explanation.

andrewhancock91 · Feb 16, 2019, 6:13 PM

Okay so what I get from that thread is that something on my network is connecting to a know bad domain that Anibus has redirected to one of their sinkholes for threat investigation, thats all fine and I can appreciate what they are trying to do. My problem is trying to find out which device or devices are connecting to the domain being sinkholed. I suppose I could do a reverse DNS on all 64 addresses in that block and compile a list of all domains that resolve there then look in my DNS logs for what device is requesting one of those domains?

NogBadTheBad · Feb 18, 2019, 2:02 PM

If you use unbound add "log-queries: yes" as a custom option, you then may be able to tie together the snot alert and host by looking at the timestamps.

This will create a lot of entries, I send my logs via syslog to my Synology NAS.

You'll see entries for queries from the clients like this:-

Feb 17 11:04:06 unbound 9828:3 info: 172.16.2.10 1.0.16.172.in-addr.arpa. PTR IN
Feb 17 11:03:45 unbound 9828:3 info: 2a02:xxxx:xxxx:xxxx::29 gateway.fe.apple-dns.net. A IN
Feb 17 11:03:45 unbound 9828:3 info: 2a02:xxxx:xxxx:xxxx::29 gateway.icloud.com. A IN

NogBadTheBad · Feb 18, 2019, 2:02 PM

(Event)
sensor id: 0 event id: 3572891651 event second: 1550493993 event microsecond: 746171
sig id: 2018455 gen id: 1 revision: 5 classification: 33
priority: 1 ip source: 184.73.137.229 ip destination: XX.XX.XX.XX
src port: 53 dest port: 58142 protocol: 17 impact_flag: 0 blocked: 0
mpls label: 0 vland id: 0 policy id: 0 appid: DNS

Packet
sensor id: 0 event id: 3572891651 event second: 1550493993
packet second: 1550493993 packet microsecond: 746171
linktype: 0 packet_length: 94
[ 0] 02 00 00 00 45 00 00 5A AF F9 40 00 F0 11 36 BD ....E..Z..@...6.
[ 16] B8 49 89 E5 52 45 0F 68 00 35 E3 1E 00 46 62 81 .I..RE.h.5...Fb.
[ 32] 06 B9 84 00 00 01 00 01 00 00 00 01 02 6E 73 07 .............ns.
[ 48] 68 65 74 73 70 74 74 03 6E 65 74 02 63 6E 00 00 hetsptt.net.cn..
[ 64] 01 00 01 C0 0C 00 01 00 01 00 00 00 64 00 04 C3 ............d...
[ 80] 16 1A F8 00 00 29 06 90 00 00 80 00 00 00 .....)........```

Looks like its a dns request to ns.hetsptt.net.cn that's doing it.

Seems to happen when my Nest smoke alarms get a bit chatty.

More investigation needed.

andrewhancock91 · Feb 18, 2019, 2:21 PM

I didn't have a chance to work with this any more yesterday but I'm going to turn on the "log-queries: yes" option and see what I can find this morning.

NogBadTheBad · Feb 18, 2019, 8:49 PM

BTW if you set the following in Advanced Configuration Pass-Through in the Advanced Configuration Pass-Through section you can get a packet capture of the packets:-

ruletype pcap
{
type alert
output alert_syslog: LOG_AUTH LOG_ALERT 
output log_tcpdump: pcap.cap
}

The following rule would capture ping out:-

pcap icmp $HOME_NET any -> !$HOME_NET any (msg:“ICMP”; sid:20000002; rev:001;classtype:icmp-event;)

Just need to work out the rule to capture DNS requests that match what I think is causing the block.

NogBadTheBad · Feb 18, 2019, 8:53 PM

Bingo

pcap udp any any ->  any 53 ( msg:"ns.hetsptt.net.cn"; flow:to_server; byte_test:1,!&,0xF8,2; \
content: "|02|ns|07|hetsptt|03|net|02|cn|00|"; nocase; sid:20000010; rev:001;classtype:string-detect; )

https://www.securityartwork.es/2013/02/21/snort-byte_test-for-dummies-2/

NogBadTheBad · Feb 25, 2019, 9:35 AM

2019-02-25 02:35:58 Information pfsense daemon unbound [40595:3] info: 172.16.2.10 1.0.16.172.in-addr.arpa. PTR IN
2019-02-25 02:35:58 Information pfsense daemon unbound [40595:3] info: 2a02:xxxx:xxxx:2::28 tileservices-prod-events-1233069693.us-east-1.elb.amazonaws.com. A IN
2019-02-25 02:35:58 Information pfsense daemon unbound [40595:3] info: 2a02:xxxx:xxxx:2::28 tileservices-prod-events-1233069693.us-east-1.elb.amazonaws.com. AAAA IN
2019-02-25 02:35:32 Information pfsense daemon unbound [40595:3] info: 127.0.0.1 6.27.176.185.in-addr.arpa. PTR IN
2019-02-25 02:35:25 Information pfsense daemon unbound [40595:3] info: 127.0.0.1 39.26.176.185.in-addr.arpa. PTR IN
2019-02-25 02:35:22 Information pfsense daemon unbound [40595:3] info: 127.0.0.1 208.94.64.125.in-addr.arpa. PTR IN
2019-02-25 02:35:16 Information pfsense daemon unbound [40595:3] info: 172.16.2.10 1.0.16.172.in-addr.arpa. PTR IN
2019-02-25 02:35:16 Information pfsense daemon unbound [40595:3] info: 172.16.4.29 api.darksky.net. AAAA IN
2019-02-25 02:35:16 Information pfsense daemon unbound [40595:3] info: 172.16.4.29 api.darksky.net. A IN
2019-02-25 02:35:06 Information pfsense daemon unbound [40595:3] info: 127.0.0.1 212.179.9.60.in-addr.arpa. PTR IN
2019-02-25 02:35:00 Information pfsense daemon unbound [40595:3] info: 172.16.2.10 1.0.16.172.in-addr.arpa. PTR IN
2019-02-25 02:35:00 Information pfsense daemon unbound [40595:3] info: 172.16.2.10 2.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.5.0.0.0.c.e.0.6.0.1.0.8.2.0.a.2.ip6.arpa. PTR IN
2019-02-25 02:34:59 Information pfsense daemon unbound [40595:3] info: 127.0.0.1 212.179.9.60.in-addr.arpa. PTR IN
2019-02-25 02:34:56 Information pfsense daemon unbound [40595:3] info: 127.0.0.1 api.bgpview.io. AAAA IN
2019-02-25 02:34:56 Information pfsense daemon unbound [40595:3] info: 127.0.0.1 api.bgpview.io. A IN
2019-02-25 02:34:56 Information pfsense daemon unbound [40595:3] info: 172.16.2.10 1.0.16.172.in-addr.arpa. PTR IN
2019-02-25 02:34:36 Information pfsense daemon unbound [40595:3] info: 127.0.0.1 225.250.108.202.in-addr.arpa. PTR IN
2019-02-25 02:34:36 Information pfsense daemon unbound [40595:3] info: 172.16.2.10 1.0.16.172.in-addr.arpa. PTR IN

Hmm its something on the firewall is causing the alert, I can trigger the alert by doing a "host 60.9.179.212"

NogBadTheBad · Feb 25, 2019, 9:38 AM

pfBlocker trying to resolve a host that i've blocked