Hows Google getting past my alias lists?

firewalluser · Feb 2, 2015, 6:21 PM

Sorry I thought you meant or were implying their ptr record of lhr08s07-in-f3.1e100.net is not helpful when they also have other domain names running from the ip address.

The thing I cant understand is the aliases should be a domain name, so I would expect the google.co.uk domain to have been linked to that ip address somehow, but I cant find anything to suggest this yet, hence the mystery of how google managed to get out.

KOM · Feb 2, 2015, 6:50 PM

One FQDN can resolve to many different IP addresses with DNS set to give out a different one in rotating fashion for subsequent lookups. This is commonly used simple method to load-balance sites.

firewalluser · Feb 2, 2015, 7:36 PM

I can understand that, well I did a hostname lookup using Diagnostics: DNS Lookup and google.co.uk returned just the one ip address that being 216.58.208.35!

Thought that was weird so tried google.com and got
62.24.155.222/32, 62.24.155.246/32, 62.24.155.231/32, 62.24.155.232/32, 62.24.155.242/32, 62.24.155.247/32, 62.24.155.251/32, 62.24.155.217/32, 62.24.155.212/32, 62.24.155.237/32…
tried google.se and got
62.24.155.236/32, 62.24.155.212/32, 62.24.155.227/32, 62.24.155.237/32, 62.24.155.242/32, 62.24.155.226/32, 62.24.155.216/32, 62.24.155.251/32, 62.24.155.246/32, 62.24.155.221/32
so went back and tried google.co.uk and this time got not one ip address but what I was expecting, that being
62.24.155.247/32, 62.24.155.227/32, 62.24.155.241/32, 62.24.155.231/32, 62.24.155.222/32, 62.24.155.237/32, 62.24.155.236/32, 62.24.155.226/32, 62.24.155.216/32, 62.24.155.246/32

So why would Diagnostics: DNS Lookup report just one ip address to begin with before reporting the full list?
I checked in the resolver log and the only update is
Feb 2 19:29:06 filterdns: clearing entry 208.73.211.199 from table Mozzilla_firefox on host www.mozzilla.org
Feb 2 19:29:06 filterdns: clearing entry 208.73.211.191 from table Mozzilla_firefox on host www.mozzilla.org
Feb 2 19:29:06 filterdns: clearing entry 208.73.211.194 from table Mozzilla_firefox on host www.mozzilla.org
Feb 2 19:29:06 filterdns: clearing entry 208.73.210.212 from table Mozzilla_firefox on host www.mozzilla.org
Feb 2 19:29:06 filterdns: adding entry 208.73.211.165 to table Mozzilla_firefox on host www.mozzilla.org
Feb 2 19:29:06 filterdns: adding entry 208.73.210.205 to table Mozzilla_firefox on host www.mozzilla.org
Feb 2 19:29:06 filterdns: adding entry 208.73.211.163 to table Mozzilla_firefox on host www.mozzilla.org
Feb 2 19:29:06 filterdns: adding entry 208.73.211.242 to table Mozzilla_firefox on host www.mozzilla.org

Something seems odd.

doktornotor · Feb 2, 2015, 7:39 PM

Look, for huge stuff like Google, these IPs change pretty much every time you try to resolve the FQDN. Absolutely normal. This really will not get you anywhere regarding attempts to block Google.

firewalluser · Feb 2, 2015, 8:24 PM

I understand what you say, but right now I've got something trying to connect back to google and I'm trying to find out what it is.

I didnt have any webpages open except pf's own webpages (dashboard, systemlog, aliases, resolvers log and firewall rules), even my default toolbar search in firefox is set to chambers uk (dictionary people) so I'm at a loss as to how firefox can still be talking to google, especially when I run noscript which blocks all javascript until the domain/subdomain is enabled.

I'm beginning to think there must be some other code built into firefox which is talking back to google.

Edit.
This seems weird as well. In the resolver log I see

pfmechanics.com.MyDomainNameWhichWillRemainPrivate

Now why would I be seeing "pfmechanics.com" as a sub domain attached to my domain name (see the resolver logs below) or is this one of those read-the-contract-to-see-what-else-I've-signed-up-for moments?

eg

Feb 2 20:06:31 unbound: [63509:0] info: resolving MyDomainNameWhichWillRemainPrivate. DS IN
Feb 2 20:06:31 unbound: [63509:0] info: query response was NXDOMAIN ANSWER
Feb 2 20:06:31 unbound: [63509:0] info: reply from <mydomainnamewhichwillremainprivate.>184.172.157.218#53
Feb 2 20:06:31 unbound: [63509:0] info: response for pfmechanics.com.MyDomainNameWhichWillRemainPrivate. A IN
Feb 2 20:06:31 unbound: [63509:0] info: query response was ANSWER
Feb 2 20:06:31 unbound: [63509:0] info: reply from <pfmechanics.net.>192.207.126.7#53
Feb 2 20:06:31 unbound: [63509:0] info: response for ns1.pfmechanics.net. A IN
Feb 2 20:06:30 unbound: [63509:0] info: resolving b.ns.MyDomainRegistrar. AAAA IN
Feb 2 20:06:30 unbound: [63509:0] info: resolving c.ns.MyDomainRegistrar. AAAA IN
Feb 2 20:06:30 unbound: [63509:0] info: query response was REFERRAL
Feb 2 20:06:30 unbound: [63509:0] info: reply from <com.>192.35.51.30#53
Feb 2 20:06:30 unbound: [63509:0] info: response for pfmechanics.com.MyDomainNameWhichWillRemainPrivate. A IN
Feb 2 20:06:30 unbound: [63509:0] info: resolving ns1.pfmechanics.net. AAAA IN
Feb 2 20:06:30 unbound: [63509:0] info: resolving ns1.pfmechanics.com. AAAA IN
Feb 2 20:06:30 unbound: [63509:0] info: query response was REFERRAL
Feb 2 20:06:30 unbound: [63509:0] info: reply from <net.>192.48.79.30#53
Feb 2 20:06:30 unbound: [63509:0] info: response for ns2.pfmechanics.net. AAAA IN
Feb 2 20:06:30 unbound: [63509:0] info: resolving ns1.pfmechanics.com. AAAA IN
Feb 2 20:06:30 unbound: [63509:0] info: resolving ns2.pfmechanics.com. AAAA IN
Feb 2 20:06:30 unbound: [63509:0] info: query response was REFERRAL
Feb 2 20:06:30 unbound: [63509:0] info: reply from <net.>192.48.79.30#53
Feb 2 20:06:30 unbound: [63509:0] info: response for ns1.pfmechanics.net. A IN
Feb 2 20:06:30 unbound: [63509:0] info: query response was ANSWER
Feb 2 20:06:30 unbound: [63509:0] info: reply from <pfmechanics.net.>192.207.126.6#53
Feb 2 20:06:30 unbound: [63509:0] info: response for ns2.pfmechanics.net. A IN</pfmechanics.net.></net.></net.></com.></pfmechanics.net.></mydomainnamewhichwillremainprivate.>

doktornotor · Feb 2, 2015, 8:16 PM

Among others: https://developers.google.com/safe-browsing/

firewalluser · Feb 2, 2015, 8:33 PM

I'm trying to reduce my google exposure so this wont be off much use.

Having typed in https://developers.google.com/safe-browsing/ and bearing in mind all I have is google.co.uk and www.google.co.uk setup in an alias with a rule allowing supposedly just those domains, I can see right now in the bottom left of firefox it cycling through all the google domains like ajax.googleapis.com, www.googleadservices.com, fonts.googleapis.com and others before eventually loading that web page.

It kind of makes a mockery of firewalls in some ways doesnt it, as this could equally be malware getting out doing its stuff, and thus more malware coming back in.

BBcan177 · Feb 2, 2015, 8:33 PM

With pfBlockerNG, you can download lists from Hurricane Electric using the new "html" format setting. I posted about it in a different thread (See below). While it might not be suitable for each requirement, it can be used quite effectively. Just enter the Search criteria in the HE Search box.

https://forum.pfsense.org/index.php?topic=83421.msg479553#msg479553

Example of HE IP lists:

http://bgp.he.net/search?search%5Bsearch%5D=twitter&commit=Search
http://bgp.he.net/search?search%5Bsearch%5D=facebook&commit=Search
http://bgp.he.net/search?search%5Bsearch%5D=spotify&commit=Search
http://bgp.he.net/search?search%5Bsearch%5D=dropbox&commit=Search

firewalluser · Feb 2, 2015, 8:51 PM

@bbCan177, thanks for the link, I've been using them amongst others to look up and cross reference the address blocks, but I'll check out the thread as I had pfblocker iirc installed on another site to restrict access to just UK IP addresses as the customer didnt trade abroad.

doktornotor · Feb 2, 2015, 8:54 PM

@firewalluser:

I'm trying to reduce my google exposure so this wont be off much use.

Hmmm? I was not suggesting that you should use it. FF and Chrome uses Google Safebrowsing by default and downloads the databases every time you launch the browser. You can check that in %UserProfile%\AppData\Local\Mozilla\Firefox\Profiles<randomjunk.default>\safebrowsing</randomjunk.default>

firewalluser · Feb 2, 2015, 9:05 PM

Ok so for another test, I've disabled the rule which allows the google aliases comprising of google.co.uk and www.google.co.uk out onto the net, and guess what its still loaded the webpage https://developers.google.com/safe-browsing/

So I'm going to try and do an explicit block of all the assigned google cidr's becuase in the system logs amongst those that resolve to a google domain, there are plenty of ip's that dont resolve and yet apart from being logged in here there is no other webpage being looked at or service/app running getting out on this single machine on its own network.

Currently this would suggest Google has a wide range of ip addresses which do not resolve.

firewalluser · Feb 2, 2015, 9:10 PM

@doktornotor:

@firewalluser:

I'm trying to reduce my google exposure so this wont be off much use.

Hmmm? I was not suggesting that you should use it. FF and Chrome uses Google Safebrowsing by default and downloads the databases every time you launch the browser. You can check that in %UserProfile%\AppData\Local\Mozilla\Firefox\Profiles<randomjunk.default>\safebrowsing</randomjunk.default>

I can see in the folder last 3 files were modified a few minutes ago, but I do have mozilla as an allowed alias comprising of mozilla.org, www.mozzilla.org, addons.mozilla.org, bugzilla.mozilla.org, Ftp.mozilla.org

firewalluser · Feb 3, 2015, 12:18 AM

Well with an explicit block and having double checked things in firefox like send back telemetry is switched off, no history etc, the firewall log shows google is probably using some amazon cloud servers if the port numbers are anything to go by but surprising google would also be using amazon cloud severs. In another test the default deny rule is showing up with my isp's ip addresses when trying to access google, which is beginning to make me wonder just how much of the web google is not connected to.

Feb 2 21:29:33 opt1 Default deny rule IPv4 (1000000103) 192.168.2.1:60532 54.186.10.229:443
ec2-54-186-10-229.us-west-2.compute.amazonaws.com TCP:S
block/1000000103 Feb 2 21:29:32 opt1 Default deny rule IPv4 (1000000103) 192.168.2.1:60531 54.186.10.229:443
ec2-54-186-10-229.us-west-2.compute.amazonaws.com TCP:S
block/1422911107
Feb 2 21:29:31 opt1 USER_RULE opt1 to google com Aliases (1422911107) 192.168.2.1:60533 62.24.155.232:443 TCP:S
block/1000000103
Feb 2 21:29:30 opt1 Default deny rule IPv4 (1000000103) 192.168.2.1:60532 54.186.10.229:443
ec2-54-186-10-229.us-west-2.compute.amazonaws.com TCP:S
block/1000000103
Feb 2 21:29:29 opt1 Default deny rule IPv4 (1000000103) 192.168.2.1:60531 54.186.10.229:443
ec2-54-186-10-229.us-west-2.compute.amazonaws.com TCP:S
block/1422911107
Feb 2 21:29:19 opt1 USER_RULE opt1 to google com Aliases (1422911107) 192.168.2.1:60524 62.24.155.217:443 TCP:S
block/1000000103
Feb 2 21:29:18 opt1 Default deny rule IPv4 (1000000103) 192.168.2.1:60523 54.186.10.229:443
ec2-54-186-10-229.us-west-2.compute.amazonaws.com TCP:S
block/1422911107
Feb 2 21:29:13 opt1 USER_RULE opt1 to google com Aliases (1422911107) 192.168.2.1:60524 62.24.155.217:443 TCP:S
block/1000000103
Feb 2 21:29:12 opt1 Default deny rule IPv4 (1000000103) 192.168.2.1:60523 54.186.10.229:443
ec2-54-186-10-229.us-west-2.compute.amazonaws.com TCP:S
block/1422911107
Feb 2 21:29:10 opt1 USER_RULE opt1 to google com Aliases (1422911107) 192.168.2.1:60524 62.24.155.217:443 TCP:S
block/1000000103
Feb 2 21:29:08 opt1 Default deny rule IPv4 (1000000103) 192.168.2.1:60523 54.186.10.229:443
ec2-54-186-10-229.us-west-2.compute.amazonaws.com TCP:S
block/1422911107
Feb 2 21:28:57 opt1 USER_RULE opt1 to google com Aliases (1422911107) 192.168.2.1:60518 62.24.155.221:443 TCP:S
block/1422911107
Feb 2 21:28:57 opt1 USER_RULE opt1 to google com Aliases (1422911107) 192.168.2.1:60517 62.24.155.221:443 TCP:S
block/1000000103
Feb 2 21:28:56 opt1 Default deny rule IPv4 (1000000103) 192.168.2.1:60516 54.186.10.229:443
ec2-54-186-10-229.us-west-2.compute.amazonaws.com TCP:S
block/1000000103
Feb 2 21:28:56 opt1 Default deny rule IPv4 (1000000103) 192.168.2.1:60515 54.186.10.229:443
ec2-54-186-10-229.us-west-2.compute.amazonaws.com TCP:S
block/1422911107
Feb 2 21:28:51 opt1 USER_RULE opt1 to google com Aliases (1422911107) 192.168.2.1:60518 62.24.155.221:443 TCP:S
block/1422911107
Feb 2 21:28:51 opt1 USER_RULE opt1 to google com Aliases (1422911107) 192.168.2.1:60517 62.24.155.221:443 TCP:S
block/1000000103
Feb 2 21:28:50 opt1 Default deny rule IPv4 (1000000103) 192.168.2.1:60516 54.186.10.229:443
ec2-54-186-10-229.us-west-2.compute.amazonaws.com TCP:S
block/1000000103
Feb 2 21:28:50 opt1 Default deny rule IPv4 (1000000103) 192.168.2.1:60515 54.186.10.229:443
ec2-54-186-10-229.us-west-2.compute.amazonaws.com TCP:S
block/1422911107
Feb 2 21:28:48 opt1 USER_RULE opt1 to google com Aliases (1422911107) 192.168.2.1:60518 62.24.155.221:443 TCP:S
block/1422911107
Feb 2 21:28:48 opt1 USER_RULE opt1 to google com Aliases (1422911107) 192.168.2.1:60517 62.24.155.221:443 TCP:S
block/1000000103
Feb 2 21:28:47 opt1 Default deny rule IPv4 (1000000103) 192.168.2.1:60516 54.186.10.229:443
ec2-54-186-10-229.us-west-2.compute.amazonaws.com TCP:S
block/1000000103
Feb 2 21:28:47 opt1 Default deny rule IPv4 (1000000103) 192.168.2.1:60515 54.186.10.229:443
ec2-54-186-10-229.us-west-2.compute.amazonaws.com TCP:S

Anyway still testing as I need to find out how google got out earlier but not now.

stephenw10 · Feb 3, 2015, 2:28 PM

That's pretty cool. :) Even if not perfect it going to stick a sufficiently large spanner in most peoples browsing they'll probably give up and do something useful instead.

Edit: Ooops. Referring to BBcan177's suggestion.

Steve

Supermule · Feb 3, 2015, 2:34 PM

This is EXACTLY why Layer7 is needed in pfSense….

Unless we are talking https traffic, then you need some sort of domain blacklist or IP range as BBcan17 is saying...

heper · Feb 3, 2015, 4:19 PM

@Supermule:

This is EXACTLY why Layer7 is needed in pfSense….

Unless we are talking https traffic, then you need some sort of domain blacklist or IP range as BBcan17 is saying...

almost all popular webservices use https … layer7 won't help, nor will squid without being an evil admin.

firewalluser · Feb 3, 2015, 4:50 PM

Have to say, if you do block all of googles ip addresses, you'll find most web pages painfully slow to load as it takes the browser a long time to timeout when its pulling code or stuff from google.

When a webpage has something to do with google beit using their ajax, google apis or what have you, a webpage can easily take a few minutes to load destrying any user experience, the delay depends on how many links to google there are, the more links the longer the delay.

If Google went off line, you'd have a riot on your hands with people getting fed up waiting for web pages to load and lots of businesses could suffer as well, but their techniques to make their services work irrespective of how badly configured a network maybe is quite illuminating when considering ways to tackle malware thats not been identified by any major AV company.

I've yet to analyse how many websites use google services of sorts, but I suspect Western countries are very heavy users of their services compared to other parts of the world not neccesarily english speaking.

Referring to BBcan177's suggestion, it looks like mismatches will occur as the sources might not be up to date with whatever resolver might have got from the various gTLD servers.

Still havent figured out how this came to be though.
Feb 2 20:06:30 unbound: [63509:0] info: response for pfmechanics.com.MyDomainNameWhichWillRemainPrivate. A IN

I've not created a "pfmechanics.com" subdomain for my domains so trying to find out why pfsense is doing this? Any ideas?

TIA

GruensFroeschli · Feb 3, 2015, 5:29 PM

Alternatively you could block this at DNS level on the pfSense itself.

Create an NAT rule on your LAN interface with destination any, port 53. Redirect destination 127.0.0.1, port 53.
–> This will force all DNS lookups no matter to which DNS server to your pfSense.

In the DNS forwarder config you can add something like:
address=/google.com/127.0.0.1

This will resolve for all google.com domains and subdomains to 127.0.0.1.
Replace 127.0.0.1 with a local server and you will see on it when something is sent to it.

See also: https://doc.pfsense.org/index.php/Wildcard_Records_in_DNS_Forwarder

stephenw10 · Feb 3, 2015, 5:36 PM

@firewalluser:

Have to say, if you do block all of googles ip addresses, you'll find most web pages painfully slow to load as it takes the browser a long time to timeout when its pulling code or stuff from google.

It's for this reason that most popular adblockers replace the requests with some locally served data instead.

Steve

kejianshi · Feb 3, 2015, 5:44 PM

I like GruensFroeschli's suggestion. I'd consider it the default method of controlling DNS queries because It doesn't even matter if people on your net manually configure alternate DNS servers on their machines, it will still only get pfsense DNS. (unbound or whatever you are running)