HackerList for pfBlockerNG

anttechs

I would love to learn how people harvest the ips ?
If i knew how it was done I would do it as well as I work from home.
I have never seen software that gets these lists or methods shown online but id love to learn it.
Also in the UK where I am we get a lot of spying now from the Government so id love to learn how to grab the right ips and put them into lists.
If anyone can tell me how its done I would gladly put a lot of time into doing it.

sorenstoutner

@anttechs:

I would love to learn how people harvest the ips ?
If i knew how it was done I would do it as well as I work from home.
I have never seen software that gets these lists or methods shown online but id love to learn it.
Also in the UK where I am we get a lot of spying now from the Government so id love to learn how to grab the right ips and put them into lists.
If anyone can tell me how its done I would gladly put a lot of time into doing it.

I get mine from the Apache logs. For example, look at the following section of a log:

185.137.19.212 - - [31/May/2017:01:51:28 -0700] "GET /wp-content/uploads/2017/03/Onion-Search-Engine-576x1024.png HTTP/1.1" 200 62173
185.137.19.212 - - [31/May/2017:01:51:50 -0700] "-" 408 -
185.137.19.212 - - [31/May/2017:01:51:50 -0700] "-" 408 -
185.137.19.212 - - [31/May/2017:01:51:50 -0700] "-" 408 -
176.119.231.202 - - [31/May/2017:01:54:54 -0700] "GET /privacy-browser HTTP/1.1" 301 -
176.119.231.202 - - [31/May/2017:01:54:54 -0700] "GET /privacy-browser/ HTTP/1.1" 200 18565
176.119.231.202 - - [31/May/2017:01:54:55 -0700] "GET /wp-content/themes/twentyseventeen/style.css?ver=4.7.5 HTTP/1.1" 200 15347
176.119.231.202 - - [31/May/2017:01:54:55 -0700] "GET /wp-content/themes/twentyseventeen/assets/js/skip-link-focus-fix.js?ver=1.0 HTTP/1.1" 200 416
176.119.231.202 - - [31/May/2017:01:54:55 -0700] "GET /wp-includes/js/jquery/jquery.js?ver=1.12.4 HTTP/1.1" 200 33766
176.119.231.202 - - [31/May/2017:01:54:55 -0700] "GET /wp-includes/js/jquery/jquery-migrate.min.js?ver=1.4.1 HTTP/1.1" 200 4014
176.119.231.202 - - [31/May/2017:01:54:55 -0700] "GET /wp-includes/js/wp-emoji-release.min.js?ver=4.7.5 HTTP/1.1" 200 4230
176.119.231.202 - - [31/May/2017:01:54:56 -0700] "GET /wp-content/themes/twentyseventeen/assets/js/global.js?ver=1.0 HTTP/1.1" 200 2606
176.119.231.202 - - [31/May/2017:01:54:56 -0700] "GET /wp-content/themes/twentyseventeen/assets/js/jquery.scrollTo.js?ver=2.1.2 HTTP/1.1" 200 2409
176.119.231.202 - - [31/May/2017:01:54:56 -0700] "GET /wp-includes/js/wp-embed.min.js?ver=4.7.5 HTTP/1.1" 200 751
176.119.231.202 - - [31/May/2017:01:54:56 -0700] "GET /wp-content/themes/twentyseventeen/assets/js/navigation.js?ver=1.0 HTTP/1.1" 200 1164
176.119.231.202 - - [31/May/2017:01:54:56 -0700] "GET /wp-content/uploads/2016/02/cropped-Privacy-Browser-2-192x192.png HTTP/1.1" 200 14224
176.119.231.202 - - [31/May/2017:01:54:56 -0700] "GET /wp-content/uploads/2016/02/cropped-Privacy-Browser-1.png HTTP/1.1" 200 46149
176.119.231.202 - - [31/May/2017:01:54:56 -0700] "GET /wp-content/themes/twentyseventeen/assets/images/header.jpg HTTP/1.1" 200 114854
176.119.231.202 - - [31/May/2017:01:54:57 -0700] "GET /wp-content/uploads/2016/02/cropped-Privacy-Browser-2-192x192.png HTTP/1.1" 200 14224
185.137.19.212 - - [31/May/2017:01:51:27 -0700] "GET /wp-content/uploads/2017/03/Green-URL-Bar-576x1024.png HTTP/1.1" 200 401589
185.137.19.212 - - [31/May/2017:01:51:25 -0700] "GET /wp-content/uploads/2017/04/Custom-Domain-Settings-Highlight-576x1024.png HTTP/1.1" 200 398460
50.62.176.35 - - [31/May/2017:01:57:37 -0700] "POST /xmlrpc.php HTTP/1.1" 200 206
50.62.176.35 - - [31/May/2017:01:57:39 -0700] "POST /xmlrpc.php HTTP/1.1" 200 206
50.62.176.35 - - [31/May/2017:01:57:40 -0700] "POST /xmlrpc.php HTTP/1.1" 200 206
50.62.176.35 - - [31/May/2017:01:57:41 -0700] "POST /xmlrpc.php HTTP/1.1" 200 206
50.62.176.35 - - [31/May/2017:01:57:42 -0700] "POST /xmlrpc.php HTTP/1.1" 200 206
50.62.176.35 - - [31/May/2017:01:57:44 -0700] "POST /xmlrpc.php HTTP/1.1" 200 206
50.62.176.35 - - [31/May/2017:01:57:45 -0700] "POST /xmlrpc.php HTTP/1.1" 200 206
50.62.176.35 - - [31/May/2017:01:57:46 -0700] "POST /xmlrpc.php HTTP/1.1" 200 206
50.62.176.35 - - [31/May/2017:01:57:47 -0700] "POST /xmlrpc.php HTTP/1.1" 200 206
50.62.176.35 - - [31/May/2017:01:57:49 -0700] "POST /xmlrpc.php HTTP/1.1" 200 206
92.200.50.254 - - [31/May/2017:02:00:20 -0700] "GET /feed/ HTTP/1.1" 200 10942
213.251.182.110 - - [31/May/2017:02:00:52 -0700] "GET /feed/ HTTP/1.1" 200 10942
94.254.22.166 - - [31/May/2017:02:06:26 -0700] "GET /feed/ HTTP/1.1" 200 10942
2.247.254.48 - - [31/May/2017:02:07:46 -0700] "GET /category/roadmap/ HTTP/1.1" 200 20000
2.247.254.48 - - [31/May/2017:02:07:48 -0700] "GET /wp-content/themes/twentyseventeen/style.css?ver=4.7.5 HTTP/1.1" 200 15347
2.247.254.48 - - [31/May/2017:02:07:50 -0700] "GET /wp-content/uploads/2016/02/cropped-Privacy-Browser-1.png HTTP/1.1" 200 46149
2.247.254.48 - - [31/May/2017:02:07:50 -0700] "GET /wp-content/themes/twentyseventeen/assets/images/header.jpg HTTP/1.1" 200 114854
2.247.254.48 - - [31/May/2017:02:08:07 -0700] "GET /wp-content/uploads/2016/02/cropped-Privacy-Browser-2-32x32.png HTTP/1.1" 200 1697
2.247.254.48 - - [31/May/2017:02:08:08 -0700] "GET /category/roadmap/ HTTP/1.1" 200 20001
2.247.254.48 - - [31/May/2017:02:08:09 -0700] "GET /category/roadmap/ HTTP/1.1" 200 20001
2.247.254.48 - - [31/May/2017:02:08:15 -0700] "GET /wp-content/uploads/2016/02/cropped-Privacy-Browser-2-192x192.png HTTP/1.1" 200 14224
2.247.254.48 - - [31/May/2017:02:08:41 -0700] "-" 408 -
2.247.254.48 - - [31/May/2017:02:08:44 -0700] "-" 408 -
2.247.254.48 - - [31/May/2017:02:09:19 -0700] "GET /category/roadmap/ HTTP/1.1" 200 20000
2.247.254.48 - - [31/May/2017:02:09:22 -0700] "GET /category/roadmap/ HTTP/1.1" 200 20000
213.251.182.110 - - [31/May/2017:02:10:39 -0700] "GET /feed/ HTTP/1.1" 200 10942
85.252.132.184 - - [31/May/2017:02:10:48 -0700] "GET /privacy-browser/changelog/ HTTP/1.1" 200 23976
85.252.132.184 - - [31/May/2017:02:10:49 -0700] "GET /wp-content/themes/twentyseventeen/style.css?ver=4.7.5 HTTP/1.1" 200 15347
85.252.132.184 - - [31/May/2017:02:10:50 -0700] "GET /wp-content/themes/twentyseventeen/assets/images/header.jpg HTTP/1.1" 200 114854
85.252.132.184 - - [31/May/2017:02:10:52 -0700] "GET /wp-content/uploads/2016/02/cropped-Privacy-Browser-1.png HTTP/1.1" 200 46149
85.252.132.184 - - [31/May/2017:02:10:56 -0700] "GET /wp-content/uploads/2016/02/cropped-Privacy-Browser-2-192x192.png HTTP/1.1" 200 14224
85.252.132.184 - - [31/May/2017:02:10:56 -0700] "GET /wp-content/uploads/2016/02/cropped-Privacy-Browser-2-32x32.png HTTP/1.1" 200 1697
85.252.132.184 - - [31/May/2017:02:11:11 -0700] "-" 408 -
198.71.225.147 - - [31/May/2017:02:11:12 -0700] "POST /xmlrpc.php HTTP/1.1" 200 206
198.71.225.147 - - [31/May/2017:02:11:13 -0700] "POST /xmlrpc.php HTTP/1.1" 200 206
198.71.225.147 - - [31/May/2017:02:11:15 -0700] "POST /xmlrpc.php HTTP/1.1" 200 206
198.71.225.147 - - [31/May/2017:02:11:16 -0700] "POST /xmlrpc.php HTTP/1.1" 200 206
198.71.225.147 - - [31/May/2017:02:11:18 -0700] "POST /xmlrpc.php HTTP/1.1" 200 206
198.71.225.147 - - [31/May/2017:02:11:19 -0700] "POST /xmlrpc.php HTTP/1.1" 200 206
198.71.225.147 - - [31/May/2017:02:11:21 -0700] "POST /xmlrpc.php HTTP/1.1" 200 206
198.71.225.147 - - [31/May/2017:02:11:22 -0700] "POST /xmlrpc.php HTTP/1.1" 200 206
198.71.225.147 - - [31/May/2017:02:11:24 -0700] "POST /xmlrpc.php HTTP/1.1" 200 206
198.71.225.147 - - [31/May/2017:02:11:25 -0700] "POST /xmlrpc.php HTTP/1.1" 200 206
77.158.78.98 - - [31/May/2017:02:11:44 -0700] "GET /category/roadmap/ HTTP/1.1" 200 20000
194.51.15.61 - - [31/May/2017:02:11:45 -0700] "GET /wp-content/themes/twentyseventeen/style.css?ver=4.7.5 HTTP/1.1" 200 15347
194.51.15.61 - - [31/May/2017:02:11:46 -0700] "GET /wp-content/uploads/2016/02/cropped-Privacy-Browser-1.png HTTP/1.1" 200 46149
77.158.78.98 - - [31/May/2017:02:11:46 -0700] "GET /wp-content/themes/twentyseventeen/assets/images/header.jpg HTTP/1.1" 200 114854
45.64.194.66 - - [31/May/2017:02:12:50 -0700] "-" 408 -
92.200.50.254 - - [31/May/2017:02:15:20 -0700] "GET /feed/ HTTP/1.1" 200 10942
213.251.182.110 - - [31/May/2017:02:20:41 -0700] "GET /feed/ HTTP/1.1" 200 10942
62.163.247.22 - - [31/May/2017:02:24:30 -0700] "GET /privacy-browser/changelog/ HTTP/1.1" 200 23976
62.163.247.22 - - [31/May/2017:02:24:31 -0700] "GET /wp-content/themes/twentyseventeen/style.css?ver=4.7.5 HTTP/1.1" 200 15347
62.163.247.22 - - [31/May/2017:02:24:31 -0700] "GET /wp-includes/js/jquery/jquery-migrate.min.js?ver=1.4.1 HTTP/1.1" 200 4014
62.163.247.22 - - [31/May/2017:02:24:31 -0700] "GET /wp-content/themes/twentyseventeen/assets/js/skip-link-focus-fix.js?ver=1.0 HTTP/1.1" 200 416

This is from a WordPress installation that only has two valid users with logins. 176.119.231.202 is an example of what normal browsing behavior looks like, where it loads a page and then fetches all of the CSS, JavaScript, and images associated with that page.

Because there are only two valid users, and the one with the vast majority of the traffic comes from my internal IP address, there should very rarely be POST commands that I don't recognize, especially not multiple posts in a row. 50.62.176.35 is attempting to abuse xmlrpc.php to hack the system, as seen by the multiple posts to it in quick succession. (xmlrpc.php is where many vulnerabilities in WordPress have been found. They are probing it to see if I am running an older version of WordPress that can be easily hacked.) 198.71.225.147 is trying the same trick.

Sometimes I will also see behavior where an IP address tries to login multiple times in a row. In that case, they are trying a dictionary attack with common logins and passwords.

anttechs

What a smart idea ;) Very Interesting thank you for replying.
I have a few wordpress sites that get attacked about 80 times a day so I could definitely look into this method.

I have copied your post onto my computer so I won't lose this ;)

I Suppose from there if you wanted to put them into categories like Government, Windows, Apple and big lists like iplist.com do then you would have to Whois every single one of the ips. Thats a lot of work!

Very interesting many thanks for taking the time to give me an example. ;)

n3by

My lists are automatically made by Suricata blocking privileged ports tcp/udp [0-1023] and few other well known services/servers ports like RDP, VNC,RADMIN,mySQL,SIP…
Once Suricata block them it also save them and automatically import them to a pfblocker alias list after a day ( just in case I need to delete an IP from Suricata list ).

So here you have my lists from two servers in two different countries:

pl_snort2c_30-05-2017.txt.gz
ro_snort2c_30-05-2017.txt.gz

TyphooN

Thank you for the list - will be adding it to my ipv4 feeds. How often do you recommend we update the list? Just trying to determine which feed I place it in and didn't want to hammer your server harder than necessary :)

sorenstoutner

@ecfx:

My lists are automatically made by Suricata blocking privileged ports tcp/udp [0-1023] and few other well known services/servers ports like RDP, VNC,RADMIN,mySQL,SIP…
Once Suricata block them it also save them and automatically import them to a pfblocker alias list after a day ( just in case I need to delete an IP from Suricata list ).

So here you have my lists from two servers in two different countries:

That is a good solution for ports where you are not running a valid service. It doesn't work in situations where there is a web server trying to differentiate between legitimate and illegitimate traffic.

Do you happen to have these lists posted somewhere they can be automatically updated by pfSense?

sorenstoutner

@TyphooN:

Thank you for the list - will be adding it to my ipv4 feeds. How often do you recommend we update the list? Just trying to determine which feed I place it in and didn't want to hammer your server harder than necessary :)

I update the list once or twice a week, so setting pfBlocker to update once a week would make sense.

I appreciate your consideration of the load on my server. Recent versions of pfBlocker use a HEAD command before a GET command to download the lists. The HEAD (header) command checks the date of the file to see if it has changed and takes minimul bandwidth (although there is all the overhead of establishing a HTTPS connection over TCP/IP first).

anttechs

Many Thanks for the lists btw ;)

In my search for finding out how some people get their lists I contacted a security company who had lists and asked them how they got there lists also asking how to get lists of companies you want to block like the Goverment, Windows, Apple, ISP's, BBC, CNN, Captia and so on.
In the UK we have a big problem with companies spying on you so I was very interested in finding out how to create lists like iplists.com

I got a very interesting reply witch Ill share on here as you might find it interesting.

–-----
If you are looking for the IP addresses allocated to ISPs you may check
this page:

. http://bgp.he.net

They have a global report per country:

. http://bgp.he.net/country/GB

You just need to get the individual announcements from those UK ASNs,
for instance:

. http://bgp.he.net/AS8220#_prefixes

I did check it out and its very good for tracking and finding ips to companies.
Of course I am in the UK so he gave me a UK example.

Very interesting ;)

TyphooN

Is this no longer being hosted? I have been getting the following the last couple of days:

Could not resolve host: pfblockerlists.smallbusinesstech.net Retry in 5 seconds…
. cURL Error: 6
Could not resolve host: pfblockerlists.smallbusinesstech.net Retry in 5 seconds...
. cURL Error: 6
Could not resolve host: pfblockerlists.smallbusinesstech.net Retry in 5 seconds...
.. Permission denied

Or is this a problem on my end with DNS resolution? I am using dns.watch for my DNS resolution at the moment.

sorenstoutner

@TyphooN:

Is this no longer being hosted? I have been getting the following the last couple of days:

Could not resolve host: pfblockerlists.smallbusinesstech.net Retry in 5 seconds…
. cURL Error: 6
Could not resolve host: pfblockerlists.smallbusinesstech.net Retry in 5 seconds...
. cURL Error: 6
Could not resolve host: pfblockerlists.smallbusinesstech.net Retry in 5 seconds...
.. Permission denied

Or is this a problem on my end with DNS resolution? I am using dns.watch for my DNS resolution at the moment.

As far as I can tell, everything is good with my server as well as with my DNS nameservers. Attached is a screenshot from mxtoolbox.com showing current DNS queries. However, I have receive two other reports today from people who were not able to access my servers, so something must have gone down in the DNS world. My guess is that the problem will sort it out over the next several hours, but if it doesn't you might try using a different DNS server to see if it makes a difference.

DNS.png_thumb

sorenstoutner

I did a little more digging and it looks like there must be some issue between them and Namecheap (my registrar) and some resolvers.

soren@soren-desktop:~$ nslookup pfblockerlists.smallbusinesstech.net 84.200.69.80
Server:         84.200.69.80
Address:        84.200.69.80#53

** server can't find pfblockerlists.smallbusinesstech.net: SERVFAIL

soren@soren-desktop:~$ nslookup pfblockerlists.smallbusinesstech.net 4.2.2.2
Server:         4.2.2.2
Address:        4.2.2.2#53

Non-authoritative answer:
Name:   pfblockerlists.smallbusinesstech.net
Address: 68.14.213.194

soren@soren-desktop:~$ nslookup pfblockerlists.smallbusinesstech.net 8.8.8.8
Server:         8.8.8.8
Address:        8.8.8.8#53

** server can't find pfblockerlists.smallbusinesstech.net: SERVFAIL

84.200.69.80 is dns.watch's main resolver. 4.2.2.2 is a resolver hosted by Level 3 Communications. 8.8.8.8 is a resolver hosted by Google.

sorenstoutner

I contacted Namecheap. They said their upstream DNS provider (whoever that is) had done some maintenance which had caused problems with DNSSEC. It should now be resolved.

TyphooN

@Soren:

I contacted Namecheap. They said their upstream DNS provider (whoever that is) had done some maintenance which had caused problems with DNSSEC. It should now be resolved.

I can now resolve and update the list. Thank you for your much valued work :)