How/Can pfBlocker process this Ransomware list?



  • Is there any way to process this list with pfBlocker?

    http://ransomwaretracker.abuse.ch/downloads/RW_URLBL.txt

    Here's a small sample of what the entries look like:

    http://1000i.co/87yfhc
    http://101natural.com/t76f3g
    http://103.27.52.92/874ghv3
    http://103.27.52.92/t67bg
    http://108.174.196.88/8dpg3
    http://109.108.129.43/t76f3g
    http://109.73.234.241/dgq01p
    http://109.73.234.241/ehprln
    http://10minutesto1.net/d05k5d
    http://11011020.web.fc2.com/ets19pre
    http://111.86.142.67/~h_fujii/95hdienf
    http://112.213.84.94/t67bg
    http://1140746.net/kjg56f7
    http://117.239.70.228/874ghv3
    http://121.83.206.211/~ftp-yama/9z6nu
    http://122.15.8.163/7fg3g
    http://125ru.web.fc2.com/09u9jn87
    http://12hourenergy.com.au/ty6yhd
    http://139.162.29.193/g67eihnrv
    

    It is one of the lists found here:
    ransomwaretracker.abuse.ch/blocklist/

    From what I can see, this list has to be broken in two parts:

    • The hard coded IP urls need to be put in an IPv4 list
    • The domain names need to be put in the DNSBL

    Alternatively is there any way that I can import a list from the local file system?

    Wouldn't be hard to write a bit of python to d/l and make two text files that pfBlocker could import.





  • @RonpfS:

    https://ransomwaretracker.abuse.ch/downloads/RW_DOMBL.txt
    and https://ransomwaretracker.abuse.ch/downloads/RW_URLBL.txt can be used in DNSBL

    This one is for IPV4 https://ransomwaretracker.abuse.ch/downloads/RW_IPBL.txt

    Thanks for the reply RonpfS….

    I've already loaded those, but when I looked at http://ransomwaretracker.abuse.ch/downloads/RW_URLBL.txt it seemed to contain a lot of new/different content from the other two lists which is why I was eager to include it.


  • Moderator

    You have two choices:

    1. Add the source to both an IPv4 alias and a DNSBL group and it will collect either the IPv4 addresses or Domains as required.

    2. In DNSBL, when a feed contains IPv4 addresses, you can enable the DNSBL IP option to collect any IPv4 address that it finds. All IPs are combined into a single DNSBL_IP alias that can be used in your firewall rules.

    Also you can add a pfSense local file as a source. Click on the blue infoblock icons for further details.



  • Thanks for the reply and all your great work on this package BBcan177

    The URL http://ransomwaretracker.abuse.ch/downloads/RW_URLBL.txt produces the following output for IPv4:

    103.27.52.92/8
    103.27.52.92
    108.174.196.88/8
    109.108.129.43
    109.73.234.241
    109.73.234.241
    111.86.142.67
    112.213.84.94
    117.239.70.228/8
    121.83.206.211
    122.15.8.163/7
    139.162.29.193
    158.195.68.10/8
    158.195.68.10
    158.195.68.10
    158.199.158.185
    172.246.84.150
    176.58.124.197
    176.9.41.156
    178.78.87.8
    178.78.87.8
    192.138.189.69/8
    192.138.189.69
    194.28.172.166
    198.1.95.93
    202.210.189.111
    209.41.183.242
    210.118.170.181/8
    210.240.104.2/8
    210.240.104.2
    211.115.110.218
    211.18.200.4
    212.26.129.68
    212.26.129.68
    213.228.128.12
    213.228.128.12
    216.104.183.199
    216.104.188.249/8
    216.104.188.249
    217.172.226.2
    217.26.70.200
    217.64.197.138
    218.228.19.9
    50.28.211.199
    64.207.144.148
    64.22.100.95/7
    66.147.244.210
    67.23.226.139
    69.162.74.116/8
    69.162.74.116
    69.61.11.216
    70.32.93.234
    72.47.222.40
    79.96.153.93/8
    79.96.153.93
    79.96.153.93
    80.109.240.71
    80.241.232.207
    81.218.219.227/8
    83.235.64.44
    83.235.64.44
    85.92.144.157
    87.106.38.204
    87.244.17.86
    88.150.144.236/7
    89.145.78.9
    94.127.33.126/7
    98.131.20.17
    98.131.20.17
    

    As you can see this is a bit dangerous (/7 /8 is a disaster waiting to happen)!
    There are some very funky URLs that start with /7 and /8 that are messing things up.  Possibly the regex needs a bit of tweaking to make delimiters white space or NOT [A-Za-z0-9].  That would likely fix this problem.

    Is there any way to hook a custom downloader?
    If it can't be done already, How about a directory similar to rc.d that runs a script with the name of the Group or the list after it is fetched, but before it is loaded?
    I would think that this would still be secure (as long as the code installed is secure) and I would require code be installed by ssh/scp which presupposes credentials and a minimal level of skill.

    Alternatively can I specify a source from the local file system someway?
    A little harder to work with, but then you don't have to touch pfSense and I can do whatever I want.

    I've had several cases where I couldn't use lists with pfBlocker lists because overrides were too difficult (Try overriding a /18 with /32 & /24 - grepping and pulling the offending line(s) would be so simple):

    I wanted also wanted to try out FIREHOL Level 1 directly from GitHub instead of downloading all the separate lists. I tried it, and it totally killed my system - I think it was because the list contained broadcast addresses that were floating around my network due to double NAT or IOT devices… didn't bother to figure out what the problem was just pulled the list because I expected that override would be way to hard or impossible anyway.

    The FIREHOL anon list is also one that I would like to load, but can't due to the difficulty of unblocking my VPN provider (/18s and multiple ASNs).  Again grepping and removing lines would be easy.

    Comments / suggestions / work arounds / have I overlooked something?

    To be clear, no criticism, just a desire to get info so I can make better use of a great package (and possibly suggest an improvement for a future release if it would be of help to a large enough user group.)


  • Moderator

    I will get one of my beta testers to post the new regex to fix that.

    Also there is an IPv4 tunable to limit CIDRs already in the pkg.

    I am away until next week so have limited access to review code.



  • Edit /usr/local/pkg/pfblockerng/pfblockerng.inc around line 4378 you should find :

    	#################################################
    	#	Download and Collect IPv4/IPv6 lists	#
    	#################################################
    
    	// IPv4 REGEX Definitions
    	$pfb['range']	= '/((?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?))-((?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?))/';
    	$pfb['ipv4']	= '/(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)(\/(3[012]|[12]?[0-9]))?/';
    
    	// IPv6 REGEX Definitions - Reference: http://labs.spritelink.net/regex
    

    Change to this

    	#################################################
    	#	Download and Collect IPv4/IPv6 lists	#
    	#################################################
    
    	// IPv4 REGEX Definitions
    	$pfb['range']	= '/((?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?))-((?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?))/';
    	//$pfb['ipv4']	= '/(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)(\/(3[012]|[12]?[0-9]))?/';
    	$pfb['ipv4']	= '/(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)((\/(3[012]|[12]?[0-9]))?(?![-0-9a-zA-Z]))/';
    
    	// IPv6 REGEX Definitions - Reference: http://labs.spritelink.net/regex
    

    it should produce this now

    103.27.52.92
    108.174.196.88
    109.108.129.43
    109.73.234.241
    111.86.142.67
    112.213.84.94
    121.83.206.211
    122.15.8.163
    139.162.29.193
    158.195.68.10
    158.199.158.185
    172.246.84.150
    176.58.124.197
    176.9.41.156
    178.78.87.8
    192.138.189.69
    194.28.172.166
    198.1.95.93
    202.210.189.111
    209.41.183.242
    210.118.170.181
    210.240.104.2
    211.115.110.218
    211.18.200.4
    212.26.129.68
    213.228.128.12
    216.104.183.199
    216.104.188.249
    217.26.70.200
    217.64.197.138
    218.228.19.9
    50.28.211.199
    64.207.144.148
    64.22.100.95
    67.23.226.139
    69.162.74.116
    69.61.11.216
    70.32.93.234
    72.47.222.40
    80.241.232.207
    81.218.219.227
    85.92.144.157
    87.106.38.204
    87.244.17.86
    88.150.144.236
    89.145.78.9
    94.127.33.126
    

    BTW These are also present in pfB_DNSBLIP .


  • Banned

    @RonpfS:

    Change to this

    And that was the "regex is easy" example of the day.



  • @RonpfS:

    Edit /usr/local/pkg/pfblockerng/pfblockerng.inc around line 4378 you should find :

    Thanks RonpfS…. good job... made the edit and it worked just fine.  Had to search for the code block, but on my system there was about 300+ fewer lines:

    4037        // IPv4 REGEX Definitions

    I assume you must be using a newer/development version?

    Anyway great job... thanks.



  • Yes I am running the development version.

    I was just helping BBcan177 on this one as he doesn't have easy access to the code.


  • Moderator

    Thanks for testing the updated regex :)


  • Banned


  • Moderator

    @dcol:

    Why not just use
    https://ransomwaretracker.abuse.ch/downloads/RW_IPBL.txt

    They are two different Feeds…

    The URL and DOM feeds should be used in DNSBL as it contains Domain names.... There are also IPs mixed in, so enabling the DNSBL IP option will also pull those IPs...


Log in to reply