Firehol level 1 list blocking LAN resources
-
I had similar issues with firehol lvl 1, but with broadcast packets.
See https://forum.pfsense.org/index.php?topic=138877.0
-
I ended up taking BBcan's advice (good advice it always is) and simply put the lists that comprise firehol1 into pfblocker (minus the bogons list of course).
-
I ended up taking BBcan's advice (good advice it always is) and simply put the lists that comprise firehol1 into pfblocker (minus the bogons list of course).
Using the Lvl 1 Feed is going to cause grief… Just don't do it ;) ;)
-
The level1 list looks like this, excluding the bogons.
(I really hope that I understood the firehol list content correctly. If that is the case, now someone else don't have to recreate this wheel again.)Here goes:
http://www.spamhaus.org/drop/drop.txt http://www.spamhaus.org/drop/edrop.txt http://feeds.dshield.org/block.txt https://feodotracker.abuse.ch/blocklist/?download=ipblocklist https://sslbl.abuse.ch/blacklist/sslipblacklist.csv https://zeustracker.abuse.ch/blocklist.php?download=badips http://osint.bambenekconsulting.com/feeds/c2-ipmasterlist.txt -
Try to use HTTPS for all sites that support it…
-
The level1 list looks like this, excluding the bogons.
(I really hope that I understood the firehol list content correctly. If that is the case, now someone else don't have to recreate this wheel again.)Here goes:
http://www.spamhaus.org/drop/drop.txt http://www.spamhaus.org/drop/edrop.txt http://feeds.dshield.org/block.txt https://feodotracker.abuse.ch/blocklist/?download=ipblocklist https://sslbl.abuse.ch/blacklist/sslipblacklist.csv https://zeustracker.abuse.ch/blocklist.php?download=badips http://osint.bambenekconsulting.com/feeds/c2-ipmasterlist.txtiorx, thank you for compiling and posting that.
Is the above list sufficient for the residential user? Or are there additional sources I should add?On the other hand, given the great work BBcan has done with the "suppression" option etc, is it safe to just use the firehol level 1 list just in case the sources it pulls from change over time?
(just one small piece of 1) protecting the kids online and 2) protecting our networked computers from what the kids may inadvertently do online!)
Thank you all.
BBCan – donation coming your way.Ari
-
This is the IP lists I’m using. I started with a few and over time added more and more lists I’ve found. I use a script that download all lists with wget, unpack those that need and put all of them in one file. The resulting file is filtered leaving only ip addresses and checked and cleaned for duplicates and sorted. Finally, the file is cleaned from RFC 1918 and RFC 4193 addresses and some common important ip addresses such as 8.8.8.8 8.8.4.4 and so on. The finished file is put on a small internal webserver. Pfblocker get the file from there. I do the same with DNSBL lists, I can post all DNSBL lists I have if someone is interested. The ones here is only IP lists.
I put the resulting IP list and DNSBL list on an external webserver as well currently updated once a week. You have them here: http://dnsbl.dyndns.org:9080/MyBlocklist.txt and http://dnsbl.dyndns.org:9080/mydnsblfeed.txt The DNSBL file is big so it can take a while to download.https://gist.githubusercontent.com/BBcan177/d7105c242f17f4498f81/raw/90eb2ac8bdc01af3008d728b7c0f10dc7b2506b4/MS-3
https://gist.githubusercontent.com/BBcan177/bf29d47ea04391cb3eb0/raw/b344ebc9475acdea1fae38a12c4ea9332838a184/MS-1
https://rules.emergingthreats.net/fwrules/emerging-Block-IPs.txt
https://www.spamhaus.org/drop/drop.txt
https://pfblockerlists.smallbusinesstech.net/hackerlist.txt
https://ransomwaretracker.abuse.ch/downloads/RW_IPBL.txt
https://rules.emergingthreats.net/blockrules/compromised-ips.txt
http://www.abuseat.org/iotcc.txt
https://raw.githubusercontent.com/firehol/blocklist-ipsets/master/firehol_level1.netset
https://raw.githubusercontent.com/firehol/blocklist-ipsets/master/firehol_level2.netset
http://cinsscore.com/list/ci-badguys.txt
https://ransomwaretracker.abuse.ch/downloads/LY_PS_IPBL.txt
https://ransomwaretracker.abuse.ch/downloads/TL_C2_IPBL.txt
https://zeustracker.abuse.ch/blocklist.php?download=badips
https://raw.githubusercontent.com/firehol/blocklist-ipsets/master/firehol_level3.netset
http://danger.rulez.sk/projects/bruteforceblocker/blist.php
https://lists.blocklist.de/lists/all.txt
http://malc0de.com/bl/IP_Blacklist.txt
https://feodotracker.abuse.ch/blocklist/?download=ipblocklist
https://raw.githubusercontent.com/firehol/blocklist-ipsets/master/firehol_level4.netset
http://feeds.dshield.org/top10-2.txt
https://feeds.dshield.org/block.txt
http://list.iblocklist.com/?list=dgxtneitpuvgqqcpfulq&fileformat=p2p&archiveformat=gz
http://list.iblocklist.com/?list=dgxtneitpuvgqqcpfulq&fileformat=p2p&archiveformat=gz
http://list.iblocklist.com/?list=dgxtneitpuvgqqcpfulq&fileformat=p2p&archiveformat=gz
http://list.iblocklist.com/?list=llvtlsjyoyiczbkjsxpf&fileformat=p2p&archiveformat=gz
http://list.iblocklist.com/?list=llvtlsjyoyiczbkjsxpf&fileformat=p2p&archiveformat=gz
http://list.iblocklist.com/?list=llvtlsjyoyiczbkjsxpf&fileformat=p2p&archiveformat=gz
http://list.iblocklist.com/?list=usrcshglbiilevmyfhse&fileformat=p2p&archiveformat=gz
http://list.iblocklist.com/?list=usrcshglbiilevmyfhse&fileformat=p2p&archiveformat=gz
http://list.iblocklist.com/?list=usrcshglbiilevmyfhse&fileformat=p2p&archiveformat=gz
http://list.iblocklist.com/?list=xpbqleszmajjesnzddhv&fileformat=p2p&archiveformat=gz
http://list.iblocklist.com/?list=xpbqleszmajjesnzddhv&fileformat=p2p&archiveformat=gz
http://list.iblocklist.com/?list=xpbqleszmajjesnzddhv&fileformat=p2p&archiveformat=gz
http://www.spamhaus.org/drop/edrop.txt
https://sslbl.abuse.ch/blacklist/sslipblacklist.csv
http://osint.bambenekconsulting.com/feeds/c2-ipmasterlist.txtIf you want to use the Firehol level 1 only you could use the same approach with a script to filter out those addresses you don't want.
[EDIT]
added three more lists to my script from this thread
Forgot that the webserver is on port 9080 Fixed now -
I put the resulting IP list and DNSBL list on an external webserver as well currently updated once a week. You have them here: https://dnsbl.dyndns.org/modules/mymod/MyBlocklist.txt and https://dnsbl.dyndns.org/modules/mymod/mydnsblfeed.txt The DNSBL file is big so it can take a while to download.
The links are not working.
Connection timed out after 15039 milliseconds Retry in 5 seconds... . cURL Error: 28 Connection timed out after 15015 milliseconds Retry in 5 seconds... . cURL Error: 28 Connection timed out after 15021 milliseconds Retry in 5 seconds... .. unknown http status code Download FAIL [ 11/20/17 08:12:54 ] Firewall and/or IDS are not blocking download. The Following list has been REMOVEDThanks for sharing. Let us know when the webserver is working again.
-
@Presbuteros:
The links are not working.
Sorry, forgot that the server is on port 9080. Changed in the original post.
[edit]
Changed to http port 9080 -
Tried again with updated URL
https://dnsbl.dyndns.org:9443/modules/mymod/MyBlocklist.txt
[ Comp ] Downloading update . **Saving configuration [ 11/20/17 09:21:40 ] ... cURL Error: 51 SSL: no alternative certificate subject name matches target host name 'dnsbl.dyndns.org' Retry in 5 seconds... . cURL Error: 51 SSL: no alternative certificate subject name matches target host name 'dnsbl.dyndns.org' Retry in 5 seconds... . cURL Error: 51 SSL: no alternative certificate subject name matches target host name 'dnsbl.dyndns.org' Retry in 5 seconds... .. unknown http status code -
This got far more complicated than I thought ….. Now I've put up a new small webserver for external access so I don't have to mess around with our production systems for this.
http://dnsbl.dyndns.org:9080/MyBlocklist.txt
http://dnsbl.dyndns.org:9080/mydnsblfeed.txtI hope everything ok now. I modified my previous posts :P
Info about all the public available IP and DNSBL lists I'm using
http://dnsbl.dyndns.org:9080/info.txt -
Thank you seanr22a – that's very generous of you to compile, filter, host and post. Very generous of iorx as well, above.
I put together a comparison in Excel of the three lists (FireHol lvl 1, iorx's compilation above, and seanr22a's compilation above) and the initial conclusions I draw are:
37% of FireHol Lvl 1's IPs are in iorx's list and 100% are in seanr22a's list
99% of iorx's values are reflected in FireHol's and 65% of the ips are reflected in seanr22a's.
Seanr22a's list may be more comprehensive. It is a 3MB download and has almost 200,000 IPs (the dbsbl list is a 45MB download :o - may reflect lots of effort on his part).Excel file attached.
My questions, as a layman, would be:
With the larger list, is there a substantially increased potential for false-positives?
Will this larger list slow down the pfSense box?Ari
-
Yes, thanks to all who are contributing here.
With the larger list, is there a substantially increased potential for false-positives?
Will this larger list slow down the pfSense box?
AriI loaded the list from seanr22a. I took a while to download and compile. It did not "appear" to slow down the pfSense box. My mobo is a Gigabyte GA-J1900N-D3V so a Celeron quad-core 2Ghz and 8 GB of RAM. What I did notice was a lot of issues loading news sites like cnn, foznews, drudgereport, etc. Videos were stalling out and certain essential elements of the page would not load. However, I would like to hear from others on their use of the list and if they had any obvious issues. I simply disabled the list until I have more time to test later.
-
@Presbuteros:
Yes, thanks to all who are contributing here.
I loaded the list from seanr22a. I took a while to download and compile. It did not "appear" to slow down the pfSense box. My mobo is a Gigabyte GA-J1900N-D3V so a Celeron quad-core 2Ghz and 8 GB of RAM. What I did notice was a lot of issues loading news sites like cnn, foznews, drudgereport, etc. Videos were stalling out and certain essential elements of the page would not load. However, I would like to hear from others on their use of the list and if they had any obvious issues. I simply disabled the list until I have more time to test later.
Check the Pfblocker logs for what is blocked related to the sites you visit. You have logs for both DNSBL and IP lists.
As you can see in the info.txt file the lists are made from public available lists maintained by many different people and organizations. Unfortunately there is no list that fits everyone. Simply whitelist those sites that causes you problem, I've done that for many ip's and domains to make it work for me so start dig in to the logs :) -
-
http://dnsbl.dyndns.org
@seanr22a: i give a "thank you"
and by the way: before importing your banlist, I checked the IP provenience behind your dyndns address (sorry for that).
Interstingly I did note a Spamhaus ZEN blocklist entry for that IP ;) (just for the record: I imported and I will thankfully update your list weekly). -
I ended up taking BBcan's advice (good advice it always is) and simply put the lists that comprise firehol1 into pfblocker (minus the bogons list of course).
Using the Lvl 1 Feed is going to cause grief… Just don't do it ;) ;)
Even with Suppression enabled?
I'm interested to know as well.
Also, when someone refers to whitelist the LAN_NET (and other vlans I guess) where does one actually whitelist those?
-
In Firewall > Aliases > IP > pfBlockerNGSuppress ?
-
In Firewall > pfBlockerNG > IPv4 > Create a new "Permit Outbound" list and add them there? Ensuring that this new list is on top of every other list (similarly to the firewall rules) as to ensure that it is applied before any "Deny" lists?"
Thanks a lot!
-
-
This got far more complicated than I thought ….. Now I've put up a new small webserver for external access so I don't have to mess around with our production systems for this.
http://dnsbl.dyndns.org:9080/MyBlocklist.txt
http://dnsbl.dyndns.org:9080/mydnsblfeed.txtI hope everything ok now. I modified my previous posts :P
Info about all the public available IP and DNSBL lists I'm using
http://dnsbl.dyndns.org:9080/info.txtFor all of you using this please see the updated info.txt file. There is a few new ipblock lists added and a few dnsbl lists. There is more than 200 downloads every day so apperently it's not only me finding them useful :)
Info about all the public available IP and DNSBL lists I'm using:
http://dnsbl.dyndns.org:9080/info.txtThe resulting lists:
http://dnsbl.dyndns.org:9080/MyBlocklist.txt
http://dnsbl.dyndns.org:9080/mydnsblfeed.txt -
Hi seanr22a, I've been using your lists for a bit now and they have been just what i've been after.. however, I wonder if you could help with a little issue which is causing poor connectivity, it seems one (or more) of the lists you use are blocking certain ranges within the cdn's of Cloudflare, CloudFront and aws. they seem to vary daily as the lists are updated.. Possibly for good reason, however, given that the ranges are generally from anywhere in the entire world assigned randomly to a site, the blocking is causing connectivity issues to certain websites, which dnbl whitelisting is having no effect on, as the IP's change all the time.
I wondered if you could process out the ranges? at the moment i'm having to go through the alerts after an update to see if there are any blocked ranges attempting to connect to/from a server or client. Having the ranges in pf suppression doesn't always seem to work for some reason.
ranges are:
http://d7uri8nf7uskq.cloudfront.net/tools/list-cloudfront-ips
https://www.cloudflare.com/ips-v4
https://ip-ranges.amazonaws.com/ip-ranges.jsonAlso, (you or a mod) perhaps make a separate thread for your list to save hijacking this one further? :)
-
I've been struggling with level 1 list as well. Unfortunately, this forum was down for maintenance during my struggles so I came up with the following. Please let me know if I took the wrong approach.
At first I was screwing around with the block inbound/outbound settings and with the auto rule order (which kept doing it wrong by the way). Also, I use separators to keep a good overview of which rules are for what. Obviously pfblocker doesn't understand which separators are for which set of rules, so it kept reordering in a way that I don't want/like. Another problem was that, probably due to my incompetence, I was unable to override some blocked IP's until I changed the order (or changed a rule from 'any' to 'out' but a reload would change the order and reload the rules back to their default.
So, I simply changed all the IP lists to alias native and created my own LAN rules in the LAN tab with a whitelist rule for false positives above it.
I feel this is actually a better way because it gives more control and doesn't mess with my rule order.
Any tips/comments on this approach?
Thanks.
