IP-Blocklist

tommyboy180

So far I am pretty impressed and happy. :)

I don't mean to sound picky, but is there a way to whitelist? Or will there be in the future?

That is coming in the next couple updates. I haven't had time to even sit down and look at the package lately due to other priorities.
As soon as I have a good amount of time to dedicate to the project I will update the package.

screech

Hello, I'm french, so sorry for my bad english ;)

I use Pfsense V2 and Ip-Blocklist.

I try to put level1 and level2 list in ip-blocklist

When I Save/Upload I have this message :

Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 81 bytes) in /etc/inc/util.inc on line 780

As said in previous message i try these thing :

Add memory_limit = 128M in /usr/local/lib/php.ini and /usr/local/etc/php.ini -> It disapear after reboot
Add ini_set("memory_limit","128M"); in /etc/inc/config.inc -> Maybe it work… Or not... It doesn't shown in /usr/local/lib/php.ini or /usr/local/etc/php.ini
I think 128M is not enough so I try to put 256M -> Exactly the same
_From I-Blocklist, lvl1 is about 814983437 IPs and lvl2 is about 330844784 IPs. After the error message, I have only 326376 IPs Blocked and the Status is Running.

An other thing is strange, when I "Purge All" The 2 lists stay in the "Current List(s)"
When 1 Add just lvl1 list, it's ok. The "Current List(s)" show me 1 list..

And the last strange thing (but maybe it's normal), when I go to Status: Package logs, it say : "No packages with logging facilities are currently installed." (And the "Enable Logging" is check and saved in Ip-Blocklist of course)

I Tryed to uninstall and reinstall the package several time. Nothing work.

I Have these package installed :

Cron
NUT
Snort
IP-Blocklist of course

Can you help me please ?

Add : I use Firefox ;) and the list are these in the exemple : http://iblocklist.whitacrecomputers.com/files/bt_level1.gz_

tommyboy180

@screech:

Hello, I'm french, so sorry for my bad english ;)

I use Pfsense V2 and Ip-Blocklist.

I try to put level1 and level2 list in ip-blocklist

When I Save/Upload I have this message :

Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 81 bytes) in /etc/inc/util.inc on line 780

As said in previous message i try these thing :

Add memory_limit = 128M in /usr/local/lib/php.ini and /usr/local/etc/php.ini -> It disapear after reboot

Add ini_set("memory_limit","128M"); in /etc/inc/config.inc -> Maybe it work… Or not... It doesn't shown in /usr/local/lib/php.ini or /usr/local/etc/php.ini

I think 128M is not enough so I try to put 256M -> Exactly the same
_From I-Blocklist, lvl1 is about 814983437 IPs and lvl2 is about 330844784 IPs. After the error message, I have only 326376 IPs Blocked and the Status is Running.

An other thing is strange, when I "Purge All" The 2 lists stay in the "Current List(s)"
When 1 Add just lvl1 list, it's ok. The "Current List(s)" show me 1 list..

And the last strange thing (but maybe it's normal), when I go to Status: Package logs, it say : "No packages with logging facilities are currently installed." (And the "Enable Logging" is check and saved in Ip-Blocklist of course)

I Tryed to uninstall and reinstall the package several time. Nothing work.

I Have these package installed :

Cron

NUT

Snort

IP-Blocklist of course

Can you help me please ?

Add : I use Firefox ;) and the list are these in the exemple : http://iblocklist.whitacrecomputers.com/files/bt_level1.gz

Adding large lists will result in the php memory error. Just wait for the 'spinning animation' in FF stop before you continue.
When it says you are blocking X amount of IPs it's supposed to say X amount networks/IPs. Then number is correct just the wrong wording.
Purge all basically deletes your settings. You may try refreshing to see if your settings have been deleted. If it's not working on BETA 2 then you may have to delete one by one.
Logging doesn't go to the package logging section, all logging goes to the firewall log.

Updates will be coming soon for those who are waiting. Keep on the look out for email alerts, bug fixes, and feature additions._

screech

Thank you for your help.

Large file do the Error but it's load -> OK ;)
Number of IP Blocked = Number of Network/IP Blocked -> OK ;)
Purge All Work, it's just a "Cache" Problem of my Firefox -> Ok ;)
Logging go to the Firewall Logging -> Hum… Maybe... How can I be sure Ipblocklist block something ? Is there a way to see if a log is from Ip-Blocklist rules or if it's from Firewall Rules ? Can I see the name resolution of IP (List have a name for each IP, so it could be great if I can see the name instead of the IP ;) )

I can't wait the update ;) Thank you very much for your work.

tommyboy180

@screech:

Thank you for your help.

Large file do the Error but it's load -> OK ;)

Number of IP Blocked = Number of Network/IP Blocked -> OK ;)

Purge All Work, it's just a "Cache" Problem of my Firefox -> Ok ;)

Logging go to the Firewall Logging -> Hum… Maybe... How can I be sure Ipblocklist block something ? Is there a way to see if a log is from Ip-Blocklist rules or if it's from Firewall Rules ? Can I see the name resolution of IP (List have a name for each IP, so it could be great if I can see the name instead of the IP ;) )

I can't wait the update ;) Thank you very much for your work.

Right now the only way to tell what was blocked by the package is to click on the red x next to the entry. That returns a popup with the line number of the firewall rule. You can then open the firewall rules and see what line IPblocklist is on. I know it's a terrible way to track what get's blocked, but I don't have control over how pfsense displays logs. A solution would be to setup a remote logging server and filter that.
Thank you for your support! I look forward to the updates soon too!

DigitalJer

Love this package, TB. Some good ideas in there, looking fwd to whitelisting etc as well - excellent groundwork tho. No complaints whatsoever.

tommyboy180

@DigitalJer:

Love this package, TB. Some good ideas in there, looking fwd to whitelisting etc as well - excellent groundwork tho. No complaints whatsoever.

Thank you! So am I. I just need to find some time to sit down and get it done.

BenKenobe

Is there any way to get this to look inwards i.e. to a local server instead of a URL, it seems crazy to point it out at a URL when that URL is my servers URL. I have the gz files or txt files that I can host locally, I don't want to be dependant upon 'outside' sources.

Also the IP addresses blocked count - is this a count of IP addresses in the files or actively blocked addresses?

tommyboy180

@BenKenobe:

Is there any way to get this to look inwards i.e. to a local server instead of a URL, it seems crazy to point it out at a URL when that URL is my servers URL. I have the gz files or txt files that I can host locally, I don't want to be dependant upon 'outside' sources.

Also the IP addresses blocked count - is this a count of IP addresses in the files or actively blocked addresses?

It's possible now. I have lists hosted on an internal server here which IPblocklist is pointed at. Just add the appropriate url. For Example, my internal url is http://172.16.x.x/lists/myprivatelists.txt

The IP addresss blocked count is the count of networks and IPs. Keep in mind it includes networks (I have to update the wording in my next update).

screech

How can i "activate" or "desactivate" IPBlocklist on special port ?

Ex : if i put the "edu" list, lots of university are blocked. But I want to surf on university website (port 80) but it's blocked by IpBlocklist in input and output…

Is ther a way to put a white list like "...:80" ?

or a way to mod the php you give to us ?

I again thank you, your work is great ;)

Hint : for those how have an error with big file (it's work, but i doesn't like error :p), i found a easy way ;) :

I do a script like that :

fetch -o /usr/local/www/packages/ipblocklist/temp/level1.gz "http://list.iblocklist.com/?list=bt_level1"
fetch -o /usr/local/www/packages/ipblocklist/temp/level2.gz "http://list.iblocklist.com/?list=bt_level2"
fetch -o /usr/local/www/packages/ipblocklist/temp/edu.gz "http://list.iblocklist.com/?list=bt_edu"
fetch -o /usr/local/www/packages/ipblocklist/temp/ads.gz "http://list.iblocklist.com/?list=bt_ads"
fetch -o /usr/local/www/packages/ipblocklist/temp/spyware.gz "http://list.iblocklist.com/?list=bt_spyware"

So my list in IpBlocklist is like :

/usr/local/www/packages/ipblocklist/temp/level1.gz
/usr/local/www/packages/ipblocklist/temp/level2.gz
/usr/local/www/packages/ipblocklist/temp/antiinfrigement
…

I cron the script every 6h ;)
Then I cron /usr/local/etc/rc.d/IP-Blocklist.sh too ;)

tommyboy180

@screech:

How can i "activate" or "desactivate" IPBlocklist on special port ?

Ex : if i put the "edu" list, lots of university are blocked. But I want to surf on university website (port 80) but it's blocked by IpBlocklist in input and output…

Is ther a way to put a white list like "...:80" ?

or a way to mod the php you give to us ?

I again thank you, your work is great ;)

Hint : for those how have an error with big file (it's work, but i doesn't like error :p), i found a easy way ;) :

I do a script like that :

fetch -o /usr/local/www/packages/ipblocklist/temp/level1.gz "http://list.iblocklist.com/?list=bt_level1"
fetch -o /usr/local/www/packages/ipblocklist/temp/level2.gz "http://list.iblocklist.com/?list=bt_level2"
fetch -o /usr/local/www/packages/ipblocklist/temp/edu.gz "http://list.iblocklist.com/?list=bt_edu"
fetch -o /usr/local/www/packages/ipblocklist/temp/ads.gz "http://list.iblocklist.com/?list=bt_ads"
fetch -o /usr/local/www/packages/ipblocklist/temp/spyware.gz "http://list.iblocklist.com/?list=bt_spyware"

So my list in IpBlocklist is like :

/usr/local/www/packages/ipblocklist/temp/level1.gz
/usr/local/www/packages/ipblocklist/temp/level2.gz
/usr/local/www/packages/ipblocklist/temp/antiinfrigement
…

I cron the script every 6h ;)
Then I cron /usr/local/etc/rc.d/IP-Blocklist.sh too ;)

One of the updates that is coming will give the user the option to block outbound or allow it.
Excellent idea for the long lists! Thanks for sharing!

nst6563

I tried using the fetch method listed above, yet when I look at the IP Blocklist page it shows enabled, but that it's blocking 0 IP's (networks). Is that correct or is there a bug in the koolaid?

Also - will entries in the whitelist over-ride the entries in the blocklist? Just curious because I've noticed come sites that we normally access (online banking for example) are blocked in the level1 list.

Sort of confused….

thanks

tommyboy180

@nst6563:

I tried using the fetch method listed above, yet when I look at the IP Blocklist page it shows enabled, but that it's blocking 0 IP's (networks). Is that correct or is there a bug in the koolaid?

Also - will entries in the whitelist over-ride the entries in the blocklist? Just curious because I've noticed come sites that we normally access (online banking for example) are blocked in the level1 list.

Sort of confused….

thanks

Your list format is probably wrong. Look at the first post for the FAQ.
Entires in the the Whitelist do over-ride the blacklist.

tommyboy180

Got a really good question from ToxIcon!

@ToxIcon:

tommyboy can i use this list with Ipblocklist

http://maliciousnetworks.org/fire-blocklist.txt

The answer is no. You the list syntax must follow COMMENT:xxx.xxx.xxx.xxx-xxx.xxx.xxx.xxx (The peerguardian format).
But I have good news it's really easy to convert your lists with Regex tools. I like to use http://gskinner.com/RegExr/ to convert my lists.

Example:
The lists above has the format like:
109.108.130.188 AS8553
109.196.134.41 AS39150
109.196.143.33 AS39150
109.196.143.60 AS39150

So using http://gskinner.com/RegExr/ do a find and replace. Paste your list into the body.
Do a find for \b\d{1,3}.\d{1,3}.\d{1,3}.\d{1,3}\b
and replace with \nSPAM:$&-$&

We end up with:
SPAM:109.108.130.188-109.108.130.188 AS8553
SPAM:109.196.134.41-109.196.134.41 AS39150
SPAM:109.196.143.33-109.196.143.33 AS39150
SPAM:109.196.143.60-109.196.143.60 AS39150

Now you need to remove the AS8553 part, so we do another find and replace with the output you have (copy the output back into the main body).
Do a find for AS
and replace with #$&

Your output should look like:
SPAM:109.108.130.188-109.108.130.188 #AS8553
SPAM:109.196.134.41-109.196.134.41 #AS39150
SPAM:109.196.143.33-109.196.143.33 #AS39150
SPAM:109.196.143.60-109.196.143.60 #AS39150

So there you have it ToxIcon. Using regex you can convert any lists into the proper format that is supported by the package.

NoahVail

IPBlocklist works great on 2 of my pfSense boxes.

One other pfSense box is a problem. I can't seem to get IPBlocklist going.

All are pfSense 2.0-BETA4.
The problem box is currently 2.0-BETA4 (i386)
built on Sat Nov 6 00:43:13 EDT 2010

On the problem box, I've install IP-Blocklist every couple of months or so (back to pfSense 2.0-BETA1). Problem box remains a problem.

I use FF. The package installs without error.
I then…
1. Paste my list link into the form and press '+'.
( ex: http://iblocklist.charlieprice.org/files/bt_spyware.gz - I've tried several good .gz links.)
2. Check 'Enable IP-Blocklist.
3. Click Save/Update.

Then the 'Current List' disappears entirely.
(I can bring it back by pressing '+', but that doesn't effect anything.)

My Status is always....

Current Status = Running

You are blocking 0 IPs

I have tried every possible combination of buttons; Save, Save/Update, etc.
I have uninstalled Country Block (the only other package) and tried IPBlocklist by itself.
I have tried rebooting between installs and configs.
I have tried a number of different versions of pfSense 2.0-BETA.

My results have been 100% consistent. Status Running - Blocking 0 IPs.

I'm uncertain on what to try, or where to look next.
If you or anyone else, have any thoughts, I'd be grateful to hear them.

Thanks.
NV

PS - a note:
At one location, spam rates are down by over 50%.
At another; I haven't had a single drive-by malware episode (were regular occurrences).
I credit CountryBlock as an important part of that result.
Thanks again.

tommyboy180

@NoahVail:

IPBlocklist works great on 2 of my pfSense boxes.

One other pfSense box is a problem. I can't seem to get IPBlocklist going.

All are pfSense 2.0-BETA4.
The problem box is currently 2.0-BETA4 (i386)
built on Sat Nov 6 00:43:13 EDT 2010

On the problem box, I've install IP-Blocklist every couple of months or so (back to pfSense 2.0-BETA1). Problem box remains a problem.

I use FF. The package installs without error.
I then…
1. Paste my list link into the form and press '+'.
( ex: http://iblocklist.charlieprice.org/files/bt_spyware.gz - I've tried several good .gz links.)
2. Check 'Enable IP-Blocklist.
3. Click Save/Update.

Then the 'Current List' disappears entirely.
(I can bring it back by pressing '+', but that doesn't effect anything.)

My Status is always....

Current Status = Running

You are blocking 0 IPs

I have tried every possible combination of buttons; Save, Save/Update, etc.
I have uninstalled Country Block (the only other package) and tried IPBlocklist by itself.
I have tried rebooting between installs and configs.
I have tried a number of different versions of pfSense 2.0-BETA.

My results have been 100% consistent. Status Running - Blocking 0 IPs.

I'm uncertain on what to try, or where to look next.
If you or anyone else, have any thoughts, I'd be grateful to hear them.

Thanks.
NV

PS - a note:
At one location, spam rates are down by over 50%.
At another; I haven't had a single drive-by malware episode (were regular occurrences).
I credit CountryBlock as an important part of that result.
Thanks again.

Are all boxes using the same build version? I tested the list and it works fine. The only thing that I can think of is that pfSense 2.0-BETA4 is just that, BETA so I can't vouch for how the package is being handled in that particular version.

I don't have a 2.0BETA4 box to test with so unfortunately you or someone else is going to have to test. I'm glad that SPAM is down and thank you for your support.
I wish I had a better answer for you but I don't have any resources to test 2.0BETA4 right now.

NoahVail

@tommyboy180:

@NoahVail:

IPBlocklist works great on 2 of my pfSense boxes.

One other pfSense box is a problem. I can't seem to get IPBlocklist going.

All are pfSense 2.0-BETA4.
The problem box is currently 2.0-BETA4 (i386)
built on Sat Nov 6 00:43:13 EDT 2010 …

Are all boxes using the same build version? I tested the list and it works fine. The only thing that I can think of is that pfSense 2.0-BETA4 is just that, BETA so I can't vouch for how the package is being handled in that particular version.

I don't have a 2.0BETA4 box to test with so unfortunately you or someone else is going to have to test. I'm glad that SPAM is down and thank you for your support.
I wish I had a better answer for you but I don't have any resources to test 2.0BETA4 right now.

OK. That is an answer, if not precisely one I was fishing for.

FYI, all 3 are different build dates; all within 40 days of now.

Today, I moved the drive from a PIII ThinkPad to a P4 Inspiron and wiped/reloaded pfSense.
After the preliminary setup, I loaded IP-Blocklist as the first package and… same result.

At least we know it can work in pfSense 2. I suspect one day we'll also know how.

Thanks for the reply.

NV

tommyboy180

~~WARNING: Duplicate entries per list cancel out

I just found something that could be a problem if you make your own lists. If you make your own list like I do and you accidentally have a duplicate entry in the list it will cancel out and will not be blocked. This is a bug that is caused by the Perl script that interprets your lists.

Example duplication:
unkn:194.71.107.25-194.71.107.25
ad:194.71.107.25-194.71.107.25

This will result in the IP not being blocked. The Fix: Just be careful when creating your own lists. If your paranoid run you "sort happybirthday.txt | uniq
" your lists!

Again this will only apply to people that make their own lists. Publishers of official IP lists filter for duplicates for you.~~

This bug is fixed. Duplicate lists are not an issue any longer!

JSmorada

I've been having an issue after installing IP-Blocklist. I can download a list (level_1 in my case), but when I check "enable" and "save/update", the machine will throw an error, "Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 35 bytes) in /etc/inc/util.inc on line 380" 5-10 minutes afterwards. I suspect my machine is at fault since it's an old Dell PII maxed out at 512MB of RAM. The only other addons I'm running are cron, Country Block and Snort. Incidentally, IP-Blocklist was failing before I installed Snort and cron.

Thanks,
Jon

tommyboy180

@nipstech:

I've been having an issue after installing IP-Blocklist. I can download a list (level_1 in my case), but when I check "enable" and "save/update", the machine will throw an error, "Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 35 bytes) in /etc/inc/util.inc on line 380" 5-10 minutes afterwards. I suspect my machine is at fault since it's an old Dell PII maxed out at 512MB of RAM. The only other addons I'm running are cron, Country Block and Snort. Incidentally, IP-Blocklist was failing before I installed Snort and cron.

Thanks,
Jon

This has been addressed in the previous posts. Basically just let it run until the "working animation" in your browser stops. For more specifics look back on some previous posts.