IP-Blocklist

[jikjik101]

tnx. no more error on the syslog with ipblock… however, i have tx underrun. need to search the forum about that one.
Thanks again. ;)

[mikesamo]

Hello,

samething with countryblock.sh

Thanks,

[MrEmbedded]

@tommyboy180:

@Systex:

@tommyboy180:

@Systex:

I'm running an embedded install of 2.0Beta5 and tried to make the changes listed in this thread in order to get it to work. However, I accidentally forgot a semicolon after the first line I added (require_once('config.inc')) and when I went to refresh the IP-Blocklist page I got the following error in the textbox:

Parse error: syntax error, unexpected T_ECHO in /usr/local/www/packages/ipblocklist/ipblocklist_list.php on line 5

Since then, I've tried everything I can to get back to a cleanly-installed state: I've undone the edits I made to that file, and I've even uninstalled and reinstalled the IP-Blacklist package, but whatever I do, I now always get that error. How can I get back to a clean slate without completely reinstalling pfsense?

My best advice is to uninstall the package, check the file system (/usr/local/www/ipblocklist) and delete it, and then reboot. That should help you start from a clean slate.

Hmm, after uninstalling I looked in /usr/local/www/ and there was no file or directory ipblocklist. I then rebooted, and reinstalled the package. Same exact error as before. The strange thing, is that if I look at the file /usr/local/www/packages/ipblocklist/ipblocklist_list.php, it's the stock, unmodified form (i.e., correct syntax)!? I'm very confused as to what's going on.

Sorry. The directory is /usr/local/www/packages/ipblocklist/
Uninstall and check to make sure that it's deleted.
Try executing the .sh manually in your console if you get the same error again.

I can add to this.  I was having the same issue with a mistype such that on the main tab there was a similar error message to Systex.  An uninstall-manual flush/reinstall did nothing.  I did not reboot however.  What I have realized is that everything is fine but the message was stale and showing in the frame and this never refreshed.  I right clicked on the frame and manually reloaded the frame where the error message was and this fixed everything for me as far as the display and the error message is concerned.

[nesense]

I got it installed with a couple of lists, when I check syslog i always get this message: "root: IP-Blocklist was found not running", if i goto Firewall > IP-Blocklist status says its running with a number of networks blocked, any idea what the issue here is?

Thank you

[tommyboy180]

@nesense:

I got it installed with a couple of lists, when I check syslog i always get this message: "root: IP-Blocklist was found not running", if i goto Firewall > IP-Blocklist status says its running with a number of networks blocked, any idea what the issue here is?

Thank you

I think I may know what the issue is. I'm still doing work on the package but the binary files are no longer in the repo. This will be fixed very soon!

[n0vnm]

How does one create an importable whitelist? I play Guild Wars and NC interactive is blocked with Level-1 blacklist. I want to whitelist the subnets for Guild Wars, but have ended up beating my head on a brick wall trying to import. I don't have sufficient skills to edit within the Linux shell. Any other suggestions?

NC INTERACTIVE, INC:206.127.144.0-206.127.159.255

Thanks!

[ToxIcon]

pfsense 2.0 R3
tommy can you fix the log for ipblocklist the
Enable Logging? does not stay enable once you click
save/update.

Also was just wondering if you can make a
dashboard widget that can display the log files for
ipblocklist and country block so that we
can see what is being block thanks. something like to one for snort
and havp.

[knight313]

pfsense 2.0 Rc3
built on Tue Jun 21 16:50:25 EDT 2011
Package does not block any IP.
System log (root: IP-Blocklist was found not running)

[ToxIcon]

Pfsense 2.0-RC3

tommyboy180  ip-blocklist will not update the list form iblocklist is it because they made some changes to the update url
like this address no longer seem to work http://iblocklist.aodcommunity.com/files/wujnnzxrgppvpdujdetm/bt_ads.gz 
but if enter into a into a browser it downloads.
their new update format looks like this
http://list.iblocklist.com/?list=bt_ads&fileformat=p2p&archiveformat=gz
but when added to  ip-blocklist it also does not download
their are no errors in the log for ip-blocklist.

Thanks tommyboy180 for all the great work and great packages

[tommyboy180]

@ToxIcon:

Pfsense 2.0-RC3

tommyboy180  ip-blocklist will not update the list form iblocklist is it because they made some changes to the update url
like this address no longer seem to work http://iblocklist.aodcommunity.com/files/wujnnzxrgppvpdujdetm/bt_ads.gz 
but if enter into a into a browser it downloads.
their new update format looks like this
http://list.iblocklist.com/?list=bt_ads&fileformat=p2p&archiveformat=gz
but when added to  ip-blocklist it also does not download
their are no errors in the log for ip-blocklist.
 
Thanks tommyboy180 for all the great work and great packages

http://iblocklist.taltos.org/files/wujnnzxrgppvpdujdetm/bt_ads.gz is the url for that file. To find the real url just download the file like normal in your browser and go back into your downloads and look at the url in your browser history.

I'm not sure how often the url scheme changes on that site so checking the url or saving it to a website you own may help with stability.

Thanks for your support! If you know php then perhaps we could work together on the project!

[wm408]

Tommyboy,

I have yet to upgrade to 2.0.x, as I will probably wait for final release (however I did test it and it looks great).  This means I am still running 1.2.3.

I have recently worked with your latest, 0.3.2.  As we've talked before in this thread, I do use the nanobsd (embedded) version and I do have a SATA 160GB drive mounted separately of the OS in my lil Soekris box.  With the debugging/output that I think is now available (versus previous versions), I was able to modify all the scripts that were getting an error, not being able to write to the embedded file system, ie: read-only.  Now they're all writing to my SATA drive and the IPblocklist package has been running for 72 hours without a crash.  This is a lot better than before where the service would crash, or it would act silly, or whatever.

The files that needed to be written were files like: ipfw.ipfw, whitelist, and wlists.txt  (i think thats all of them).  (again once IPblocklist could write somewhere, it worked fine… maybe for embedded systems you could write it all to tmp?  And then just have it reload them to tmp if the system gets reset?, not sure.), or where in the scripts it needs to write these files, invoke the /etc/rc.conf_mount_rw and then back /etc/rc.conf_mount_ro when done?  (just a thought)

I am using a custom list that I created which basically just blocks 98% of all countries that I load from an internal webserver.

Maybe by now this whole thread includes this kind of information, or you probably already know.

I just wanted to say thanks.   :)

[tommyboy180]

@wm408:

Tommyboy,

I have yet to upgrade to 2.0.x, as I will probably wait for final release (however I did test it and it looks great).  This means I am still running 1.2.3.

I have recently worked with your latest, 0.3.2.  As we've talked before in this thread, I do use the nanobsd (embedded) version and I do have a SATA 160GB drive mounted separately of the OS in my lil Soekris box.  With the debugging/output that I think is now available (versus previous versions), I was able to modify all the scripts that were getting an error, not being able to write to the embedded file system, ie: read-only.  Now they're all writing to my SATA drive and the IPblocklist package has been running for 72 hours without a crash.  This is a lot better than before where the service would crash, or it would act silly, or whatever.

The files that needed to be written were files like: ipfw.ipfw, whitelist, and wlists.txt  (i think thats all of them).  (again once IPblocklist could write somewhere, it worked fine… maybe for embedded systems you could write it all to tmp?  And then just have it reload them to tmp if the system gets reset?, not sure.), or where in the scripts it needs to write these files, invoke the /etc/rc.conf_mount_rw and then back /etc/rc.conf_mount_ro when done?  (just a thought)

I am using a custom list that I created which basically just blocks 98% of all countries that I load from an internal webserver.

Maybe by now this whole thread includes this kind of information, or you probably already know.

I just wanted to say thanks.   :)

Excellent. Would you be interested in helping write code and take this package to the next level? I'm getting more busy every day with normal day-to-day things and my school work. I'm actually in an Oracle class right now away from home. If you want to help out and dedicate a couple hours a week to this project that would super awesome.

Have you considered using Countryblock to block just countries? You should find it more stable for embedded installs. There are a couple hosting companies that are using countryblock and they have found it to be quite stable.

When I am able to find the time I will release a new version that will detect if embedded and put the files that must be written to and read from in /tmp.

[wm408]

Tommyboy,

Thanks for the coding offer.  I just did a few, little hacks in your scripts in response to debugging output that was provided.  I am taking Linux admin and some programming classes, but I am not uber like you.  ;D

Let me know what you have in mind and maybe I can help, (pm ?)

As far as writing to tmp, that'd be a cool option I think.  Maybe we could rig a cron job or something to help the IPblocklist reload after a power cycle.

And maybe the country block would have been a better choice.  (doh).

[tommyboy180]

@wm408:

Tommyboy,

Thanks for the coding offer.  I just did a few, little hacks in your scripts in response to debugging output that was provided.  I am taking Linux admin and some programming classes, but I am not uber like you.   ;D

Let me know what you have in mind and maybe I can help, (pm ?)

As far as writing to tmp, that'd be a cool option I think.  Maybe we could rig a cron job or something to help the IPblocklist reload after a power cycle.

And maybe the country block would have been a better choice.  (doh).

The latest version of ipblocklist reloads after a filter reload. It's actually really nice because you don't have to setup email notifications or a cron job anymore. It's stable now.
I do set the file system to read/write if on embedded but I guess the files need the same permission so /tmp is an excellent place.

I will be able to get to this update within the next couple of weeks. Thank you so much and when you feel comfortable with PHP send me a PM. I don't want to leave the package DEV scene but it just takes me a little longer to get updates out than users want.

[wm408]

Ahhh OK I see.  After a filter reload sounds better.

Thanks!

@tommyboy180:

@wm408:

Tommyboy,

Thanks for the coding offer.  I just did a few, little hacks in your scripts in response to debugging output that was provided.  I am taking Linux admin and some programming classes, but I am not uber like you.   ;D

Let me know what you have in mind and maybe I can help, (pm ?)

As far as writing to tmp, that'd be a cool option I think.  Maybe we could rig a cron job or something to help the IPblocklist reload after a power cycle.

And maybe the country block would have been a better choice.  (doh).

The latest version of ipblocklist reloads after a filter reload. It's actually really nice because you don't have to setup email notifications or a cron job anymore. It's stable now.
I do set the file system to read/write if on embedded but I guess the files need the same permission so /tmp is an excellent place.

I will be able to get to this update within the next couple of weeks. Thank you so much and when you feel comfortable with PHP send me a PM. I don't want to leave the package DEV scene but it just takes me a little longer to get updates out than users want.

[tommyboy180]

New IP-Blocklist poll at http://forum.pfsense.org/index.php/topic,39649.0.html

Everyone please vote on the issue. This directly affects the next release of the package. Thank you.

[Woger]

Hi,
I am new to pfsense but not completely new to freebsd. I have pfsense 2.0RC3 latest svn running with ip-blocklist installed. IP-blocklist seems to work fine. The problem I have is in the list. When I install sh-drop.tgz it installs about 430 IP's. If I install bt_hijjacked.gz only I get about 370 IP's. When I install both, I get just 70 IP's It seems that the script crashes as soon as it sees double IP ranges. I did read earlier about this and that doubles ones are completely removed, but it seems that the script stops completely.

I also get these errors:

[2.0-RC3][admin@pfsense.localdomain]/usr/local/www/packages/ipblocklist(39): ./convert-execute.sh
1 table deleted.
1 table deleted.
Use of uninitialized value in split at /usr/local/www/packages/ipblocklist/convert.pl line 63, <infile>line 469.
Use of uninitialized value in split at /usr/local/www/packages/ipblocklist/convert.pl line 36, <infile>line 469.
Use of uninitialized value in split at /usr/local/www/packages/ipblocklist/convert.pl line 36, <infile>line 469.
Use of uninitialized value in split at /usr/local/www/packages/ipblocklist/convert.pl line 63, <infile>line 469.
Use of uninitialized value in split at /usr/local/www/packages/ipblocklist/convert.pl line 36, <infile>line 469.
Use of uninitialized value in split at /usr/local/www/packages/ipblocklist/convert.pl line 36, <infile>line 469.
Use of uninitialized value in split at /usr/local/www/packages/ipblocklist/convert.pl line 63, <infile>line 469.
Use of uninitialized value in split at /usr/local/www/packages/ipblocklist/convert.pl line 36, <infile>line 469.
Use of uninitialized value in split at /usr/local/www/packages/ipblocklist/convert.pl line 36, <infile>line 469.
Use of uninitialized value in split at /usr/local/www/packages/ipblocklist/convert.pl line 63, <infile>line 469.
Use of uninitialized value in split at /usr/local/www/packages/ipblocklist/convert.pl line 36, <infile>line 469.
Use of uninitialized value in split at /usr/local/www/packages/ipblocklist/convert.pl line 36, <infile>line 469.
rm: /usr/local/www/packages/ipblocklist/lists/ipfw.ipfwTEMP: No such file or directory
rm: Wlists/whitelistTEMP: No such file or directory
rm: /tmp/rules.debug.tmp: No such file or directory
rm: /tmp/rules.debug.tmp: No such file or directory
0
1
2
3
4

Besides this: I am planning to run a few web and mailservers behind this pfsense machine. Which lists do you recommend to use?

Thanks,

Woger</infile></infile></infile></infile></infile></infile></infile></infile></infile></infile></infile></infile>

[tommyboy180]

70 seems like a really low number. What normally happens is the IP range is converted into CIDR and duplicates are removed. The number 70 doesn't mean 70 IP addresses it means 70 Networks since it's all converted to CIDR.

It's possible that the reading is correct. One way to see is to look at the ipfw.ipfw file. You can then see the raw list and determine why that reading is so low.

The next release will be CIDR native. This will eliminate weird stuff like this. If you want vote on the discussion to help me get a better understanding of what my userbase needs are at http://forum.pfsense.org/index.php/topic,39649.0.html

As far as lists go I recommend the spam, drop, and the forum spam list. Also I highly recommend running countryblock and blocking the major spamming countries.

[Woger]

Well it's 137 to be exactly. :)
The last line in ipfw.ipfw is 95.64.9.0/24 which is the second last line in sh_drop. The last line in sh_drop is DROP:95.64.98.0-95.64.99.255 . Exactly the same range is in bt_hijacked somewhere in the middle.
A wc -l on all three files gives:

358 bt_hijacked
    139 ipfw.ipfw
    469 sh_drop
Looks like it drops most off the lines during installation of the list files.

[Woger]

OK,
Probably found the problem.
both lists use a lot off the same ranges which are removed. This is because you use uniq -u in your convert script. The -u removes all duplicates. If I remove the -u it leaves 1 behind and I get 570 networks instead of 137. Is there a reason why you use the -u ?

Greetings,
Woger