Country Block

tommyboy180

Errors:
topspammers Tab -> /usr/local/pkg/pfblocker_topspammers.xml does not exists on line 175

I think top pfblocker_topspammers.xml was left out of pfblocker.xml

 <additional_files_needed>http://www.pfsense.org/packages/config/pf-blocker/pfblocker_topspammers.xml
<prefix>/usr/local/pkg/</prefix>
<chmod>0755</chmod></additional_files_needed>

Also do we want to limit the ipblocklist functionality to CIDR lists only? I took a poll on the package forums asking users if they wanted CIDR only or to have the normal PG2 lists. If we use CIDR only then we can eliminate future problems.

I was talking a look at your pfblocker_Range2CIDR function and it looks good but it's a long process for very large files like Level1 and Level2 lists.

marcelloc

Almost all free lists from ip-blocked are p2p, so I think we can suggest CIDR but keep p2p feature.

I will fix missing file and test rpc sync.

marcelloc

topspammers call fixed.

Can you test how many time does rules apply takes when you do not change urls?

tommyboy180

iblocklist.com lets your download all lists in p2p, dat, and CIDR now. I talked with fakhir not that long ago about the ip-blocklist package and moving to CIDR.

marcelloc

when I choose cidr format, all lists become 'subscription'.

tommyboy180

Copy the list URL and manually change the fileformat to "cidr". It works for all links.
If we can figure out how to get fetch to follow links then our users won't have to input the direct URL anymore.

marcelloc

But there are other compatible lists, and custom lists can be done too.

I understand that CIDR is better.

I've tested level1 file and both(p2p and CIDR) gets errors while appying to pfsense.

The block2CIDR works very fine, the extra time you see is downloading time. I did a lot of tests before including it to pfBlocker.

tommyboy180

I didn't think about that. You're right.
I'll see if I can figure out why large lists are failing.

tommyboy180

The large lists take up the default memory space. I was able to overcome this in IPblocklist version 3.5 by adding:

set limit table-entries 900000

I put this at the top of rules.debug. You can enter any amount that you think will be big enough but 900000 should cover all scenarios.
Hopefully there is a smooth way to add this option to the rules.debug file.

marcelloc

@tommyboy180:

The large lists take up the default memory space. I was able to overcome this in IPblocklist version 3.5 by adding:
set limit table-entries 900000
I put this at the top of rules.debug. You can enter any amount that you think will be big enough but 900000 should cover all scenarios.
Hopefully there is a smooth way to add this option to the rules.debug file.

I have no idea how to do this, but I will take a look on it.

~~For a stand alone use, all my tests went fine. tomorrow I will check sync problems~~.

All tests with pfBlocker, including sync between pfsenses boxes were sucessfull.

Did you liked the new options in gui? It was what you were thinking?

Cino

Great work!! I am having an issue tho: i'm getting this error with i click on the top spammers tab:

Warning: fopen(/usr/local/pkg/pfblocker_topspammers.xml): failed to open stream: No such file or directory in /etc/inc/xmlparse.inc on line 175

Warning: Invalid argument supplied for foreach() in /usr/local/www/pkg_edit.php on line 418

I did go thru and removed every entry of countryblock and pfblocker in my config and file system before doing installing the package again.. same results

running:
2.1-DEVELOPMENT (i386)
built on Tue Sep 13 17:28:43 EDT 2011
FreeBSD 8.1-RELEASE-p4

P.S my last git sync was this morning

marcelloc

Reinstall the package, there was a missing file in pfBlocker.xml

I've fixed this few hours ago. Btw i'll test this in a vm.

Cino

I did another re-install. Same results.. Let me know how your test turns out… Wondering if its because i'm running 2.1

marcelloc

@Cino:

I did another re-install. Same results.. Let me know how your test turns out… Wondering if its because i'm running 2.1

Does 2.1 uses same pkg repo as 2.0?

Cino

@marcelloc:

@Cino:

I did another re-install. Same results.. Let me know how your test turns out… Wondering if its because i'm running 2.1

Does 2.1 uses same pkg repo as 2.0?

as far as i know, yes

marcelloc

I found the error.

I'm fixing it right now.

Cino

@marcelloc:

I found the error.

I'm fixing it right now.

that did it.. Thank you!!! Only using the country blocking right now, over the weekend, i'll give the ip-list side a tried

marcelloc

pfBlocker 0.1.2 is out.

Now with widget.

tommyboy180

I we can't figure out how to increase the table size limits for large lists like level1 then perhaps we can break the lists into multiple tables.

Example:
table limit will be 10,000 entries. More entries will go into another table. So if Level1 has 35,000 lines the we will have 3 tables

pfblocker - 10000 lines
pfblocker1 - 10000 lines
pfblocker2 - 10000 lines
pfblocker3 - 5000 lines

What do you think? I hope we can find a way to increase the memory limit.

marcelloc

Could be a solution for this problem.

I'm still looking for a way to do it without multiple tables or file hacks.

Does it happens on x64 versions with for example 4g ram?

And thanks for widget clue. :)