PfBlockerNG

BBcan177 · Feb 19, 2015, 6:35 AM

The hour setting seems to just be the hour when the user wants all the schedules to "come together" - i.e. if you put "4" then at 4am all the 1,2,3,4,6,8,12 hourly schedules will go off together and then it rolls around hour by hour from there.

Thanks Phil… Yes you are correct... The Cron event will always be called each hour and the code will check to see if the user changed the Base Start Hour and adjust accordingly.

So pf3000, the Cron event will not show the Base Hour. It will always be "*"

I will revert that commit! :)

So for example :

Base Hour of ( 0 ) with a 4hr Freq. will download @ 0,4,8,12,16,20
Base Hour of ( 1 ) with a 4hr Freq. will download @ 1,5,9,13,17,21

I recommend that people change the Base Min and Base Hour settings so that the List providers are getting hit at various times to avoid a surge with everyone at the same Cron settings.

pf3000 · Feb 19, 2015, 6:45 AM

Hi BBcan177… ~~Great, it works now~~ okay
At the moment it's 1hr Frequency. When I was tinkering what I really wanted to achieve was - I would like for it to be 4 or X hours. Something like "*/12" (?)

phil.davis · Feb 19, 2015, 7:06 AM

Looking at this hour-frequency scheduling, I also noticed that nothing will happen at the zero hour. For example, if you put "4" as the CRON base hour, then the 2-hourly schedule list calculated in the code becomes:
"4","6","8","10","12","14","16","18","20","22","24","2"
but hours returned by PHP date() function are in the range 0 to 23.
So when it runs at like 00:15 it will not match hour "24" and so the expected 0 schedule will not run.
If you put CRON base hour "0" it is OK - the "0" gets put straight in as the first element in the list, it is only if you use a non-zero base hour that also wraps some schedules through the zero hour.

Proposed fix here: https://github.com/phil-davis/pfsense-packages/commit/c5b497d4ea370e8f076bd95af5259d547894f2fa

Review it, test it, and feel free to include it yourself in the next bugfix version.

st4t1c · Feb 19, 2015, 7:26 AM

Hello,

Thank you for the thorough answer, so my guess is that if an IP exists in two lists - one configured with "Deny Inbound" and one with "Deny Both", the "Deny Both" is the one that remains after deduplication?

@BBcan177:

@st4t1c:

Are the deduplicated IPs fall under the "Deny Inbound" or "Deny Both" rule since after the deduplication they're not "left" on the second list?

Hi st4t1c,

De-duplication works as follows ( using the tool grepcidr )

pfBNG will download any Country/Continent selections (No de-duplication as they are all unique already - However, de-dup will occur if you select a Country and then select it again in the TOP 20 Tab)

As each Alias/List is downloaded, it will compare each IPv4 Address to a masterfile. If the address exists or is already being blocked by a CIDR address, it will not be added. This will continue for each list downloaded.

When Cron runs, any list that need to be updated will have its IP addresses removed from the master database and a new de-duplication validation is done on all the new IPs in the recently downloaded file.

So an IP that might originally be listed in one List, might be listed in a different list after a Cron event.

I recommend that a "Force Reload" is run when users change Country Blocking, or add/remove Aliases/Lists. This will re-sync the whole Database and lists and make it more efficient.

If you want to have a list specific to a Firewall rule, you will need to use the "Alias Native" List Action, which does not use de-duplication or not enable de-duplication (Which I would not recommend)

fakemoth · Feb 19, 2015, 7:43 AM

Hello and thanks for working on this package! BBcan177 & wbennett77, can you please offer an estimation for the release of the final version? Or at least make a BIG post (an edit on the first page of the thread, something…) when you think it's production ready?

It's just me: I am just starting with this and for now I am angry with snort :) , so if I throw anything else it would surely make a mess on my setup.

GJ!

capitangiaco · Feb 19, 2015, 4:36 PM

I've upgraded to 2.2
I see pfblocker in the menu', but if I try to access it I see ERROR: No valid package defined.!
I've installed pfblockerng but it cannot start it cause it says:
The Package 'pfBlocker' is currently Enabled. Either Disable pfBlocker, or 'Disable Validation Check' in pfBlockerNG
(checkin the Disable Validation Check remove the error but pfblockerng doesn't insert rules in PF)
How can I disable the old pfblocker ?

thanks

Giacomo

BBcan177 · Feb 19, 2015, 4:54 PM

@capitangiaco:

How can I disable the old pfblocker ?

HI capitangiaco,

You can manually edit the config file /conf/config.xml

<config><enable_cb>on</enable_cb>
<enable_log><inbound_interface>wan</inbound_interface>
<inbound_deny_action>block</inbound_deny_action>
<outbound_interface>lan</outbound_interface>
<outbound_deny_action>reject</outbound_deny_action>
<credits><donation></donation></credits></enable_log></config>

and change the <enable_cb>on</enable_cb> to

<enable_cb></enable_cb>

I will be adding a conversion and a proper uninstall to correct this issue going forward. The previous pfBlocker version is not maintained by me.

Please make a backup before manually editing that file!

BBcan177 · Feb 19, 2015, 5:01 PM

@fakemoth:

can you please offer an estimation for the release of the final version?

Hi fakemoth,

I am the Developer of pfBlockerNG.

wbennett77 just posted the first post in this Thread which I Hijacked (with his permission of course :) )

The package is released. I set it as "Beta" until I was sure that it was stable. I hope to change that to "Stable" in the upcoming weeks. But there is no reason why you cannot install it in production. I have not seen any serious bugs so far and any bugs that are reported, have been promptly fixed.

BBcan177 · Feb 19, 2015, 5:05 PM

@st4t1c:

Thank you for the thorough answer, so my guess is that if an IP exists in two lists - one configured with "Deny Inbound" and one with "Deny Both", the "Deny Both" is the one that remains after deduplication?

Hi st4t1c,

De-duplication has nothing to do with the Firewall Rule processing order.

Basically, pfSense Floating Rules are processed first (top to bottom), then the Interfaces are processed (top to bottom)… So on a first match of the Firewall Rule criteria, the firewall will act on the settings in that particular rule.

If you want to have certain Rules to use a specific set of Alias/Lists, then you need to make "Alias Native" Rules to be able to fine-tune that functionality.

BBcan177 · Feb 19, 2015, 7:16 PM

@phil.davis:

Review it, test it, and feel free to include it yourself in the next bugfix version.

Thanks Phil for spotting that… I ran it thru some testing and it looks correct now. Will get that into a commit.

Here is a table to show the various Base Hour/Frequency Settings.


Base Hour [ 0 ] 
2hr  [0,2,4,6,8,10,12,14,16,18,20,22]
3hr  [0,3,6,9,12,15,18,21]
4hr  [0,4,8,12,16,20]
6hr  [0,6,12,18]
8hr  [0,8,16]
12hr [0,12]

Base Hour [ 1 ] 
2hr  [1,3,5,7,9,11,13,15,17,19,21,23]
3hr  [1,4,7,10,13,16,19,22]
4hr  [1,5,9,13,17,21]
6hr  [1,7,13,19]
8hr  [1,9,17]
12hr [1,13]

Base Hour [ 2 ] 
2hr  [2,4,6,8,10,12,14,16,18,20,22,0]
3hr  [2,5,8,11,14,17,20,23]
4hr  [2,6,10,14,18,22]
6hr  [2,8,14,20]
8hr  [2,10,18]
12hr [2,14]

Base Hour [ 3 ] 
2hr  [3,5,7,9,11,13,15,17,19,21,23,1]
3hr  [3,6,9,12,15,18,21,0]
4hr  [3,7,11,15,19,23]
6hr  [3,9,15,21]
8hr  [3,11,19]
12hr [3,15]

Base Hour [ 4 ] 
2hr  [4,6,8,10,12,14,16,18,20,22,0,2]
3hr  [4,7,10,13,16,19,22,1]
4hr  [4,8,12,16,20,0]
6hr  [4,10,16,22]
8hr  [4,12,20]
12hr [4,16]

Base Hour [ 5 ] 
2hr  [5,7,9,11,13,15,17,19,21,23,1,3]
3hr  [5,8,11,14,17,20,23,2]
4hr  [5,9,13,17,21,1]
6hr  [5,11,17,23]
8hr  [5,13,21]
12hr [5,17]

Base Hour [ 6 ] 
2hr  [6,8,10,12,14,16,18,20,22,0,2,4]
3hr  [6,9,12,15,18,21,0,3]
4hr  [6,10,14,18,22,2]
6hr  [6,12,18,0]
8hr  [6,14,22]
12hr [6,18]

Base Hour [ 7 ] 
2hr  [7,9,11,13,15,17,19,21,23,1,3,5]
3hr  [7,10,13,16,19,22,1,4]
4hr  [7,11,15,19,23,3]
6hr  [7,13,19,1]
8hr  [7,15,23]
12hr [7,19]

Base Hour [ 8 ] 
2hr  [8,10,12,14,16,18,20,22,0,2,4,6]
3hr  [8,11,14,17,20,23,2,5]
4hr  [8,12,16,20,0,4]
6hr  [8,14,20,2]
8hr  [8,16,0]
12hr [8,20]

Base Hour [ 9 ] 
2hr  [9,11,13,15,17,19,21,23,1,3,5,7]
3hr  [9,12,15,18,21,0,3,6]
4hr  [9,13,17,21,1,5]
6hr  [9,15,21,3]
8hr  [9,17,1]
12hr [9,21]

Base Hour [ 10 ] 
2hr  [10,12,14,16,18,20,22,0,2,4,6,8]
3hr  [10,13,16,19,22,1,4,7]
4hr  [10,14,18,22,2,6]
6hr  [10,16,22,4]
8hr  [10,18,2]
12hr [10,22]

Base Hour [ 11 ] 
2hr  [11,13,15,17,19,21,23,1,3,5,7,9]
3hr  [11,14,17,20,23,2,5,8]
4hr  [11,15,19,23,3,7]
6hr  [11,17,23,5]
8hr  [11,19,3]
12hr [11,23]

Base Hour [ 12 ] 
2hr  [12,14,16,18,20,22,0,2,4,6,8,10]
3hr  [12,15,18,21,0,3,6,9]
4hr  [12,16,20,0,4,8]
6hr  [12,18,0,6]
8hr  [12,20,4]
12hr [12,0]

Base Hour [ 13 ] 
2hr  [13,15,17,19,21,23,1,3,5,7,9,11]
3hr  [13,16,19,22,1,4,7,10]
4hr  [13,17,21,1,5,9]
6hr  [13,19,1,7]
8hr  [13,21,5]
12hr [13,1]

Base Hour [ 14 ] 
2hr  [14,16,18,20,22,0,2,4,6,8,10,12]
3hr  [14,17,20,23,2,5,8,11]
4hr  [14,18,22,2,6,10]
6hr  [14,20,2,8]
8hr  [14,22,6]
12hr [14,2]

Base Hour [ 15 ] 
2hr  [15,17,19,21,23,1,3,5,7,9,11,13]
3hr  [15,18,21,0,3,6,9,12]
4hr  [15,19,23,3,7,11]
6hr  [15,21,3,9]
8hr  [15,23,7]
12hr [15,3]

Base Hour [ 16 ] 
2hr  [16,18,20,22,0,2,4,6,8,10,12,14]
3hr  [16,19,22,1,4,7,10,13]
4hr  [16,20,0,4,8,12]
6hr  [16,22,4,10]
8hr  [16,0,8]
12hr [16,4]

Base Hour [ 17 ] 
2hr  [17,19,21,23,1,3,5,7,9,11,13,15]
3hr  [17,20,23,2,5,8,11,14]
4hr  [17,21,1,5,9,13]
6hr  [17,23,5,11]
8hr  [17,1,9]
12hr [17,5]

Base Hour [ 18 ] 
2hr  [18,20,22,0,2,4,6,8,10,12,14,16]
3hr  [18,21,0,3,6,9,12,15]
4hr  [18,22,2,6,10,14]
6hr  [18,0,6,12]
8hr  [18,2,10]
12hr [18,6]

Base Hour [ 19 ] 
2hr  [19,21,23,1,3,5,7,9,11,13,15,17]
3hr  [19,22,1,4,7,10,13,16]
4hr  [19,23,3,7,11,15]
6hr  [19,1,7,13]
8hr  [19,3,11]
12hr [19,7]

Base Hour [ 20 ] 
2hr  [20,22,0,2,4,6,8,10,12,14,16,18]
3hr  [20,23,2,5,8,11,14,17]
4hr  [20,0,4,8,12,16]
6hr  [20,2,8,14]
8hr  [20,4,12]
12hr [20,8]

Base Hour [ 21 ] 
2hr  [21,23,1,3,5,7,9,11,13,15,17,19]
3hr  [21,0,3,6,9,12,15,18]
4hr  [21,1,5,9,13,17]
6hr  [21,3,9,15]
8hr  [21,5,13]
12hr [21,9]

Base Hour [ 22 ] 
2hr  [22,0,2,4,6,8,10,12,14,16,18,20]
3hr  [22,1,4,7,10,13,16,19]
4hr  [22,2,6,10,14,18]
6hr  [22,4,10,16]
8hr  [22,6,14]
12hr [22,10]

Base Hour [ 23 ] 
2hr  [23,1,3,5,7,9,11,13,15,17,19,21]
3hr  [23,2,5,8,11,14,17,20]
4hr  [23,3,7,11,15,19]
6hr  [23,5,11,17]
8hr  [23,7,15]
12hr [23,11]

ntct · Feb 20, 2015, 1:06 AM

Hello

I use multiple subnet for interface, use deny inbound and Enable Suppression

For example ,LAN have 192.168.1/24 and 163.22.51.126/25 (use Virtual IPs for multiple subnet)

IN alerts tabs, i found most suppress button is not in source but in destination

Any idea?

Thanks~

BBcan177 · Feb 20, 2015, 2:05 AM

@ntct:

I use multiple subnet for interface, use deny inbound and Enable Suppression
IN alerts tabs, i found most suppress button is not in source but in destination

Hi ntct,
I have a fix, if you are able to test it, send me a PM and I will give some instructions.

ntct · Feb 20, 2015, 5:18 AM

I load new pfblockerng_alerts.php

It's corrected

Thank BBcan177!!

pf3000 · Feb 20, 2015, 11:16 AM

@BBcan177:

Thanks Phil for spotting that… I ran it thru some testing and it looks correct now. Will get that into a commit.

Here is a table to show the various Base Hour/Frequency Settings.
Base Hour [ 0 ] 
2hr  [0,2,4,6,8,10,12,14,16,18,20,22]
3hr  [0,3,6,9,12,15,18,21]
4hr  [0,4,8,12,16,20]
6hr  [0,6,12,18]
8hr  [0,8,16]
12hr [0,12]

How can I change Frequency? I can't find it anywhere.

phil.davis · Feb 20, 2015, 11:29 AM

When you add an Alias with a bunch of lists, you get to choose the Update Frequency. So for each Alias you can decide how often to check for updates.

pf3000 · Feb 20, 2015, 12:18 PM

I mean the frequency of the CRON Event found in the Alerts tab. It seems to run every 60 minutes. Should I be bothered about that?

BBcan177 · Feb 20, 2015, 12:37 PM

@pf3000:

I mean the frequency of the CRON Event found in the Alerts tab. It seems to run every 60 minutes. Should I be bothered about that?

The Alerts Tab is "hard-coded" to refresh every "60 seconds" if you click the "Auto-Refresh" checkbox. You shouldn't need to refresh more often. After a refresh, you should probably be analyzing the alerts :)

pf3000 · Feb 20, 2015, 1:05 PM

All my lists are set to update weekly.

The "NEXT Scheduled CRON Event will run at"…. it runs every 60 minutes. Is that normal?

Eg: Now it shows
NEXT Scheduled CRON Event will run at 17:00 in 1 Minutes
After few minutes it shows
NEXT Scheduled CRON Event will run at 18:00 in 59 Minutes

BBcan177 · Feb 20, 2015, 1:14 PM

@pf3000:

All my lists are set to update weekly.

I would expect most lists to change daily and any good blacklists to be updated on an hourly basis. I can however add an option in the future to edit the cron settings (min/hour/day etc)

pf3000 · Feb 20, 2015, 2:43 PM

I'd be concerned if the hourly cron is actually pinging/contacting the block list servers every hour.

iBlocklist for example changed their free download policy to weekly. "…10 TERAbytes of data for us each month......free users will only be able to download updated lists once per week." At that time I was using old pfblocker and it had only a maximum of 24 hrs update interval for lists. Then I disabled the update option and created a custom weekly cron job.