PfBlockerNG

BBcan177

All my lists are set to update weekly.

I would expect most lists to change daily and any good blacklists to be updated on an hourly basis. I can however add an option in the future to edit the cron settings (min/hour/day etc)

pf3000

I'd be concerned if the hourly cron is actually pinging/contacting the block list servers every hour.

iBlocklist for example changed their free download policy to weekly. "…10 TERAbytes of data for us each month......free users will only be able to download updated lists once per week." At that time I was using old pfblocker and it had only a maximum of 24 hrs update interval for lists. Then I disabled the update option and created a custom weekly cron job.

phil.davis

@pf3000:

All my lists are set to update weekly.

The "NEXT Scheduled CRON Event will run at"…. it runs every 60 minutes. Is that normal?

Eg: Now it shows
NEXT Scheduled CRON Event will run at 17:00 in 1 Minutes
After few minutes it shows
NEXT Scheduled CRON Event will run at 18:00 in 59 Minutes

Yes, it is normal. The cron job is set to run every hour. It checks what aliases have what frequency of update and runs updates for the ones that are due.
In our case, if you only have Aliases that are set to update weekly, then the cron job will wake up 24*7=168 times a week, but 167 times it will quickly find it has nothing to do and exit.

pf3000

@phil.davis:

Yes, it is normal. The cron job is set to run every hour. It checks what aliases have what frequency of update and runs updates for the ones that are due.
In our case, if you only have Aliases that are set to update weekly, then the cron job will wake up 24*7=168 times a week, but 167 times it will quickly find it has nothing to do and exit.

Okay great; thanks for the clarification : )

BBcan177

@pf3000:

I'd be concerned if the hourly cron is actually pinging/contacting the block list servers every hour.

To add onto what Phil posted above, the first step is to compare the Alias Frequency setting to the current hour and if they are a match (ie needs to be updated) then the second step is to compare the previously downloaded files timestamp to the List providers URL. If the file remains unchanged, then the update for that list is skipped.

iBlocklist for example changed their free download policy to weekly.

Just so you know that the post you linked is from 2013. I have not seen any issues with IBlock restricting Downloads for the free service.

I would also recommend that there are several other alternative lists available. If you are updating once per week, you are missing a lot of current threat analysis that these Threat Source providers have. I did post a few other Sources in this thread. There are lots of others. Just need to do your homework. :) But if you are just using pfBlockerNG for IBlock lists, then you are doing yourself a big injustice in my opinion.

Bummer

Is anyone familiar with Block List at https://www.blocklist.de/en/index.html ? They look very interesting.

I run Fail2Ban on my servers. It's set with a hair trigger that after 2 failed login attempts via SSH, FTP, etc the IP is blocked for 2 hours. It then sends out an email for the IP and ownership to me. Block List gets their IPs from servers running Fail2Ban. Their lists are updated every 30 minutes and it's free. So if I get a hack attempt and I have Fail2Ban notifiy Block List, they will then update their lists in 30 minutes. Tf I have pfBlockerNG checking every hour, that IP is automatically blocked from my entire network.

At least the above sounds it could be too good to be true. So has anyone had any experience with them? Are there any holes in my theory?

BBcan177

Yes I use that list also but I do not participate in the IP sharing part.

I use Ossec (part of my Security Onion deployment) on my servers which is similar to what fail2ban is doing. If you can get fail2ban to output the IP to a local logfile, You can have pfBlockerNG pull that file on an hourly basis to block that IP for your whole network. I actually block the whole /24. My mail server can get a spam email or a brute force attack from several different IPs in the same /24 over a short period of time. I really hate spammers and pfBNG really has cut down on my Inbound Spam traffic. (like over 95% reduction)… ;D

doktornotor

@Bummer:

Is anyone familiar with Block List at https://www.blocklist.de/en/index.html ? They look very interesting.

Yeah it works pretty well… Using it for mailservers.

Bummer

Hey BBScan and doktornotor,

Thanks for the input.

I like that idea of subscribing to them because of the other IPs reported to them by Fail2Ban.

Can you point me to instructions on how to prevent the duplication of IPs from several different lists? I've seen the de-duplication box to check, but surely it can't be that easy. There must be more to that.

BTW, the new pfBlockerNG looks really great. It has a lot more bells and whistles I want to play with. It's just learning the terminology and playing with them. Much of this I've never messed wtih before.

BBcan177

@Bummer:

Can you point me to instructions on how to prevent the duplication of IPs from several different lists? I've seen the de-duplication box to check, but surely it can't be that easy. There must be more to that.

Yup.. One checkbox to enable de-duplication. Then hit "Force Reload"

BTW, the new pfBlockerNG looks really great. It has a lot more bells and whistles I want to play with. It's just learning the terminology and playing with them. Much of this I've never messed wtih before.

Thanks. It was a lot of work. I hope users try to use more of the advanced features (like Reputation) to help protect their networks…

Bummer

Hey BBScan177,

Okay, now for stupid question #2…

Yup.. One checkbox to enable de-duplication. Then hit "Force Reload"

If I have the lists set to update hourly, daily, or whatever, the de-duplication remains? If that's the case, it's simple simon, too good to be true!

Yes, I've seen the back and forth on reputation. I'm still not sure what it does, I can be a little slow on things like this. But once I get it down, them I'm on it!

stuck

@marcelloc:

Check if you still have aliases and rules from pfBlocker version.
If so, remove it.

Thanks marcelloc,
I did find a firewall rule that referenced a pfBlocker version and removed it. I don't see any aliases that were used by the old pfBlocker.
This still caused the same error.

BBcan177

It's strange that there are no aliases. (url tables) listed? Are you sure you checked it properly?

Try to run this command from the shell :

pfctl -sa (and at the end of the output, do you see any pfblocker tables?)

Also try to delete all files in

/var/db/aliastables (then reboot)

panz

I just upgraded from pfSense 2.1.5 to 2.2 and pfblocker disappeared from Firewall menu; however, all my blocklists are still there, because pfblocker was setup to use aliases only.

Now:

how do I install pfblockerNG? Is pfblockerNG going to recover my config or will I have to enter the settings again? I'm asking just to be sure I'm not going to make a mess…
18 block rules are present in my WAN interface Firewall setup (1 row for each IBlockList list I'm using) do I have to manually delete them before installing pfblockerNG?

BBcan177

You will need to re-configure, as it doesn't share the same settings.

I would recommend that you create aliases with more than one list. No real need to have a rule per list.

If you want to stay with "Alias" rules, you can modify the existing rules from pfblocker, and change the "alias table" setting and change the description to :

pfb_ then the name of the aliastable
(more details in the alias settings tab)

panz

I created an Alias for each list because, with one "big" list, pfblocker crashed with pfSense's message "PF was wedged/busy and has been reset".

So, I'm going to follow this procedure:

delete any previous firewall rule created by pfblocker;
delete any Alias also created by pfblocker;
enter the setting again in the new package pfblockerNG.

BBcan177

@panz:

I created an Alias for each list because, with one "big" list, pfblocker crashed with pfSense's message "PF was wedged/busy and has been reset".

Enable "De-duplication" in the General Tab. You will see the total size of the Block lists shrink substantially.

Bummer

Hey guys,

I set up a few lists from BlockList.de and have it set to run via cron. I'm getting the errors below. What am I doing wrong?

CRON PROCESS START [ 02/21/15 13:00:00 ]
Updates Found
Updates Found
UPDATE PROCESS START

[ pfB_Africa_v4 ] exists, Reloading File

[ pfB_Africa_v6 ] exists, Reloading File

[ pfB_Top_v4 ] exists, Reloading File
[ pfB_Top_v6 ] exists, Reloading File

[ White Listed IPs_custom ] exists, Reloading File

[ http://lists.blocklist.de/lists/ssh.txt ]
** TERMINATED - Header contains Blank/International/Special or Spaces

[ SSH-BlockList_custom ] Loading Custom File
–--------------------------------------------------------
Original Masterfile Outfile [ Post Duplication count ]
–--------------------------------------------------------
952 952 952 [ Passed ]
–--------------------------------------------------------

[ http://lists.blocklist.de/lists/ssh.txt ]
** TERMINATED - Header contains Blank/International/Special or Spaces
\ Email-BlockList_custom ] exists, Reloading File

===[ Aliastables / Rules ]================================

No Changes to Firewall Rules, Skipping Filter Reload
Updating: pfB_SSHBlockList
no changes.
UPDATE PROCESS ENDED

BBcan177

Make sure you set the "Header" in the IPv4 tab for each list. The log usually has all the details to diagnose issues.

** TERMINATED - Header contains Blank/International/Special or Spaces

Also - try to use. "https" when possible.

EDIT:

You can't use "-" in the Header field…

Bummer

Okay, BBScan177,

OKay, call me stupid. Okay, very stupid. How do I set the header for each list? When making the list is says to make a unique header. I saw the area and put the same thing in the "header" area as I did for the "List Description" and "List Name". So now that I know that's not correct, what do I do?

Sorry, I feel stupid.

Thanks!