PfBlocker

taryezveb

Is this fix something that will eventually be added to pfBlocker? Or fixed in pfSense? Will this fix have to be reapplied if pfSense is updated (obviously yes for a new install)?

marcelloc, said yes for x64:
@marcelloc:

The ini_set("memory_limit", "250M") will be merged to next release and used when x64 instalation is detected.

When pfBlocker package is updated. All that should be needed is to reinstall pfBlocker package.

squirreldip

@squirreldip:

@marcelloc:

Did you saw posts about memory limit?

Thanks for the post - I'll give it a shot when I get home tonight.

Is this fix something that will eventually be added to pfBlocker? Or fixed in pfSense? Will this fix have to be reapplied if pfSense is updated (obviously yes for a new install)?

If not then maybe something should be added to the wiki speaking to list limits and possibly something pointing to a "use at your own risk" fix.

I made the changes from Page 21 and tested: I found an IP in the list that returned a ping. When using Level 3 list the blocker worked (would not return ping) but doing the same with the Level 1 list did not work (the ping returned).

I also increased my "Firewall Maximum Table Entries" from 1,000,000 to 100,000,000 and still no luck.

marcelloc

@squirreldip:

I found an IP in the list that returned a ping. When using Level 3 list the blocker worked (would not return ping) but doing the same with the Level 1 list did not work (the ping returned).

what action did you choose to level1 list and what ip did you tested?

marcelloc

@LinuxTracker:

SpamIPs_0-69 http://pastebin.com/MTds2fik
SpamIPs_70-179 http://pastebin.com/w0ZDtMym
SpamIPs_180-255 http://pastebin.com/QPi4PtMN
CorporateSpam http://pastebin.com/95xvHnk9

I've published your lists on my packages repo

http://e-sac.siteseguro.ws/pfBlocker/lists/

Thanks again LinuxTracker for your collaboration.

Note:

For those who will use these lists, config update to at least once a day as it's manually uploaded.

tommyboy180

@LinuxTracker:

I'd like to offer up a copy of the Spam List I've been compiling for over a year.
I began compiling it for use in IPBlocklist (used with Country IP Blocks). It's now optimized for pfBlocker.

Here's the details/disclaimer:

505 CIDRs as of today.

Based on spams received at the mail servers I care for.

It's focused on US Spammers but includes CIDRs from some countries I couldn't block outright. Non-US CIDRs are noted.
3a) A few countries I eventually gave up on and just country blocked outright (ie Poland, Peru); so there may be some inconsistency.

I converted it to CIDR format 2 weeks ago. That took a long time (orig PG ranges were 1-254).

For each spam IP, I carefully examined it's host to determine the appropriate range.
5a) Criteria includes bot spams, dynIP ranges and scummy hosting companies.
5b) For a single IP, it may take 15+ minutes of careful research before I can decide what range to block.

I generally do 2-4 update sessions every month.

Use At Your Own Risk. I'd review it first for possible editing, if I were you.

I've broken the list up into 3 because it became unwieldy.
I've recently broken off corporate spam (ie: Linkedin, Constant Contact, exacttarget) into a fourth list. I prob still need to shift some IPs into it.

I thought Pastebin (Private link - 1 Month expiration) would be the most transparent option for publishing this.

SpamIPs_0-69 http://pastebin.com/MTds2fik
SpamIPs_70-179 http://pastebin.com/w0ZDtMym
SpamIPs_180-255 http://pastebin.com/QPi4PtMN
CorporateSpam http://pastebin.com/95xvHnk9

MODS: If this violates forum protocol, please delete the post and forgive me.
If a mod wants the constantly updated live url (.gz format), please PM me.
The update URL is under my personal domain so I can't otherwise distribute it. Sorry.

Thanks.

edit: added screencap - 8 Hours of spam hits - domain w/ ~10 email accounts.

pfCustomSpamList = 3 SpamIP lists above.

Great SPAM list!

LinuxTracker

@tommyboy180:

Great SPAM list!

Thank you. I really owe you guys. I'm not in a position to pay back properly, but I'm glad to contribute something.

I put a lot of time into keeping the list up to date. It's rewarding to know a wider audience can make use of it.

I have this one email account I keep alive as a honeypot.
The former employee must have left their address everywhere, seems like every spammer in the US hits on it.

This is 12p-9p today on the same server.

After 5p the numbers ramped way up.
(I think 70%-80% were SMTP attempts, so I adjusted the rules tonight. My future numbers will all be SMTP.)

And again - Thanks.
Especially to you TB180, because I was well served by CountryBlock and IPBlocklist for 18 months.
But also to marcelloc, JimP and other devs who've spent scads of their own time developing an open Enterprise Class Security Suite; that I couldn't possibly afford.

LinuxTracker

First up:
I noticed a couple of interesting things happen at the top of the hour.

During the pfBlocker list update; my filters drop offline for a few seconds.
There's often a brief spike in spam activity at the top of the hour. Just from certain hard core spammers (ie: whoever is at 91.205.234.240-91.205.234.245)
That spike lasts from about a minute before the hour till a minute after.

The result is I'll have 2-8 spams get through.

Any changes to the lists in pfBlocker also drops my filters for 3sec-5sec.
It happens at the change, not when I navigate to General Tab and press save.

Notes:
The box is a P4 w/ HT, 1GB and an SATA drive. It runs pfBlocker, squid/squidGuard and Unbound.
It runs really well. RAM use averages 40% and CPU is mostly under 5%.

I did put in NTOP last night. It seemed to weigh heavy on the system, so I uninstalled it tonight.
I'll see if that makes the difference.

Secondly - about the Level1 Blocklist:
After pfBlocker converts it and loads into the table, it seems overly huge to me.
I've thought about streamlining the post-conversion CIDR ranges.
But doing that manually would be tedious. I also wouldn't know how to update it afterward.
I'm not sure anything can be done, but I thought I'd toss that out there.

And yes. I know my P4 is old and slow.
However:
To me, all code Must Be Efficient. It's never the hardware's fault; till it is.

squirreldip

@marcelloc:

@squirreldip:

I found an IP in the list that returned a ping. When using Level 3 list the blocker worked (would not return ping) but doing the same with the Level 1 list did not work (the ping returned).

what action did you choose to level1 list and what ip did you tested?

Thanks for your help!

I've backed up my VM and re-installed a fresh copy - modified according to your posts. The large lists now block.

Interesting is that trying to list the list the pfBlocker table still replies a blank page.

marcelloc

@squirreldip:

Interesting is that trying to list the list the pfBlocker table still replies a blank page.

The diagnostics -> table does not apply the 256 MB php memory limit. You may need to apply the DO AT YOUR OWN RISK patch there too.

marcelloc

@LinuxTracker:

During the pfBlocker list update; my filters drop offline for a few seconds.

There's often a brief spike in spam activity at the top of the hour. Just from certain hard core spammers (ie: whoever is at 91.205.234.240-91.205.234.245)
That spike lasts from about a minute before the hour till a minute after.

The result is I'll have 2-8 spams get through.

Any changes to the lists in pfBlocker also drops my filters for 3sec-5sec.
It happens at the change, not when I navigate to General Tab and press save.

Both are related to same thing.

To prevent max table memory limit errors, pflBlocker cleans it's tables before reapply, that's why you have some seconds without "protection"

If you choose to update list every hour, then every hour you will have some seconds off.

biogoon

I think I've found an issue with the password field in the XMLRPC Sync pane. The field seems to truncate some passwords as it does not escape at least a subset of symbols. As a result, configs don't get synced properly across systems.

Thanks again for such a great package!

LinuxTracker

@marcelloc:

To prevent max table memory limit errors, pflBlocker cleans it's tables before reapply, that's why you have some seconds without "protection"

If you choose to update list every hour, then every hour you will have some seconds off.

OK. That's helpful to know. I'll normally have the lists set for daily updates. It looks like that happens at 23:00.
I'll write a script to cycle the mail server service during that time.

Thanks.

marcelloc

@biogoon:

I think I've found an issue with the password field in the XMLRPC Sync pane. The field seems to truncate some passwords as it does not escape at least a subset of symbols. As a result, configs don't get synced properly across systems.

Thanks again for such a great package!

Sync problem you did was related to old package's version or password symbols?

biogoon

Password symbols in the newest version (1.0.1)

marcelloc

@LinuxTracker:

OK. That's helpful to know. I'll normally have the lists set for daily updates. It looks like that happens at 23:00.
I'll write a script to cycle the mail server service during that time.

Also take a look on other great packages to get even better antispam protection. ;D

Postfix forwarder and mailscanner.

LinuxTracker

@marcelloc:

Also take a look on other great packages to get even better antispam protection.

Postfix forwarder and mailscanner.

sigh I suppose I have to grow up and play with the big-boy toys sometime.

I have a ?
Is there a way to force pfBlocker to manually pull a list update? For the life of me, I can't figure out how to do it.
I keep having to wait for the scheduled updates to occur.

ccb056

This package seems not to work with multiple WAN interfaces. When more than 1 inbound interface is selected the firewall rules are only added to the 1 WAN interface.

ccb056

Oi marcello, I got the multiple WAN interfaces working by adding a dummy rule and then starting pfblocker.

One thing I notice is when lists are added they consume RAM but when the list is removed the RAM is not returned.

Obrigado!

marcelloc

If you have no rules on inbound interfaces, you are already blocking everything.

The memory usage is controled by freebsd, there is no code to keep lists on memory after apply config is finished.

tommyboy180

Memory consumed by the system will go inactive when not in use, like when you stop pfblocker or delete tables. That physical memory is still allocated by the OS for a period of time before it's released back into a shared pool.

That memory will return back to the OS after a little.