RFC (make up a number not in use) - Blueprint for setting up snort + pfblocker
-
URGENT UPDATE:
GPLv2 community rules: also ignore this rule:
2417 PROTOCOL-FTP format string attemptDISABLED:36
Saw that rule start firing up yesterday. It's a rule from a vulnerability reported in 2007, for XM Easy FTP Server (tsk tsk the software people install these days….). I'm betting 30 years from now, the rule is still there and will be added on my lists. Any takers on that? Seeing the rule here and then go on deleting it is cheating ;)@lpallard:
After adding a few custom lists in pfblocker, and activating pfblocker, it created several rules in the WAN & LAN rule lists. See screenshots. There are 2 problems with my setup that I'd like to fix..
It shouldn't create any rules. That means you did not select alias only in the list.
@lpallard:1. I tried accessing some websites that are supposed to be blocked by pfblocker (China, Russia, etc) but I can still access them flawlessly from any machine on my LAN. Why is that?
Blocking by country ranges is something you should never do. Please see:http://forum.pfsense.org/index.php/topic,70453.0.html
@lpallard:2. pfblocker did not create any rules for my custom lists even if I selected "Alias Only" in the list options. How do I add my custom lists to the firewall rules and on which interface? LAN or WAN?
That's the expected behavior. When selecting alias only, pfblocker knows that it shouldn't create any rule, and all rule setup should be handled by you.
You need a block rule on WAN:
Action: Block
Disabled: NOT ticked
Interface: $wan_interface (select your interface)
TCP/IP Version: VERSION 4!!!! Remember, IPs in those lists are v4 IPs. Don't go creating another rule for v6.
Protocol: Any
Source: not NOT ticked (I hope that makes sence).
In the box under source start typing pfblocker. A tooltip should pop up with an entry in the form pfBlockerCustomAliasName. Select that
Destination; Any
Log: NOT ticked. (there are occasions when you do need to log attempted connections from banned sources, general public use is not one of them)
Description: A description to help you identify this rule.
Hit save.No go into your LAN (or DMZ,OPT1,SATELLITECONNECTIONTOTHEMOON1 etc..etc..) interface and repeat the above, this time with:
Action: Reject (I don't have to go into why you shouldn't wait for timeouts for the LAN side, you need your applications (browser, remote control exploit etc…etc....) to know they can't connect to a destination immediately. Hitting a bad website will get you an immediate couldn't open the website for example). Oh wait, I did go into.....)
Interface: $lan_interfaceI just looked at your screenshots. As far as the countries go, please remove them and stop using pfblocker for that purpose. The pfBlockerTopSpammers rule was created because you didn't select alias only on the list. Please select alias only, remove the rule (if it's not done already) and re-create it using my instructions above.
Why Blocking By Country IP Lists Is Always A Bad Idea.
Back in the early experimental ARPANET days, we pretty much knew Joe was node 1, Clark was node 2, and that's it pretty much.
A couple years later, universities started wanting to get on the (insert ZOMG!!!1111oneeleven look here) Internet Of Things. We reluctanly gave them access, and things went downhill from there on.
Fast forward a couple (or more) decades forward, and the IP pools started getting exhausted. We used to give out /8s back then. Now its a /32 (yea yea, a /32 cannot exist since its a single host etc..etc… look at it from the customer's perspective) IF you are lucky, or told to get access using NAT on an existing IP. The stupidly stubborn Industry Leaders refuse to learn how to use SNI (hosting multiple SSLs on a single IP) and make our work much easier...er... harder HARDER I meant ;). There are even occasions when we actually take IPs from a regional authority (there are 6 and 2 covert ones if I remember them right, too lazy to sideload google) and give them to another. Eg. Africa's one. It hardly used its allocated IPs, while the North American one is almost exhausted. A company acquiring a sizable IP allocation (no, your run of the mill hosting provider is not one of them, coughour creation Google, is cough. Must be getting cold) is free to have that allocation and move it's IPs from country to country. An IP belonging to a European country for example could end up in an Asian country. It takes a lot of effort to keep the lists updated, and is not really worth it. This was, is and will always be Rule Number 1 of why you shouldn't block by country ranges.
There are no such things as Chinese State Sponsored Hackers, or the likes. They don't have the training, and don't put in the effort required to be called a hacker. Hackers are people I have tremendous respect for. Any state sponsored script kiddies are not people I have respect for. Includes the American counterpart, the Pakistan(ese?) counterpart and the Russian counterpart. And pretty much every state sponsored script kiddies. Trust me, the Chinese are not the end all be all ZOMG!!!111oneeleven uber-hackers they are hyped up to be, and no they have no interest in (or actually any way of knowing) your secrets. Up to this point, the good guys are still ahead in the game ;) This is Rule Number 2.
Rule Number 3 is standard. I'm always right, unless I state otherwise.To recap:
Rules of why blocking by country ranges is bad:- Lists must be kept up to date for the functionality to be maintained.
- No such thing as state sponsored hackers. State sponsored script kiddies yes, but that doesn't align with our Keeping You Scared policies. Imagine the news anchor saying "They can't even program a simple "Hello World" program...dramatic pause but they can launch a downloaded exploit against you. We'll take a look into why your computer is not safe...." Doesn't give you that.....how to describe it.... "Chinese dramatic raise of voice hackers end dramatic raise are coming for your data. They have already accessed our country's most TOP SECRET data and are in the process of using sophisticated attacks against commercial entities to get our engineering technology secrets. We'll take a look into why no one is safe against the next generation of Chinese State Sponsored Hackers." See? that did it. Lost my train of thought....were was I?
oh yea,3) I'm always right, unless I state otherwise.
EDIT: Typos
EDIT2: Forgot the disabled rule count. -
@jflsakfja:
Saw you writing somewhere that you should reduce your lenghty texts. Please don't, I enjoy reading your curious thoughts very much ;D ;D And as they say, a laugh a day keeps the things away.
-
Hey, thanks for replying!
You really should condense everything you said, and make a tutorial out of it. You explain extrelemy well and it seems that you know what you are talking about, which are two good qualities to have at the same time ;)
That means you did not select alias only in the list.
FYI, I did. I however uninstalled pfblocker 2 or 3 times in the last few weeks and upgraded pfsense to latest release in between. It appeatrs to me as if my install was more or less stuck with the old rules created by the initial install…
If I remember chronologically:
I installed pfblocker
Added my custom lists
Selectrd Deny Both in ALL lists
Confirmed pfblocker had created rules based on my selections and lists
...
Uninstalled pfblocker
Upgraded pfsense
Reinstalled pfblocker
Re-created my lists from your initital post on this thread
Selected Alias only for ALL listsThen I discovered that it had re-created rules on the WAN & LAN IF while also creating aliases in the Aliases menu..
This is where I am as of now.
Question: Would it be safe to dlete all the rules (except the ones automatically created by pfsense) and re-create them based on your instructions?
Also I wanted to tell you: Something's wrong in the lists you are suggesting (at least from my geographical location) or maybe its the fact that I am blocking entire countries but immediately after I enabled pfblocker, my internet ceased immediately to function, I cannot even connect to Google.com from Canada.
That bothers me a lot to say the least. I thought (and really honestly thought) that the whole point of pfblocker was to allow you to block countries by IP ranges.... Apparently, its not the case.
Tonight, I will try to make this work.
-
OK as I said, I tried to make it work.
I did a lot of house cleaning… Removed pretty much every custom rules, made sure EVERYTHING in pfblocker was set to "Alias Only" (and you were right jflsakfja, some of them were still at Deny both).. My mistake, I guess trying to make this stuff work while being tired is useless...
Then pfblocker created the Aliases properly, and I added the custom firewall rules to WAN & LAN as you suggested (WAN with Block, LAN with reject).
And I applied the new settings. Waited for a few minutes (that pfsense box is slow like sh**) and then I picked a few random IP's in the custom pfblocker lists that are supposedly being blocked (or rejected) by my custom firewall rules. I tried pinging these IP's from my LAN, and I can successfully ping all of them.
Shouldn't I normally find that ping gets rejected by the firewall and issue something like 100% packet lost????
I dont think its working after all.
What have I done wrong?!
-
@lpallard:
OK as I said, I tried to make it work.
I did a lot of house cleaning… Removed pretty much every custom rules, made sure EVERYTHING in pfblocker was set to "Alias Only" (and you were right jflsakfja, some of them were still at Deny both).. My mistake, I guess trying to make this stuff work while being tired is useless...
Then pfblocker created the Aliases properly, and I added the custom firewall rules to WAN & LAN as you suggested (WAN with Block, LAN with reject).
And I applied the new settings. Waited for a few minutes (that pfsense box is slow like sh**) and then I picked a few random IP's in the custom pfblocker lists that are supposedly being blocked (or rejected) by my custom firewall rules. I tried pinging these IP's from my LAN, and I can successfully ping all of them.
Shouldn't I normally find that ping gets rejected by the firewall and issue something like 100% packet lost????
I dont think its working after all.
What have I done wrong?!
Your story is my story; I must have tried at least 100 times to get pfBlocker to work doing the same as you did; for me also it has never blocked any IP's at all, even though the tables are filled with IP's. There's something buggy somewhere I guess.
-
Hollander: It's just that sometimes I feel I'm going off in lengthy tangents and sometimes (most times actually) people miss my point.
lpallard: are you sure the alias selected in the rules you created starts with pfBlocker?
For example, I create the list BadPeers and put those lists in it, alias only for the list BadPeers. pfBlocker then creates that alias, but appends (prepends?) pfBlocker in front of it. So in pfsense's rules I have to select pfBlockerBadPeers as the alias, if I type BadPeers, then it will not function, since that alias doesn't exist.
Go into your firewall rules, and hover over the alias. WAN side, source should be your alias. Hover over it with the mouse (it would be akward to hover yourself over a screen :o) and see if a tooltip pops up with IPs in it.
A newly created blocking(or rejecting) rule could allow packets to flow through it, unless you clear the firewall states (Diagnostics>States>Reset) but I don't think this is the case since you selected random IPs to ping.If it allows traffic then the rule is not set up properly, pfblocker is currently updating the lists (so the table is empty, just to make sure go Diagnostics>Tables>select the pfblocker table (could take a while) and check that it is populated), source/destination/protocol do not match. Or a horrific bug exists somewhere, but I haven't found anything to support this.
I'll attach a couple of screenshots to help explanation.
-
Hi there! I have found several anomalies in my setup.
First of all, yes, the aliases are all properly created, and I also created the rule properly. As a matter of fact, I am also selecting the pfblocker**** alias in the rules as you said (see screenshots).
The story got uglier when I went to the Diagnositc > Table:
1. All tables are populated except the iblocklist custom pfblocker list. I dont know why. I went to the list settings in pfblocker, and tried removing a few list entries that I had suspicions about their validity… Then I hit Save but my router takes ages to return to the homepage, sometimes completely dies, sometimes, freezes, its a POS.. If you (or someone else feels like helping me pick up a new box, plz feel free to weight in on my thread (http://forum.pfsense.org/index.php/topic,70534.msg384951.html#msg384951)
2. There is a bug with my "alienvault" list. When I select it in the Diagnostics > Tables, the pages disappears and I end up on a totally blank page, kinda like if the pfsense web server died..
So in a nutshell:
-list alienvault doesnt display in Diagnostics > Tables
-iblocklist is empty
-iblocklist takes 10-20 minutes+ to save when modified, usually requires a hard firewall reset. Quite frankly, every time I modify tables, rules, aliases or whatever else that touches the firewall core, the entire box dies. Posting this reply took me over an hour for Heaven's Sake!!!! Sorry... I had to reboot (hard reset by unplugging the power cord) the box twice.I have seen a LOT of messages such as these in the "Filter Reload " page: Most of the time, they hang up indefinitely....
Processing early nat rules for package /usr/local/pkg/squid.inc...
Processing early nat rules for package /usr/local/pkg/tftp.inc...
Then the pfsense box stops responding from the web browser, the internet goes down, the phone goes down, the SSH session no longer works
...Do I have a more general firewall issue you figure??
Thanks!!
-
Blank pages if I remember correctly (this is a non-transferable license granted to the reader of this post, to correct me) is php running out of memory.
The aliases look ok, rules look ok, but the memory usage on that page is insane! I'm using 25% with all snort rules, those lists, on 2GB of RAM. Just saw squid on the services, never used it, can't comment on that. (this is also a non-transferable license granted to the reader of this post, to correct me)If the router dies at random times,freezes etc.. etc.. then something is definately wrong. I'm suspecting a hardware issue. Can you check Diagnostics>SMART status > Information/Tests select Attributes, and make sure that:
Reallocated sector count = 0
Current pending sector = 0
Offline Uncorrectable = 0 ?If that comes out positive (no other value than those) check your RAM with memtest86.
If that checks out ok, check PSU voltages (NOT responsible for anything etc…etc....)
If that checks out ok, visually inspect the motherboard for blown caps (capacitors,you'll know how they look when their top has a hole with metal bulging outwards).
An atom box is not that old, shouldn't have given up the spirit yet.
The lists taking a long time to populate means that the list is huge and is being downloaded, the download was OK but the box runs out of memory populating them, or the download failed.
Re-reading your reply, I'm now thinking it's more a out of memory problem than anything else. But you could perform the tests I showed above, just to make sure. Is that 80something% with the lists populated? -
is php running out of memory.
Make sense with so much RAM usage…
Snort is using about 80% of the RAM.... Before snort is started, my memory usage is around 10% (with Squid, SG, HAVP, etc) all running then I start Snort and it goes up to 85-88%...
SMART returns no errors, I had already checked this one often.
RAM could be faulty, never checked it. Out of 4 sticks, 2 were brand new at the time I purchased this Foxconn barebone machine, then I added another older stick, and finally another stick salvaged from an old machine... RAM could (and probably) is faulty
Lets talk about the PSU shall we ;)
About 6 months after I bought the machine, the PSU fan started to make faint grinding noises. At that time I thought it was out of alignment due to wear (since running 24/7) and because of the PSU's quality.. I decided to cut off the steel mesh protecting the PSU's fan thinking it was hitting it. Didnt't help at all... Fast forward 1 year later, the grinding noise is so intense it sounds almost like a real grinder.
Then one night I was woken up by a strong burning electronic smell... The fan had stopped turning completely and the PSU was probably in the 200degrees range (seriously I burnt my finger touching the PSU case..
No doubt the PSU is dying and I am borrowing time here...
Also, I need a rackmount enclosure... Quite frankly, I'd keep the CPU and Mobo and would only add more RAM, put it in a 1U enclosure and change the PSU and be done with it, but this Atom platform doesnt allow adding more RAM so is it really worthwhile to spend money to change the PSU and be stuck with this CPU/mobo?
Other than that, mobo seems OK. I also blame the Realtek NIC for some anomalies (cant restore WAN public IP after Power outage, Squid gets hung up on the WAN interface and requires rebooting the box)... You can search for my name on this forum, you'll see how much I had problems to make this thing run smoothly or reliably.
To a certain extent, I wonder how much the Realtek NIC and RAM are responsible for my misadventures!?
How can you use less than 2GB RAM with all the rules you suggested on this thread!?
Oh & I am forgeting! THis box didnt come with a dual NIC (surprise surprise!) and has only a PCI slot, so I added a cheap second hand PCI ethernet adapter (cant remember the brand/model). Maybe this compoennt is also defective..
I think the main thing to remember is that I built this box when I had no idea what I was doing and I was looking only at Watts (hence the choice for an Atom). Now with my better knowledge of pfsense and hardware reliability, if I had to restart fresh (which is what I am thinking to do), I'd start with making sure the box comes with 2 Intel NICs, supports more than 4GB RAM, and has at least 1 PCIE for future expansion, and finally has a reliable 24/7 rated PSU..
This box has none of these features.
-
When I try to load this list: http://doc.emergingthreats.net/pub/Main/RussianBusinessNetwork/RussianBusinessNetworkIPs.txt I get a php error from pfblocker.inc line 262. Increasing max table entries doesn't seem to help. Is there another setting I need to tweak?
Did anyone get this list to build? All the others are fine, this one just won't take. Ideas? fixes?
Thanks,
Rick -
When I try to load this list: http://doc.emergingthreats.net/pub/Main/RussianBusinessNetwork/RussianBusinessNetworkIPs.txt I get a php error from pfblocker.inc line 262. Increasing max table entries doesn't seem to help. Is there another setting I need to tweak?
Did anyone get this list to build? All the others are fine, this one just won't take. Ideas? fixes?
Thanks,
RickYou did select the txt Format and not the gz Format?
-
You did select the txt Format and not the gz Format?
Yep,
deleted it, rebuilt it. Didn't trust the <url>copy so I went to the url, saw the list, copied the known working url and still can't get this list to build.I have 7 others, all done the same way… some GZ, some TXT, this one just does not want to populate.
Rick</url>
-
lpallard: saw your other thread about hardware recommendations. Like I said on this thread, your problems sound like a hardware problem. I'll be replying to that thread for hardware, to keep this one somewhat on topic.Dunno when, since I'm really busy these days, but I promise I will.
As for the memory usage, I'm using right now…24% of 2026 MB, which comes out to...486.24. So, snort with all rules enabled CAN run on a machine with less than a gig of ram. Highest I've seen it rise was to 30% (don't forget the machine does all the other pfsense duties). Make sure you select AC-BNFA-NQ under snort interface settings> Detection performance > Search method. I could write a 500 page essay on why, but to Hollander's disappointment, I will not :D. What did we say about correcting me? (rhetorical question)
Ramosel: see if this helps: http://forum.pfsense.org/index.php?topic=42543.300. Can't remember if I had to do that on my systems, but it was definately not needed going 2.0 to 2.1. The list does download on mine (just re-checked, since all lists are in a single table, took 5 random IPs from that list and searched for them in the table, they were there.
-
@jflsakfja:
Ramosel: see if this helps: http://forum.pfsense.org/index.php?topic=42543.300. Can't remember if I had to do that on my systems, but it was definately not needed going 2.0 to 2.1. The list does download on mine (just re-checked, since all lists are in a single table, took 5 random IPs from that list and searched for them in the table, they were there.
Thanks, saw that earlier and left alone as I am running a 2.1 64bit pfsense. I've built all the pfblocker lists using the links your provided at the end of the first run. All built successfully except this RBN one. I don't understand it?
Sent you PM, don't know if you wanted to respond or not??
Rick
-
The bigger the problem, the bigger the hammer? :o
Replied to your PM.
-
As was privately requested, here's detailed instructions on how to set up an alias only list and use that in your rules.
Setting up the alias list
First you need to create the alias, and then populate it with IPs. This process is handled by pfblocker. Install the pfblocker package, go to Firewall>Pfblocker.
General tab:
Enable: ticked
Logging: ticked
Inbound interfaces: $your_wan_interface ctrl + click for multi selection
Inbound deny action: block
Outbound interfaces: $your_lan_dmz_interfaces, ctrl + click for multi selection
Outbound deny action: block
Don't actually know if this is needed since we will be creating an alias list, but doesn't hurt to try it.
Lists tab:
Press the + button
Alias name: your alias name CARE!!! the alias will be pfBlockerYourName
Description: blah blah
lists: add the lists I posted making sure you select the proper type (txt or gz). To add more, there's a + button below the box you enter the lists.
IMPORTANT! List action: Alias only
Update frequency: Once a day
scroll down, hit save, wait for the lists to be populated. A coffee break is recommended.Firewall rule with your alias list:
from a previous post (self-quoting will be the new rage in 2014):"You need a block rule on WAN:
Action: Block
Disabled: NOT ticked
Interface: $wan_interface (select your interface)
TCP/IP Version: VERSION 4!!!! Remember, IPs in those lists are v4 IPs. Don't go creating another rule for v6.
Protocol: Any
Source: not NOT ticked (I hope that makes sence).
In the box under source start typing pfblocker. A tooltip should pop up with an entry in the form pfBlockerCustomAliasName. Select that
Destination; Any
Log: NOT ticked. (there are occasions when you do need to log attempted connections from banned sources, general public use is not one of them)
Description: A description to help you identify this rule.
Hit save.No go into your LAN (or DMZ,OPT1,SATELLITECONNECTIONTOTHEMOON1 etc..etc..) interface and repeat the above, this time with:
Action: Reject (I don't have to go into why you shouldn't wait for timeouts for the LAN side, you need your applications (browser, remote control exploit etc…etc....) to know they can't connect to a destination immediately. Hitting a bad website will get you an immediate couldn't open the website for example). Oh wait, I did go into.....)
Interface: $lan_interface"The thing I missed in that post was under WAN source (or LAN destination), right bellow the NOT tickbox is a dropdown menu. MAKE SURE YOU SELECT "Single host or alias" THEN start typing pfblocker…. it should show a popup with all the pfblocker aliases, select the one you created above
I posted screenshots of the rules, and pfblocker lists a couple posts up, please refer to them.
-
@jflsakfja:
The bigger the problem, the bigger the hammer? :o
Replied to your PM.
I'm about to hammer this thing…
I've moved my Firewall Max Tables and Firewall Max Table Entries to 10,000,000 per the discussion
I've built separate lists in pfBlocker for all the recommended lists and they all populate... Except the Russian Business Networks.
I've torn down all the separate list and used the single alias model and threw multiple lists together... they all populate except the Russian Business Networks.What am I missing? I've looked at my logs and saw nothing (not saying I looked in the right place).
Rick
-
OK, no responses and no luck fixing this. Something with this list and my system just don't jibe and I'm not the only one. Doing a search here, at least 2 other postings from folks having trouble with this specific list.
Did some research and it looks like this list is almost 2 years old anyway… (Feb 10, 2012)
http://doc.emergingthreats.net/bin/view/Main/RussianBusinessNetworkSo that being the case is is best to abandon this list and just use the "emerging-rbn.rules" from within Snort?
Rick
-
Abandoning the list is not better, since you want to block all packets coming from those sources, not let through until snort decides that it should block.
Have you tried posting in the pfblocker thread? Maybe the guys over there can help.Technically RBN was dismantled, but the "bad guys" are still on a large number of those IPs.
-
Just as an update; it appears the lists were updated last night. Blocking cloudflare and perfectly legitime sites. I've been creating exceptions (dump them in an alias and allow that one before the block rule) for 1 hour :-[
