Taming the beasts… aka suricata blueprint
-
Everything (in life) is a work in progress.
-
However, and this is just my opinion, it seems the methods could use a little tidying up perhaps. Most of the ideas are obviously still valid and the fun lies in trying to get it to work regardless of the state of the path.
The folks in this thread are very helpful also, so there's that to keep you pumped :) jflsakfja has stated he intends on updating it eventually and BBcan177 is working on a package that will take out the script portion mentioned at the 2nd step of the guide.
I hope this isn't too jibber jabber :)
Not at all ;D
And you are right, it has become a long thread. I printed it some months ago, it was 65 pages. That takes 1 week of analysis and cross checking if you aren't as skilled (such as I am in these matters). JFL has said he will write a new tutorial, and, like you said, he will probably wait until BB is done with pfBlockerNG to incorporate parts of that package in the tuto too (I think). And he will add to the tuto (if I understood correctly) how to do mass maintenance of rules using the SID-config files Bill so kindly added to Snort.
JFL has one prominent problem: he is loaded with work and doesn't have much time ATM :-[
-
Yea, apologies to everyone for taking so long, but I've been busy with work as Mr. Jingles has stated. One can only do so much with one head, two arms and two legs. I'm not an octopus :P
-
Believe me, I understand busy. Thanks for the responses guys! Really. I'll just be patient and wait for the new items. At least I'm back to a little free time to even post on the forum.
Rick
-
With the upgrade to 2.2 and ubound being able to contact other DNS servers freely, a few FPs might have cropped up in your blocked hosts list.
Rules 2200075, 2240003 and 2102329 need to go.
-
Hi, will you update the blue prnt to set PfBlockerNG as list blocker?
Thank You
-
Of course, wasn't that why pfBlockerNG was created for in the first place? (to make lists easier to manage) ;)
The updated guide is coming. But if everybody could ease up on the "are we there yet?" questions I would appreciate it ;D
As soon as the new version is out, both topics (snort and suricata) will be updated to show where to find the new version, and a topic dedicated to discussing the new guide will be created.
-
Since I like transparency, just letting anyone know that I'm waiting for permission to go ahead and start public work on the guide. Some parts of the guide have been completed offline, waiting to be pushed when the time comes.
Here's the relevant topic. https://forum.pfsense.org/index.php?topic=88244. An email has been sent to the mentioned address as well.
Edit: brain-farting-typo
-
Thanks for the guide jflsakfja, it's obvious that you've put a lot of work into it and I look forward to seeing it completed. I have one suggestion though, when you publish the guide it might be better to use pictures (maybe pics showing the firewall rules on all the interfaces) or indenting, similar to what you did with configuring the "pfsense ports". For example, the "outgoing ports" rule creation gets a little lost in a paragraph format in my opinion.
"Head over to an interface's tab and set up a an allow rule. Source should be the interface's subnet. The destination should be any, and for the ports use the outgoing ports alias created above. Destination should be any. Otherwise identical to the webgui rule."
-
That's exactly what I'm planning to do eventually, hence github, hence github pages (a little less known feature of github) ;)
Something along the lines of: http://jflsakfja.github.io/test-page
It's gonna be good, I promise that :-)
-
@jflsakfja:
That's exactly what I'm planning to do eventually, hence github, hence github pages (a little less known feature of github) ;)
Something along the lines of: http://jflsakfja.github.io/test-page
It's gonna be good, I promise that :-)
I can't wait to see it, really. I'm learning a lot just from reading thru your examples. Forgive me if you've mentioned this before but do you have a date in mind for the full release?
- 7 days later
-
I took a little break and worked on something else to give my mind a rest. I have a newer box and am working on getting it up now :) I gotta say, it sure is nice to see PFBlockerNG in the packages list. I can't wait to get deep into the the suricata after, mwa ha ha!
-
Here's a good one for Suricata, no need for pfBlocker ;)
drop ip any any <> $HOME_NET any (msg:"GeoIP Country Block"; geoip:!US,CA,BE,CZ,FR,DE,UK,NL,DK,FI,IE,NO,CH,JP,AU,NZ,SE,IS; classtype:policy-violation; sid:7710002; rev:1;)
Feel free to add/remove countries as you wish…
F.
-
Actually that's the exact rule that the guide is recommending not to use, for a reason or two ;-)
Why spend CPU/RAM analyzing packets that you know you'll drop? Packets that by the time you've finished analyzing them, a small number will get through (suricata/snort doesn't work on the live traffic, but a copy of that traffic).
A rule like that will take most of the RAM suricata is using. If you need a 2nd interface, double it. 3rd triple it and so on. A pfsense rule though will not take that much RAM.
Blocking by countries is NOT as attractive as it sounds. Most hosting providers don't rent datacenter space/servers in the country their visitors are. Blocking the US for example (as you should, see NSA saga) will get rid of most of the "known internet". Admittedly not a bad thing to do, but.There is no date on the new guide. I need the pf guys to give me the OK to go ahead with the guide. It's their move now. I'm pretty much sitting around waiting for their answer.
-
Yea I understand it should be at the firewall level, not at the IDS one in //, but still try it… with some decent hardware; pfSense & Suricata geoIp rule, try to visit a banned country site and tell me how much data was passed before the drop/block kicked in...Also compare the memory footprint of this option vs pfblocker or an alias list...
Concerning the NSA, no need trying to fight it; they operate at a different layer...Imagine if they had to opperate at the "user" level...
Just intercept/inject bigger hardware...you will never see them, they will always catch all...
F.
-
Any news on progress with the guide?
-
Still waiting for the pfsense crew's answer.
-
@jflsakfja:
Still waiting for the pfsense crew's answer.
Looking forward to your guide, I hope they respond soon.
-
@jflsakfja:
Still waiting for the pfsense crew's answer.
Looking forward to your guide, I hope they respond soon.
Yep, I second that!
Rick
- 18 days later
-
And I will third it :)
I am in the process of installing a new pfsense firewall and v2.0 of the infamous guide would come just in handy :)
-
jflsakfja, I cannot thank you enough for this. Over the last week I read through this entire thread and I am going to have to go through and read at least the first few pages again before trying this for myself.
I am sure I am not alone in having set up Snort/Suricata piecemeal, tweaking based on the odd nugget of advice picked up here and there but always wondering "am I really doing this right?". I am looking forward to seeing the updated guide, thak you again for all your efforts.
-
I am in the process of installing a new pfsense firewall and v2.0 of the infamous guide would come just in handy :)
Agreed :D
I have a matching hardware spare so I've started a 2.2.1 build and am just going to hold tight until the guide comes out. I'm venturing into new territory with Suricata and would rather follow the knowledge. Until then, my 2.1.5 with Snort is running just fine.
Curious though, is there any "school of thought" as to order of loading Squid3, PfBlockerNG and Suricata?
Rick
- 11 days later
-
@jflsakfja any progress? I'm pretty sure you're probably done with the write up, but still waiting on the pfSense team to give the OK?
-
Negative on progress, since I still haven't got the OK. Patience is a virtue we all need ;)
-
@jflsakfja:
Patience is a virtue we all need ;)
I'll second that one too!
Rick
-
Hello,
On my home I use pfsense 2.1.5 and now I switched from Snort to Suricata, set it as recommended by jflsakfja instructions in this thread….
Looking at alert logs yesterday I found that China people try to probe/hack my home network ( probably they found that Lenovo tablet can't report home and want to see whats wrong... ) so I put them in an alias blocked and set them in firewall as permanent blocked traffic In ( WAN ) and Out ( LAN ) but I still get alert in Suricata from there IP.
And by the way I still have Pfblocker set up to block all incoming traffic from Asia, and other sources.
Isn't pfsense firewall blocking traffic before it arriving at Suricata ?
Thank you.
edit:
security by obscurity ( pictures removed ) -
@jflsakfja:
Patience is a virtue we all need ;)
After reading https://forum.pfsense.org/index.php?topic=88244 , I think we will need more then patience. I understand what you're doing/asking for.. I've seen many pfsense guides on the internet, and none of them have gotten in trouble.. They just put the standard trademark disclaimer.
I'll wait and see
-
Hello,
On my home I use pfsense 2.1.5 and now I switched from Snort to Suricata, set it as recommended by jflsakfja instructions in this thread….
Looking at alert logs yesterday I found that China people try to probe/hack my home network ( probably they found that Lenovo tablet can't report home and want to see whats wrong... ) so I put them in an alias blocked and set them in firewall as permanent blocked traffic In ( WAN ) and Out ( LAN ) but I still get alert in Suricata from there IP.
And by the way I still have Pfblocker set up to block all incoming traffic from Asia, and other sources.
Isn't pfsense firewall blocking traffic before it arriving at Suricata ?
Thank you.
edit:
security by obscurity ( pictures removed )Short answer is nope :).
Long answer is: Think of the way packets are processed as a realtime copy of the packet stream. pf processes the actual packets, while at the same time copying them and sending them to suricata (actually logging the stream as is, and suricata sniffing that copy, but that's contrary to my stupidly simple explanations, so ignore it).Here's what happens and why you still get alerts:
A packet arrives from IP 1.1.1.1. That packet is copied and processed, sent on its way to your computer. At the exact time (relative, play along) that the packet is copied, suricata picks it up and starts processing the copy. The actual packet has long reached your computer, suricata is munching on a copy. After it decides that IP 1.1.1.1 is bad, it adds it to the blocked hosts. The next time a packet from that IP arrives, both pf and suricata will pick it up (original+copy). While pf is still deciding what to do with the packet (it will drop it, since the IP is known bad), suricata will still process a copy of it, and generate an alert.The downside: wasted processing. I tried working around with it with suricata's BPF (an exercise for the extremely stubborn out there) but could not, not without significant performance penalties. Basically BPF says "ignore packets from these IPs". There is no reason to analyze a packet that's known to be blocked anyway.
The upside: each time the bad IP generates an alert, the alert timestamp is updated. If it keeps doing it while being inside the ban timeframe, then the IP is perpetually kept on blocked hosts (well, until something like a reboot kicks it (temporarily) out).
All in all, don't worry if you are still seeing alerts from blocked IPs. As long as your rules are set up correctly (easy to test) then you are set :)
-
On my home I use pfsense 2.1.5 and now I switched from Snort to Suricata, set it as recommended by jflsakfja instructions in this thread….
Unless you are writing your own rules or you are an ISP dealing with 30k users, right now you are better with Snort than Suricata. Using ET open and Snort VRT rulesets, youll get more coverage of threat protection with Snort (All VRT rules compatible with the engine)
I personnaly prefer Suricata for a few reasons. IP protocol detection is better tuned…it will actually alert on some IPv6, Hop-by-Hop, etc... And when writing rules and outputting to syslog, I get more info on why the rules didnt load... Plus, the loggin is more detailed with Suricata.
But the Snort OpenAppID is promising and give you a good overlook of apps on your network/ports -do you know whats on your port TCP 443 ?- And the IP reputation with Snort is alot more user friendly than Suricata. Also, the host attribute table feature of Snort can be usefull depending on the size of your network and uniqueness of your users. Again, I find Snort better at detecting packet overlapping, while you could spend days fine tuning Surita Stream engine and end up just disabling the Stream rules ;)
30$/year VRT with Snort is a great deal of protection for the money. ET Pro is more expensive…which they had a home user pricing...
Then again, if you have time up your sleeve, try them both Suricata/Snort and find out for yourself their strengths. But honestly, on pfSense, for protecting a home or SOHO network w/o getting too much into technical stuff; go with the Snort package.
F.
- 22 days later
-
@jflsakfja:
…
The upside: each time the bad IP generates an alert, the alert timestamp is updated. If it keeps doing it while being inside the ban timeframe, then the IP is perpetually kept on blocked hosts (well, until something like a reboot kicks it (temporarily) out).All in all, don't worry if you are still seeing alerts from blocked IPs. As long as your rules are set up correctly (easy to test) then you are set :)
Hi to all and thanks for creating this thread !
I'm following your suggestion to set up a suricata ids,
Can someone explain how can I check the list of "blocked hosts" and their timers?Thanks,
Chris–-
I figured it out: services -> suricata -> blocks gives me all the info I need. -
Unless you are writing your own rules or you are an ISP dealing with 30k users, right now you are better with Snort than Suricata. Using ET open and Snort VRT rulesets, youll get more coverage of threat protection with Snort (All VRT rules compatible with the engine)
There was talk (here and elsewhere) many months back that Snort as a long term product was dead… or dying... in the wake of Suricata and its resources. Has there been a change to that thought? New people involved?
Just curious?
Rick
-
There was talk (here and elsewhere) many months back that Snort as a long term product was dead… or dying... in the wake of Suricata and its resources. Has there been a change to that thought? New people involved?
Just curious?
Rick
I think a couple of things have happened. First, thus far the Cisco purchase of Snort has not resulted in the open source project side being squashed. That was a fear early on after the purchase. Second, Snort 3.0-BETA supports multi-threading. So once v3.0 goes from BETA to RELEASE, the argument about Suricata's performance advantages will lose some steam.
I think both systems are fine. Each has its own unique features. Suricata can grab and log a lot more information than Snort can at the moment (all the JSON stuff, TLS cert exchanges, etc.), but Snort sports the new OpenAppID functionality.
Bill
-
New user here, bear with me.
Following the first post and reading it over and over, I don't understand the part about floating rules.
Here's what I did(also see screenshots)
- Created new interface called DMZ(did this to test on my current system)
- Created Floating Rule, as described, but ONLY for the interface DMZ
- Created allow rule for everything on the interface tab for DMZ(started out with DNS only, but nothing went through, so I changed it to any)
Testing with ping = failed
Testing NSLOOKUP = failedWhen disabling the floating rule, all traffic pass, as expected.
I'm sure it's me messing this up in some way, but I don't see how/why.
Assistance greatly appreciated.
BR Jim -
Can I see a screenshot of the floating rule in question?
If you are talking about the "block all" floating rule, it should only apply to traffic destined for pfsense's ports (that's why there is a giant red warning under it).
-
Thank you for the post - I've been walking through it and adjusting as necessary. I have a server at a colo accessed via a tunnel so some adjustment is necessary.
About that floating rule, the first one your mention where you write in large red "DON'T CHANGE DESTINATION PORT RANGE!!!". If I follow that example EXACTLY as you write it, rule #1 :P, then I end up blocking all outgoing traffic. Here is the float rule: (attached) Do you really intend to block ALL? I'm corn-fused?! Maybe I missed a step?
Thx.
-
Most common question I had to answer so far ;D
The rule you show will block all traffic.
The rule you want will block all traffic destined for pfsense's ports!. That's where the "don't change ports" part comes in.
Adjust that rule to destination pfsense's ports and it will be OK ;)
-
@jflsakfja:
Can I see a screenshot of the floating rule in question?
If you are talking about the "block all" floating rule, it should only apply to traffic destined for pfsense's ports (that's why there is a giant red warning under it).
Thanks for your reply. I guess the post above concerns the same confusion. Don't get me wrong, but you write the following:
Next up Floating tab:
Set up a rule but make these changes:
Action Block
Quick TICKED!!!
Interface Hold CTRL and click on all interfaces EXCEPT LAN(admin) and SYNC
Direction any
Source any
Destination anyIf you read this directly(as I did, since I'm absolute beginner), your rule will block everything in/out on all interfaces, except "LAN".
I did this, and got confused. I could not wrap my head around, how on earth a Floating block ANY ANY ANY to all interfaces would possibly allow any traffic to pass through.
My suggestion is to clarify(maybe more red big letters) that this floating block rule is ONLY for the ports you specify as being web interface and SSH(which makes good sense).
Thanks for your guide, I'm looking forward to following the next steps.
BR Jim
-
I agree, the text is a bit confusing, but it was meant to say "you created the rule, now head over to the floating tab and set up an identical rule to it, making these changes".
It's getting changed in the next version anyways (since I can't edit old posts) when I finally find the time to finish it. Caught up with work (a LOT of it) these days, and the guide is pushed back on my priorities list.
-
Sounds great, thanks for the clarification. Couldn't wrap my head around that weird floating rule :)
Looking forward to Version2 8)
-
Ok – reading through 28(!) pages is not my idea of fun. Is there a good summary for current (May, 2015) setups from scratch on 2.2.2 of PFSense, and using Suricata and any other helpful stuff for a colo'd LAN offering services to folks on the Internet?