Taming the beasts… aka suricata blueprint
-
How can I test if suricata is working? I tried for example in the "files.rules" to activate all (for example jpg) and try to access jpg files on the web, and no alert where triggered, shouldn't suricata deny those files?
-
It depends if those rules are a GET or a POST JPG rules…if they are a GET, then it should trigger after 2-3 webpages max, if they are a POST, then dont worry.
Best advice, use ET ruleset, go visit the same webpages or generate the same traffic you are use to see when you were using Snort, make sure same rules are active and compare...
Every year I see some articles about plp doing some lab test comparing Snort and Suricata, they throw some malware and trojan in, see who catch what...and most of the time Suricata wins...
F.
-
Precision:
Snort is an IDS engine and a propriatary ruleset, also compatible with ET ruleset.
Suricara is an IDS engine and no propriatary ruleset, also compatible with ET ruleset, but not optimized….
So its important, when comparing, to understand if you are comparing an engine to detect a vulnerability or an engine to detect a vulnerability made on their syntax of a rule...
Suricata come naked, but it can perform alot better than Snort in some cases if you have a appropriate ruleset...
Hope I hvent confused the topic ;)
F.
-
@fsansfil Snort is an IDS. As soon as people started working on the IPS part, the IPS part blew apart. Development on snort-inline has ceased (the IPS) and the devs have moved on to suricata. This is what gave birth to suricata (well, that and the fact that snort wasn't scalable enough for NSA). I don't understand the "not optimized" part. Suricata has been seen analyzing 10Gbps connections (multiple, if anyone cares), something that snort can (and forever will) only dream about.
Snort VRT on the other hand are the rules that are designed for snort. Some of them work with suricata. Personally, I avoid them.
@soloam: This guide has been tested time and time again. If you set it up according to the guide, then suricata will alert correctly. Just to let everyone know, the transition was also tested. A cluster using snort was migrated to suricata by uninstalling snort, installing suricata and configuring it identically to this topic.
Did you restart after removing snort? After installing suricata?Is the suricata interface even running? Any logs showing some more info?What's the HOME net that suricata sees?
-
Hi jflsakfja, I don't doubt that the system works, I was just trying to see if I did something wrong, or if there is something wrong I uninstalled snort but I can't remember if I restarted after uninstalling and before installing suricata. So probably not, can it cause problems? I restarted after installing suricata.
In the home network I see the same entry's that I did in snort, my lan addresses and my wan address.
One thing. I didn't used the script to make the alias to the block ip's, I had everything set with pfblocker (from your previews tutorial) and I didn't mess with it. I only unistalled snort and installed suricata.
One question, any idea way Suricata doesn't have portscan alerts? I looked on the web and it's not supported. Any reason to way is that?
Thank You
Best Regards
Soloam -
Judging from the things you said, suricata should work. Are you sure there are no errors in the logs, and that the interface was indeed started? 9 out 10 times someone has a problem with snort/suricata is because the interface was either not started, or failed silently.
It doesn't support the portscan preprocessor, it doesn't mean it doesn't support alerts based on portscans. A rule designed to detect a portscan will still function, but a rule designed for X number of ports scanned on Y hosts will not (that's what the preprocessor does).
Doesn't matter if you used pfblocker or custom aliases, that's outside the suricata part.
-
Hello jflsakfja, thank you for your replay. Yes, all is ok, no errors on the logs and the interface is running. I don't know if something is wrong. I only find it weird that I don't have no alerts in 6 days, but then again, probably your rules blue print is that good and I don't get any false positives :P.
The only thing that I would like is to force some kind of attack or error that would trigger the alert in suricata just to be sure that everything is ok.
I will take advantage of this post to ask if you think that it's a good practice to have suricata on the lan ports? On snort I had it working, and a rarely get any alerts, but it helped me once to diagnose some malware in some computers. Snort started blocking all the connections to the WAN because it detected something suspicious.
Best Regards
Soloam -
Without knowing how closely you followed the guide (for example, did you set up unused ports rules?) I can't say how you can easily test it. If you left the ping rules enabled for example, a simple ping from outside the network to your WAN should show an alert.
I often get asked if it's a good idea to have snort/suricata enabled on LANs. I'll flat out say my answer: I don't think it's a good idea to set up identical setups on WAN and LANs and expect an increased sense of security. Some rules aren't designed to be used like that. snort/suricata on an internal interface (the way I set them up) only uses custom rules (IF the interface requires an IDS in place).
Some people are already taking aim with their stones at hand, so I have to be quick to explain that. 3 interface setup, extrapolate from there: WAN, LAN1, LAN2:
LAN1 is your admin interface.
LAN2 is your general "browse the net" type of interface.LAN1 should ONLY have access to the admin ports. Hosts on this interface shouldn't be able to access anything else. Ideally it's a single system that is on this interface. Easy to identify a compromised system, things should show up in the logs. 99.9999% of installations are NOT set up like this.
LAN2 clients should be able to browse the net, so connectivity to anything NOT LAN1 (get it? ;-)) should be allowed. Since these are client boxes, incoming connections to them shouldn't be allowed. The pfsense ports are already taken care of by the floating rule. Clients on this interface can only do these things:
1)Get compromised through a website/downloaded program
2)Compromise other systems on LAN2
3)Compromise foreign systems on WAN.in the case of 1) nothing you can do about that, other than keeping things up to date. WAN side snort/suricata will handle the alerting. If it's a 0-day exploit that's used against you, that means you pissed off the wrong persons ;-).
in the case of 2) No matter how you set up snort/suricata, the gateway NEVER SEES TRAFFIC INTERNAL TO THE NETWORK. Host1 wants to talk to Host2. Gateway1 won't see any of their direct packets. Without seeing their packets, snort/suricata will not alert. The only thing you can do about this, is have internal firewalls on each of the hosts (as you should) and keep them up to date.
in the case of 3) WAN side snort/suricata should catch the bad packets and generate an alert. At this point communication to the remote host has been cut off. The infected internal host, can't compromise what it can't see. This is where most people misunderstand things.
Ideally rules should be designed to fire up early in an "attack", not wait for the attack to be successful to generate an alert. Yes I do admit that it's difficult to do this, and not all rules follow it.
Since the remote host has been cut off, and you noticed the alert, you manually set up the packet capture on the internal interface to identify what host is doing the blah blah with the remote host (attempting to, but can't). You then proceed to unplug that host from the network and repair it.
Talking from a provider's POV, most things these days are using TLS/some other form of encryption. Unless you are actively MITMing the traffic that's passing through the gateway (if you do, you are a bad a person, shame on you), it doesn't actually matter if you have a WAN IDS, or a WAN and a million LAN IDSs. They still can't see inside the traffic.
My advice is don't focus too much on the network defenses and forget the other defenses. As mentioned in the guide, design your defenses in a layered form. Work from the inside out, not the outside in. A strong network defense, but a 13 year old webserver (yes I've seen in in real life by so called "industry experts") will not get you far down the line before someone compromises it. A weak network defense but an up to date and properly hardened server can withstand a whole lot of battering before showing any signs of weakness.
Again, don't forget that the same principles applied to servers can be applied to client machines as well. For example, raise hands everybody, who uses a separate browser profile to keep everything related to pfsense (cookies, cached files) separate from the rest of the browsing? You at the back? No? You in the front row? No?
Design security systems from the heart, or core, outwards. The strongest defenses should be at the center, even if they are never reached from the outside.
A recent (relative, more on this later) compromise of a customer's website has shown that internal security measures can be even more effective than strong perimeter ones. A few dozen backdoored files were uploaded to our servers, after going through a malware/antivirus scan (which missed the backdoor). The files were taken directly from the previous hosting company, downloaded, scanned, and uploaded (directly by me if anyone asks).
A week later, alerts started going off. The servers were attempting to send a very large volume of emails, coming from that domain. A quick look at them showed about 400 queued emails, sat there for the past 4 hours (an email server almost never queues emails more than a few minutes, let alone queue 400 emails from a domain that rarely sees 5 emails a day).
So how did the queued emails end up there? The server upon seeing an increased number of emails wanting to be sent, started slowly backing off on the sending rate for that domain. Suricata on the other hand saw that something was up (a few custom rules I use ;-)) and started banning remote servers.
The aftermath? A hidden spambot backdoor managed to send about 40 emails. Needless to say that the previous hosting company is still spamming (to this day) the hell out of everyone out there. When further analysis of the backdoor took place (and subsequent cleaning up of every infected file, by hand), the backdoor was found to be 2 different forms. One of them being early 2013 (yes almost 2 years ago) code. I don't care who is on either side. A 2 year old compromise (that is still undetected to this date) and a compromise completely fixed hours after it was used (NOT found out, used!), says a LOT.
Had this depended entirely on suricata with the default ruleset, I'm sure the servers would be at least on one (likely on all of them) of the 88 email blacklists that are out there :-). As I said, suricata isn't the tweak this knob, tweak that knob, OK, you now have the best security on the planet. Suricata combined with other security layers, yes that is the best you can get. Ideally suricata should be the first thing you install (well, after pfsense), but the last thing you finish setting up.
Back to my cave.
-
Great explanation jflsakfja.
I had a bad conclusion, my suricata is not working, I tried (temporary test) to allow ping to my wan port from a external address, and they passed with success, on suricata alert, still empty. =/
-
I guess I have my answer, looks it seems that suricata is not compatible with PPPOE :/
https://forum.pfsense.org/index.php?topic=73906.15
-
I guess I have my answer, looks it seems that suricata is not compatible with PPPOE :/
https://forum.pfsense.org/index.php?topic=73906.15
That's true for now. It is a limitation of the Suricata binary itself. FreeBSD presents a data link type for PPPoE that Suricata does not recognize at the moment. Snort will work, though if you want to try it instead. The DAQ module in Snort correctly handles the FreeBSD PPPoE data link type.
Bill
-
I used snort without any problem for some time, I changed to Suricata to test it out. Back to Snort for now.
Thank You
-
The only problem that I had with Snort, that lead me searching for a new IDS was the fact that Snort keep blocking my WAN ip, even that the ip is on the passing list (it's dynamic ip, its always changing, but the list is updated).
-
I just wanted to thank you all for the great write up! I really appreciate you taking the time to provide all of this information.
After following the guide I got everything working except for some of the PfIP Reputation widget output. Also, BBcan177 refers to version 2.3.4 of the pfiprep script but I am only able to find version 2.3.3. I feel I may be behind the times due to other updates that have taken place, but this was a great starting point for me.
I am running pfSense 2.1.5 and everything is going well. If there have been other changes due to software updates since this post began; would someone be able to point me in the right direction to learn about the changes/updates?
Thanks for all the great information!
Dan
-
A new guide is in the works, as well as a better "keeping the guide up-to-date" procedure is being worked on.
For now, this thread is a great starting point. Most updates in the new guide have to do with making it easier to set everything up, but the outline basically follows this guide.
I know I promised the new guide a while back. I haven't given up on, it's just that work has been keeping me a bit busy lately.
-
Thanks for the quick response. I definitely understand how wok can take up all your time!
Like I said, after following this guide everything seems to be running great except for some of the widget stuff to monitor the IP Lists. I need notice on the forums that BBcan177 is developing a pfBlockerNG package so I may look into that going forward. It would be nice to have that streamlined and able to backup.
Do you know of a way to backup the Suricata settings? I am not sure if they are getting backed up with the regular pfSense backup. I do not see them in the restore options. Also, anytime my box is power cycled I lose my Suricata block list. Is that the proper behaviour?
Your ET rule list on github is great! I have run into a few rules that trigger due to things I use day-to-day such as Cisco VPN phones and what not. Would you mind if I forked your list just so I can keep up with my findings?
I am also getting into writing some custom rules. I had to read your posts a few times but I think it has finally "clicked". Very exciting stuff!
Again…thanks for sharing with the community.
-
Thanks for the quick response. I definitely understand how wok can take up all your time!
Like I said, after following this guide everything seems to be running great except for some of the widget stuff to monitor the IP Lists. I need notice on the forums that BBcan177 is developing a pfBlockerNG package so I may look into that going forward. It would be nice to have that streamlined and able to backup.
Yes, BBcan177 is pushing towards a pfBlockerNG package. Should be very good when it's released. I'll leave it up to him for more info.
@dancwilliams:Do you know of a way to backup the Suricata settings? I am not sure if they are getting backed up with the regular pfSense backup. I do not see them in the restore options. Also, anytime my box is power cycled I lose my Suricata block list. Is that the proper behaviour?
Make sure you visit Suricata>Global settings> and tick the checkbox to keep the settings when the package is removed. That should take care of keeping settings between package upgrades/reinstallations if something goes wrong. As for transfering the complete settings over to a "new" system, bmeeks should be able to point out more details for it.
Yes, the blocked hosts get cleaned when you reboot. It's an annoying "feature" ;-)
Your ET rule list on github is great! I have run into a few rules that trigger due to things I use day-to-day such as Cisco VPN phones and what not. Would you mind if I forked your list just so I can keep up with my findings?
You are only permitted to fork the list if you contact ET and tell them the rules on the list have long been broken and should be removed/updated. Then you are free to do what you want with it. Except one thing, claim copyrights and sue me later :P
@dancwilliams:I am also getting into writing some custom rules. I had to read your posts a few times but I think it has finally "clicked". Very exciting stuff!
Again…thanks for sharing with the community.
Not a problem. I'll send an invoice later ;)
-
@jflsakfja:
Yes, the blocked hosts get cleaned when you reboot. It's an annoying "feature" ;-)
You really have to love those features. They are saving you from yourself you know….ha! :)
@jflsakfja:
You are only permitted to fork the list if you contact ET and tell them the rules on the list have long been broken and should be removed/updated. Then you are free to do what you want with it. Except one thing, claim copyrights and sue me later :P
I will definitely start hounding them. You are right…there are some rules in there that are just ridiculous! And no copyrights will be claimed...can't be too careful these days!
@jflsakfja:
Not a problem. I'll send an invoice later ;)
Of course! I just appreciate the chance to get to learn this stuff. ;)
-
Also, BBcan177 refers to version 2.3.4 of the pfiprep script but I am only able to find version 2.3.3.
Thanks Dan.. I moved my gist to the following URL.. Unfortunately I can't edit my original post.
https://gist.github.com/BBcan177/3cbd01b5b39bb3ce216a
If anyone else is using my pfIPrep script, please PM me and I will move you to the new pfBlockerNG package. As my script is a manual process, I can guide you on how to cleanly remove it.
-
So, I have fallen deep into this Suricata hole…so much you can do!
Is there a way to pass custom variables into Suricata through the GUI? I see under the interface "WAN Variables" there are some static definitions that can be adjusted, but I am curious about adding a few custom variables.
Thanks!
Dan
