Taming the beasts… aka suricata blueprint
-
@jflsakfja:
1. Legacy behavior where every alert results in a block unless the offender is in a Pass List. This is how things work today.
That would be a nice option.
@bmeeks:2. Only blocking when the PRIORITY field in the associated rule is less than or equal to a set value. For example, block on Priority 1 alerts but only alert for Priority 2 and 3 alerts. Still not sure how useful this option would really be, nor how reliable the Priority field values are.
Not at all useful, since I've seen priorities that really don't match the rules.
@bmeeks:3. A more conventional IPS setup where all rules are only alerts (with no blocks) unless the admin explicitly changes the rule action from "alert" to "drop". This could be handled fairly easily using the new SID MGMT tab feature. It would still require the admin to do a lot of work, though.
If you turn that around, I'd love to have the functionality. Instead of all rules being alert and you had to change them to drop, I would prefer all rules to be drop and you can change them to alert, if need be. Makes a lot more sense when the inline parts are finally available.
That's what I would like to see, alert/blocking wise. I didn't even gave it much thought, until I saw your post.
What I would like though (WANTED FEATURE ALERT!) is a way to turn off the category + priority from the syslog logs. It's driving me (and our loggers) insane. Every IDS/IPS/firewall log should be:
Source IP - Source Port - Destination IP - Destination Port - Protocol - Rule ID - Reason for blocking/dropping/rule explanation.Thank you for the comments. Option #1 is sort of a no-brainer since that is how things work today and no changes in any code would be required.
Your assessment of Option #2 reinforces what I suspected – the assignment of the Priority value in a rule is not necessarily perfect.
Your comment on Option #3 regarding turning the logic around (making all alerts DROP by default and allowing changing some to only ALERT) would require rewriting some of the underlying binary code in both Suricata and Snort. All of the internal routines are geared to act on the action keyword (either ALERT or DROP) in the rule. So without essentially rewriting the core mechanisms of both binaries, implementing as you suggest would not be possible. And for several very valid reasons, the pfSense team wants packages to stay in sync with upstream to the maximum amount possible. That means no or very limited patching for use on pfSense.
Turning off fields in the syslog output would also require upstream changes in the binary. Today the fields are hard-coded in that Suricata source code module.
Bill
-
I am attempting the red pill. However it is not going down all that well. ;) I am attempting the very first portion of the tutorial, blocking everything and setting up explicit "whitelist rules" I am using version 2.1.5.
- First setup your port aliases - check.
- Next setup your anti-lockout rule - check. (and it works… phew)
- Next setup a floating rule that blocks basically everything in/out on WAN - check.
- Next setup rule(s) that allow LAN services access to services (DNS, DHCP, NTP) - check. I get a DHCP address and I can resolve DNS.
- Finally, setup your explicit outbound ports rule to allow traffic out from the LAN - No Joy.
Here's what I suspect is happening - can someone help me figure this out?
Floating rules are parsed before rules on other interfaces. Thus, if a packet matches a floating rule and the Quick option is active (as we were told to do) on that rule, pfsense will not attempt to filter that packet against any rule on any other interface. Thus in step 5 the explicit outbound rules are not being hit because the floating "block everything in/out" has "quick" checked.
Does this make any sense? Has anyone else followed the first post in the thread and got working? I'm stumped.
Cheers,
Dan
-
I'm thinking that you too got confused by the floating rule. The floating "block everything" rule isn't meant to be a "block everything" rule. It's meant to be a rule to block everything to pfsense's ports!, hence the giant red warning. I'm working on version 2.0 of this guide, which will clear up some confusion (and possibly create confusion elsewhere :p).
-
I'm still confused. If you "DON'T CHANGE DESTINATION PORT RANGE!!!" on a new rule then you end up with "any" on the rule. It looks like the attached. And yes, it blocks everything. ;)
So instead I should use the alias for the pfSense ports? i.e. instead of "DON'T CHANGE DESTINATION PORT RANGE!!!" on a new rule it should say "Destination port range 'pfsense_ports' as above"?
a 2.0 guide would be much appreciated - this thread is a long hard slog. ;D
edit: I did find what I was looking for somewhere in the middle of this thread you give a very clear explanation:
Re-reading it does make sense on why it blocked out traffic. I meant to say create a new floating rule, based on the previous allow rule, but this time around change the pass to a block, keeping the destination ports the same.
A)1 normal pass rule for the ports active on the interface you want to administer pfsense from.
B)1 floating rule block rule for ALL interfaces EXCEPT the one you want to administer pfsense from.
-
What I would like to do is migrate both the Suricata and Snort packages to a new paradigm where you have alerts and blocks, but not a block from every alert like you have today. In a true IPS, you manually edit rules to change the "alert" action to "drop" only for those rules that detect really bad threats. For other things a simple alert to notify you is enough. However, in both Suricata and Snort today on pfSense, the output plugin that handles blocking simply keys off every alert. So any alert results in a block (unless the offending IP is in a Pass List).
I have been running some ideas around in my head on a method for presenting this new paradigm in the GUI. I am thinking about a series of behavior options the admin can choose from. Maybe something along these lines:
1. Legacy behavior where every alert results in a block unless the offender is in a Pass List. This is how things work today.
2. Only blocking when the PRIORITY field in the associated rule is less than or equal to a set value. For example, block on Priority 1 alerts but only alert for Priority 2 and 3 alerts. Still not sure how useful this option would really be, nor how reliable the Priority field values are.
3. A more conventional IPS setup where all rules are only alerts (with no blocks) unless the admin explicitly changes the rule action from "alert" to "drop". This could be handled fairly easily using the new SID MGMT tab feature. It would still require the admin to do a lot of work, though.
4. A refinement of #3 above where choosing one of the Snort VRT IPS policy settings results in blocks from all alerts originating from any of the IPS Policy rules, while other rules (non-IPS policy ones) simply alert. Of course the admin would still have the ability to manually edit any of the other rules to block (drop) by changing the action keyword from "alert" to "drop".
This would set both packages up to be ready to implement true IPS mode when the pfSense kernel is ready to support it. As both packages exist today where any alert results in a block, things could be a pain to administer if those blocks were true inline traffic drops.
I would like the community's input on the above. What changes would you like to see in the blocking behavior?
Thanks,
BillWell I'm alot like you on those feelings.
1,2,3 - Maybe the answer is in the "log" operator. Alert, Drop = block, Pass = Pass and … Log = Log; No drop, No block, just log.. There are many rules these days that I would want to be "log" only in pfsense. Also...all those complains about false positives, you can switch them manualy to "log" only. What I hate the most is when a rule is disabled because of too many false positives....well hell, let me decide whats FP...
"Pass" is sweet too, I have a set of rules that are true firewall rules with barely any payload inspection. "Pass" comes handy in this ruleset...
I personnaly replace all Alert with Drop and monitor the traffic, then, I change single rules to log only, one by one.
Keep up the good work Bill!
Cheers.
F.
-
What I would like to do is migrate both the Suricata and Snort packages to a new paradigm where you have alerts and blocks, but not a block from every alert like you have today. In a true IPS, you manually edit rules to change the "alert" action to "drop" only for those rules that detect really bad threats. For other things a simple alert to notify you is enough. However, in both Suricata and Snort today on pfSense, the output plugin that handles blocking simply keys off every alert. So any alert results in a block (unless the offending IP is in a Pass List).
I have been running some ideas around in my head on a method for presenting this new paradigm in the GUI. I am thinking about a series of behavior options the admin can choose from. Maybe something along these lines:
1. Legacy behavior where every alert results in a block unless the offender is in a Pass List. This is how things work today.
2. Only blocking when the PRIORITY field in the associated rule is less than or equal to a set value. For example, block on Priority 1 alerts but only alert for Priority 2 and 3 alerts. Still not sure how useful this option would really be, nor how reliable the Priority field values are.
3. A more conventional IPS setup where all rules are only alerts (with no blocks) unless the admin explicitly changes the rule action from "alert" to "drop". This could be handled fairly easily using the new SID MGMT tab feature. It would still require the admin to do a lot of work, though.
4. A refinement of #3 above where choosing one of the Snort VRT IPS policy settings results in blocks from all alerts originating from any of the IPS Policy rules, while other rules (non-IPS policy ones) simply alert. Of course the admin would still have the ability to manually edit any of the other rules to block (drop) by changing the action keyword from "alert" to "drop".
This would set both packages up to be ready to implement true IPS mode when the pfSense kernel is ready to support it. As both packages exist today where any alert results in a block, things could be a pain to administer if those blocks were true inline traffic drops.
I would like the community's input on the above. What changes would you like to see in the blocking behavior?
Thanks,
BillWell I'm alot like you on those feelings.
1,2,3 - Maybe the answer is in the "log" operator. Alert, Drop = block, Pass = Pass and … Log = Log; No drop, No block, just log.. There are many rules these days that I would want to be "log" only in pfsense. Also...all those complains about false positives, you can switch them manualy to "log" only. What I hate the most is when a rule is disabled because of too many false positives....well hell, let me decide whats FP...
"Pass" is sweet too, I have a set of rules that are true firewall rules with barely any payload inspection. "Pass" comes handy in this ruleset...
I personnaly replace all Alert with Drop and monitor the traffic, then, I change single rules to log only, one by one.
Keep up the good work Bill!
Cheers.
F.
Utilizing LOG and PASS in addition to ALERT and DROP are excellent suggestions. I now have more options to be thinking about …
Bill
-
Hi, I made the transition from snort to suricata, I followed your tutorial from the beginning, and I think that I have all working. But one thing is wired, on snord I had several alerts a day, some were identified as false positives, others I wasn't able to determine, so a keep them blocked until I identified the real source. On Suricata, I have it running now at 5 days, and not even one alert. Should I go back to snort in my production system until Suricata is in a more mature state?
-
Suricata is more than mature…
The problem is, some of Snort VRT rules arent compatible, out of the box, with suricata.
Majority of ET rules are. So the alerts you were seeing with Snort and not, right now, with suricata are most probably coming from Snort VRT ruleset.
Take those rules you were seeing, paste them in the custom box of your suricata interfaces and youll get a warning on why they arent compatible.
Bingo...just modify the rules now according to the warning and you all set.
You can even use the SID Mgmt tab to mod many rules at the same time.
Cheers.
F.
-
How can I test if suricata is working? I tried for example in the "files.rules" to activate all (for example jpg) and try to access jpg files on the web, and no alert where triggered, shouldn't suricata deny those files?
-
It depends if those rules are a GET or a POST JPG rules…if they are a GET, then it should trigger after 2-3 webpages max, if they are a POST, then dont worry.
Best advice, use ET ruleset, go visit the same webpages or generate the same traffic you are use to see when you were using Snort, make sure same rules are active and compare...
Every year I see some articles about plp doing some lab test comparing Snort and Suricata, they throw some malware and trojan in, see who catch what...and most of the time Suricata wins...
F.
-
Precision:
Snort is an IDS engine and a propriatary ruleset, also compatible with ET ruleset.
Suricara is an IDS engine and no propriatary ruleset, also compatible with ET ruleset, but not optimized….
So its important, when comparing, to understand if you are comparing an engine to detect a vulnerability or an engine to detect a vulnerability made on their syntax of a rule...
Suricata come naked, but it can perform alot better than Snort in some cases if you have a appropriate ruleset...
Hope I hvent confused the topic ;)
F.
-
@fsansfil Snort is an IDS. As soon as people started working on the IPS part, the IPS part blew apart. Development on snort-inline has ceased (the IPS) and the devs have moved on to suricata. This is what gave birth to suricata (well, that and the fact that snort wasn't scalable enough for NSA). I don't understand the "not optimized" part. Suricata has been seen analyzing 10Gbps connections (multiple, if anyone cares), something that snort can (and forever will) only dream about.
Snort VRT on the other hand are the rules that are designed for snort. Some of them work with suricata. Personally, I avoid them.
@soloam: This guide has been tested time and time again. If you set it up according to the guide, then suricata will alert correctly. Just to let everyone know, the transition was also tested. A cluster using snort was migrated to suricata by uninstalling snort, installing suricata and configuring it identically to this topic.
Did you restart after removing snort? After installing suricata?Is the suricata interface even running? Any logs showing some more info?What's the HOME net that suricata sees?
-
Hi jflsakfja, I don't doubt that the system works, I was just trying to see if I did something wrong, or if there is something wrong I uninstalled snort but I can't remember if I restarted after uninstalling and before installing suricata. So probably not, can it cause problems? I restarted after installing suricata.
In the home network I see the same entry's that I did in snort, my lan addresses and my wan address.
One thing. I didn't used the script to make the alias to the block ip's, I had everything set with pfblocker (from your previews tutorial) and I didn't mess with it. I only unistalled snort and installed suricata.
One question, any idea way Suricata doesn't have portscan alerts? I looked on the web and it's not supported. Any reason to way is that?
Thank You
Best Regards
Soloam -
Judging from the things you said, suricata should work. Are you sure there are no errors in the logs, and that the interface was indeed started? 9 out 10 times someone has a problem with snort/suricata is because the interface was either not started, or failed silently.
It doesn't support the portscan preprocessor, it doesn't mean it doesn't support alerts based on portscans. A rule designed to detect a portscan will still function, but a rule designed for X number of ports scanned on Y hosts will not (that's what the preprocessor does).
Doesn't matter if you used pfblocker or custom aliases, that's outside the suricata part.
-
Hello jflsakfja, thank you for your replay. Yes, all is ok, no errors on the logs and the interface is running. I don't know if something is wrong. I only find it weird that I don't have no alerts in 6 days, but then again, probably your rules blue print is that good and I don't get any false positives :P.
The only thing that I would like is to force some kind of attack or error that would trigger the alert in suricata just to be sure that everything is ok.
I will take advantage of this post to ask if you think that it's a good practice to have suricata on the lan ports? On snort I had it working, and a rarely get any alerts, but it helped me once to diagnose some malware in some computers. Snort started blocking all the connections to the WAN because it detected something suspicious.
Best Regards
Soloam -
Without knowing how closely you followed the guide (for example, did you set up unused ports rules?) I can't say how you can easily test it. If you left the ping rules enabled for example, a simple ping from outside the network to your WAN should show an alert.
I often get asked if it's a good idea to have snort/suricata enabled on LANs. I'll flat out say my answer: I don't think it's a good idea to set up identical setups on WAN and LANs and expect an increased sense of security. Some rules aren't designed to be used like that. snort/suricata on an internal interface (the way I set them up) only uses custom rules (IF the interface requires an IDS in place).
Some people are already taking aim with their stones at hand, so I have to be quick to explain that. 3 interface setup, extrapolate from there: WAN, LAN1, LAN2:
LAN1 is your admin interface.
LAN2 is your general "browse the net" type of interface.LAN1 should ONLY have access to the admin ports. Hosts on this interface shouldn't be able to access anything else. Ideally it's a single system that is on this interface. Easy to identify a compromised system, things should show up in the logs. 99.9999% of installations are NOT set up like this.
LAN2 clients should be able to browse the net, so connectivity to anything NOT LAN1 (get it? ;-)) should be allowed. Since these are client boxes, incoming connections to them shouldn't be allowed. The pfsense ports are already taken care of by the floating rule. Clients on this interface can only do these things:
1)Get compromised through a website/downloaded program
2)Compromise other systems on LAN2
3)Compromise foreign systems on WAN.in the case of 1) nothing you can do about that, other than keeping things up to date. WAN side snort/suricata will handle the alerting. If it's a 0-day exploit that's used against you, that means you pissed off the wrong persons ;-).
in the case of 2) No matter how you set up snort/suricata, the gateway NEVER SEES TRAFFIC INTERNAL TO THE NETWORK. Host1 wants to talk to Host2. Gateway1 won't see any of their direct packets. Without seeing their packets, snort/suricata will not alert. The only thing you can do about this, is have internal firewalls on each of the hosts (as you should) and keep them up to date.
in the case of 3) WAN side snort/suricata should catch the bad packets and generate an alert. At this point communication to the remote host has been cut off. The infected internal host, can't compromise what it can't see. This is where most people misunderstand things.
Ideally rules should be designed to fire up early in an "attack", not wait for the attack to be successful to generate an alert. Yes I do admit that it's difficult to do this, and not all rules follow it.
Since the remote host has been cut off, and you noticed the alert, you manually set up the packet capture on the internal interface to identify what host is doing the blah blah with the remote host (attempting to, but can't). You then proceed to unplug that host from the network and repair it.
Talking from a provider's POV, most things these days are using TLS/some other form of encryption. Unless you are actively MITMing the traffic that's passing through the gateway (if you do, you are a bad a person, shame on you), it doesn't actually matter if you have a WAN IDS, or a WAN and a million LAN IDSs. They still can't see inside the traffic.
My advice is don't focus too much on the network defenses and forget the other defenses. As mentioned in the guide, design your defenses in a layered form. Work from the inside out, not the outside in. A strong network defense, but a 13 year old webserver (yes I've seen in in real life by so called "industry experts") will not get you far down the line before someone compromises it. A weak network defense but an up to date and properly hardened server can withstand a whole lot of battering before showing any signs of weakness.
Again, don't forget that the same principles applied to servers can be applied to client machines as well. For example, raise hands everybody, who uses a separate browser profile to keep everything related to pfsense (cookies, cached files) separate from the rest of the browsing? You at the back? No? You in the front row? No?
Design security systems from the heart, or core, outwards. The strongest defenses should be at the center, even if they are never reached from the outside.
A recent (relative, more on this later) compromise of a customer's website has shown that internal security measures can be even more effective than strong perimeter ones. A few dozen backdoored files were uploaded to our servers, after going through a malware/antivirus scan (which missed the backdoor). The files were taken directly from the previous hosting company, downloaded, scanned, and uploaded (directly by me if anyone asks).
A week later, alerts started going off. The servers were attempting to send a very large volume of emails, coming from that domain. A quick look at them showed about 400 queued emails, sat there for the past 4 hours (an email server almost never queues emails more than a few minutes, let alone queue 400 emails from a domain that rarely sees 5 emails a day).
So how did the queued emails end up there? The server upon seeing an increased number of emails wanting to be sent, started slowly backing off on the sending rate for that domain. Suricata on the other hand saw that something was up (a few custom rules I use ;-)) and started banning remote servers.
The aftermath? A hidden spambot backdoor managed to send about 40 emails. Needless to say that the previous hosting company is still spamming (to this day) the hell out of everyone out there. When further analysis of the backdoor took place (and subsequent cleaning up of every infected file, by hand), the backdoor was found to be 2 different forms. One of them being early 2013 (yes almost 2 years ago) code. I don't care who is on either side. A 2 year old compromise (that is still undetected to this date) and a compromise completely fixed hours after it was used (NOT found out, used!), says a LOT.
Had this depended entirely on suricata with the default ruleset, I'm sure the servers would be at least on one (likely on all of them) of the 88 email blacklists that are out there :-). As I said, suricata isn't the tweak this knob, tweak that knob, OK, you now have the best security on the planet. Suricata combined with other security layers, yes that is the best you can get. Ideally suricata should be the first thing you install (well, after pfsense), but the last thing you finish setting up.
Back to my cave.
-
Great explanation jflsakfja.
I had a bad conclusion, my suricata is not working, I tried (temporary test) to allow ping to my wan port from a external address, and they passed with success, on suricata alert, still empty. =/
-
I guess I have my answer, looks it seems that suricata is not compatible with PPPOE :/
https://forum.pfsense.org/index.php?topic=73906.15
-
I guess I have my answer, looks it seems that suricata is not compatible with PPPOE :/
https://forum.pfsense.org/index.php?topic=73906.15
That's true for now. It is a limitation of the Suricata binary itself. FreeBSD presents a data link type for PPPoE that Suricata does not recognize at the moment. Snort will work, though if you want to try it instead. The DAQ module in Snort correctly handles the FreeBSD PPPoE data link type.
Bill
-
I used snort without any problem for some time, I changed to Suricata to test it out. Back to Snort for now.
Thank You
