Taming the beasts… aka suricata blueprint
-
There is no such thing as a trusted network. As I said somewhere else, the only thing you should trust is the little voice inside your head. Everything/everyone out there is out to get you. And no, it's not paranoia when they ARE out to get you.
NSA aside, the use case is when trying to limit a local (internal) host from spreading something nasty on the internet. Other than that, there is no use for it, since it can still spread the "nasty" to other hosts on its subnet. When each host is on a different subnet, that's a different story, it effectively blocks it from accessing the entire network, but that is a highly specialized use case (ie not something you will encounter outside of an ISP).
Another use case that pops into my mind is when providing internet access to others (free wifi anyone?). You can't tell what their systems are infected with, so to limit your liability you could block it from accessing the internet.
As you can see from the guide, most weight is put into securing the network from outside threats, with some weight given on somewhat limiting outgoing connections. My recommendation for a normal home network is to use your trusted network example. Perfectly normal paranoia aside, it's very unlikely the NSA has broken into your house/computer-while-it-was-shipped and installed a backdoor. Incoming "nasties" are blocked. Since they are blocked, and you use some common sense (a shocking truth, but that hottie that keeps sending you naked images of her, is actually trying to get your email client to parse a command obfuscated in the image, ala firefox (yes a browser) jpg code execution), your network is (mostly) safe. Yes there are other attack vectors (eg browser memory mismanagement) but those are outside the scope of this guide :D.
-
Here is a report from The Solutionary Security Engineering Research Team (SERT) Quarterly Threat Intelligence Report for Q2 2014
http://www.solutionary.com/research/threat-reports/quarterly-threat-reports/sert-threat-intelligence-q2-2014/
Key findings include:
The top 10 ISPs represented the source of 52% of the malware identified in the new period.Amazon-hosted malware nearly triples in first half of 2014.
56% of malware captured was hosted in the United States, a 12% increase from Q4’13
Malicious SSH activity continues to be a popular method to gain administrative access to target systems.
-
Not going to register to download a report that I'm responsible of producing at my work :p
Let me guess, they propagate the Chinese 1337 hackers meme?
-
@jflsakfja:
Not going to register to download a report that I'm responsible of producing at my work :p
Let me guess, they propagate the Chinese 1337 hackers meme?Its great that you are cataloging these Malicious Attempts in your network. But its important to have a well balanced Source of Threats. Some Sources are good for Mail Servers, others Malware, others SSH attack etc. By combing threat sources from a multitude of Threat Sources, you can better determine their IP Reputation.
I typically don't like to have to register for Reports also. But looking at the Solutionary Website, there are other reports available online without needing to register.
http://www.solutionary.com/research/threat-reports/monthly-threat-reports/2014/06/security-threat-report-june-2014/
Looking at my current suggested List of Blocklist Sources (See data below),
grep "98.124.198.1" pf/*
pf/ET_IPrep.txt:98.124.198.1
pf/e_tcnc:98.124.198.1
pf/e_tdrop:98.124.198.1grep "85.159.211.119" pf/*
pf/ET_IPrep.txt:85.159.211.119
pf/e_tblackhole:85.159.211.119
pf/e_tcnc:85.159.211.119grep "85.25.148.6" pf/*
pf/IBlock_BT_Spy.txt:85.25.148.6
grep "217.12.207.151" pf/*
pf/IBlock_BT_Spy.txt:217.12.207.151
grep "192.99.6.61" pf/*
no results
grep "192.99.6.0" pf/*
no results
grep "^192.99." pf/*
pf/ToastedSpam.txt:192.99.0.0/16Game Over Zeus Botnet (goz)
http://garwarner.blogspot.ca/2014/07/new-gameover-zeus-variant-uses-fastflux.html?m=1Only ET IQRisk Blocklist picks up all of these IPs.
I have seen this spam hit my Network "E-ZPass" (I forwarded them to SpamCop also)
http://garwarner.blogspot.ca/2014/07/e-zpass-spam-leads-to-location-aware.html?m=1Only ET IQRisk Blocklist picks up all of these IPs.
-
Right, I finally have a little bit more time to work on this now :P
Could I ask one more question, JFL?
This is what I am struggling with:
1. You said in one thread (I think it was your previous thread about how to set up Snort) said to ditch Snort for Suricata;
2. Some Snort rules (some 600 I believe Bill said) Suricata can not interpret.
3. Your current instructions (this thread) only deal with the ET-rules, not with the Snort rules.
4. I take it ET doesn't completely overlap the Snort (paid subscription, which is what I have) rules, so there is/might be value in using both the Suricata- and Snort rules in Suricata.What would be wisdom here? Take your old instructions about the Snort rules to know what to disable, or do you consider these instructions obsolete perhaps? If so, ditch Snort all together? (but then point 4?) If not ditch Snort, how do you cope with the rules Suricata can't handle (the Snort rules)? There are 600 it seems, so [a1] how to find out which ones and [a2] after disabling these, aren't we missing out on important protection?
Or: have both Snort and Suricata inspect the interfaces at the same time (Snort with the full subscriber rule set, Suricata with the ET rule set)?
Yes, vague and complex for a stupid economist like me, I know :-[
( ;D )
Thank you again very much :P
-
@Hollander:
1)The only reason suricata is better than snort is that snort's inline has been abandoned and the developers moved to suricata, so if you want an IPS then you have to use suricata. We are talking about the general majority of users out there, there are other differences, but most users will never notice them. There is NO other software out there that matches its capabilities right now, pay,free,donated-to-us-by-aliens. It is actually the top of the line, absolute best IDS/IPS available. It bloody hell should be the best, it was funded directly by the U.S. government, I'm sure those guys want to get their money's worth.2)As far as the rules that suricata cannot interpret you only have these 2 choices:a)don't use them or b)manually rewrite the rule so that it is accepted by suricata.
3)This thread is about suricata, so there is no point in dealing with the snort rules, since those rules are not used. My opinion as to whether they are updated, or not, is my opinion only and does not reflect the opinion of The Company, etc… etc...
4)If you followed the guide to the letter and actually set up your suricata as suggested, you should be seeing about 90% banned hosts from custom rules (simple port scanning rules that detect port scans even the ET rules miss). Do you want to focus on the 1% that is banned hosts from the snort rules (assuming you are using snort with suricata, and filtered out the not working rules), the 9% from ET rules, or the 90% from custom rules? ;)
The trick to setting up an IDS/IPS is thinking like an attacker. What would you do to compromise an unknown system? These are the steps (most) attackers worth their salt follow (note: Does NOT refer to Chinese and Korean so called "state sponsored hackers", I don't have the time to deal with script kiddies):
a) Identify who the system belongs to. If it belongs to a letter soup agency, have the drill ready (you do have a "In the case of emergency, drill HERE" sticker that when followed shatters the HDD's platters right? Easy access to RAM modules (cold attack) is a bonus.
b) Identify used services. Usually a full portscan follows that identifies the used ports. This is the first trap we set up. Following this guide, ALL privileged ports are now under constant check. A portscan (even so called "sophisticated" portscans, using actual TCP handshakes (not trying to trick the system into responding)) WILL hit our trap if it's destined for privileged ports (probability it is:infinite).
c) In the case of true "state sponsored hackers" I personally expect them NOT to perform portscans. They should have been at one of my classes so far (if you are a state sponsored hacker and have missed my training classes, please contact your supervisor for a re-schedule) and should have known by now than when attacking, you NEVER alert the victim. And yes, I am talking to you United States Army Information Systems Control (now integrated into NETCOM). How many times have you seen a cat running around screaming "ARE THERE ANY MICE HERE??????" It crawls silently next to its victim, then attacks. The same strategy should be implemented here: Perform a legitimate connection to a used port, and see what information you can gather from it. If it spews out that the webserver used is a known buggy version, then find the exploit for it and attack it. It's not rocket science. This is were security by obscurity comes in handy. If you don't allow the attacker to understand the version of a software used, then the attacker is more likely to screw up by trying to use known large scale exploitable bugs (bugs that should be corrected by your updates long ago. You are updating without delays right?). Suricata will fire up an alert on old exploits, since I will not bother with the latest and greatest exploits. If my upstream distribution patches the flaw within 8 hours of it being public, and ET publishes a rule update at the end of the day, I'll take my upstream distribution over ET any time. What I'm trying to say here is, there is no point in trying to keep up with exploits using rules, you should instead keep up with exploits using server admin best practices. 99.999% of so called "industry leaders" have absolutely NO idea when the last time their servers were updated. I've seen SSL certificates being expired for a WHOLE YEAR being used by a so called "leading web hosting provider", and I believe my, and everybody's, conclusion is that if you did not see that your SSL certificate has been expired for the past year, there is at least a 1 year gap between the last time you checked and updated the server and now. 1 year of NOT patching a system is the difference between a compromise and not.
d) The attacker has successfully exploited a known flaw and is now on your system. This is were inward facing interfaces come in handy. If a webserver starts scanning your firewall for open ports, then this is a clear sign of a compromised system. If that same webserver starts sending out reflection attacks out of nowhere, it too is a clear sign of a compromise. Ideally in this case the IDS/IPS will block the server. Having a few angry customers is better if you handle your compromise transparently. An announcement somewhere along the lines of "We have detected signs of a system compromise and have to shut down a system pending further investigation" will go a long way instead of, I don't know, SITTING ON A COMPROMISE FOR 2 MONTHS? coughebaycough pardon my Vogons. Seeing your webserver ending up on your IDS/IPS's banned hosts list is your hint to pick up the phone, give me a call and say "hello, are you offering forensic analysis services? Yes, we have had a system compromise. Ok I'll see you on monday". You should never ask how much a forensic analysis of a compromised host costs. It's a case of a "if you have to ask how much it costs, you cannot afford it".Summary because I just previewed the post and my $deity I write long posts :p.
Use the IDS/IPS to detect and prevent network attacks. Use server best practices to STOP host attacks. If you are using snort/suricata for a home network, most of these points are invalid, since the only way in is:
1)browser bug
2)email client bug
3)irc/messenger/voip client bug
4)"any service that performs connections with a remote network" bugTrust me, if I can't see it, I can't shoot at it. Light bending around planets to look behind them (not the keep a google tab open type, if I can't type its name out of my head, I don't bother) does not apply in network/system security.
Ideally the bugs should be handled by your upstream (commercial company,beer drunk open source developers, depends on the case) swiftly and your exposure will be minimized. This is where common sense comes in. I will never delay updating a critical software to a newer version that fixes a bug because I don't like the newer version's interface. A little hint here: based on my professional experience, updating a system first, THEN checking what/if-something doesn't work is the best way to maintain a system. This also depends on the distro you are using. If you are using distro X because "ZOMGWTFBBL it uses the latest version of text editor nobody uses!!!ZOMGWTFLOLBEES" then ditch it and start using distros created before the other distro's creators were even born. In other words, if you are using a debian derivative, STOP as soon as possible (yesterday would be nice) and start using debian. Use the money you saved from unscheduled downtimes, things breaking right and left, and support financially debian. If I was a developer and was starving to death, I wouldn't sit down and code the bug fix for you. If you offered me a piece of pizza, then sure as hell I would sit down. I would even fix that other bug that is affecting you but isn't so serious. -
Only ET IQRisk Blocklist picks up all of these IPs.
Would be great to be able to get access to this at a Snort consumer licence or similar price. I asked a few months back and was told ET got rid of it due to abuse by enterprise customers abusing the offer.
One thing that would be a reassuring addition for the dashboard widget would be a date/timestamp when the rules were last updated.
I'm very impressed with Suricata and this threads guidance, thank you.
-
@irj972:
Only ET IQRisk Blocklist picks up all of these IPs.
Would be great to be able to get access to this at a Snort consumer licence or similar price. I asked a few months back and was told ET got rid of it due to abuse by enterprise customers abusing the offer.
Would be good if they change their minds. More People need to speak up and ask them for a better package.
One thing that would be a reassuring addition for the dashboard widget would be a date/timestamp when the rules were last updated.
This is already part of my latest widget.
https://forum.pfsense.org/index.php?topic=78062.msg432181#msg432181
-
@Hollander:
This is what I am struggling with:
1. You said in one thread (I think it was your previous thread about how to set up Snort) said to ditch Snort for Suricata;
2. Some Snort rules (some 600 I believe Bill said) Suricata can not interpret.
3. Your current instructions (this thread) only deal with the ET-rules, not with the Snort rules.
4. I take it ET doesn't completely overlap the Snort (paid subscription, which is what I have) rules, so there is/might be value in using both the Suricata- and Snort rules in Suricata.Suricata will not load the Snort Shared Object Rules. You can disable all of the Snort SO Categories for each Interface in the Categories Tab, or just leave it as is, and Suricata will just skip the Snort SO Categories. It will still load the remainder of the Snort Categories. You can refer to the previous Snort Recommended Disable List, or just disable as you see false positives.
What you don't want to do is run both Snort and Suricata in Blocking Mode at the same time which will cause issues. If you want to run both, only one can be in Blocking Mode.
-
@jflsakfja:
@Hollander:
1)The only reason suricata is better than snort is that snort's inline has been abandoned and the developersA joy to read once again, JFL ;D
Armed with this knowledge, and the reply from BB right above about which Snort rules to disable (thanks again BB :P ) I reserved two hours to start disabling and enabling. Unfortunately, it seems Bill still didn't have time to make the enabling all and then disabling some-routine slightly more comfortable. He announced he will do that, so I will have to wait for that. Because the current way of working will require me to weeks of from work :-X
(1. Enable all -> 2. start to disable the exceptions -> 2a. disable first exception, it will not be disabled but will have the bright red cross -> wait for pfSense to process and respond back -> 2b. click the same cross once again & wait for pfSense to respond after which it will have the desired light yellow cross. That is not doable in a few hours with all the many rules I will have to manually disable).
-
I didnt know Suricata offered inline detection and blocking in Pfsense?
For what I know it works more or less the same way as Snort does (not inline)
Enlighten me pls :)
@jflsakfja:
@Hollander:
1)The only reason suricata is better than snort is that snort's inline has been abandoned and the developers moved to suricata, so if you want an IPS then you have to use suricata. We are talking about the general majority of users out there, there are other differences, but most users will never notice them. There is NO other software out there that matches its capabilities right now, pay,free,donated-to-us-by-aliens. It is actually the top of the line, absolute best IDS/IPS available. It bloody hell should be the best, it was funded directly by the U.S. government, I'm sure those guys want to get their money's worth.2)As far as the rules that suricata cannot interpret you only have these 2 choices:a)don't use them or b)manually rewrite the rule so that it is accepted by suricata.
3)This thread is about suricata, so there is no point in dealing with the snort rules, since those rules are not used. My opinion as to whether they are updated, or not, is my opinion only and does not reflect the opinion of The Company, etc… etc...
4)If you followed the guide to the letter and actually set up your suricata as suggested, you should be seeing about 90% banned hosts from custom rules (simple port scanning rules that detect port scans even the ET rules miss). Do you want to focus on the 1% that is banned hosts from the snort rules (assuming you are using snort with suricata, and filtered out the not working rules), the 9% from ET rules, or the 90% from custom rules? ;)
The trick to setting up an IDS/IPS is thinking like an attacker. What would you do to compromise an unknown system? These are the steps (most) attackers worth their salt follow (note: Does NOT refer to Chinese and Korean so called "state sponsored hackers", I don't have the time to deal with script kiddies):
a) Identify who the system belongs to. If it belongs to a letter soup agency, have the drill ready (you do have a "In the case of emergency, drill HERE" sticker that when followed shatters the HDD's platters right? Easy access to RAM modules (cold attack) is a bonus.
b) Identify used services. Usually a full portscan follows that identifies the used ports. This is the first trap we set up. Following this guide, ALL privileged ports are now under constant check. A portscan (even so called "sophisticated" portscans, using actual TCP handshakes (not trying to trick the system into responding)) WILL hit our trap if it's destined for privileged ports (probability it is:infinite).
c) In the case of true "state sponsored hackers" I personally expect them NOT to perform portscans. They should have been at one of my classes so far (if you are a state sponsored hacker and have missed my training classes, please contact your supervisor for a re-schedule) and should have known by now than when attacking, you NEVER alert the victim. And yes, I am talking to you United States Army Information Systems Control (now integrated into NETCOM). How many times have you seen a cat running around screaming "ARE THERE ANY MICE HERE??????" It crawls silently next to its victim, then attacks. The same strategy should be implemented here: Perform a legitimate connection to a used port, and see what information you can gather from it. If it spews out that the webserver used is a known buggy version, then find the exploit for it and attack it. It's not rocket science. This is were security by obscurity comes in handy. If you don't allow the attacker to understand the version of a software used, then the attacker is more likely to screw up by trying to use known large scale exploitable bugs (bugs that should be corrected by your updates long ago. You are updating without delays right?). Suricata will fire up an alert on old exploits, since I will not bother with the latest and greatest exploits. If my upstream distribution patches the flaw within 8 hours of it being public, and ET publishes a rule update at the end of the day, I'll take my upstream distribution over ET any time. What I'm trying to say here is, there is no point in trying to keep up with exploits using rules, you should instead keep up with exploits using server admin best practices. 99.999% of so called "industry leaders" have absolutely NO idea when the last time their servers were updated. I've seen SSL certificates being expired for a WHOLE YEAR being used by a so called "leading web hosting provider", and I believe my, and everybody's, conclusion is that if you did not see that your SSL certificate has been expired for the past year, there is at least a 1 year gap between the last time you checked and updated the server and now. 1 year of NOT patching a system is the difference between a compromise and not.
d) The attacker has successfully exploited a known flaw and is now on your system. This is were inward facing interfaces come in handy. If a webserver starts scanning your firewall for open ports, then this is a clear sign of a compromised system. If that same webserver starts sending out reflection attacks out of nowhere, it too is a clear sign of a compromise. Ideally in this case the IDS/IPS will block the server. Having a few angry customers is better if you handle your compromise transparently. An announcement somewhere along the lines of "We have detected signs of a system compromise and have to shut down a system pending further investigation" will go a long way instead of, I don't know, SITTING ON A COMPROMISE FOR 2 MONTHS? coughebaycough pardon my Vogons. Seeing your webserver ending up on your IDS/IPS's banned hosts list is your hint to pick up the phone, give me a call and say "hello, are you offering forensic analysis services? Yes, we have had a system compromise. Ok I'll see you on monday". You should never ask how much a forensic analysis of a compromised host costs. It's a case of a "if you have to ask how much it costs, you cannot afford it".Summary because I just previewed the post and my $deity I write long posts :p.
Use the IDS/IPS to detect and prevent network attacks. Use server best practices to STOP host attacks. If you are using snort/suricata for a home network, most of these points are invalid, since the only way in is:
1)browser bug
2)email client bug
3)irc/messenger/voip client bug
4)"any service that performs connections with a remote network" bugTrust me, if I can't see it, I can't shoot at it. Light bending around planets to look behind them (not the keep a google tab open type, if I can't type its name out of my head, I don't bother) does not apply in network/system security.
Ideally the bugs should be handled by your upstream (commercial company,beer drunk open source developers, depends on the case) swiftly and your exposure will be minimized. This is where common sense comes in. I will never delay updating a critical software to a newer version that fixes a bug because I don't like the newer version's interface. A little hint here: based on my professional experience, updating a system first, THEN checking what/if-something doesn't work is the best way to maintain a system. This also depends on the distro you are using. If you are using distro X because "ZOMGWTFBBL it uses the latest version of text editor nobody uses!!!ZOMGWTFLOLBEES" then ditch it and start using distros created before the other distro's creators were even born. In other words, if you are using a debian derivative, STOP as soon as possible (yesterday would be nice) and start using debian. Use the money you saved from unscheduled downtimes, things breaking right and left, and support financially debian. If I was a developer and was starving to death, I wouldn't sit down and code the bug fix for you. If you offered me a piece of pizza, then sure as hell I would sit down. I would even fix that other bug that is affecting you but isn't so serious. -
I didnt know Suricata offered inline detection and blocking in Pfsense?
For what I know it works more or less the same way as Snort does (not inline)
Enlighten me pls :)
Why do I have to always write like I'm presenting a case to the court? Seems like even a miss-placed comma gets the author burned on the stake in these forums.
I have mentioned in the past that snort inline has been abandoned. Says so on their page if you don't believe me.
As it stands right now, the exact moment of this post, yes suricata and snort more or less run the same way on pfsense. BUT: The inline capability is there, it just cannot be used yet as pfsense needs some changes to it. They are coming soon(ish). Would you want to tear down your snort system when the changes are made, reconfigure and troubleshoot suricata, then enable it and forget it, or just log in and tick a checkbox and be done with it?
I don't see why we need to split the development effort between snort and suricata. Snort doesn't offer anything more than suricata, but offers plenty less than suricata. A couple years time down the road, the cisco announcement that snort will no longer be maintained as a community project should wake up everybody. And yes, the announcement WILL come. The same way the ebay announcement came when I was running around screaming "change your ebay passwords!!!" and everybody was looking at me puzzled.
It's unfortunate that an open source project died (snort) but it's survival of the fittest. Now if we could only convince the (about) 43298 different open source text editors to die and merge in a single project, we will be colonizing Alpha Centauri by this time next year, and permission is hereby granted to quote me on that.
-
It's unfortunate that an open source project died (snort) but it's survival of the fittest. Now if we could only convince the (about) 43298 different open source text editors to die and merge in a single project, we will be colonizing Alpha Centauri by this time next year, and permission is hereby granted to quote me on that.
Ha! Thats good stuff! ;D
-
@jflsakfja:
I didnt know Suricata offered inline detection and blocking in Pfsense?
For what I know it works more or less the same way as Snort does (not inline)
Enlighten me pls :)
Why do I have to always write like I'm presenting a case to the court? Seems like even a miss-placed comma gets the author burned on the stake in these forums.
I have mentioned in the past that snort inline has been abandoned. Says so on their page if you don't believe me.
As it stands right now, the exact moment of this post, yes suricata and snort more or less run the same way on pfsense. BUT: The inline capability is there, it just cannot be used yet as pfsense needs some changes to it. They are coming soon(ish). Would you want to tear down your snort system when the changes are made, reconfigure and troubleshoot suricata, then enable it and forget it, or just log in and tick a checkbox and be done with it?
I don't see why we need to split the development effort between snort and suricata. Snort doesn't offer anything more than suricata, but offers plenty less than suricata. A couple years time down the road, the cisco announcement that snort will no longer be maintained as a community project should wake up everybody. And yes, the announcement WILL come. The same way the ebay announcement came when I was running around screaming "change your ebay passwords!!!" and everybody was looking at me puzzled.
It's unfortunate that an open source project died (snort) but it's survival of the fittest. Now if we could only convince the (about) 43298 different open source text editors to die and merge in a single project, we will be colonizing Alpha Centauri by this time next year, and permission is hereby granted to quote me on that.
I don't think Supermule meant to burn you, JFL ;D
I didn't interpret it as if you wrote that Suricata has inline currently, but I do have the background knowledge you provided earlier in this fine forum (Suricata doesn't have it now either, but giving pfSense going FreeBSD 10.0 it most likely will have - in the future). I could understand that Supermule thought you suggested otherwise.
But let's not fight and keep the FreeBSD spirit high in this great community :-*
I am a 100% behind you concerning the text in green, btw. Of course cisco will go that route. Of course.
-
@Hollander:
A joy to read once again, JFL ;D
Armed with this knowledge, and the reply from BB right above about which Snort rules to disable (thanks again BB :P ) I reserved two hours to start disabling and enabling. Unfortunately, it seems Bill still didn't have time to make the enabling all and then disabling some-routine slightly more comfortable. He announced he will do that, so I will have to wait for that. Because the current way of working will require me to weeks of from work :-X
(1. Enable all -> 2. start to disable the exceptions -> 2a. disable first exception, it will not be disabled but will have the bright red cross -> wait for pfSense to process and respond back -> 2b. click the same cross once again & wait for pfSense to respond after which it will have the desired light yellow cross. That is not doable in a few hours with all the many rules I will have to manually disable).
Yeap, the change to the way rules are disabled makes sense for an already established and working system, but a new system that needs to be set up from the ground up is a nightmare to set up. That and you don't know when an ET rule "maintainer" will get the bright idea that he needs to fiddle with a no longer maintained rule, just for the hell of it, and your system starts blocking everybody because the rule was at its default off but the "maintainer" decided to flip it on.
The way I would like it:
- Enable all button
- Start clicking on rules I want to be disabled. These rules should be changed to user disabled (pale yellow).
- Move down the list
- Apply and wait.
- Move to next category and start at 1.
@Hollander and @Supermule
Oh, I didn't mean to sound hostile, it was a failed (as can be seen) attempt to humor :D. No hard feelings :) -
@jflsakfja:
Why do I have to always write like I'm presenting a case to the court?
That is the question you should answer, since this is entirely your choice.
@jflsakfja:
It's unfortunate that an open source project died (snort) but it's survival of the fittest.
Did it really die yet?
Last I checked, latest sources still available and there are active discussions in dev community…Being on the pessimistic side - yes, there is a chance Cisco will kill it, but I do not think this happened yet.
-
That wasnt what I was saying…..
Pls. read my post again.
You speak as Suricata is better and it maybe is...but as of yet, it offers nothing over Snort regarding PFsense and protection...
Can we agree on that? ;)
@jflsakfja:
I didnt know Suricata offered inline detection and blocking in Pfsense?
For what I know it works more or less the same way as Snort does (not inline)
Enlighten me pls :)
Why do I have to always write like I'm presenting a case to the court? Seems like even a miss-placed comma gets the author burned on the stake in these forums.
I have mentioned in the past that snort inline has been abandoned. Says so on their page if you don't believe me.
As it stands right now, the exact moment of this post, yes suricata and snort more or less run the same way on pfsense. BUT: The inline capability is there, it just cannot be used yet as pfsense needs some changes to it. They are coming soon(ish). Would you want to tear down your snort system when the changes are made, reconfigure and troubleshoot suricata, then enable it and forget it, or just log in and tick a checkbox and be done with it?
I don't see why we need to split the development effort between snort and suricata. Snort doesn't offer anything more than suricata, but offers plenty less than suricata. A couple years time down the road, the cisco announcement that snort will no longer be maintained as a community project should wake up everybody. And yes, the announcement WILL come. The same way the ebay announcement came when I was running around screaming "change your ebay passwords!!!" and everybody was looking at me puzzled.
It's unfortunate that an open source project died (snort) but it's survival of the fittest. Now if we could only convince the (about) 43298 different open source text editors to die and merge in a single project, we will be colonizing Alpha Centauri by this time next year, and permission is hereby granted to quote me on that.
-
Did it really die yet?
Last I checked, latest sources still available and there are active discussions in dev community…Being on the pessimistic side - yes, there is a chance Cisco will kill it, but I do not think this happened yet.
Ok, I'll bite.
An IDS isn't actually useful. It is useful when it's the only thing you have. An IPS on the other hand is immensely useful. Let's be clear on that one. Snort will NEVER have the inline capability that suricata has. Why? I don't know, but a hint as to why is that most (all?) inline maintainers for snort decided to kill the project and move on to suricata.
Seeing the latest source being available doesn't balance the 10 year outdated rules. Or the lack of fundamental understanding on how the Internet works, which is needed when writing a system to protect you on the Internet (for the love of god, if someone from the last dozen developers still reads this, please I'm down on my knees begging you, remove the "simple http" request rule. Please. There are millions of kids suffering because of it, billions of people starving because of it. I know that having an unknown guy tell you how to do your job is universally understood as THE most annoying thing, and you will resist all attempts to change the way you work, but it's THE most idiotic rule I've ever seen. Please I'm begging you remove it). 50 years later, come and re-read this post. If snort is still "maintained" you will see that the "simple http" request rule is still there. That's not how an alive project behaves.
Is ET better? HELL NO. Read the comments on the github list and you will too realize that even they have rules that should have long be deleted. For example doing a search on duckduckgo for "2010228 emerging" will show the 4th result with a title of "Global Causes of Death to 2020 – NEW YORK, Dec. 12, 2012 ...". See? I"m not crazy. This rule has been one of the leading causes for deaths worldwide. You know why? Because it identifies windows 7. Let's see, windows 7 RTM hit MSDN somewhere in the middle of 2009. So in 3 years time (result's title) more sysadmins have died to it than cancer, accidents, meteor stikes, COMBINED. When will rule maintainers realize that this is not a game, and people are actually getting killed because of their rules?
-
That wasnt what I was saying…..
Pls. read my post again.
You speak as Suricata is better and it maybe is...but as of yet, it offers nothing over Snort regarding PFsense and protection...
Can we agree on that? ;)
Nope ;D
Right now it (suricata) offers a more efficient packet analysis (suricata's AC behaves like snort's AC-BNFA). Ability to keep running even if a rule has a typo. Detailed logs. And that's just the simple things. I'll not go "under the hood" sort of speak, because not everyone will notice the difference, and frankly nobody will use them :p
-
If you all were happening to be in my neigborhood I would have invited you all in for some BBQ and some of the crappiest 'beer' you've ever drunk ('Heineken', one of the things many Dutch feel ashamed about. Freddie Heineken was a great marketeer but a lousy beer brewer). After which we would all be angry, and after which I would have grabbed in to my secret stash of serious beer - which comes from Belgium and goes by names such as 'Duvel', Westmalle Triple', and 'Kasteelbier'.
After which you all would be asking each other to marry each other, and the mission would be accomplished.
If you catch my drift ;D
-
@jflsakfja:
Ok, I'll bite.
Don't.
@jflsakfja:
An IDS isn't actually useful. It is useful when it's the only thing you have. An IPS on the other hand is immensely useful. Let's be clear on that one. Snort will NEVER have the inline capability that suricata has. Why? I don't know, but a hint as to why is that most (all?) inline maintainers for snort decided to kill the project and move on to suricata.
We are talking about current situation with Snort/Suricata on pfSense - no inline for either :( so it does not make a big difference.
@jflsakfja:
Seeing the latest source being available doesn't balance the 10 year outdated rules.
…..........I don't know, but I do not really have any issues with current set of rules. They some thinking first, but then entire thing is pretty manageable, especially if you use predefined policy.
-
@dgcom If you don't have a problem with the current rules you are either not using enough rules, or you followed my snort guide. Take it from the guy that broke the snort package by enabling too many rules (bmeeks can confirm this if you don't believe me). There is a picture I posted somewhere on these forums with the forum's banned hosts record. If I remember right, it was about 12.5K banned hosts. Most of that comes from custom rules, so (in a jokingly manner) I do know WTF I'm talking about ;)
If you don't want to use suricata, don't use it, I don't have any problem with that. One less installation to troubleshoot ;D
-
I use those rules, which I think useful for me.
And I never said that I do not want to use Suricata. Once it has inline functionality in pfSense, I'll migrate.
-
That was my point….
We need the inline functionality to work....
And it doesnt. Yet....
I use those rules, which I think useful for me.
And I never said that I do not want to use Suricata. Once it has inline functionality in pfSense, I'll migrate.
-
I think having both Snort and Suricata around will promote a healthier environment. From what I have seen, I don't think that Cisco will close Snort to "closed Source", they may add new features that are only for Paid Subscribers or for their own Cisco Equipment.
While I think that ET and OISF are capable of the same behavior.
What we really need is a simpler way to manage the Rules. Enabling/Disabling one at a time is too cumbersome and leads to human error and is too taxing on Time (Which we all wish we had more of .. :) )
I hope the Devs listen and give us the right tools for the job so we can manage the IDS/IPS of our own Choosing in the most efficient way.
-
What we really need is a simpler way to manage the Rules. Enabling/Disabling one at a time is too cumbersome and leads to human error and is too taxing on Time (Which we all wish we had more of .. :) )
I agree. If there was a function or feature to show relationships between or how different rules affect others, that would be very useful and could possibly prevent human error. Especially for those less experienced or knowledgeable.
-
I think having both Snort and Suricata around will promote a healthier environment. From what I have seen, I don't think that Cisco will close Snort to "closed Source", they may add new features that are only for Paid Subscribers or for their own Cisco Equipment.
While I think that ET and OISF are capable of the same behavior.
What we really need is a simpler way to manage the Rules. Enabling/Disabling one at a time is too cumbersome and leads to human error and is too taxing on Time (Which we all wish we had more of .. :) )
I hope the Devs listen and give us the right tools for the job so we can manage the IDS/IPS of our own Choosing in the most efficient way.
If the rule maintainers even said for once "yes, I've read that crazy guy's list", even for a single time, and corrected the rules I've been screaming about for years, then we wouldn't have all those rules to disable. An example? See the previous post about the win 6.1 rule. It's an invalid rule that must be deleted, end of story. If the rule maintainer doesn't delete it, I will figure out a way to work around that limitation by disabling the rule. I will not stop using the software just because a single person out of the thousands of persons that have to do with suricata doesn't realize he made a mistake.
Standing up to your actions and saying "I'm sorry, I made a mistake" is magnitudes of order more respectable than saying "I will not delete the rule because kids,cyber,terrorism,syria,russia,china,ukraine".Want an easier process to setting up suricata? Convince upstream to actually delete false positive rules. Not disable them, delete them. If they don't want to delete the win 6.1 rule, give microsoft a call, then put them on conference with the rule maintainer next to you and ask: "Is windows 7 actually windows 6.1?". Even after they hear the answer from microsoft they insist on not deleting the rule, a public name and shame is the industry accepted practice of dealing with it. Bonus points if you get it on video. Until then, I'm personally happy with maintaining my own false positives list and manually disabling the rules. Yes the disabling process as mentioned earlier should be changed, but it sort of works for now. Even if nobody else maintains the rules, I have my own rules for suricata to do what I want it to do.
A huge thanks to bmeeks (and all others that helped in the past) for getting snort/suricata packages were they are. I'm not playing down any of that work, if they didn't make the packages, we wouldn't be here arguing today on what rules and how we should disable them. Think about it. When IDS/IPS systems first started getting imagined (not created, imagined) they came with a price of a few million $. Now it's just a click and follow the guide away.
This is where the economics comes in (Hollander will be thrilled :P): If there is a single user of your software, don't expect much in the sense of support from it (donations/subscriptions). If everybody out there is using your software, and you are not a greedy SoB and actually charge a logical price for your software (eg subscriptions), a whole lot of people will chime in to support you. Yes this is the ideal scenario, and no it's not what actually happens in reality. It should happen though, because it's a shame to see projects die because of lack of user support. No sympathy to openssl. As Linus would have said:"The sooner the developers die in a horrific car accident, the better". Not my words, and put the phone down, it's not a "terrorist threat".
There is a (not so) fine line between fixing bugs and ignoring feature requests, and implementing feature requests but ignoring bugs. OpenSSL went over that line years ago.
If you see a man running around waving his arms up high, screaming "CHARGE FOR SOFTWARE????ARE YOU F***ING INSANE???SOFTWARE SHOULD ALWAYS BE FREE", it's RMS, please ignore him. His words have no relation to reality (no pun intended).
Theoretically speaking now, don't want to offend anyone, but I'll go ahead and do so anyway: If a certain IDS/IPS offers me all I want, then I'll contribute to its creation (documentation,donations,beer for the developers, hookers and blow (let's see how many got that joke) and so on). If said IDS/IPS's rule maintainer doesn't listen to me when I say a certain rule is buggy and must be corrected, don't expect said maintainer to get any support from me. Just saying, don't want to offend anyone.
And yes, I've put my money where my mouth is, remember the donations for snort's sync to slave system. It was a feature I wanted badly, the package maintainer impemented it without any noticable side effects to the general usage, and I've donated to those that made it happen. Everybody should do the same and stop sucking on RMS's meme that software should always be free and developers must die of hunger.
-
My vision for both Suricata and Snort includes a true inline operation option. For that to happen we need some changes to the pfSense kernel code. I think those will come, but it will take a little while. Obviously changes down at that level should not just be rushed out. So please continue to have patience.
As for some improvements that can actually happen without pfSense kernel changes, here are some highlights from my internal road map of new features I have planned for each package.
Suricata:
Migration to the new 2.0.2 binary code. This offers several new detection/inspection features, especially around DNS requests. There are also a multitude of other changes and improvements compared to the existing 1.4.6 binary code base in the current package.Adding the same XMLRPC sync feature to Suricata that currently exists in Snort so you can synchronize Suricata setups across multiple firewalls.
Fixing the list of bugs you guys have submitted over the last few months. There are several of them.
–---------------------------------------------------------------------------------
Snort:
Adding support for some of the new file capture features that have been added to the Snort binary. These mimic in large part what Suricata does in this area.Both Suricata and Snort:
Backporting significant functionality changes between Snort and Suricata. This means reverting to the old force rule enable/disable icon behavior that some folks so desperately want (I'm looking at you jflsakfja… ;))Adding a filter option to the ALERTS tab so you can see only what you want. This will be modeled after the firewall log filter functionality.
Providing rule management functionality equivalent to PulledPork with regex matches for enabling or disabling rules. In other words, the ability to read and interpret enablesid.conf, modifysid.conf and disablesid.conf files. You would be able to edit these offline and upload to the firewall, or edit in place using the same interface as I implemented for the IP REP lists management tab in Snort.
Bill
-
Hi Bill,
This is Fantastic News. Lots to look forward to. We are all deeply in your debt! ;)
We've never had a poll for the Best Package Maintainer! But I think we should start…
If you need any help to Test, I am sure several of us will throw our hats into the ring.
-
Just wondering…
Would it be feasible/easy/doable to implement squid like behavior in Suricata/Snort to make it independant of the inline capabilities of Pfsense?
Instead of Snort inspecting copies of packets, then going through as a transparent proxy, then the inline could be there?
Doable?
-
My vision for both Suricata and Snort includes a true inline operation option. For that to happen we need some changes to the pfSense kernel code. I think those will come, but it will take a little while. Obviously changes down at that level should not just be rushed out. So please continue to have patience.
As for some improvements that can actually happen without pfSense kernel changes, here are some highlights from my internal road map of new features I have planned for each package.
Suricata:
Migration to the new 2.0.2 binary code. This offers several new detection/inspection features, especially around DNS requests. There are also a multitude of other changes and improvements compared to the existing 1.4.6 binary code base in the current package.Adding the same XMLRPC sync feature to Suricata that currently exists in Snort so you can synchronize Suricata setups across multiple firewalls.
Fixing the list of bugs you guys have submitted over the last few months. There are several of them.
–---------------------------------------------------------------------------------
Snort:
Adding support for some of the new file capture features that have been added to the Snort binary. These mimic in large part what Suricata does in this area.Both Suricata and Snort:
Backporting significant functionality changes between Snort and Suricata. This means reverting to the old force rule enable/disable icon behavior that some folks so desperately want (I'm looking at you jflsakfja… ;))Adding a filter option to the ALERTS tab so you can see only what you want. This will be modeled after the firewall log filter functionality.
Providing rule management functionality equivalent to PulledPork with regex matches for enabling or disabling rules. In other words, the ability to read and interpret enablesid.conf, modifysid.conf and disablesid.conf files. You would be able to edit these offline and upload to the firewall, or edit in place using the same interface as I implemented for the IP REP lists management tab in Snort.
Bill
If The Company was hiring, you would be one of the first people it would hire, trust me on that one ;D
Many thanks for helping the community with those packages. I'm sure as the features everybody wants are added we can all chime in to make it worth a bit for the time you've spent on them.
WRT the blue text, maybe we should agree on a bare minimum of disabled rules (the absolute lowest number of general use rules) that should be disabled on all installations and somehow integrate that into newer installs? So people don't have to get choked into configuring snort/suricata right away after install, but actually get on the internet and get some help if they have a problem, without the IDS/IPS banning everything (coughsimple http requestcough).
-
@jflsakfja:
WRT the blue text, maybe we should agree on a bare minimum of disabled rules (the absolute lowest number of general use rules) that should be disabled on all installations and somehow integrate that into newer installs? So people don't have to get choked into configuring snort/suricata right away after install, but actually get on the internet and get some help if they have a problem, without the IDS/IPS banning everything (coughsimple http requestcough).
Not a bad idea. When I get to that point, I will be in touch. That could be offered as an option with the default being "yes" (meaning, auto-disable some bare minimum of community-agreed upon quasi-useless rules).
Bill
-
Not a bad idea.
Can I apply for having a good idea too?
( ;D )
The checkboxes that we have in the firewall rules to select multiple rules: also in Snort/Suricata? So you can quickly select multiple rules in a category and then mass enable/disable them?
Select the rules -> click 'disable' -> click 'apply', and move on to the next category.
Or a CLI-operation where you can edit a *.conf quicker than one by one as currently is the case?
Let me guess, probably not good ideas from the eternal noob :-[
( ;D )
Thanks for all you do, Bill :P
-
@Hollander:
The checkboxes that we have in the firewall rules to select multiple rules: also in Snort/Suricata? So you can quickly select multiple rules in a category and then mass enable/disable them?
bangs head on desk Why didn't I think of that? ;D Selecting a dozen rules then hitting disable selected rules would be very nice indeed.
-
@Hollander:
Not a bad idea.
Can I apply for having a good idea too?
( ;D )
The checkboxes that we have in the firewall rules to select multiple rules: also in Snort/Suricata? So you can quickly select multiple rules in a category and then mass enable/disable them?
Select the rules -> click 'disable' -> click 'apply', and move on to the next category.
Or a CLI-operation where you can edit a *.conf quicker than one by one as currently is the case?
Let me guess, probably not good ideas from the eternal noob :-[
( ;D )
Thanks for all you do, Bill :P
[/quote]Excellent idea. Let me see if I can find enough real estate to show the checkboxes. For non-widescreen configurations, I'm running out of space on the RULES tab when displaying the rules. When everyone migrates to pfSense 2.2, it has widescreen as a built-in theme. That will help out greatly.
I do also plan to offer what is a kind of CLI interface for this. Do a Google search for enablesid.conf, modifysid.conf and disablesid.conf. You should get some hits showing how these work with packages such as PulledPork. The idea for the Suricata and Snort packages is to be able to edit these files offline and then upload them to the firewall.
Bill
-
Ran into a small problem with bbcan177's script. Was working wonderfully then quit updating after the last Alpha update. Running pfSense 2.2 alpha.
Here's the error:
SSL options: 81004bff Peer verification enabled Using CA cert file: /etc/ssl/cert.pem Certificate verification failed for /O=www.projecthoneypot.org/OU=Domain Control Validated/CN=www.projecthoneypot.org 675141692:error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed:/usr/pfSensesrc/src/secure/lib/libssl/../../../crypto/openssl/ssl/s3_clnt.c:1169: fetch: https://www.projecthoneypot.org/list_of_ips.php?t=d&rss=1: Authentication error
Anyone else run into this?
NEVER MIND, I'm stupid… LOL..
-
Well thank you Bill and JFL for boosting my self esteem ;D ;D ;D
But now… 8)
( ;D )
I am struggling with a dnsmasq problem (blocking ad servers via dnsmasq, can't get the script to run in cron - although manually it works), and on the site that provides the lists I also noted a format for Snort:
http://pgl.yoyo.org/adservers/
(Select the second drop down - list ad servers IP adresses): there is also a Snort format of the list there.
Would it make sense to use Snort to block ad servers using that list?
I know there will be reservations; 'noob, shut up'.
Ok, I will :P
-
@Hollander:
Well thank you Bill and JFL for boosting my self esteem ;D ;D ;D
But now… 8)
( ;D )
I am struggling with a dnsmasq problem (blocking ad servers via dnsmasq, can't get the script to run in cron - although manually it works), and on the site that provides the lists I also noted a format for Snort:
http://pgl.yoyo.org/adservers/
(Select the second drop down - list ad servers IP adresses): there is also a Snort format of the list there.
Would it make sense to use Snort to block ad servers using that list?
I know there will be reservations; 'noob, shut up'.
Ok, I will :P
When you can't get a script to run via cron, the #1 cause is forgetting to provide the complete and entire path to all files used in the script. When a cron task executes, it does not inherit the user environment you have when working at a shell prompt (CLI). Some examine the script and make sure full paths are provided for all referenced files.
Bill
-
Ran into a small problem with bbcan177's script. Was working wonderfully then quit updating after the last Alpha update. Running pfSense 2.2 alpha.
Here's the error:
SSL options: 81004bff Peer verification enabled Using CA cert file: /etc/ssl/cert.pem Certificate verification failed for /O=www.projecthoneypot.org/OU=Domain Control Validated/CN=www.projecthoneypot.org 675141692:error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed:/usr/pfSensesrc/src/secure/lib/libssl/../../../crypto/openssl/ssl/s3_clnt.c:1169: fetch: https://www.projecthoneypot.org/list_of_ips.php?t=d&rss=1: Authentication error
Anyone else run into this?
NEVER MIND, I'm stupid… LOL..
SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
The server certificate is invalid, either because it is signed by an invalid CA (internal CA, self signed,…), doesn't match the server's name or because it is expired.Hi wcrowder, is this still an issue for you?
-
I do also plan to offer what is a kind of CLI interface for this. Do a Google search for enablesid.conf, modifysid.conf and disablesid.conf. You should get some hits showing how these work with packages such as PulledPork. The idea for the Suricata and Snort packages is to be able to edit these files offline and then upload them to the firewall.
This will make the Rules Configuration Process a Breeze. All we would have to do, is copy the text based conf file to any other Snort/Suricata Interface or to another Box. Rules can be disabled or enabled by SID, Category or by PCRE (ie : Autodesk or Adobe - Which will disable/enable any rules that has any of those Keywords in them. This way even as new rules are added, the PCRE will disable them at each rule-update automatically. :)