Taming the beasts… aka suricata blueprint
-
@jflsakfja:
The trick to proving that industry leaders are complete and utter idiots is knowing your job and constantly thrive to be the best at it. The BEST! A "take no prisoners" approach to training/learning at your job, is the only secure way of achieving your goals.
Thank God I'm retired and no longer have to deal with this. YOU are SO right.
What I found to be most frustrating was that its not just the industry leaders… It's our/your own IT department VPs and AVPs too. Professional Executives who are the reason for PEBKAC and have NO IT history/experience. And some are such "utter idiots" that even when you prove them to be so, they don't see it. Don't confuse them with the facts, their minds are made up. And If you do get the upper hand on the point in case their fallback is always budget. Everything has to go through the budget filter. What they say: "We want to build the safest, most secure system for our customers". What they mean: "We want to build the cheapest, worst system we can get away with". And then when that wasn't enough... the real scumbag execs and lawyers brought in "arm's length". Don't listen to your talented "in-house" IT folks - Hire a consultant so when we do it "our way" (read cheap) and it does get compromised/falls apart/doesn't work, you can blame the consultant!!
Sorry, it was that time; Rant:30
<soap box="" mode="" off="">So now, I leave the training/learning to you folks and sit here and absorb it. Thank you. Its a pleasure.
Rick</soap>
-
@jflsakfja:
@Hollander:
I am sure I am once again stupid, but: Interface Groups do exactly the same?
For example, I have MultiWAN, in which both my WAN's are. In the MultiWAN-tab I have set some rules that apply to both WAN's at the same time. Floating rules can do that too, so when would one use Interface Groups, and when would one use Floating Rules ???
Floating rules are rules that can be active on multiple interfaces at the same time, by just merely CTRL+clicking multiple interfaces (extremely easy to add a new interface to an existing rule).
Your posts are always a pleasure to read, Jfl :P
As to your ARM-example; I do recognize it. I am an economist, but I have been involved in huge ERP-projects for over a decade. Trust me, ERP-projects need more economists; a retrained history student that suddenly, after 4 weeks of courses, is a 'senior financial consultant', can se-rious-ly mess up global ERP-projects in which we need to implement the full scale of accounting and controlling, all the way up to consolidation and hyperinflation accounting and resulting in financial analytics. If you want to have the servers and the ERP-systems to do that au-to-ma-ti-cally (which you want, if you have say a 1000 locations/100 different GAAP's/150 different currencies/ around the globe that need to report their results 3 days after month end), you don't get anywhere with a former history student that says that a simple journal entry is too complex and we 'need to discuss this on a higher, more strategical level').
Many times companies turned to me for second opinions on blueprints. For a simple reason: because they couldn't believe the crap all these 'global system integrators' were saying (and yes, these are the tens of thousands employees type of companies I am referring to :-[ ). The clients take them 'because they have global coverage' (needed if you have to do a global roll out), and then they are bravely waiting for their turn to be screwed. The biggest scam of all, the one I always saw returning, is that the blueprint consists, for 50-90%, of this line: 'not possible in the standard system, custom development is needed'. Then you have this situation: a Fortune500 company invests 500 million in the project (hardware (global, again), licenses, consultants to set it all up). Accepts that it will take 5-10 years to roll it all out. And after signing the contract finally, after a long and difficult, frustrating birth, gets a blueprint in which it is said that that top of the bill ERP-system can not send customer profitability in real time to the data warehouse. 'This is custom development, because the standard system can not do that'.
I've made many enemies over the years when I came in for second opinions: usually, I just made sure both the customer and the former history student-now 'sr. financial consultants' were in the same room, after which I opened a development box and customized the required functionality on the scene.
I've made ma-ny enemies over the years with all these 'integrators' who have 23 years old 'senior' punks on staff who know nothing except for saying that they have 'tens of thousands of years of experience' ;D ;D ;D
Of course, I also learned over the years that most VP's are either corrupt or incompetent or both; in the end, there's always somebody in the top who doesn't care about the truth, who doesn't care that the company is ripped off. You don't need to be an Einstein to understand what's behind that. (From an economical point of view, btw: it all boils down to ownership of the company. So called 'agency problems'. Nobody gives a sh*t if he is playing with not his own money, and gets away with it).
Anyway, so: I recognize your ARM-example all too well :P
If I may, JFL: as much as I agree with your point of view on doing things efficiently (I have to, I am an economist ;D ), sofar I still don't see a difference re Floating Rules versus Interface Groups: as easily as I can add an interface to a an existing Floating Rule, just as easily can I add an interface to an existing Interface Group (to which then the rules for that Group also apply). And since then the Floating Rule won't handle it but the Interface Group will, it is not a matter of pushing rules through the Firewall landscape, as the Floating Rule doesn't handle the packet, but the first stop will be the Interface Group.
You know by now that I am the eternal noob, famous for asking stupid questions ;D
-
@Hollander:
1. Does it make sense that if IR_PRI1 is blocking that Snort is also still blocking IP's based on Dshield (which is in IR_PRI1)?
You can "uncheck" these ET rules as pfIPRep already includes these lists. No need for Snort/Suricata to be looking at these. You should also enable the "Port Scanning Pre-Processor" to ban IPs that scan your IP for Open Ports.
I am sure that jflsakfja has already said to disable them, but I noticed from your post that you indicated "dShield".
(Pro Rules)
etpro-botcc.portgrouped.rules
etpro-botcc.rules
etpro-ciarmy.rules
etpro-compromised.rules
etpro-drop.rules
etpro-dshield.rules(These are discontinued and are blank, so you can skip these also)
etpro-rbn-malvertisers.rules
etpro-rbn.rules(Subscriber Rules)
emerging-botcc.portgrouped.rules
emerging-botcc.rules
emerging-ciarmy.rules
emerging-compromised.rules
emerging-dshield.rulesemerging-rbn-malvertisers.rules
emerging-rbn.rulesYou will also notice in the pfiprep script the Collect line for Snort/Suricata. This will load all of the IPs in those Rulesets and remove any duplicates. I am noticing that there are some IPs in these Rules that are not in their ET COMP, ET Block or ET IPREP. I have asked their support desk for the reason why, but still waiting for a reply.
So I would recommend enabling the Snort/Suricata (i386/AMD64) rule accordingly. Only need to have one of them enabled.
-
@Hollander:
If I may, JFL: as much as I agree with your point of view on doing things efficiently (I have to, I am an economist ;D ), sofar I still don't see a difference re Floating Rules versus Interface Groups: as easily as I can add an interface to a an existing Floating Rule, just as easily can I add an interface to an existing Interface Group (to which then the rules for that Group also apply). And since then the Floating Rule won't handle it but the Interface Group will, it is not a matter of pushing rules through the Firewall landscape, as the Floating Rule doesn't handle the packet, but the first stop will be the Interface Group.
I've always set up the floating rules instead of interfaces groups. If the rules are evaluated as you say (first, multiple interfaces,simple to maintain) then I'm assuming it's the same thing as a floating rule, minus the create a new interface group part before using it.
If someone more experienced says that's the way to go, then by all means I stand corrected and that's the way to go.
I posted the easiest way to achieve what we need to achieve, from my point of view. If others want to interject please go ahead offering your opinion, I won't bite :D
-
Abuse.ch has launched a new Blocklist service. SSL Blacklist (SSLBL) is designed to aid in detecting botnet traffic that uses SSL to communicate
You can add the conservative or aggressive IP Blocklist to pfiprep to help mitigate these SSL risks.
See the following link:
https://sslbl.abuse.ch/blacklist/Conservative IP list:
https://sslbl.abuse.ch/blacklist/sslipblacklist.csvAggressive IP list:
https://sslbl.abuse.ch/blacklist/sslipblacklist_aggressive.csv -
You can "uncheck" these ET rules as pfIPRep already includes these lists. No need for Snort/Suricata to be looking at these. You should also enable the "Port Scanning Pre-Processor" to ban IPs that scan your IP for Open Ports.
I am sure that jflsakfja has already said to disable them, but I noticed from your post that you indicated "dShield".
Thanks BB ;D
I have not yet disabled the Snort/Suricata rules, as doing that currently is cumbersome given the way this works (have to click each rule twice, wait for pfSense, etc: it will take hours and hours this way). JFL commented the same in his setup post, and asked if Bmeeks could change that. I understood from Bill that he will change it, so I awaiting that change.
-
Abuse.com has launched a new Blocklist service. SSL Blacklist (SSLBL) is designed to aid in detecting botnet traffic that uses SSL to communicate
You can add the conservative or aggressive IP Blocklist to pfiprep to help mitigate these SSL risks.
See the following link:
https://sslbl.abuse.ch/blacklist/Conservative IP list:
https://sslbl.abuse.ch/blacklist/sslipblacklist.csvAggressive IP list:
https://sslbl.abuse.ch/blacklist/sslipblacklist_aggressive.csvThat's great, BB ;D
The noob at it again: to which categories do you need to add these in the script? Simply copy line X in the script and change the URL?
-
I was wondering, is there some wisdom in what to enable on what interface? I know Bmeeks uses Snort both on LAN and WAN, albeit different kind of rules on both types of interfaces. Enabling on LAN makes sense as you can see which LAN client triggers an alert, enabling on WAN obviously makes sense to prevent stuff coming in. At the same time, it is said that setting the same rules on WAN and LAN is overkill, so I arrive at the question: which rules should one set on WAN, and which one on LAN?
Thank you from the eternal noob for the zillion-th time ;D
-
In order to determine what rules to enable for lan facing and wan facing interfaces, you need to go through the rules one by one, looking at their sources and destinations. I'm having a hard time keeping up as it is, imagine that as well :p
The easy way out is to enable 2 identical interfaces, one on wan and one on lan. We've already trimmed down suricata enough for that to run OK, assuming you have at least a decent amount of RAM (4GB).
Whether an internal facing interface is really needed, it's another question. Most "malware" rules have rules for both directions (in + out) so the wan one should spot something fishy easily. If you have multiple lan interfaces, that add yet another question. How will pfsense analyze traffic from LAN1 to LAN2? It needs an internal facing interface.
Questions, questions. Answer 1 and 1000 will pop up :p
-
@jflsakfja:
In order to determine what rules to enable for lan facing and wan facing interfaces, you need to go through the rules one by one, looking at their sources and destinations. I'm having a hard time keeping up as it is, imagine that as well :p
The easy way out is to enable 2 identical interfaces, one on wan and one on lan. We've already trimmed down suricata enough for that to run OK, assuming you have at least a decent amount of RAM (4GB).
Whether an internal facing interface is really needed, it's another question. Most "malware" rules have rules for both directions (in + out) so the wan one should spot something fishy easily. If you have multiple lan interfaces, that add yet another question. How will pfsense analyze traffic from LAN1 to LAN2? It needs an internal facing interface.
Questions, questions. Answer 1 and 1000 will pop up :p
Thank you ;D
Well, if the rulesets are available somewhere for import in MS Excel, and if I would really understand what I need to look for (the part in bold in your reply, which I don't quite understand yet, as in: what do I need to look for exactly?), I could try to filter out in Excel the rules that need to go on WAN, versus the ones that need to go on LAN.
I'm more than willing to attempt an effort at that, but you would need to give this noob slightly more detailed instructions on what to look for :P
-
Text copied from edit rules tab, so the columns are:
SID PROTOCOL SOURCE DESTINATION EXPLANATIONWAN facing rule:
2017397 http $EXTERNAL_NET any $HOME_NET any ET DOS Apple CoreText Exploit Specific string
Notice the red text in the source. It means a source that is not us(= every other on the Internet => WAN facing)LAN facing rule:
2017920 udp $HOME_NET 123 $EXTERNAL_NET any ET DOS Possible NTP DDoS Multiple MON_LIST Seq 0 Response Spanning Multiple Packets IMPL 0x02
Notice the blue text in the source. It means a source that is us (=> internally facing)BOTH facing rule: (WAN+LAN, must be duplicate)
2017919 udp any any any 123 ET DOS Possible NTP DDoS Inbound Frequent Un-Authed MON_LIST Requests IMPL 0x03
Notice the green text in the source. It means a source that is any(ie we are not sure if it's us or them) => it needs to be duplicated to both internally and externally facing interface/s. -
That's great, BB ;D
The noob at it again: to which categories do you need to add these in the script? Simply copy line X in the script and change the URL?
You can add a new line below the Abuse Palevo list as follows:
collect "$sch4" "AbuseSSLBL|tier1|https://sslbl.abuse.ch/blacklist/|sslipblacklist.csv|process|yes|yes|faf"
I am showing the conservative list in this example, you can also use the aggressive one if you choose. I am away until early next week so I can't provide any advice until I test it my self. There are not that many IPs in the aggressive file, so even if there are FPs, probably wouldn't be many to deal with.
In regards to disabling those rules. The ones I posted above, you can disable the whole category in the category tab for the WAN interface, you don't need to do this one rule at a time.
-
@Hollander, you have to jump in and learn some Linux CLI like the "grep" command. Google it and you will see how powerful this command is.
The rules files are located in these folders depending on your OS type and Snort/Suricata:
Snort64. /usr/pbi/snort-amd64/etc/snort/rules
Snort32. /usr/pbi/snort-i386/etc/snort/rules
Suri64. /usr/pbi/suricata-amd64/etc/suricata/rules
Suri32. /usr/pbi/suricata-i386/etc/suricata/rules
So as an example:
grep "$EXTERNAL_NET" /usr/pbi/suricata-amd64/etc/suricata/rules/*
Note to add a "" to escape special characters, as $ is used as a variable name.
-
If non-transparent proxy is being used, there is no real point in running Snort or Suricata with blocking enabled on LAN interface - all connections will be between internal hosts and LAN interface IP where proxy is listening. By default, those are whitelisted and if you decide to block internal hosts, you will cut off them from the internet completely…
-
If non-transparent proxy is being used, there is no real point in running Snort or Suricata with blocking enabled on LAN interface - all connections will be between internal hosts and LAN interface IP where proxy is listening. By default, those are whitelisted and if you decide to block internal hosts, you will cut off them from the internet completely…
I'm not understanding this. Are you saying that they be able to alert correctly since they are going thru a proxy? I run a non-transparent proxy, using wpad.. Snort and Suricata have been alerting correctly and block external IPs when needed. During testing, ET Policy were the block of my alerts on the LAN interface..
-
When internal client goes through non-transparent proxy, Suricata will only see internal IPs on LAN interface.
If there is some alert - let's say malicious JavaScript - it will alert, but since all IPs are local (ex. 192.168.1.1:55555 -> 192.168.1.1:8080), it won't block anything. For this to block, you have to catch this connection on WAN interface. -
If non-transparent proxy is being used, there is no real point in running Snort or Suricata with blocking enabled on LAN interface - all connections will be between internal hosts and LAN interface IP where proxy is listening. By default, those are whitelisted and if you decide to block internal hosts, you will cut off them from the internet completely…
Are you talking about a transparent firewall running snort/suricata? There are uses for this, no it's not for everyone. By default snort/suricata has difficulty determining its IPs when running in transparent mode, so no hosts will be whitelisted. Ask yourself. What would you pick? System compromise or system getting cut off from the internet? If an internal server is infected somehow and starts spewing exploits here and there, the acceptable way of dealing with it is cutting off its network access. Care! There have been cases (supposedly) in the past where malware was (allegedly) designed to that if you pulled the ethernet cable it would start destroying the system's filesystem.
A transparent snort/suricata system comes in handy when you don't want hosts on the network being able to access it in any way. How will you attack the upstream gateway if you don't know there IS an upstream gateway?
-
Hmm… I thought, I clearly wrote:
If non-transparent proxy is being used…
Let me try one more time - if you do NOT run proxy (Squid or anything else) in transparent mode - which means internal machine browser have to specify proxy in the browser, which, in turn, means that internal machine will connect to proxy in order to request any web page - in this case, running Snort/Suricata riles, which check any protocols, which proxy can rely (ex. HTTP, HTTPS, FTP) is possible in alert mode only. Even if you check to block, by default connections will not be blocked, because they are only between internal machine and LAN interface IP and those IPs are whitelisted by Snort/Suricata as internal network.
See example in the attached screenshot. In there, 192.168.15.56 is the internal IP of the pfSense, which has Snort listening on port 8080 and 192.168.15.174 is the client, receiving obfuscated JavaScript in the web response. Snort will not block this because all IPs are internal.
In this case, if such blocking is required, appropriate rules should be activated on WAN interface, but you will not see internal IP in the alerts.
-
Transparent or not the proxy will still not be blocked, since it's a known IP by snort/suricata. (WAN interface).
It will block the external host though, as long as it can see inside the protocol (ie no TLS).
If you want to block internal hosts, then just change the pass list to a different one. Snort/suricata will think all systems are to be blocked (even internal IP ones) if you have it listening on an internal interface.
EDIT:brain fart :p
-
Transparent or not the proxy will still not be blocked, since it's a known IP by snort/suricata. (WAN interface).
Sorry, not sure I understand you - do you have your proxy listening on WAN interface?
If you want to block internal hosts, then just change the pass list to a different one. Snort/suricata will think all systems are to be blocked (even internal IP ones) if you have it listening on an internal interface.
What is "it" in here? Proxy?
Entire internal network is in the pass list by default.
-
Snort/suricata WAN interface. No matter what proxy you run internally, eventually it must connect to the outside to pull data in. That's when the alerts will fire up. Snort/suricata will try to ban both source+destination (if you did not set it up like that, you should) and since the destination (proxy) is listed on its (snort/suricata's) pass list, it (proxy) will not be blocked. The outside IP will though.
WRT the listening, I was talking about snort/suricata.
"Entire internal network is in the pass list by default." I am aware of that, that's why I said change the pass list. If you tell suricata these are the networks you should NOT ban, but do NOT include the private IPs in that list, suricata will notice the alert, check to see if a private IP is on the pass list, NOT see the IP there, therefore will proceed to ban the internal IP. Most likely the proxy (pfsense) will be banned as well, so that IP should be on the pass list.
If I still fail to see the point, my apologies, it was a long day.
-
Yeah, I think we speak different languages.
I am not talking about WAN interface, not sure why you brought it in.
I am talking about Snort/Suricata instance on LAN interface. In case of non-transparent proxy it does not see external IPs, period.
-
WAN is brought in because it is needed to connected to the outside world. If you have the same rules on both interfaces and an alert pops up; the WAN interface will block the the external IP. interface. Since you should have the same alert for both interfaces, you can cross reference the WAN/LAN alert logs to find the internal IP on your LAN interface. I get what you're saying now about the LAN not blocking the the internal IP since its already white-listed. I haven't had any LAN clients stumble any malicious javascript yet.
-
Yeah, I think we speak different languages.
I am not talking about WAN interface, not sure why you brought it in.
I am talking about Snort/Suricata instance on LAN interface. In case of non-transparent proxy it does not see external IPs, period.
snort/suricata interface on LAN should show the same alert that snort/suricata on the WAN will show. If you did change the pass list, then both the internal (private) IP and the external (malicious site) IP will be blocked. If you still think that pfsense will not see the internal IP because it somehow "travels through the proxy" then you are missing a critical piece of information. snort/suricata are running the interfaces as promiscuous. That means they see every single packet that hits them, even if it's not destined for them. snort/suricata will see the packets from the laptop to the proxy (assuming proxy on pfsense), the proxy's answer to the laptop (both on LAN interface) and the request from the proxy to the malicious site, and the malicious site answer to the proxy (WAN interface). Anything that flags an alert with any of that will result in an IP being blocked, even internal IPs. Period. (throwing it in there, since something clicks in my head when someone says period to me, then everything turns black).
-
If you have the same rules on both interfaces and an alert pops up; the WAN interface will block the the external IP. interface. Since you should have the same alert for both interfaces, you can cross reference the WAN/LAN alert logs to find the internal IP on your LAN interface. I get what you're saying now about the LAN not blocking the the internal IP since its already white-listed. I haven't had any LAN clients stumble any malicious javascript yet.
Yes, that is the point I am trying to make - in such case you have to have appropriate rules enabled on WAN interface and, as you said, there is a need to find which internal IP it was, on the LAN interface as well… But that is a lot of manual labor to cross-reference alert logs :( And the additional memory used for duplicate rules...
For me, Snort http preprocessor with its built-in inspection rules (like (http_inspect) HTTP RESPONSE HAS UTF CHARSET WHICH FAILED TO NORMALIZE) is too noisy and blocks normal sites often.
-
@jflsakfja:
Anything that flags an alert with any of that will result in an IP being blocked, even internal IPs.
Don't know about you, but in my setup internal IPs are in the safe list by default (did I repeat it 3rd time?).
Oh, and I am not complaining about that. -
@jflsakfja:
Anything that flags an alert with any of that will result in an IP being blocked, even internal IPs.
Don't know about you, but in my setup internal IPs are in the safe list by default (did I repeat it 3rd time?).
Oh, and I am not complaining about that.I also repeated 3 times that you can actually change the PASS list so they are not.
-
And why would I need to remove my internal network from the pass list?
What is the use case? May be for some public hotspot it may make sense, but not for the trusted network… -
There is no such thing as a trusted network. As I said somewhere else, the only thing you should trust is the little voice inside your head. Everything/everyone out there is out to get you. And no, it's not paranoia when they ARE out to get you.
NSA aside, the use case is when trying to limit a local (internal) host from spreading something nasty on the internet. Other than that, there is no use for it, since it can still spread the "nasty" to other hosts on its subnet. When each host is on a different subnet, that's a different story, it effectively blocks it from accessing the entire network, but that is a highly specialized use case (ie not something you will encounter outside of an ISP).
Another use case that pops into my mind is when providing internet access to others (free wifi anyone?). You can't tell what their systems are infected with, so to limit your liability you could block it from accessing the internet.
As you can see from the guide, most weight is put into securing the network from outside threats, with some weight given on somewhat limiting outgoing connections. My recommendation for a normal home network is to use your trusted network example. Perfectly normal paranoia aside, it's very unlikely the NSA has broken into your house/computer-while-it-was-shipped and installed a backdoor. Incoming "nasties" are blocked. Since they are blocked, and you use some common sense (a shocking truth, but that hottie that keeps sending you naked images of her, is actually trying to get your email client to parse a command obfuscated in the image, ala firefox (yes a browser) jpg code execution), your network is (mostly) safe. Yes there are other attack vectors (eg browser memory mismanagement) but those are outside the scope of this guide :D.
-
Here is a report from The Solutionary Security Engineering Research Team (SERT) Quarterly Threat Intelligence Report for Q2 2014
http://www.solutionary.com/research/threat-reports/quarterly-threat-reports/sert-threat-intelligence-q2-2014/
Key findings include:
The top 10 ISPs represented the source of 52% of the malware identified in the new period.Amazon-hosted malware nearly triples in first half of 2014.
56% of malware captured was hosted in the United States, a 12% increase from Q4’13
Malicious SSH activity continues to be a popular method to gain administrative access to target systems.
-
Not going to register to download a report that I'm responsible of producing at my work :p
Let me guess, they propagate the Chinese 1337 hackers meme?
-
@jflsakfja:
Not going to register to download a report that I'm responsible of producing at my work :p
Let me guess, they propagate the Chinese 1337 hackers meme?Its great that you are cataloging these Malicious Attempts in your network. But its important to have a well balanced Source of Threats. Some Sources are good for Mail Servers, others Malware, others SSH attack etc. By combing threat sources from a multitude of Threat Sources, you can better determine their IP Reputation.
I typically don't like to have to register for Reports also. But looking at the Solutionary Website, there are other reports available online without needing to register.
http://www.solutionary.com/research/threat-reports/monthly-threat-reports/2014/06/security-threat-report-june-2014/
Looking at my current suggested List of Blocklist Sources (See data below),
grep "98.124.198.1" pf/*
pf/ET_IPrep.txt:98.124.198.1
pf/e_tcnc:98.124.198.1
pf/e_tdrop:98.124.198.1grep "85.159.211.119" pf/*
pf/ET_IPrep.txt:85.159.211.119
pf/e_tblackhole:85.159.211.119
pf/e_tcnc:85.159.211.119grep "85.25.148.6" pf/*
pf/IBlock_BT_Spy.txt:85.25.148.6
grep "217.12.207.151" pf/*
pf/IBlock_BT_Spy.txt:217.12.207.151
grep "192.99.6.61" pf/*
no results
grep "192.99.6.0" pf/*
no results
grep "^192.99." pf/*
pf/ToastedSpam.txt:192.99.0.0/16Game Over Zeus Botnet (goz)
http://garwarner.blogspot.ca/2014/07/new-gameover-zeus-variant-uses-fastflux.html?m=1Only ET IQRisk Blocklist picks up all of these IPs.
I have seen this spam hit my Network "E-ZPass" (I forwarded them to SpamCop also)
http://garwarner.blogspot.ca/2014/07/e-zpass-spam-leads-to-location-aware.html?m=1Only ET IQRisk Blocklist picks up all of these IPs.
-
Right, I finally have a little bit more time to work on this now :P
Could I ask one more question, JFL?
This is what I am struggling with:
1. You said in one thread (I think it was your previous thread about how to set up Snort) said to ditch Snort for Suricata;
2. Some Snort rules (some 600 I believe Bill said) Suricata can not interpret.
3. Your current instructions (this thread) only deal with the ET-rules, not with the Snort rules.
4. I take it ET doesn't completely overlap the Snort (paid subscription, which is what I have) rules, so there is/might be value in using both the Suricata- and Snort rules in Suricata.What would be wisdom here? Take your old instructions about the Snort rules to know what to disable, or do you consider these instructions obsolete perhaps? If so, ditch Snort all together? (but then point 4?) If not ditch Snort, how do you cope with the rules Suricata can't handle (the Snort rules)? There are 600 it seems, so [a1] how to find out which ones and [a2] after disabling these, aren't we missing out on important protection?
Or: have both Snort and Suricata inspect the interfaces at the same time (Snort with the full subscriber rule set, Suricata with the ET rule set)?
Yes, vague and complex for a stupid economist like me, I know :-[
( ;D )
Thank you again very much :P
-
@Hollander:
1)The only reason suricata is better than snort is that snort's inline has been abandoned and the developers moved to suricata, so if you want an IPS then you have to use suricata. We are talking about the general majority of users out there, there are other differences, but most users will never notice them. There is NO other software out there that matches its capabilities right now, pay,free,donated-to-us-by-aliens. It is actually the top of the line, absolute best IDS/IPS available. It bloody hell should be the best, it was funded directly by the U.S. government, I'm sure those guys want to get their money's worth.2)As far as the rules that suricata cannot interpret you only have these 2 choices:a)don't use them or b)manually rewrite the rule so that it is accepted by suricata.
3)This thread is about suricata, so there is no point in dealing with the snort rules, since those rules are not used. My opinion as to whether they are updated, or not, is my opinion only and does not reflect the opinion of The Company, etc… etc...
4)If you followed the guide to the letter and actually set up your suricata as suggested, you should be seeing about 90% banned hosts from custom rules (simple port scanning rules that detect port scans even the ET rules miss). Do you want to focus on the 1% that is banned hosts from the snort rules (assuming you are using snort with suricata, and filtered out the not working rules), the 9% from ET rules, or the 90% from custom rules? ;)
The trick to setting up an IDS/IPS is thinking like an attacker. What would you do to compromise an unknown system? These are the steps (most) attackers worth their salt follow (note: Does NOT refer to Chinese and Korean so called "state sponsored hackers", I don't have the time to deal with script kiddies):
a) Identify who the system belongs to. If it belongs to a letter soup agency, have the drill ready (you do have a "In the case of emergency, drill HERE" sticker that when followed shatters the HDD's platters right? Easy access to RAM modules (cold attack) is a bonus.
b) Identify used services. Usually a full portscan follows that identifies the used ports. This is the first trap we set up. Following this guide, ALL privileged ports are now under constant check. A portscan (even so called "sophisticated" portscans, using actual TCP handshakes (not trying to trick the system into responding)) WILL hit our trap if it's destined for privileged ports (probability it is:infinite).
c) In the case of true "state sponsored hackers" I personally expect them NOT to perform portscans. They should have been at one of my classes so far (if you are a state sponsored hacker and have missed my training classes, please contact your supervisor for a re-schedule) and should have known by now than when attacking, you NEVER alert the victim. And yes, I am talking to you United States Army Information Systems Control (now integrated into NETCOM). How many times have you seen a cat running around screaming "ARE THERE ANY MICE HERE??????" It crawls silently next to its victim, then attacks. The same strategy should be implemented here: Perform a legitimate connection to a used port, and see what information you can gather from it. If it spews out that the webserver used is a known buggy version, then find the exploit for it and attack it. It's not rocket science. This is were security by obscurity comes in handy. If you don't allow the attacker to understand the version of a software used, then the attacker is more likely to screw up by trying to use known large scale exploitable bugs (bugs that should be corrected by your updates long ago. You are updating without delays right?). Suricata will fire up an alert on old exploits, since I will not bother with the latest and greatest exploits. If my upstream distribution patches the flaw within 8 hours of it being public, and ET publishes a rule update at the end of the day, I'll take my upstream distribution over ET any time. What I'm trying to say here is, there is no point in trying to keep up with exploits using rules, you should instead keep up with exploits using server admin best practices. 99.999% of so called "industry leaders" have absolutely NO idea when the last time their servers were updated. I've seen SSL certificates being expired for a WHOLE YEAR being used by a so called "leading web hosting provider", and I believe my, and everybody's, conclusion is that if you did not see that your SSL certificate has been expired for the past year, there is at least a 1 year gap between the last time you checked and updated the server and now. 1 year of NOT patching a system is the difference between a compromise and not.
d) The attacker has successfully exploited a known flaw and is now on your system. This is were inward facing interfaces come in handy. If a webserver starts scanning your firewall for open ports, then this is a clear sign of a compromised system. If that same webserver starts sending out reflection attacks out of nowhere, it too is a clear sign of a compromise. Ideally in this case the IDS/IPS will block the server. Having a few angry customers is better if you handle your compromise transparently. An announcement somewhere along the lines of "We have detected signs of a system compromise and have to shut down a system pending further investigation" will go a long way instead of, I don't know, SITTING ON A COMPROMISE FOR 2 MONTHS? coughebaycough pardon my Vogons. Seeing your webserver ending up on your IDS/IPS's banned hosts list is your hint to pick up the phone, give me a call and say "hello, are you offering forensic analysis services? Yes, we have had a system compromise. Ok I'll see you on monday". You should never ask how much a forensic analysis of a compromised host costs. It's a case of a "if you have to ask how much it costs, you cannot afford it".Summary because I just previewed the post and my $deity I write long posts :p.
Use the IDS/IPS to detect and prevent network attacks. Use server best practices to STOP host attacks. If you are using snort/suricata for a home network, most of these points are invalid, since the only way in is:
1)browser bug
2)email client bug
3)irc/messenger/voip client bug
4)"any service that performs connections with a remote network" bugTrust me, if I can't see it, I can't shoot at it. Light bending around planets to look behind them (not the keep a google tab open type, if I can't type its name out of my head, I don't bother) does not apply in network/system security.
Ideally the bugs should be handled by your upstream (commercial company,beer drunk open source developers, depends on the case) swiftly and your exposure will be minimized. This is where common sense comes in. I will never delay updating a critical software to a newer version that fixes a bug because I don't like the newer version's interface. A little hint here: based on my professional experience, updating a system first, THEN checking what/if-something doesn't work is the best way to maintain a system. This also depends on the distro you are using. If you are using distro X because "ZOMGWTFBBL it uses the latest version of text editor nobody uses!!!ZOMGWTFLOLBEES" then ditch it and start using distros created before the other distro's creators were even born. In other words, if you are using a debian derivative, STOP as soon as possible (yesterday would be nice) and start using debian. Use the money you saved from unscheduled downtimes, things breaking right and left, and support financially debian. If I was a developer and was starving to death, I wouldn't sit down and code the bug fix for you. If you offered me a piece of pizza, then sure as hell I would sit down. I would even fix that other bug that is affecting you but isn't so serious. -
Only ET IQRisk Blocklist picks up all of these IPs.
Would be great to be able to get access to this at a Snort consumer licence or similar price. I asked a few months back and was told ET got rid of it due to abuse by enterprise customers abusing the offer.
One thing that would be a reassuring addition for the dashboard widget would be a date/timestamp when the rules were last updated.
I'm very impressed with Suricata and this threads guidance, thank you.
-
@irj972:
Only ET IQRisk Blocklist picks up all of these IPs.
Would be great to be able to get access to this at a Snort consumer licence or similar price. I asked a few months back and was told ET got rid of it due to abuse by enterprise customers abusing the offer.
Would be good if they change their minds. More People need to speak up and ask them for a better package.
One thing that would be a reassuring addition for the dashboard widget would be a date/timestamp when the rules were last updated.
This is already part of my latest widget.
https://forum.pfsense.org/index.php?topic=78062.msg432181#msg432181
-
@Hollander:
This is what I am struggling with:
1. You said in one thread (I think it was your previous thread about how to set up Snort) said to ditch Snort for Suricata;
2. Some Snort rules (some 600 I believe Bill said) Suricata can not interpret.
3. Your current instructions (this thread) only deal with the ET-rules, not with the Snort rules.
4. I take it ET doesn't completely overlap the Snort (paid subscription, which is what I have) rules, so there is/might be value in using both the Suricata- and Snort rules in Suricata.Suricata will not load the Snort Shared Object Rules. You can disable all of the Snort SO Categories for each Interface in the Categories Tab, or just leave it as is, and Suricata will just skip the Snort SO Categories. It will still load the remainder of the Snort Categories. You can refer to the previous Snort Recommended Disable List, or just disable as you see false positives.
What you don't want to do is run both Snort and Suricata in Blocking Mode at the same time which will cause issues. If you want to run both, only one can be in Blocking Mode.
-
@jflsakfja:
@Hollander:
1)The only reason suricata is better than snort is that snort's inline has been abandoned and the developersA joy to read once again, JFL ;D
Armed with this knowledge, and the reply from BB right above about which Snort rules to disable (thanks again BB :P ) I reserved two hours to start disabling and enabling. Unfortunately, it seems Bill still didn't have time to make the enabling all and then disabling some-routine slightly more comfortable. He announced he will do that, so I will have to wait for that. Because the current way of working will require me to weeks of from work :-X
(1. Enable all -> 2. start to disable the exceptions -> 2a. disable first exception, it will not be disabled but will have the bright red cross -> wait for pfSense to process and respond back -> 2b. click the same cross once again & wait for pfSense to respond after which it will have the desired light yellow cross. That is not doable in a few hours with all the many rules I will have to manually disable).
-
I didnt know Suricata offered inline detection and blocking in Pfsense?
For what I know it works more or less the same way as Snort does (not inline)
Enlighten me pls :)
@jflsakfja:
@Hollander:
1)The only reason suricata is better than snort is that snort's inline has been abandoned and the developers moved to suricata, so if you want an IPS then you have to use suricata. We are talking about the general majority of users out there, there are other differences, but most users will never notice them. There is NO other software out there that matches its capabilities right now, pay,free,donated-to-us-by-aliens. It is actually the top of the line, absolute best IDS/IPS available. It bloody hell should be the best, it was funded directly by the U.S. government, I'm sure those guys want to get their money's worth.2)As far as the rules that suricata cannot interpret you only have these 2 choices:a)don't use them or b)manually rewrite the rule so that it is accepted by suricata.
3)This thread is about suricata, so there is no point in dealing with the snort rules, since those rules are not used. My opinion as to whether they are updated, or not, is my opinion only and does not reflect the opinion of The Company, etc… etc...
4)If you followed the guide to the letter and actually set up your suricata as suggested, you should be seeing about 90% banned hosts from custom rules (simple port scanning rules that detect port scans even the ET rules miss). Do you want to focus on the 1% that is banned hosts from the snort rules (assuming you are using snort with suricata, and filtered out the not working rules), the 9% from ET rules, or the 90% from custom rules? ;)
The trick to setting up an IDS/IPS is thinking like an attacker. What would you do to compromise an unknown system? These are the steps (most) attackers worth their salt follow (note: Does NOT refer to Chinese and Korean so called "state sponsored hackers", I don't have the time to deal with script kiddies):
a) Identify who the system belongs to. If it belongs to a letter soup agency, have the drill ready (you do have a "In the case of emergency, drill HERE" sticker that when followed shatters the HDD's platters right? Easy access to RAM modules (cold attack) is a bonus.
b) Identify used services. Usually a full portscan follows that identifies the used ports. This is the first trap we set up. Following this guide, ALL privileged ports are now under constant check. A portscan (even so called "sophisticated" portscans, using actual TCP handshakes (not trying to trick the system into responding)) WILL hit our trap if it's destined for privileged ports (probability it is:infinite).
c) In the case of true "state sponsored hackers" I personally expect them NOT to perform portscans. They should have been at one of my classes so far (if you are a state sponsored hacker and have missed my training classes, please contact your supervisor for a re-schedule) and should have known by now than when attacking, you NEVER alert the victim. And yes, I am talking to you United States Army Information Systems Control (now integrated into NETCOM). How many times have you seen a cat running around screaming "ARE THERE ANY MICE HERE??????" It crawls silently next to its victim, then attacks. The same strategy should be implemented here: Perform a legitimate connection to a used port, and see what information you can gather from it. If it spews out that the webserver used is a known buggy version, then find the exploit for it and attack it. It's not rocket science. This is were security by obscurity comes in handy. If you don't allow the attacker to understand the version of a software used, then the attacker is more likely to screw up by trying to use known large scale exploitable bugs (bugs that should be corrected by your updates long ago. You are updating without delays right?). Suricata will fire up an alert on old exploits, since I will not bother with the latest and greatest exploits. If my upstream distribution patches the flaw within 8 hours of it being public, and ET publishes a rule update at the end of the day, I'll take my upstream distribution over ET any time. What I'm trying to say here is, there is no point in trying to keep up with exploits using rules, you should instead keep up with exploits using server admin best practices. 99.999% of so called "industry leaders" have absolutely NO idea when the last time their servers were updated. I've seen SSL certificates being expired for a WHOLE YEAR being used by a so called "leading web hosting provider", and I believe my, and everybody's, conclusion is that if you did not see that your SSL certificate has been expired for the past year, there is at least a 1 year gap between the last time you checked and updated the server and now. 1 year of NOT patching a system is the difference between a compromise and not.
d) The attacker has successfully exploited a known flaw and is now on your system. This is were inward facing interfaces come in handy. If a webserver starts scanning your firewall for open ports, then this is a clear sign of a compromised system. If that same webserver starts sending out reflection attacks out of nowhere, it too is a clear sign of a compromise. Ideally in this case the IDS/IPS will block the server. Having a few angry customers is better if you handle your compromise transparently. An announcement somewhere along the lines of "We have detected signs of a system compromise and have to shut down a system pending further investigation" will go a long way instead of, I don't know, SITTING ON A COMPROMISE FOR 2 MONTHS? coughebaycough pardon my Vogons. Seeing your webserver ending up on your IDS/IPS's banned hosts list is your hint to pick up the phone, give me a call and say "hello, are you offering forensic analysis services? Yes, we have had a system compromise. Ok I'll see you on monday". You should never ask how much a forensic analysis of a compromised host costs. It's a case of a "if you have to ask how much it costs, you cannot afford it".Summary because I just previewed the post and my $deity I write long posts :p.
Use the IDS/IPS to detect and prevent network attacks. Use server best practices to STOP host attacks. If you are using snort/suricata for a home network, most of these points are invalid, since the only way in is:
1)browser bug
2)email client bug
3)irc/messenger/voip client bug
4)"any service that performs connections with a remote network" bugTrust me, if I can't see it, I can't shoot at it. Light bending around planets to look behind them (not the keep a google tab open type, if I can't type its name out of my head, I don't bother) does not apply in network/system security.
Ideally the bugs should be handled by your upstream (commercial company,beer drunk open source developers, depends on the case) swiftly and your exposure will be minimized. This is where common sense comes in. I will never delay updating a critical software to a newer version that fixes a bug because I don't like the newer version's interface. A little hint here: based on my professional experience, updating a system first, THEN checking what/if-something doesn't work is the best way to maintain a system. This also depends on the distro you are using. If you are using distro X because "ZOMGWTFBBL it uses the latest version of text editor nobody uses!!!ZOMGWTFLOLBEES" then ditch it and start using distros created before the other distro's creators were even born. In other words, if you are using a debian derivative, STOP as soon as possible (yesterday would be nice) and start using debian. Use the money you saved from unscheduled downtimes, things breaking right and left, and support financially debian. If I was a developer and was starving to death, I wouldn't sit down and code the bug fix for you. If you offered me a piece of pizza, then sure as hell I would sit down. I would even fix that other bug that is affecting you but isn't so serious. -
I didnt know Suricata offered inline detection and blocking in Pfsense?
For what I know it works more or less the same way as Snort does (not inline)
Enlighten me pls :)
Why do I have to always write like I'm presenting a case to the court? Seems like even a miss-placed comma gets the author burned on the stake in these forums.
I have mentioned in the past that snort inline has been abandoned. Says so on their page if you don't believe me.
As it stands right now, the exact moment of this post, yes suricata and snort more or less run the same way on pfsense. BUT: The inline capability is there, it just cannot be used yet as pfsense needs some changes to it. They are coming soon(ish). Would you want to tear down your snort system when the changes are made, reconfigure and troubleshoot suricata, then enable it and forget it, or just log in and tick a checkbox and be done with it?
I don't see why we need to split the development effort between snort and suricata. Snort doesn't offer anything more than suricata, but offers plenty less than suricata. A couple years time down the road, the cisco announcement that snort will no longer be maintained as a community project should wake up everybody. And yes, the announcement WILL come. The same way the ebay announcement came when I was running around screaming "change your ebay passwords!!!" and everybody was looking at me puzzled.
It's unfortunate that an open source project died (snort) but it's survival of the fittest. Now if we could only convince the (about) 43298 different open source text editors to die and merge in a single project, we will be colonizing Alpha Centauri by this time next year, and permission is hereby granted to quote me on that.
