Problem with rules
-
Again, thank you for this answer.
At the time of the alert, I was downloading and updating some (free) software packages, installed on my windows 10 pc, so no malware or other problem here.
Looks like most windows downloads trigger the alert.Took your advice and added this to the suppress list (created an empty suppress list (didn't exist yet); assigned it to the interface; suppressed the alert, using the icons in the alert view - plus icon next to the red x icon):
#ET POLICY PE EXE or DLL Windows file download
suppress gen_id 1, sig_id 2000419Thanks for your time, effort and detailed responses.
-
The way you handled it is best. The key to understand is that rule does not mean that something "bad" is happening. It is simply a trigger that says some host is downloading an EXE or DLL file. As you said, not all (in fact, really just a handful) of such files are bad. Most are entirely a normal part of updating software from the web. But some folks, in say a Corporate environment with locked down machines where all software updates are handled from internal (as in LAN) sources, might want to know if an employee is trying to download some executable from the web instead of from the internal software distribution servers. That rule would fire for them.
It also has implications as a "flag setting rule" that other rule signatures depend on. That's where the flowbits part comes in. Flowbits are a super critical piece of an IDS/IPS to understand, but few admins take the time to investigate them. I suspect less than 20% of IDS admins can tell you what flowbits are or what they do. Failure to understand their role in the proper configuration and operation of the IDS/IPS can be a costly error!
Bill
-
Bill, I use SID Mgmt to disable rules because I can document save the settings to a file and document the reason for disabling if I like.
In your opinion, is it better to disable them in the UI? Suppress them?Also, I am using a brute-force approach to Snort: I am enabling all the categories and disabling the false-positives. This approach concerns me given the rules are updated every day.
When I start blocking rules, is administration of this likely to become overwhelming, having to constantly monitor the alerts and unblock clients blocked by false-positives?
Is there a better approach that is easier to maintain?
-
Bill, I use SID Mgmt to disable rules because I can document save the settings to a file and document the reason for disabling if I like.
In your opinion, is it better to disable them in the UI? Suppress them?Also, I am using a brute-force approach to Snort: I am enabling all the categories and disabling the false-positives. This approach concerns me given the rules are updated every day.
When I start blocking rules, is administration of this likely to become overwhelming, having to constantly monitor the alerts and unblock clients blocked by false-positives?
Is there a better approach that is easier to maintain?
Enabling all rules is generally a bad idea for a number of reasons. Chief among them are resource consumption on your firewall and lots of false positives to slog through. You should instead study and become familiar with each rule category and the types of rules within each. You want to run the rules that are pertinent to the assets you are protecting, and disable the rules that designed to protect things you don't have. For example, most home users have no public-facing web server, no public-facing mail server, no public-facing DNS server and no public-facing SQL Server. So why burden up your firewall by enabling all the rule categories designed to protect those assets that a home user does not have? Even many small business networks don't contain these kinds of assets because they use remote hosting services for mail and web services. You gain nothing by running the rules which detect an attempted compromise of a DNS server if you don't run a DNS server that faces the public Internet.
Another thing that many people do not realize is that tons of the rules from both Snort and Emerging Threats are actually default disabled. I think a lot of admins assume that all the rules in any given category will be "turned on" when they enable that category. That is not true. When you enable a category in the GUI, then only the enabled rules in that category are used. See, the rule authors will comment out (or disable) rules in a category for any number of reasons. Perhaps the rule is for older assets (say some exploit for Windows 98) that is not likely to be needed today. Maybe the rule is prone to false positives generically and really only applies to very specific setups. If you study through the rules in each category from Snort and ET, you will find quite a number of them disabled. You can of course force any of them to be enabled if they apply to your network. The ides is you know enough about your network, the hosts on it, and what the rules do so you can enable only the ones you need.
Running an IDS is not the same as running an anti-virus client. It requires detailed knowledge of networking, of your network hosts and the applications they run, and an understanding of how the rules work and what they are designed to protect. Running an IDS is not just "enable all the rules, set them to auto-update, and forget about it". That is a recipe for a large headache!
I suggest getting a Snort VRT subscription if you don't already have one, and then running with either the "Connectivity" or "Balanced" Snort IPS Policy enabled and nothing else. Those rules will protect you from most of the bad stuff while not generating a lot of false positives.
Bill
-
As always, Bill's advice isn't something you ignore…
To help me determine witch rules to use, I recently found and use a list, published by another user.
You can find it here: https://raw.githubusercontent.com/jflsakfja/suricata-rules/master/list.txtBill mentioned in another post:
<quote>@jflsakfja's list of Suricata rules can be a good starting place.</quote> -
Thanks Bill,
I've been having to track down 1:2000419 a couple of times a week recently. Seems something in YouTube keeps triggering this on the WAN. I guess Google is bouncing YouTube around all their sfo*.net servers. So far I've been tracking them down and suppressing and tracking by IP but I may just suppress the SID.Rick
-
Hi Bill,
I'm sorry I keep asking (probably stupid newbie) questions, but here I am again.
1. Flowbits: My understanding of the way suricata works.
If both flowbits and a specific rule set are enabled, there is a possibility (given "set" or "setx" flowbit option with the matching flowbit name) that additional rules are automatically enabled (that is what this topic is about - 1:2000419 ET POLICY PE EXE or DLL Windows file download).What happens if disablesid.conf is used on a specific rule or ruleset, do the flowbit rules still gets activated for this rule or ruleset?
If NOT, is it better to suppress rules instead of disabling them (and not use disablesid.conf at all)?
The down side of course (if I'm right) is that suppressed rules still require processing, thus impacting throughput, they just don't show up in the alert list.2. you mentioned in another topic enabling some rules on your WAN interface.
<quote>Just so I can see Snort doing something, I enabled the emerging-ciarmy and emerging-dshield rules on my WAN just because they frequently trigger</quote>
If I understand suricata right (using it in IPS mode), the incoming packet is evaluated by suricata before it is passed to the firewall.
I noticed that suricata catches a lot of alerts (block NOT yet enabled), using the above rule sets, however, I can always identify a matching block in the firewall log.The question, witch one is better:
- don't use these rule sets (also recommended by jflsakfja (see https://raw.githubusercontent.com/jflsakfja/suricata-rules/master/list.txt) -> DO NOT USE) and let the firewall handle this incoming (bad) traffic.
- use the rule sets, but keep the alert setting
- use the rule sets, and enable blocking in dropsid.conf.
Again, this question seems to have a performance impact; witch is better, handle (bad) incoming traffic with suricata or with the firewall?
Thanks again.
-
Enabling all rules is generally a bad idea for a number of reasons. Chief among them are resource consumption on your firewall and lots of false positives to slog through. You should instead study and become familiar with each rule category and the types of rules within each. You want to run the rules that are pertinent to the assets you are protecting, and disable the rules that designed to protect things you don't have.
Where can I find a good reference for the rule categories? I have looked at both the Snort and ET websites and have not found clear definitions.
For example, most home users have no public-facing web server, no public-facing mail server, no public-facing DNS server and no public-facing SQL Server. So why burden up your firewall by enabling all the rule categories designed to protect those assets that a home user does not have? Even many small business networks don't contain these kinds of assets because they use remote hosting services for mail and web services. You gain nothing by running the rules which detect an attempted compromise of a DNS server if you don't run a DNS server that faces the public Internet.
My firewall protects public facing web/ftp servers and is the portal for all traffic to the Internet from our Intranet.
Running an IDS is not the same as running an anti-virus client. It requires detailed knowledge of networking, of your network hosts and the applications they run, and an understanding of how the rules work and what they are designed to protect. Running an IDS is not just "enable all the rules, set them to auto-update, and forget about it". That is a recipe for a large headache!
I am not being haphazard in my approach. My knowledge of networking is extensive and my knowledge of our hosts and applications is complete–it should be since I wrote and configured nearly all of it. I have studied the workings of Snort pretty thoroughly and believe I have a pretty good understanding. But I don't have a thorough understanding of all of the rule groups, and given the thousands of individual rules, do not have the time to become intimate with all of them.
I suggest getting a Snort VRT subscription if you don't already have one, and then running with either the "Connectivity" or "Balanced" Snort IPS Policy enabled and nothing else. Those rules will protect you from most of the bad stuff while not generating a lot of false positives.
I was tempted to use the "Balanced" IPS policy but need to disable the blacklist rules since I use pfBlockerNG as well. In addition, this setting only applies to the VRT rules and not ET rules. These still have to be selected manually. Is there a list of categories you would suggest enabling that would give sufficient protection for the services we provide? All the recommendations I can find seem to be for home-based routers and not for a commercial web portal. Would you recommend the list mentioned above for my situation?
Steve
-
Enabling all rules is generally a bad idea for a number of reasons. Chief among them are resource consumption on your firewall and lots of false positives to slog through. You should instead study and become familiar with each rule category and the types of rules within each. You want to run the rules that are pertinent to the assets you are protecting, and disable the rules that designed to protect things you don't have.
Where can I find a good reference for the rule categories? I have looked at both the Snort and ET websites and have not found clear definitions.
For example, most home users have no public-facing web server, no public-facing mail server, no public-facing DNS server and no public-facing SQL Server. So why burden up your firewall by enabling all the rule categories designed to protect those assets that a home user does not have? Even many small business networks don't contain these kinds of assets because they use remote hosting services for mail and web services. You gain nothing by running the rules which detect an attempted compromise of a DNS server if you don't run a DNS server that faces the public Internet.
My firewall protects public facing web/ftp servers and is the portal for all traffic to the Internet from our Intranet.
Running an IDS is not the same as running an anti-virus client. It requires detailed knowledge of networking, of your network hosts and the applications they run, and an understanding of how the rules work and what they are designed to protect. Running an IDS is not just "enable all the rules, set them to auto-update, and forget about it". That is a recipe for a large headache!
I am not being haphazard in my approach. My knowledge of networking is extensive and my knowledge of our hosts and applications is complete–it should be since I wrote and configured nearly all of it. I have studied the workings of Snort pretty thoroughly and believe I have a pretty good understanding. But I don't have a thorough understanding of all of the rule groups, and given the thousands of individual rules, do not have the time to become intimate with all of them.
I suggest getting a Snort VRT subscription if you don't already have one, and then running with either the "Connectivity" or "Balanced" Snort IPS Policy enabled and nothing else. Those rules will protect you from most of the bad stuff while not generating a lot of false positives.
I was tempted to use the "Balanced" IPS policy but need to disable the blacklist rules since I use pfBlockerNG as well. In addition, this setting only applies to the VRT rules and not ET rules. These still have to be selected manually. Is there a list of categories you would suggest enabling that would give sufficient protection for the services we provide? All the recommendations I can find seem to be for home-based routers and not for a commercial web portal. Would you recommend the list mentioned above for my situation?
Steve
I don't mean to insult an IDS/IPS admin nor impune anyone's abilities. There are lots of questions asked here and generally the background, training and skill level of the poster is unknown. Sometimes those of us replying make erroneous assumptions, and for that I apologize in advance (and in hindsight if I offended with my original reply as I did not intend to offend).
My comment about "set and forget" was not meant to ridicule. I just probably let some frustration show through. I get lots of queries from folks where it is obvious from their questions and the information they provide about their setups that they think of Snort or Suricata as just another flavor of Symantec or Kasperksy AV. You install it, point it at your interfaces, enable the daily rules download and you are good to go.
There are no really good descriptions of the rules. In all honesty they appear to be a quasi-haphazard collection of responses to threats over the years. One thing you will notice if you start really examining them is they still contain signatures to detect threats that are likely not relevant today. That was a pet peeve of one of the former big posters in this sub-forum – the continued existence in the rule set of obviously outdated and in some cases even misguided rules. It's like new rules make it in, but old ones don't get regularly audited and cleaned out when no longer appropriate. At least that's my impression from the outside looking in. Maybe rules do get frequently retired and I just don't notice.
The rules are somewhat organized by the names of the category files. Suricata and Snort on pfSense simply display the filename of each category on the CATEGORIES tab. If I had public-facing web servers, then I would run the Snort web server rules. I can't remember the exact filename at the moment, but it has "server" and "web" in it. There is a similar category for Emerging Threats. Same thing for mail servers, DNS servers, etc. In terms of which specific rules to use in a category, that is going to require the boring job of reading the rules and determining what they actually trigger on. The message strings in each rule are a big help there.
-
I don't mean to insult an IDS/IPS admin nor impune anyone's abilities. There are lots of questions asked here and generally the background, training and skill level of the poster is unknown. Sometimes those of us replying make erroneous assumptions, and for that I apologize in advance (and in hindsight if I offended with my original reply as I did not intend to offend).
You didn't insult me, Bill, and I'm sorry for giving that impression. I really do appreciate the help. I was tired and frustrated with searching for information on what would seem an obvious question: what categories and rules are suggested for a fully robust IPS protecting public web, ftp, and mail servers and also acting as the portal for all of a company's access to the Internet.
I confess I'm trying to cheat, here. As IT Director, this is just one of many, many enhancements I am trying to make to our networking architecture so I just want to learn, if I can, without covering old ground, what rule categories would be suggested for my commercial network IPS.
Is there a list of categories you would suggest enabling that would give sufficient protection for the services we provide? All the recommendations I can find seem to be for home-based routers and not for a commercial web portal. Would you recommend starting with https://raw.githubusercontent.com/jflsakfja/suricata-rules/master/list.txt for my situation?
Thanks, again.
Steve -
Hi Bill,
I'm sorry I keep asking (probably stupid newbie) questions, but here I am again.
1. Flowbits: My understanding of the way suricata works.
If both flowbits and a specific rule set are enabled, there is a possibility (given "set" or "setx" flowbit option with the matching flowbit name) that additional rules are automatically enabled (that is what this topic is about - 1:2000419 ET POLICY PE EXE or DLL Windows file download).What happens if disablesid.conf is used on a specific rule or ruleset, do the flowbit rules still gets activated for this rule or ruleset?
If NOT, is it better to suppress rules instead of disabling them (and not use disablesid.conf at all)?
The down side of course (if I'm right) is that suppressed rules still require processing, thus impacting throughput, they just don't show up in the alert list.2. you mentioned in another topic enabling some rules on your WAN interface.
<quote>Just so I can see Snort doing something, I enabled the emerging-ciarmy and emerging-dshield rules on my WAN just because they frequently trigger</quote>
If I understand suricata right (using it in IPS mode), the incoming packet is evaluated by suricata before it is passed to the firewall.
I noticed that suricata catches a lot of alerts (block NOT yet enabled), using the above rule sets, however, I can always identify a matching block in the firewall log.The question, witch one is better:
- don't use these rule sets (also recommended by jflsakfja (see https://raw.githubusercontent.com/jflsakfja/suricata-rules/master/list.txt) -> DO NOT USE) and let the firewall handle this incoming (bad) traffic.
- use the rule sets, but keep the alert setting
- use the rule sets, and enable blocking in dropsid.conf.
Again, this question seems to have a performance impact; witch is better, handle (bad) incoming traffic with suricata or with the firewall?
Thanks again.
For Flowbits –
When the "use auto-flowbits" option is enabled then flowbit dependency rules will always get enabled because that logic runs after the SID MGMT stuff runs. The only exception to this is when the rule is disabled within the GUI using the red X icons. That stores the GID:SID of that rule in the config.xml file that holds the firewall configuration for Suricata. The very last processing that occurs as the suricata.rules file is written to disk is to check the config.xml file for disabled and enabled rules. Any info found here will override previous settings from SID MGMT.
I'm not sure I understand you question about IPS and the firewall completely, but here is the answer to what I think the question is –
In IPS mode with no rules set to DROP, Suricata will see traffic as it leaves the NIC and heads to the rest of pfSense. Your firewall by default will drop all unsolicited incoming traffic (the default rules do this until you specifically add permitted inbound stuff). So yes, it would be entirely normal to see Suricata alert on something and have the firewall drop the traffic via the default rule to drop unsolicited inbound traffic.
The rulesets question --
I did not go look up the old thread you mentioned, but there has been a sort of ongoing argument among users here about whitelisting versus blacklisting in general. pfBlockerNG is usually at the center of this, but Suricata and Snort get involved as well. In my personal view, with maybe the exception of a mail server, I don't see the point in loading up the firewall with hundreds of thousands of IP addresses to be blocked individually. For example, there are folks who want to put dozens and dozens of IP lists with blacklisted countries in them (China, Russia, etc.) in pfBlockerNG. Other folks say why do that? Just use the default drop rules of the firewall to block those countries. As for why I put those two rule sets on my WAN, it was purely to generate some alerts so I can see them on the ALERTS tab. There is absolutely zero security benefit to it. My personal firewall protects a small home network. Those lists generate probably a dozen alerts per hour tops on my firewall.
In terms of which is better at blocking (Suricata or the firewall), that depends on the threat. The firewall is great at recognizing that someone is attempting to access a web server at port 80 on your firewall and you don't have a "permit" rule for that. So the firewall blocks it. Same for some list of IP addresses, but my comment above about the efficacy of whitelisting over blacklisting may apply. What Suricata is great at that the firewall could never do is inspect the content of a packet to identify malicious intent. The firewall just looks at IP address and port to make a decision. Suricata can look at the actual content of the packet payload, so it can identify many more threats and attacks.
Bill
-
I don't mean to insult an IDS/IPS admin nor impune anyone's abilities. There are lots of questions asked here and generally the background, training and skill level of the poster is unknown. Sometimes those of us replying make erroneous assumptions, and for that I apologize in advance (and in hindsight if I offended with my original reply as I did not intend to offend).
You didn't insult me, Bill, and I'm sorry for giving that impression. I really do appreciate the help. I was tired and frustrated with searching for information on what would seem an obvious question: what categories and rules are suggested for a fully robust IPS protecting public web, ftp, and mail servers and also acting as the portal for all of a company's access to the Internet.
I confess I'm trying to cheat, here. As IT Director, this is just one of many, many enhancements I am trying to make to our networking architecture so I just want to learn, if I can, without covering old ground, what rule categories would be suggested for my commercial network IPS.
Is there a list of categories you would suggest enabling that would give sufficient protection for the services we provide? All the recommendations I can find seem to be for home-based routers and not for a commercial web portal. Would you recommend starting with https://raw.githubusercontent.com/jflsakfja/suricata-rules/master/list.txt for my situation?
Thanks, again.
SteveHe has a very good set of rules and is an experienced Suricata user. I have not had contact with him in quite some time. I believe he lives in Greece and suffered some serious injuries in a motorcycle or automobile accident a couple of years ago (can't remember which). Prior to that he was very active on the forum here.
Bill