Snort whitelist IP's not working, what I my doing wrong?
-
@lpallard:
Hey Bill,
let me ask you a stupid question…. When I select IPS policy (for example Connectivity or balanced), do I need to select the ET rules that are located underneath the selection menu on the page as well ?? If choosing a IPS policy automatically loads a pre-configured set of ET rules, and checking on the rules manually, wouldnt that load the rules twice?
I have set both LAN & WAN to AC-BNFA-NQ but I still use close to 100% of 8GB RAM.... I even rebooted the firewall to make sure the memory policy was indeed used.
I am looking to connect a newly configured FTP server to a third interface (OPT1) and assign it some level of protection with snort (lets say connectivity) or something similar which will provide some degree of protection for the most significant attacks while not generating tons of false posivives which will prevent users from connecting to the FTP server.
With so much RAM being used for only LAN & WAN, I hardly think this willwork with OPT1 as well...
Any idea?
As usual, thanks!!! (Im learning a LOT with you hence why I keep asking stuff) :)
The IPS Policy is a feature unique to the Snort VRT rules package. It uses only VRT rules. The VRT (Vulnerability Research Team) is the old Sourcefire team that maintains rules for Snort. ET (Emerging Threats) is a competitor of sorts that maintains is own separate set of rules that work with Snort. So you have something like Ford and Chevy to use a U.S. auto maker analogy. Both are automobiles and have the same basic features, but are sold through different outlets and have different available options. So it is with the VRT rules and the ET rules.
Quite some time ago the VRT, in an attempt to make managing rules easier for admins, started tagging different rules with a policy metadata keyword. The keyword is either "connectivity", "balanced" or "security". Not all rules are so tagged, but most are. Some even have multiple policy keywords assigned to them. When you enable this option, the Snort package looks through all the VRT rules (and only the VRT rules, since they are the only ones with the policy metadata included in them) and selects all rules that match the policy keyword you choose. This is why when you enable this option, all the VRT rule categories on the CATEGORIES tab are disabled. You are picking automatically based on the assigned policy metadata, so manual selection of VRT rules is disabled.
Since the ET rules are different from the VRT rules, and since they have no policy metadata within them, they remain available for selection. If you choose ET rules when using a VRT policy, then those ET rules will be added to the group of ET policy rules.
A lot of new users think they should select all the CATEGORIES on the tab. That is actually not true. The rules are divided into groupings that roughly match a function. For example, you have various e-mail server rules with "smtp" in the name, or database server rules with "sql" in the name. A lot of folks do not run their own e-mail or SQL database servers, so they have no reason to enable those rules. Doing so just wastes memory and processing power looking for exploits against servers you don't even have!
You should not be using nearly 100% of 8GB of RAM. Something is not right. I run the CONNECTIVITY policy and a few ET rule sets on my LAN, and several ET rule sets on my WAN along with a few on a DMZ. I use less than 2GB of RAM with those rules.
Bill
-
Now it started using a normal amount of RAM all of a sudden…. Rebooted the firewall yesterday, and Snort is just MUCH more responsive and uses 30% of 4GB which I believe is more reasonable...
Dont know why after I changed the RAM policy and rebooted the firewall it took 2 days to make a change on memory consumption but hey! Better late than never!
-
I just did some modifications on my LAN interface and activated quite a few more rules now that I can enjoy normal RAM usage thanks to your advice of using AC-BNFA-NQ…
Now starting LAN, I get these in the system logs:
snort[37401]: FATAL ERROR: /usr/pbi/snort-amd64/etc/snort/snort_34106_vtnet0/rules/snort.rules(10746) : pcre compile of "(obj.data|\object.data).+file\x3A\x2F\x2F127\x2E[0-9]" failed at offset 11 : missing opening brace after \o Oct 12 12:41:00 php: /snort/snort_interfaces.php: The command '/usr/local/bin/snort -R 34106 -D -q -l /var/log/snort/snort_vtnet034106 --pid-path /var/run --nolock-pidfile -G 34106 -c /usr/pbi/snort-amd64/etc/snort/snort_34106_vtnet0/snort.conf -i vtnet0' returned exit code '1', the output was ''Reading https://forum.pfsense.org/index.php?topic=74930.15 I understand there is maybe a typo in a regex for a rule… Can someone help me pinpoint the faulty rule preventing Snort from starting??
I tried searching for a rule with an ID of 10746 but couldnt find any..
Thanks!
**EDIT: Got it!I opened /usr/pbi/snort-amd64/etc/snort/snort_34106_vtnet0/rules/snort.rules and looked on line 10746 (this was the line number NOT the rule ID….) and found this:
alert tcp $EXTERNAL_NET $HTTP_PORTS -> $HOME_NET any (msg:"ET WEB_CLIENT Possible Microsoft Internet Explorer Dynamic Object Tag/URLMON Sniffing Cross Domain Information Disclosure Attempt"; flow:established,to_client; content:"obj"; nocase; content:"data"; nocase; within:10; content:"file|3A|//127."; nocase; within:20; pcre:"/(obj.data|\object.data).+file\x3A\x2F\x2F127\x2E[0-9]/si"; reference:url,tools.cisco.com/security/center/viewAlert.x?alertId=19873; reference:url,tools.cisco.com/security/center/viewAlert.x?alertId=20610; reference:url,www.microsoft.com/technet/security/bulletin/ms10-035.mspx; reference:url,www.coresecurity.com/content/internet-explorer-dynamic-object-tag; reference:cve,2010-0255; reference:url,doc.emergingthreats.net/2011695; classtype:attempted-user; sid:2011695; rev:4;)**So it appears that rule SID 2011695 from ET Web client is faulty. I went to the interface rules, deactivated the rule and the interface started just fine within seconds..
Bill, can you take a look and let me know if this is a system problem or there is really a problem with the REGEX in this rule?****
-
@lpallard:
Bill, can you take a look and let me know if this is a system problem or there is really a problem with the REGEX in this rule?
It is an actual problem within the rule text. I sent a notice to one of the Emerging Threats guys several months ago about it, got an acknowledgement that it was faulty and they would get it fixed. Someone seems to have dropped the ball after that.
There are many issues in all of the rule sets. There are outdated rules, rules with errors in them, etc. It's like most of the rule writers are always focused on adding new rules to the sets but not going through and periodically doing maintenance to remove outdated or just plain wrong rules. Or at least they don't seem to do the latter very often IMHO.
Bill
-
@lpallard:
Bill, can you take a look and let me know if this is a system problem or there is really a problem with the REGEX in this rule?
It is an actual problem within the rule text. I sent a notice to one of the Emerging Threats guys several months ago about it, got an acknowledgement that it was faulty and they would get it fixed. Someone seems to have dropped the ball after that.
There are many issues in all of the rule sets. There are outdated rules, rules with errors in them, etc. It's like most of the rule writers are always focused on adding new rules to the sets but not going through and periodically doing maintenance to remove outdated or just plain wrong rules. Or at least they don't seem to do the latter very often IMHO.
Bill
Hmmmm its too bad. Snort's effectiveness will likely suffer from this..
In the meantime, do you know a good way of retrieving (or listing) all rules that have been manually disabled from the alert's tab? (The ones I clicked "Force disable this rule and remove it from the current ruleset)
I have opted to disable rules but in the end I'd rather add the "offerder's" IPs to an alias rather than deactivating the rule alltogether. Browsing one category at the time and looking through thousands of rules to find the ones in pale yellow status (user disabled) seems ineffective and unreliable to me..
Cheers!
-
@lpallard:
@lpallard:
Bill, can you take a look and let me know if this is a system problem or there is really a problem with the REGEX in this rule?
It is an actual problem within the rule text. I sent a notice to one of the Emerging Threats guys several months ago about it, got an acknowledgement that it was faulty and they would get it fixed. Someone seems to have dropped the ball after that.
There are many issues in all of the rule sets. There are outdated rules, rules with errors in them, etc. It's like most of the rule writers are always focused on adding new rules to the sets but not going through and periodically doing maintenance to remove outdated or just plain wrong rules. Or at least they don't seem to do the latter very often IMHO.
Bill
Hmmmm its too bad. Snort's effectiveness will likely suffer from this..
In the meantime, do you know a good way of retrieving (or listing) all rules that have been manually disabled from the alert's tab? (The ones I clicked "Force disable this rule and remove it from the current ruleset)
I have opted to disable rules but in the end I'd rather add the "offerder's" IPs to an alias rather than deactivating the rule alltogether. Browsing one category at the time and looking through thousands of rules to find the ones in pale yellow status (user disabled) seems ineffective and unreliable to me..
Cheers!
I currently have a Pull Request for a Snort updated posted for review by the pfSense developers. Once they approve and merge the update, Snort will have the same automatic rule SID management features as Suricata. You can use text-based conf files compatible with Oinkmaster and PulledPork to enable or disable individual rules by category, GID:SID, or PCRE matching. Read up on the feature by looking at the Suricata threads in this sub-forum.
Once this is in the production Snort package, it will be the best way for you to manage Snort rules.
Bill
-
Allright Bill! I will look at Suricata's thread to familiarize myself with the concept, in the meantime, I will wait for Snort's package modifications to be implemented.
So far Snort is running VERY fine on my firewall and seems to be protecting my network fine
Big THANKS to you and all for helping and teaching how Snort works!
-
@lpallard:
Allright Bill! I will look at Suricata's thread to familiarize myself with the concept, in the meantime, I will wait for Snort's package modifications to be implemented.
So far Snort is running VERY fine on my firewall and seems to be protecting my network fine
Big THANKS to you and all for helping and teaching how Snort works!
Here is a good starting point for seeing what's in Suricata that will be coming soon to Snort: https://forum.pfsense.org/index.php?topic=80886.0
-
Im kinda going back to the title of this topic … It appears as I may not yet be a master at Snort.... :)
I have added a range of IPs in an alias called "externalservices", and regrouped this alias with others in a new alias called "trusted" (see screenshots)
Then I added this alias in a passlist under snort.
Then I added this passlist in my WAN interface.
Easy Peasy... So far...
The problem is that Snort still blocks the IPs in the range specified in the alias.
Why?? of course I restarted the interface, and even tried rebooting pfsense to no avail..
-
@lpallard:
Im kinda going back to the title of this topic … It appears as I may not yet be a master at Snort.... :)
I have added a range of IPs in an alias called "externalservices", and regrouped this alias with others in a new alias called "trusted" (see screenshots)
Then I added this alias in a passlist under snort.
Then I added this passlist in my WAN interface.
Easy Peasy... So far...
The problem is that Snort still blocks the IPs in the range specified in the alias.
Why?? of course I restarted the interface, and even tried rebooting pfsense to no avail..
Can you post screenshots of the following things?
1. The WAN SETTINGS tab scrolled down to the section where you set the PASS LIST.
2. Click the VIEW LIST button to the right of the PASS LIST drop-down and capture the contents of the pop-up window (or at least verify that all the IP addresses that should be on your PASS LIST show up there).
Bill
-
There you go..
-
Thanks for posting the screenshots. They seem to be OK. One thing that I just noticed, in your earlier screenshots you were saying the IPs were being blocked but you are showing alerts from the ALERTS tab. Are those IPs also showing up as actively blocked on the BLOCKED tab and by looking at the <snort2c>table under Diagnostics…Tables?
When an IP is in a PASS LIST, you will still get alerts from it, but those alerts should not result in a block. To see which IPs are currently being actively blocked, you look on the BLOCKED tab.
Bill</snort2c>
-
Hey Bill,
my pleasure for the screenshots.
Yes, although I was showing the alerts tab, they are indeed in the Blocked tab….
They are also in the snort2c table.
Yes, I have noticed if an IP generates an alarm and is in the passlist, only the alarm will be created and the IP will NOT get blocked. But it seems on my system this is ONLY true for my home_net and external_net variables....
Weird no?
-
@lpallard:
Yes, I have noticed if an IP generates an alarm and is in the passlist, only the alarm will be created and the IP will NOT get blocked. But it seems on my system this is ONLY true for my home_net and external_net variables….
This sentence is a bit confusing for me. Are you saying that only IP addresses in your HOME_NET and EXTERNAL_NET variables are never blocked? That is a kind of contradiction since EXTERNAL_NET on Snort is generally all addresses not in HOME_NET when using the default setting.
The default parameters for a PASS LIST will include within it all locally-attached IP subnets as well as DNS servers, gateways, and the WAN IP address. These would never be blocked. The ADDRESS alias you can add on the PASS LIST edit screen is where you specify other IP addresses that will not be blocked. You can only specify actual IP addresses in that Alias. You cannot use any FQDN type alias.
To the best of my knowledge, this feature is working for all other users.
Bill
-
Bill, I have a question regarding "(portscan) TCP Portsweep" and "(portscan) TCP Filtered Portsweep"..
To be quite honest, when I enabled the preproc for the portsweep detection, I thought this would be useful in blocking the IP's purposely performing portsweeps on my public IP (I had in mind attack servers, etc) but what ended up happening is that most (80%+) of the sites I visit are getting blovked by snort because of portsweeps.
Most of these sites are well known sites (google IP's, my ISP's portal, cnn.com, some popular forums, etc)… I must say most of the alarms are from Google's e100 servers (173.194.0.0/16, 74.125.0.0/16, 216.58.0.0/16).
So I am wondering if these alarms are "normal" and should be ignored because they originate from well known/reliable sources, but on the other hand why would forums, search engines and other public sites perform portsweeps???
I hope you can shed some light on this issue.. for now I have kept the preproc enabled and when a site doesnt load (or stops loading) I login to pfsense and remove the blocked host from the block list (PITA)..
Thanks!
-
@lpallard:
Bill, I have a question regarding "(portscan) TCP Portsweep" and "(portscan) TCP Filtered Portsweep"..
To be quite honest, when I enabled the preproc for the portsweep detection, I thought this would be useful in blocking the IP's purposely performing portsweeps on my public IP (I had in mind attack servers, etc) but what ended up happening is that most (80%+) of the sites I visit are getting blovked by snort because of portsweeps.
Most of these sites are well known sites (google IP's, my ISP's portal, cnn.com, some popular forums, etc)… I must say most of the alarms are from Google's e100 servers (173.194.0.0/16, 74.125.0.0/16, 216.58.0.0/16).
So I am wondering if these alarms are "normal" and should be ignored because they originate from well known/reliable sources, but on the other hand why would forums, search engines and other public sites perform portsweeps???
I hope you can shed some light on this issue.. for now I have kept the preproc enabled and when a site doesnt load (or stops loading) I login to pfsense and remove the blocked host from the block list (PITA)..
Thanks!
I don't have a precise answer, but my personal observations over the last year indicate the portscan preprocessor in Snort has become overly sensitive and "false-positives" frequently. I've had to set mine on the WAN to the lowest sensitivity and add a number of known safe and frequently visited sites to an Alias and then assign that Alias to the "Ignore Scanners" parameter in the portscan preprocessor configuration.
Things that open a number of HTTP connections like hitting a busy web site with a bunch of ad server lookups, for example, seem to trigger portscan alerts.
Bill
-
I also observed that most (now I am even starting to think maybe ALL) sites are generating portscans but why, that remains a mystery to me.. What I ended up doing was to lower the preproc setting sensitivity to low on the snort interface, then allow a "running-in" period where I try to visit as many sites as I usually visit and let my systems contact whatever web services they need, then when an alert is generated I add it to an alias that I assigned to Snort's interfaces…
May not be the best but it works. All I need now is a real attack from one of those "legit and trusted" sites and snort wont pick it up..
Perfection doesnt exist I guess...
This page also helped me a lot:
http://manual.snort.org/node85.html
Thanks Bill for your help once again!
