Snort whitelist IP's not working, what I my doing wrong?
-
Hello,
I am running 2 snort processes, one on LAN the other on WAN. Setup as bmeeks suggested with only a few rules on WAN (emerging-ciarmy.rules, emerging-rbn.rules) and "Use IPS policy=balanced" on LAN.
All works fine.
I was recently trying to retrieve podcasts from some websites such as the linux action show from Jupiter Broadcastings, and noticed the bash script that I use generated empty podcasts (0 bytes) all of a sudden. Not long after, I realized that Snort was the rootcause.
Basically, after several tests and confirming Snort was indeed the reason why it disnt work, I setup my pfsense box this way:
Created a new alias named "podcasters" where I entered the IP's being blocked by Snort. Not sure if entering IP's is the best way because if the originating IP changes, then my alias will no longer be valid… What should I do there? Should I use the actual RSS link such as
http://feeds.feedburner.com/BsdNowMp3?format=xmlCreated a new alias where I added all of my trusted aliases. Named this alias "trustedhosts"
In Snort interfaces (both LAN & WAN), I selected "trustedhosts" under "Pass List"
Restarted both interfaces
Doesnt work.
Upon relaunching the podcast script, I immediately see these alerts in Snort's logs:
10/04/14 14:32:23 2 TCP Attempted Information Leak 192.168.0.101 33433 69.16.232.40 80 1:2013028 ET POLICY curl User-Agent Outbound
10/04/14 14:31:18 2 TCP Attempted Information Leak 192.168.0.101 33433 93.184.215.248 80 1:2013028 ET POLICY curl User-Agent Outbound
10/04/14 14:31:10 2 TCP Attempted Information Leak 192.168.0.101 33433 204.16.245.39 80 1:2013028 ET POLICY curl User-Agent OutboundThen the "offending" IP's (204.16.245.39, 93.184.215.248 & 69.16.232.40) are added to the blocked IP's.
I tried with adding the alerts to the suppression rule, didnt work either.
Am I doing it the right way? Should I simply disable the rules regarding to "ET POLICY curl User-Agent Outbound"?
OK looking for the pro's advice!
-
This is going to be a difficult task to accomplish with a PASS LIST (or whitelist) because many of these services are provided by CDNs (content distribution networks) of one type or another where the IP address will frequently change. Snort cannot use a fully-qualified domain name (FQDN) for an Alias because real time lookup is just too expensive. You would not want traffic held up for potentially several seconds while Snort waited for DNS servers to respond, so Snort instead works only with IP addresses. That means if you have a host with a lot of potential IP addresses it can use, adding them all to a PASS LIST becomes a difficult task.
In your case, I would suggest either disabling the rule entirely (click the red X under the GID:SID column on the ALERTS tab), or suppressing the rule when the source IP address is 192.168.0.101. To do that, click the small plus (+) sign by the IP address in the SRC column of the ALERTS tab. After you do that, make sure that you then actually have the SUPPRESS LIST assigned to the interface. Check that on the INTERFACE SETTINGS tab for the interface.
If this is a home network, then I would suggest not using any of the ET POLICY rules. Those are more relevant in corporate environments where certain activity is more strictly controlled or forbidden. Another words, a typical office cubicle worker in a Fortune 500 company probably has no business running a curl script. But for a home network, that constraint probably does not exist.
Bill
-
Hey Bill,
worked like a champ (adding the suppression by source IP)!
2 questions regarding the snort process.. First of all, I have allocated 8GB of RAM to pfsense. I have HAVP, Squid, SG, pfBlocker and Snort running on LAN & WAN. Right now (at the moment Im writing this) its using 96% of 8170MB of RAM!!! Is this normal??
Before (yesterday) I was runnign the identical SAME config and packages on 4GB RAM and all was fine (although snort seemed a bit slow), I gradually added RAM (from 4GB to 6GB then to 8GB) and every time its using 90-98% RAM…
SO I conclude Snort is using all available RAM and adapts itself to how much RAM is availble? (Kinda like ZFS)?
Other question is stopping the LAN process. When I go in interfaces, and click the green arrow to stop the LAN process, it doesnt stop. At first I thought it took only a long time so I waited something like 10 minutes, frequently refreshing the interface tab. After that period of waiting, I tried to shut it down once more but it didnt work.
I went to the main pfsense portal page, clicked on the stop button next to snort, waited a few minutes then went back to snort interfaces only to see it down (the red X),
I am pretty sure it used to stop when I clicked on the Green arrow in the snort Interfaces...
What do you think?
-
Hi lpallard,
Try using this memory setting for Snort.
AC-BNFA-NQ
-
Try the memory setting recommended by BBcan177 and that should reduce Snort's memory consumption.
The icon on the INTERFACES tab will stop Snort. Based on the behavior you saw, and the amount of memory you saw Snort using, I'm going to guess you had somehow started duplicate Snort processes. That is very easy to do if you click the START icon more than once. When you have lots of rules enabled, Snort can take many, many seconds to start up. If you get impatient and click start again, it is possible to launch a duplicate Snort process. So instead of one Snort instance per interface, you wind up with two. Some folks who have been really impatient even wound up with three. That is one scenario that can cause very high memory consumption.
This problem of duplicate processes appears to more pronounced on boxes where lots and lots of rules are enabled with more limited CPU horsepower. This combination leads to very long Snort process start times, and users think nothing is happening and click the icon again. That can launch a duplicate process.
Bill
-
Thanks guys for the replies!
I guess that makes sense that re-clicking on the start icon would start another process..
What I dont get is the fact that right after rebooting the pfsense VM, I immediately see all RAM being used. There wouldn't be duplicate snort processes being started at boot time??
The performance policy "AC-BNFA" is apparently recommended for low end systems… Would 2 cores @3.1GHz with 8GB DDR3 RAM being considered low end with IPS policy selection set at "Balanced" and ALL "ET Open Rules" selected?
I am looking for optimal performance and no visible impact on the LAN clients. I am currently using AC as the memory performance policy but maybe I would get the exact same performance with much more reasonable RAM usage on AC-BNFA..?
-
@lpallard:
Thanks guys for the replies!
I guess that makes sense that re-clicking on the start icon would start another process..
What I dont get is the fact that right after rebooting the pfsense VM, I immediately see all RAM being used. There wouldn't be duplicate snort processes being started at boot time??
The performance policy "AC-BNFA" is apparently recommended for low end systems… Would 2 cores @3.1GHz with 8GB DDR3 RAM being considered low end with IPS policy selection set at "Balanced" and ALL "ET Open Rules" selected?
I am looking for optimal performance and no visible impact on the LAN clients. I am currently using AC as the memory performance policy but maybe I would get the exact same performance with much more reasonable RAM usage on AC-BNFA..?
Several folks have tested all the memory options, including some users here on the Forum, and there is essentially no difference in anything except memory consumption. There is no significant performance advantage in terms of speed with one setting versus any other. As you see, some of the other settings do nothing but eat memory like crazy. I think you will find all the "power users" here using AC-BNFA or AC-BNFA_NQ. I use the latter on my firewalls.
Bill
-
Hey Bill,
let me ask you a stupid question…. When I select IPS policy (for example Connectivity or balanced), do I need to select the ET rules that are located underneath the selection menu on the page as well ?? If choosing a IPS policy automatically loads a pre-configured set of ET rules, and checking on the rules manually, wouldnt that load the rules twice?
I have set both LAN & WAN to AC-BNFA-NQ but I still use close to 100% of 8GB RAM.... I even rebooted the firewall to make sure the memory policy was indeed used.
I am looking to connect a newly configured FTP server to a third interface (OPT1) and assign it some level of protection with snort (lets say connectivity) or something similar which will provide some degree of protection for the most significant attacks while not generating tons of false posivives which will prevent users from connecting to the FTP server.
With so much RAM being used for only LAN & WAN, I hardly think this willwork with OPT1 as well...
Any idea?
As usual, thanks!!! (Im learning a LOT with you hence why I keep asking stuff) :)
-
@lpallard:
Hey Bill,
let me ask you a stupid question…. When I select IPS policy (for example Connectivity or balanced), do I need to select the ET rules that are located underneath the selection menu on the page as well ?? If choosing a IPS policy automatically loads a pre-configured set of ET rules, and checking on the rules manually, wouldnt that load the rules twice?
I have set both LAN & WAN to AC-BNFA-NQ but I still use close to 100% of 8GB RAM.... I even rebooted the firewall to make sure the memory policy was indeed used.
I am looking to connect a newly configured FTP server to a third interface (OPT1) and assign it some level of protection with snort (lets say connectivity) or something similar which will provide some degree of protection for the most significant attacks while not generating tons of false posivives which will prevent users from connecting to the FTP server.
With so much RAM being used for only LAN & WAN, I hardly think this willwork with OPT1 as well...
Any idea?
As usual, thanks!!! (Im learning a LOT with you hence why I keep asking stuff) :)
The IPS Policy is a feature unique to the Snort VRT rules package. It uses only VRT rules. The VRT (Vulnerability Research Team) is the old Sourcefire team that maintains rules for Snort. ET (Emerging Threats) is a competitor of sorts that maintains is own separate set of rules that work with Snort. So you have something like Ford and Chevy to use a U.S. auto maker analogy. Both are automobiles and have the same basic features, but are sold through different outlets and have different available options. So it is with the VRT rules and the ET rules.
Quite some time ago the VRT, in an attempt to make managing rules easier for admins, started tagging different rules with a policy metadata keyword. The keyword is either "connectivity", "balanced" or "security". Not all rules are so tagged, but most are. Some even have multiple policy keywords assigned to them. When you enable this option, the Snort package looks through all the VRT rules (and only the VRT rules, since they are the only ones with the policy metadata included in them) and selects all rules that match the policy keyword you choose. This is why when you enable this option, all the VRT rule categories on the CATEGORIES tab are disabled. You are picking automatically based on the assigned policy metadata, so manual selection of VRT rules is disabled.
Since the ET rules are different from the VRT rules, and since they have no policy metadata within them, they remain available for selection. If you choose ET rules when using a VRT policy, then those ET rules will be added to the group of ET policy rules.
A lot of new users think they should select all the CATEGORIES on the tab. That is actually not true. The rules are divided into groupings that roughly match a function. For example, you have various e-mail server rules with "smtp" in the name, or database server rules with "sql" in the name. A lot of folks do not run their own e-mail or SQL database servers, so they have no reason to enable those rules. Doing so just wastes memory and processing power looking for exploits against servers you don't even have!
You should not be using nearly 100% of 8GB of RAM. Something is not right. I run the CONNECTIVITY policy and a few ET rule sets on my LAN, and several ET rule sets on my WAN along with a few on a DMZ. I use less than 2GB of RAM with those rules.
Bill
-
Now it started using a normal amount of RAM all of a sudden…. Rebooted the firewall yesterday, and Snort is just MUCH more responsive and uses 30% of 4GB which I believe is more reasonable...
Dont know why after I changed the RAM policy and rebooted the firewall it took 2 days to make a change on memory consumption but hey! Better late than never!
-
I just did some modifications on my LAN interface and activated quite a few more rules now that I can enjoy normal RAM usage thanks to your advice of using AC-BNFA-NQ…
Now starting LAN, I get these in the system logs:
snort[37401]: FATAL ERROR: /usr/pbi/snort-amd64/etc/snort/snort_34106_vtnet0/rules/snort.rules(10746) : pcre compile of "(obj.data|\object.data).+file\x3A\x2F\x2F127\x2E[0-9]" failed at offset 11 : missing opening brace after \o Oct 12 12:41:00 php: /snort/snort_interfaces.php: The command '/usr/local/bin/snort -R 34106 -D -q -l /var/log/snort/snort_vtnet034106 --pid-path /var/run --nolock-pidfile -G 34106 -c /usr/pbi/snort-amd64/etc/snort/snort_34106_vtnet0/snort.conf -i vtnet0' returned exit code '1', the output was ''Reading https://forum.pfsense.org/index.php?topic=74930.15 I understand there is maybe a typo in a regex for a rule… Can someone help me pinpoint the faulty rule preventing Snort from starting??
I tried searching for a rule with an ID of 10746 but couldnt find any..
Thanks!
**EDIT: Got it!I opened /usr/pbi/snort-amd64/etc/snort/snort_34106_vtnet0/rules/snort.rules and looked on line 10746 (this was the line number NOT the rule ID….) and found this:
alert tcp $EXTERNAL_NET $HTTP_PORTS -> $HOME_NET any (msg:"ET WEB_CLIENT Possible Microsoft Internet Explorer Dynamic Object Tag/URLMON Sniffing Cross Domain Information Disclosure Attempt"; flow:established,to_client; content:"obj"; nocase; content:"data"; nocase; within:10; content:"file|3A|//127."; nocase; within:20; pcre:"/(obj.data|\object.data).+file\x3A\x2F\x2F127\x2E[0-9]/si"; reference:url,tools.cisco.com/security/center/viewAlert.x?alertId=19873; reference:url,tools.cisco.com/security/center/viewAlert.x?alertId=20610; reference:url,www.microsoft.com/technet/security/bulletin/ms10-035.mspx; reference:url,www.coresecurity.com/content/internet-explorer-dynamic-object-tag; reference:cve,2010-0255; reference:url,doc.emergingthreats.net/2011695; classtype:attempted-user; sid:2011695; rev:4;)**So it appears that rule SID 2011695 from ET Web client is faulty. I went to the interface rules, deactivated the rule and the interface started just fine within seconds..
Bill, can you take a look and let me know if this is a system problem or there is really a problem with the REGEX in this rule?****
-
@lpallard:
Bill, can you take a look and let me know if this is a system problem or there is really a problem with the REGEX in this rule?
It is an actual problem within the rule text. I sent a notice to one of the Emerging Threats guys several months ago about it, got an acknowledgement that it was faulty and they would get it fixed. Someone seems to have dropped the ball after that.
There are many issues in all of the rule sets. There are outdated rules, rules with errors in them, etc. It's like most of the rule writers are always focused on adding new rules to the sets but not going through and periodically doing maintenance to remove outdated or just plain wrong rules. Or at least they don't seem to do the latter very often IMHO.
Bill
-
@lpallard:
Bill, can you take a look and let me know if this is a system problem or there is really a problem with the REGEX in this rule?
It is an actual problem within the rule text. I sent a notice to one of the Emerging Threats guys several months ago about it, got an acknowledgement that it was faulty and they would get it fixed. Someone seems to have dropped the ball after that.
There are many issues in all of the rule sets. There are outdated rules, rules with errors in them, etc. It's like most of the rule writers are always focused on adding new rules to the sets but not going through and periodically doing maintenance to remove outdated or just plain wrong rules. Or at least they don't seem to do the latter very often IMHO.
Bill
Hmmmm its too bad. Snort's effectiveness will likely suffer from this..
In the meantime, do you know a good way of retrieving (or listing) all rules that have been manually disabled from the alert's tab? (The ones I clicked "Force disable this rule and remove it from the current ruleset)
I have opted to disable rules but in the end I'd rather add the "offerder's" IPs to an alias rather than deactivating the rule alltogether. Browsing one category at the time and looking through thousands of rules to find the ones in pale yellow status (user disabled) seems ineffective and unreliable to me..
Cheers!
-
@lpallard:
@lpallard:
Bill, can you take a look and let me know if this is a system problem or there is really a problem with the REGEX in this rule?
It is an actual problem within the rule text. I sent a notice to one of the Emerging Threats guys several months ago about it, got an acknowledgement that it was faulty and they would get it fixed. Someone seems to have dropped the ball after that.
There are many issues in all of the rule sets. There are outdated rules, rules with errors in them, etc. It's like most of the rule writers are always focused on adding new rules to the sets but not going through and periodically doing maintenance to remove outdated or just plain wrong rules. Or at least they don't seem to do the latter very often IMHO.
Bill
Hmmmm its too bad. Snort's effectiveness will likely suffer from this..
In the meantime, do you know a good way of retrieving (or listing) all rules that have been manually disabled from the alert's tab? (The ones I clicked "Force disable this rule and remove it from the current ruleset)
I have opted to disable rules but in the end I'd rather add the "offerder's" IPs to an alias rather than deactivating the rule alltogether. Browsing one category at the time and looking through thousands of rules to find the ones in pale yellow status (user disabled) seems ineffective and unreliable to me..
Cheers!
I currently have a Pull Request for a Snort updated posted for review by the pfSense developers. Once they approve and merge the update, Snort will have the same automatic rule SID management features as Suricata. You can use text-based conf files compatible with Oinkmaster and PulledPork to enable or disable individual rules by category, GID:SID, or PCRE matching. Read up on the feature by looking at the Suricata threads in this sub-forum.
Once this is in the production Snort package, it will be the best way for you to manage Snort rules.
Bill
-
Allright Bill! I will look at Suricata's thread to familiarize myself with the concept, in the meantime, I will wait for Snort's package modifications to be implemented.
So far Snort is running VERY fine on my firewall and seems to be protecting my network fine
Big THANKS to you and all for helping and teaching how Snort works!
-
@lpallard:
Allright Bill! I will look at Suricata's thread to familiarize myself with the concept, in the meantime, I will wait for Snort's package modifications to be implemented.
So far Snort is running VERY fine on my firewall and seems to be protecting my network fine
Big THANKS to you and all for helping and teaching how Snort works!
Here is a good starting point for seeing what's in Suricata that will be coming soon to Snort: https://forum.pfsense.org/index.php?topic=80886.0
-
Im kinda going back to the title of this topic … It appears as I may not yet be a master at Snort.... :)
I have added a range of IPs in an alias called "externalservices", and regrouped this alias with others in a new alias called "trusted" (see screenshots)
Then I added this alias in a passlist under snort.
Then I added this passlist in my WAN interface.
Easy Peasy... So far...
The problem is that Snort still blocks the IPs in the range specified in the alias.
Why?? of course I restarted the interface, and even tried rebooting pfsense to no avail..
-
@lpallard:
Im kinda going back to the title of this topic … It appears as I may not yet be a master at Snort.... :)
I have added a range of IPs in an alias called "externalservices", and regrouped this alias with others in a new alias called "trusted" (see screenshots)
Then I added this alias in a passlist under snort.
Then I added this passlist in my WAN interface.
Easy Peasy... So far...
The problem is that Snort still blocks the IPs in the range specified in the alias.
Why?? of course I restarted the interface, and even tried rebooting pfsense to no avail..
Can you post screenshots of the following things?
1. The WAN SETTINGS tab scrolled down to the section where you set the PASS LIST.
2. Click the VIEW LIST button to the right of the PASS LIST drop-down and capture the contents of the pop-up window (or at least verify that all the IP addresses that should be on your PASS LIST show up there).
Bill
-
There you go..
-
Thanks for posting the screenshots. They seem to be OK. One thing that I just noticed, in your earlier screenshots you were saying the IPs were being blocked but you are showing alerts from the ALERTS tab. Are those IPs also showing up as actively blocked on the BLOCKED tab and by looking at the <snort2c>table under Diagnostics…Tables?
When an IP is in a PASS LIST, you will still get alerts from it, but those alerts should not result in a block. To see which IPs are currently being actively blocked, you look on the BLOCKED tab.
Bill</snort2c>
