Snort and Interface Enable/Disable
-
Hi-
I'm looking at Snort, and its acting a bit funky. If I restart the service manually, it says that the service is started and the log shows that it is inspecting packets correctly for my interfaces, but the interfaces show Snort as disabled and Barnyard2 Enabled. It looks as if somehow the Snort Services page is reporting the wrong status for the interface?
See attached images if you're curious what I'm seeing.
EDIT: Forgot something important. Running pfSense 2.0.1-RELEASE (i386), Snort 2.9.2.3 pkg v. 2.5.2
EDIT2: And somehow it starts out looking as if it is enabled, and then turns off. I'm not sure what is turning it off though, as nothing is showing up in the log! ideas?
-
Lets talk something even WEIRDER. Restarting Snort seems to also be resetting the active rules. I turn off iTunes user agent blocking (for example) and then the next day I come in and find that the rule is back on! Anyone else seen this sort of thing?
-
Yes….when I turn off a rule, and the rules get updated, then its back on and blocking...That way one can shut yourself out if youre unlucky.
It doesnt respect that you disabled the rule...
-
This seems like a real mixed thing, as it means you need to run some sort of script to update the rules back to your desired preferences after every rules update. I wonder how PulledPork deals with this.
It seems like a diff or just respecting your rules choices would be a better choice. If you lock yourself out of the box, that seems like it would require other ways to fix. Except that, as you mentioned in your post, the whitelists aren't being respected…
-
Exactly :) So in a coorporate environment, it has issues that needs urgent attention if colocating or remote sites uses snort as well.
-
supermule- got it in one. We can't use it here at our office for just this reason, right now. The integration just isn't there.
at least we can use it on the WAN side of things to block stuff from the outside. It seems a bit backward to need a separate Snort box to get stuff done.
-
Do you know who is developing Snort as a package?
-
No unfortunately not.
-
Well, thanks for the info and the assistance. Amazing speed on that speedtest.net report.
-
It rocks to have a nice internet throughput :)
When I come to think of it, I think its Ermal who integrates Snort in Pfsense !
-
I noticed that in the other thread! Looks like he's been quite active recently. This sort of thing really makes me want to get more involved in the developer side of things though.
-
I was I had the competence in FreeBSD to help out….but I havent :(
-
Supermule- not sure if this helps, but if you really want things running, some people have done some interesting stuff:
-
Thx :)
-
Supermule,
From looking at your screenshots snort IS running on the selected interface and barnyard2 is NOT running. You can tell by the X and the Checkmark that appear on these services. It shows the state of the service if you click the indicator not its running state.
As for rules enabling themselves or snort shutting down, I have found this to occur if you are changing setting in an interface Snort is listening on. However, we run Snort in production without any issue whatsoever. Its helped us find malware and protected us from many different attacks. Just be sure you suppress some of the warning if you use HTTP inspect and watch the alerts before turning on block. But as far as the package being broken, I have not found this to be true.
-
TJ-
I'd love to know how you got it running or your versions for everything, because I'm running with barnyard 2 active and my rulesets for the interfaces are constantly resetting. In addition, I mentioned in this thread about the icons. Even though Snort IS running, and is generating alerts, the icon for the interface is showing as a red 'X'. This isn't making any sense.
The big issue is the Emerging Threat Policy rules that keeps resetting, which would block Pandora, iTunes, and other non-risky programs. I'm currently using Snort on my WAN and is is blocking IPs and traffic generated from rules-matching for non-whitelist traffic, but I've now had the Emerging Threat rules reset on me twice.
-Josh
-
I am not the Snort package maintainer, but I have been working the last month on some fixes/updates for the Snort package and plan to post them soon for others to test in non-production setups. Here are some of the things I currently have working –
-
Auto-flowbit rule resolution like PulledPork implements. This automatically scans your enabled rules and turns on any non-active rules that are required to satisfy flowbit dependencies in your selected rules. If you don't want alerts from any of the auto-flowbit rules, you just add their SID to the Suppression List.
-
I fixed a bug in the http_inspect code that sets up the snort.conf file. This code was not properly reading and setting some values. Also in http_inspect, I've added some new settings that are part of the current Snort binary but were not available back when the original Snort package was created on pfSense. In my updates, you can individually select inspect depths for "server" and "client". Finally, I added a configurable "no_alerts" parameter that lets you utilize the http_inspect normalization without getting any alerts.
-
Fixed a gap in how the Snort package handles the critical classification.config and reference.config files during rule updates. When running both Emerging Threats and Snort VRT rules, these two files were not properly generated to contain information from both rule sets. As a result, barnyard2 logging lacked details for some alerts, and at least on my system, I saw some Snort failures and stops when it alerted on a rule but could not find the corresponding classification parameter in the classification.config file. The correct procedure when using both ET and VRT rules is to produce a combined classification.config file and reference.config file that has the entries from both rule sets.
-
The stream5 preprocessor also had a bug I fixed where it did not properly recognize the max_segs and max_queued bytes parameters. I also added a new configuration parameter for memcap in stream5. These have stopped all the log errors on big file downloads saying "…session pruned that exceeded max queued bytes...", etc.
-
I added configuration parameters for the Modbus and DNP3 SCADA preprocessors. These are used in industrial control systems.
-
I am working on implementing the Snort pre-defined IDS-Policies now available in the rules. Here is a link to the VRT blog with details on the policies and flowbit auto-resolution: http://blog.snort.org/2012/01/importance-of-pulledpork.html
I believe I also see the errors in the current package that are causing disabled rules to get enabled again. Basically the "disablesid" functionality is not getting called during rule updates. I will look at fixing that while I'm working on my other updates.
I want to touch base with the pfSense forum admins before I do, but my plan is to post my changes as files (with installation instructions) that some others can test in non-production environments. I do not know how to build the actual Snort package for installation, so all I can provide for now are the individual PHP files needed to implement my updates.
-
-
- The stream5 preprocessor also had a bug I fixed where it did not properly recognize the max_segs and max_queued bytes parameters. I also added a new configuration parameter for memcap in stream5. These have stopped all the log errors on big file downloads saying "…session pruned that exceeded max queued bytes...", etc.
Oh how I have been sooo waiting for this, :)
-
I believe I also see the errors in the current package that are causing disabled rules to get enabled again. Basically the "disablesid" functionality is not getting called during rule updates. I will look at fixing that while I'm working on my other updates.
This is the one thing pissing me off big time! :D
-
bmeeks-
Truly amazing. If you need any help testing anything or implementing any changes, let me know. We have 2 systems here, and getting this up and running really would simplify so many matters on our end. Hopefully the moderators pick this up!
-
Would be good if one could tag Ermal in this topic…. :)
-
Truly amazing. If you need any help testing anything or implementing any changes, let me know. We have 2 systems here, and getting this up and running really would simplify so many matters on our end. Hopefully the moderators pick this up!
Thanks. I use pfSense in a home network environment. I'm testing using a VMware virtual machine. So far I've tested only on 32-bit code. I suspect my changes will work fine on 64-bit, but I want to create a 64-bit VM to test with as well.
I have two pieces still in progress: (1) implementing the pre-defined IDS policies, and (2) a more robust enablesid and disablesid functionality that survives rule updates. I work on my changes during the evenings when I'm home. Hopefully I can have test files ready to post by this weekend.
-
DAMN NICE!!
-
bmeeks, thanks for your very promising work.
Would be good if one could tag Ermal in this topic….
It seems to me that Snort is a complex piece of software that requires one to have a certain amount hands-on-experience with IDS, in order to create a fully functional pfsense package.
Apparently a new binary will be nice, because 2.9.2.3 is going EoL soon.
Snort Version Released EOL
Snort 2.9.2.3 2012-05-16 2013-02-28
Snort 2.9.3.1 2012-08-12 TBD**
Snort 2.9.4.0 2012-11-30 TBD**
-
Snort is indeed a bit complex and somewhat sensitive to the settings in the snort.conf file (no big surprise there ;) )
I am quite close to being finished with my modifications and updates. Here is a screenshot of some of the new features that are working. I have corresponded with the Forum admins, and the recommended course of action for posting my code is via Github and then executing a Pull Request for Ermal or others on the pfSense team to evaluate. So I will not post modified PHP files directly to the forum, but will instead have them available on my fork of the Github pfsense-packages repository. I will post a link back here when things are posted.
Are there any particular settings for the HTTP_Inspect or Stream5 preprocessors that anyone would like to have GUI configuration parameters available for? Below you can see what will be available (if my submissions are accepted by the pfSense team).
New Preprocessors tab showing additional settings
-
Here is another screenshot of the new Categories tab showing the features I've added for IPS Policy selection and automatic flowbit resolution for selected rules.
New Categories tab (partial)
-
Last post for this evening…
For those of you that use the enablesid and disablesid feature in the Snort package, I've found some potential big bugs with its implementation in the current code base. In particular, it can't cope with multi-line rules. These are rules with a backslash ("") at the end of the line followed by a newline to signal continuation onto the next line. There are a handful of these multi-line rules in the rule sets from Emerging Threats and the Snort VRT. The current code just assumes any newline indicates the end of a rule when reading a rules file. Obviously with multi-line rules this is a mistake.
This bug, combined with the way the old code physically wrote back to disk a modified version of the rules file when enablesid or disablesid was used, risks writing a corrupt rules file when the original contains any multi-line rules.
My new procedure for handling enablesid and disablesid will fix this potential bug. I'm including the capability to "reset to defaults" any rules you may have previously disabled or enabled. The "reset to defaults" feature will return all the rules in a particular category to their default state, effectively wiping the slate clean of any modifications you previously made.
Of course I'm also adding the necessary hooks to ensure that any disablesid or enablesid customizations are preserved across rule updates... ;D
-
Is it possible that this could go into 2.0.3 as well??
Since current Snort package EOL??
Pls. push Jim and Ermal for this if you can!
-
Is it possible that this could go into 2.0.3 as well??
Since current Snort package EOL??
My changes are entirely contained within the handful of *.PHP files that comprise the GUI piece of the Snort package. They require no changes to the Snort binary nor to the underlying pfSense code in order to work, so assuming my changes are accepted by the pfSense team, a new Snort package (say v2.5.3) could be posted in the repository. It would work with 2.0, 2.0.1, 2.0.2 or even a 2.0.3 pfSense installation.
I don't have a Snort 2.9.4.2 binary (for FreeBSD) to test with, but I'm pretty confident my changes will work fine with the latest Snort binary. I've reviewed the 2.9.4.2 docs and see nothing that should be a problem in terms of configuration changes.
-
So basically we need to compile a new package for Pfsense?
-
So basically we need to compile a new package for Pfsense?
Yeah, that's essentially correct. Although, for someone with quite moderate "geek" skills, my updated files can be copied to an existing pfSense installation and used without recompiling anything. The Snort package itself on pfSense consists of essentially three parts: (1) the Snort binary module, (2) some support libraries for the binary module, and (3) a collection of PHP files that comprise the GUI. Really all the GUI for Snort does is peform the function of creating the snort.conf file. The snort.conf file is where all the configuration information lives for the Snort binary.
My updates and new features for Snort are really just additions and corrections to the PHP code that runs the GUI and builds the snort.conf file. So to use my updated files, you just copy them to the two sub-directories where the Snort PHP stuff resides and then everything just works. No reboot, and no compiling. Of course a rudimentary level of "geek skill" is required to copy over the files. This can be as simple as knowing how to use WinSCP, for instance… :)
The point of a formal package update would be to make the overall installation simple (just click the icon in the Packages tab). Now updating to the latest version of the Snort binary will require compiling the underlying package. However, a new binary is not necessary to start taking advantage of the changes I've made.
-
If the current package is seeing EOL anyway, then I suggest writing to Ermal or Jim to get them to compile it before it goes EOL.
-
If the current package is seeing EOL anyway, then I suggest writing to Ermal or Jim to get them to compile it before it goes EOL.
Pushing for an update to the latest 2.9.4.x Snort binary is my plan. I should finish my PHP code tweaks and have everything posted later today or by the end of the weekend to Github. I will then create the Pull Request to signal Ermal and others on the pfSense team to take a look at my changes.
-
Damn nice mate!! The world would be a fecking nice place to be if populated by folks like you :)
-
Here is a link to the Github Pull Request that has the code changes I am proposing for the Snort package.
https://github.com/bsdperimeter/pfsense-packages/pull/352
The Pull Request has a change log highlighting the major modifications. It also has the modified files with the changes highlighted. Should any of you want to test on your own system, contact me via PM and I can provide the files and instructions for using them. I recommend that any testing be done initially on non-production systems. I have been running the changes on two different pfSense VMs in VMware with no issues.
Bill
-
Here is a link to the Github Pull Request that has the code changes I am proposing for the Snort package.
This pic says its all :)
-
This pic says its all :)
Yeah, the Github summary shows each and every changed or deleted line. Sort of makes it look like more work than it really is.
I would like to recruit a few testers willing to give the changes a shake down in a non-production environment. The changes are super easy to back out if something causes a problem. To implement the changes, you just copy a few files to two folders after renaming the existing target files (to have as backups). No reboot necessary. To back out, just copy the renamed previously existing files (the backups) over top of the new files.
Bill
-
PM Sent
-
-
Seems to be working with a few caveats.
In testing the memcap setting, the most surefire way I know to trigger the "S5: Pruned 5 sessions from cache for memcap" error was to run a speedtest and max out my bandwidth. I have a 50/5 connection so your recommendation of a 32MB Stream5 memcap was still setting off that error. I raised it to 134217728 (128MB) since I have a lot of free memory and that error is now gone. :)
When I go into the Rules tab, instead of getting the Rules page, I seem to be getting the Github webpage for the Rules code but the web address is still showing as 192.168.1.1:443/snort/snort_rules.php?id=0Redownloaded from GIT and overwrote again and its working now.
Did I copy the file wrong?I also noticed that when I went to Update Rules, it completed successfully but gave an error at the bottom of the screen complaining of a line 9xx in snort.inc. I copied the line to post here but in my excitement of trying to see if Stream5 was fixed I copied some text again and it got overwritten. I've gone back to Update Rules again but I can not reproduce the error that I saw initially, might have to wait for a newer rule update that I can download.
Enabling an interface kicks off this error but System Logs show Initialization Complete and the Interface in fact does Enable.
Warning: is_dir() expects parameter 1 to be string, array given in /usr/local/pkg/snort/snort.inc on line 924 Warning: is_file() expects parameter 1 to be string, array given in /usr/local/pkg/snort/snort.inc on line 926 Warning: is_dir() expects parameter 1 to be string, array given in /usr/local/pkg/snort/snort.inc on line 924 Warning: is_file() expects parameter 1 to be string, array given in /usr/local/pkg/snort/snort.inc on line 926 Warning: Cannot modify header information - headers already sent by (output started at /usr/local/pkg/snort/snort.inc:924) in /usr/local/www/snort/snort_interfaces.php on line 129 Warning: Cannot modify header information - headers already sent by (output started at /usr/local/pkg/snort/snort.inc:924) in /usr/local/www/snort/snort_interfaces.php on line 130 Warning: Cannot modify header information - headers already sent by (output started at /usr/local/pkg/snort/snort.inc:924) in /usr/local/www/snort/snort_interfaces.php on line 131 Warning: Cannot modify header information - headers already sent by (output started at /usr/local/pkg/snort/snort.inc:924) in /usr/local/www/snort/snort_interfaces.php on line 132 Warning: Cannot modify header information - headers already sent by (output started at /usr/local/pkg/snort/snort.inc:924) in /usr/local/www/snort/snort_interfaces.php on line 133 Warning: Cannot modify header information - headers already sent by (output started at /usr/local/pkg/snort/snort.inc:924) in /usr/local/www/snort/snort_interfaces.php on line 136
