Snort 2.9.4.6 Pkg v 2.5.9

[daniela]

Thank you a lot everyone, I really appreciate.

For anyone else who may be interested or have a similar setup, Snort with lowmemory and "connectivity" inbuilt set of rules is working and being stable since 10+ hours with no perceivable slowdown even at full "speed" (if one may use the word, eh). Performance impact is not too bad (about 10% on memory and a few % on CPU, and less than a handful of extra processes) and I gave it 10MB space which is advised space for nanoBSD (may be one could increase it a little bit). Worst memory load is 57% and it's averaging about 35%, it won't melt. I decided to let it be for a while and whatever it filters and is not a false positive, I am adding to custom firewall rules. That way when I get rid of it, at least I have both learned something and improved the network a little. I got a process termination for lack of swap space, but since it did not keep happening, I think it's too amazing a miracle to tinker with, and kept my hands off it. I also realize it'd be better to reboot when doing extensive configuration changes, oh well. Also, I confirm the firewall kept working alright while enabling snort (which took about 1min per interface and I made sure to hammer the firewall).

I'll get some desktop from some closet and try Snort there, I'll activate it a couple hours a day while I am around and see what happens. I am realizing that it is not only necessary to learn Snort, but also to learn about network use of the users and what they need and use and what they don't. The users, however, seem to all have read everything about the bastard operator from hell and so don't even think of discussing needs and wishes, they also lie when asked! I am only trying to be helpful. I can't do any harm after all, I already have blocked Facebook years ago. :-D

I will move to 2.0.3, or to current version, as soon as downtime is available. I confirm to anyone on 2.0.1 that it sort of works.

[Supermule]

When I disable this:

Enable DCE/RPC2 Detection The DCE/RPC preprocessor detects and decodes SMB and DCE/RPC traffic. Default is Checked.

Then Snort wont start. I cant find any rules that is associated with this preproc. at all.

[bmeeks]

@Supermule:

When I disable this:

Enable DCE/RPC2 Detection The DCE/RPC preprocessor detects and decodes SMB and DCE/RPC traffic. Default is Checked.

Then Snort wont start. I cant find any rules that is associated with this preproc. at all.

There is a rule option somewhere that is dependent on the preprocessor.  In the system log there should be a message with these words "snort[81145]: FATAL ERROR: /usr/pbi/snort-i386/etc/snort/snort_64703_em0/rules/snort.rules(6050) Unknown rule option: 'dce_iface'." .  Snort rules can contain all manner of special rule options, and many of these options depend on certain preprocessors being enabled.  In this case, the rule option is "dce_iface".  That's why it is usually best to just run all the preprocessors except maybe Sensitive Data (SDF).  You never know when a rule update will suddenly enable a formerly disabled rule or include a new rule with the rule option.  If you have the preprocessor disabled that some new rule depends on, then Snort will fail to start after the rule update.

If you want to disable the preprocessor anyway, then at the top of the Preprocessors tab is a check box that says "Auto-Disable Text Rules".  If you check that option, then Snort will automatically disable any text rules that contain a rule option dependent upon the preprocessor you disabled.  Using your example, if you check that box you should see this warning similar to this in the system log when Snort starts –

 [Snort] Warning: auto-disabled 97 rules due to disabled preprocessor dependencies.

It's telling you that it automatically disabled 97 rules that contained options dependent upon the DCE_RPC preprocessor.  For this example, I just used the IPS Policy - Security with no Emerging Threats rules.  If you have a different set of rules enabled, you will likely see a number different from 97.

I put a View button on the Preprocessors tab that is supposed to let you open and view the disabled rules, but I just found out it does not appear to be working.  I will have to see why.  In the meantime, you will find a text log file of the auto-disabled rules in /var/log/snort.  The file will be named with the interface to make it easy to spot.

Bill

[Supermule]

Thanks Bill!!

[bmeeks]

@Supermule:

Thanks Bill!!

You're welcome.  And by the way, I found the problem with the View button not working.  A needed piece of JavaScript code got left out of the PHP page file during the last Snort package update.  I've fixed it in my base code, but I will just wait until the next scheduled update to push the fix out to the production package.  I'm really close to having the next update ready to go anyway.

Bill

[shinzo]

In 2.1 RC1.  I notice that the block list gets cleared everytime i save a setting in lets say squid.  Also if i update the firewall rules on the server, it clears the list as well.

[bmeeks]

@shinzo:

In 2.1 RC1.  I notice that the block list gets cleared everytime i save a setting in lets say squid.  Also if i update the firewall rules on the server, it clears the list as well.

Unfortunately this is something that is outside the direct control of the Snort package.  The pfSense core code clears all the packet filter tables when certain key events transpire.  The Snort block table is just a victim of this behavior.  Snort does not have its own independent block table.  It just inserts IP addresses into the packet filter that it wants blocked.

Bill

[shinzo]

Oh that's fine then.  Just wanted to make sure it wasn't a bug or anything.  As long as things get blocked  then i don't mind the table being flushed out.  I was wondering how the update was coming along and a eta on it, thanks.

[bmeeks]

@shinzo:

I was wondering how the update was coming along and a eta on it, thanks.

The package code changes are done.  I'm doing testing now trying to flush out any little bugs.  The addition of multiple configuration engine support for some of the preprocessors resulted in quite a bit of code being added/edited.  The next version will have multiple configuration support for Frag3, Stream5 and HTTP_Inspect.

I have sort of been stalling while waiting to see if the Snort port in FreshPorts gets updated to the 2.5 code from 2.9.4.6.  I wanted to include that binary update as well.

Bill

[Gradius]

@bmeeks:

@Gradius:

Just want to say the old bug is back again, it bans my OWN IP after a bit a while just looking some normal websites.

Getting this:
(http_inspect) IIS UNICODE CODEPOINT ENCODING - 08/05/13-22:46:05
(portscan) TCP Portsweep - 08/05/13-22:48:52
(ssp_ssl) Invalid Client HELLO after Server HELLO Detected - 08/05/13-22:55:55

Is your WAN IP dynamic and frequently changing?  If so it might what is causing the problem.  Are you running 2.0.3 or 2.1 pfSense?

Bill

Yes, Internet here is 100% IP dynamic whatever I power on/off my xDSL modem.

2.1-BETA1 (i386)
built on Wed May 22 08:31:46 EDT 2013
FreeBSD 8.3-RELEASE-p8

[bmeeks]

@Gradius:

Yes, Internet here is 100% IP dynamic whatever I power on/off my xDSL modem.

2.1-BETA1 (i386)
built on Wed May 22 08:31:46 EDT 2013
FreeBSD 8.3-RELEASE-p8

Snort builds the whitelist during each startup sequence.  When the WAN IP changes, pfSense usually does a good job of restarting things.  When restarted, Snort will correctly detect the new WAN IP and modify the whitelist accordingly assuming WAN IP is checked in the whitelist config (that is the default if you do not change it).  Maybe in the newer 2.1 snapshots something is not working quite right with the auto-restart of packages.

A workaround would be to manually enter an Alias containing the IP subnet that your ISP routinely issues WAN IPs to you from.  Then add this Alias to a custom whitelist for the WAN interface.  That way no matter what IP in the block you happen to get, it will be whitelisted.  This is not ideal and really should only be used as a temp workaround.  Hopefully this problem will disappear as the 2.1 snapshots continue to be tweaked.  I can also take a look to see if there is anything that could be done within Snort itself to better detect a WAN IP change.

Bill

[pfSenseRocks]

I have IPS Policy ( i.e. Snort GPLv2 Community Rules + Emerging Threats rule set) enabled on the WAN. And, all rule set minus the Snort GPLv2 Community Rules + Emerging Threats rule set enabled on the LAN interface.

Should I see 2 snort processes in this configuration, i.e. one snort process per interface? If I have IPv4 and IPv6 enabled on both the interface should I expect to see 4 processes?
Thanks!

[bmeeks]

@pfSenseRocks:

I have IPS Policy ( i.e. Snort GPLv2 Community Rules + Emerging Threats rule set) enabled on the WAN. And, all rule set minus the Snort GPLv2 Community Rules + Emerging Threats rule set enabled on the LAN interface.

Should I see 2 snort processes in this configuration, i.e. one snort process per interface? If I have IPv4 and IPv6 enabled on both the interface should I expect to see 4 processes?
Thanks!

One Snort process per interface.  So in your case you should see two Snort processes.  There was an issue with the later 2.1 Snapshots where multiple Snort processes per interface were getting kicked off on reboots.  That was the result of some changes going on with the pfSense Snapshot code, though.  Nothing has changed in the Snort package for a while.

Bill

[pfSenseRocks]

Thanks Bill. There is certainly something wonky going on, on the latest 2.1 snapshots. I have reconfigured snort for just the WAN interface IPv4 (no IPv6). Further, I only have IPS Policy ( i.e. Snort GPLv2 Community Rules + Emerging Threats rule set) enabled on the WAN. I see four (4) snort processes consuming up to 90% of the 6GB RAM and over 60% of the 16GB swap space.

Anything I can do (provide logs, traces, additional information) to debug and resolve this issue?

[gogol]

@pfSenseRocks:

Anything I can do (provide logs, traces, additional information) to debug and resolve this issue?

You could read through this thread. I already made a note about this a few pages back  ;)

[pfSenseRocks]

Thank you for the workaround. I was offering up any help I can provide (since I have a 100% & consistent repro) to debug this issue and solve it rather than just working around it.

[bmeeks]

@pfSenseRocks:

Thank you for the workaround. I was offering up any help I can provide (since I have a 100% & consistent repro) to debug this issue and solve it rather than just working around it.

I have some VMs I can test in. I have a July 4th 2.1 Snapshot that does not exhibit this behavior.  I will "snapshot" that VM and then let it upgrade to the latest 2.1 RC snapshot and see what I can determine about the multiple Snort process starts.

I've been letting Snort cook for a while with no package updates for two reasons.  First to see how things were performing for users, and to see if the FreeBSD port got updated to the 2.5.x Snort binary.  I have a new version of the Snort package ready that implements multiple engine/server configurations for the FRAG3, STREAM5 and HTTP_INSPECT preprocessors.

Bill

[bmeeks]

@pfSenseRocks:

Thank you for the workaround. I was offering up any help I can provide (since I have a 100% & consistent repro) to debug this issue and solve it rather than just working around it.

pfSenseRocks:

I upgraded a test VM to the latest 2.1RC snapshot.  I could not reproduce the multiple processes problem.  I have Snort configured on two interfaces for the VM, and I only get two Snort processes.  Now I am using my new 2.6.0 package code in the VM.  I can try reverting a VM back to the current 2.5.9 package and try again.

Bill

[pfSenseRocks]

That is great news, Bill. Thanks for the update. Let me update to the latest snapshot as well and see if I can reproduce your success.

[pfSenseRocks]

Unfortunately, I still reproduce the problem. Usually occurs after snort restarts after downloading new rules.

[2.1-RC1][admin@sense.home]/root(1): ps -ax | grep snort
23405  ??  Ss    8:25.86 /usr/pbi/snort-amd64/bin/snort -R 56048 -E -q -l /var/log/snort/snort_em0_vlan1056048 –pid-path /var/run
24490  ??  SNLs  0:28.51 /usr/pbi/snort-amd64/bin/snort -R 56048 -D -q -l /var/log/snort/snort_em0_vlan1056048 --pid-path /var/run
45765  ??  SNs    0:29.51 /usr/pbi/snort-amd64/bin/snort -R 56048 -D -q -l /var/log/snort/snort_em0_vlan1056048 --pid-path /var/run
46524  ??  Ss    0:03.79 /usr/pbi/snort-amd64/bin/snort -R 40477 -D -q -l /var/log/snort/snort_em0_vlan1140477 --pid-path /var/run
47171  ??  SNs    0:03.70 /usr/pbi/snort-amd64/bin/snort -R 40477 -D -q -l /var/log/snort/snort_em0_vlan1140477 --pid-path /var/run
47645  ??  SNs    0:03.76 /usr/pbi/snort-amd64/bin/snort -R 40477 -D -q -l /var/log/snort/snort_em0_vlan1140477 --pid-path /var/run
52671  0  S+    0:00.00 grep snort

Version 2.1-RC1  (amd64)
built on Mon Aug 19 16:16:39 EDT 2013
FreeBSD 8.3-RELEASE-p9