Snort 2.9.4.6 Pkg v 2.5.9

fragged

No VLAN's. I seem to have mistaken VIP's and VLAN's in my post.

maex

Hi,

Got the same problem on an AMD fx6300 - 6core. snort ALWAYS starts 3 instances at the same time.
I think it should be easily reproducible when you enable the maximum of rules and preprocessors. That way snort needs masses of ram (around 2GB when loaded) and takes minutes to load (almost 4GB during loading).

r, max

bmeeks

@maex:

Hi,

Got the same problem on an AMD fx6300 - 6core. snort ALWAYS starts 3 instances at the same time.
I think it should be easily reproducible when you enable the maximum of rules and preprocessors. That way snort needs masses of ram (around 2GB when loaded) and takes minutes to load (almost 4GB during loading).

r, max

Thanks for the suggestion. I will try this (loading up a Snort instance) in a VM and see if I can reproduce the multiple starts problem. Thus far I have not been able to, but I have not tried enabling all the rules at once.

When you say the "maximum of rules and preprocessors", which rule families (Snort VRT, Emerging Threats, GPLv2 Community) are you running?

Bill

pfSenseRocks

I run all families.

priller

@fragged:

I don't have any VLAN's set up.

Running 2.1-RELEASE now

and I have issues with multiple Snort instances running every now and then.

Not much to add other than I've seen this too. The initial appearance was of a memory leak. Then I notice that there were multiple Snort instances running. I restart Snort and memory utilization goes back to normal (dup processes killed).

It does seem to happen after a rules update. Now that I know what to look for, I'll track more closely.

I do not have any VLAN's.

I only run a very minimal set of rules. Just "Connectivity" and 10 ET categories.

maverick_slo

I have same problems as Bill found out in my other thread… :(
I have 2 2.1 pfsenses one is static other is pppoe.
On both OpenVPN and squid+havp and thats it.

On one there are 2 processes and on other 4.
Stopping snort, killing other live processes and then start snort seems to fix it.

bmeeks

Hopefully I have some good news on the multiple Snort processes after a reboot front. Using the suggestions here of trying a "fully loaded" configuration with everything enabled, I was finally able to consistently get two duplicate Snort processes running with each reboot of a VM. The VM had Snort configured on only one interface (the WAN), and thus should have had only one running Snort process.

The problem, I believe, is really the fault of pfSense itself. However I am still tracking this hypothesis down. I'm thinking now the problem is in the rc.newwanip script.

The good news is I think I've found a workaround to use within the snort.sh shell script that is called to start Snort on a reboot. I need to test another day or two to be sure the workaround does not break something else.

Oddly enough, there was code in the existing snort.sh script that should avoid the multiple processes on reboot. But I found out during some research that the utility I was using (pgrep) has a long-standing known bug where it only matches the first 15 characters of a process name. I modified the arguments supplied to pgrep to shorten the match string and it started working correctly.

As others have surmised, the trigger for the multiple processes is a suitably loaded Snort configuration such that it takes a long time to start up. That in turn means there is a delay in creating the PID file in /var/run. In the start routine, the shell script tries to find a running process and will send it a SIGHUP to restart instead of starting a completely new one. It tries to find a running process by first looking for a PID file, and if that fails, it uses pgrep to see if a Snort process with the same arguments is running. On a slowly starting Snort setup, the PID file may not be there yet because it is written at the end of the startup. And with the 15-character bug I mentioned earlier, pgrep was not finding the running process either. Now I've been able to get pgrep to find the running process.

Of course the real problem here is pfSense itself sending multiple package restart commands on a reboot. Snort is actually getting called to start twice during the reboot cycle by two different pfSense init processes.

Bill

maverick_slo

Awsome news and it totally makes sense!

If you need test machine, I will be more than glad to install new package/.sh to my firewall.

Regards,
M

Cino

@maverick_slo:

Awsome news and it totally makes sense!

If you need test machine, I will be more than glad to install new package/.sh to my firewall.

Regards,
M

I wouldn't mind testing it either before its deployed to the masses

bmeeks

@Cino:

@maverick_slo:

Awsome news and it totally makes sense!

If you need test machine, I will be more than glad to install new package/.sh to my firewall.

Regards,
M

I wouldn't mind testing it either before its deployed to the masses

Thanks guys for volunteering to test. Let me do some more testing of my own for a day or two, then I will post the file here for others having the multiple processes issue to test. The fix will consist of a new snort.inc file that will go in /usr/local/pkg/snort. This file contains the subroutine that creates the snort.sh rc.d script on each reboot and when saving changes in the Snort GUI.

Bill

pfSenseRocks

Bill, this is great news! I appreciate all your help, hard work and diligence. Let me know if and how I can help.

bmeeks

Guys:

Attached to this post is a test fix for the multiple Snort processes that get started on a reboot on pfSense. This problem seems most acute on 2.1 systems. This is a test file. It should do no harm, but use at your own risk!

Use WinSCP, SCP or a similar process to copy this file to the firewall and place it in /usr/local/pkg/snort.

First rename the existing snort.inc file to snort.inc.orig. This will allow you to easily rollback if something goes wrong.

Second, rename the new file from snort.inc.txt to just snort.inc

Third, go into Snort and click the Interfaces tab and then select any interface. Click the Save button to force a generation of a new /usr/local/etc/rc.d/snort.sh file.

Fourth, try a reboot and see if the multiple Snort processes are history.

IMPORTANT – The file is in UNIX format (LF only with no CR/LF). Do not edit it on a Windows machine! When you transfer it, send it over in binary format to preserve the line endings. I had to name it with a *.txt extension in order to attach it to this post.

Bill

snort.inc.txt

pfSenseRocks

At the risk of jinxing it, we have a winner! 2 reboots and counting…

bmeeks

@pfSenseRocks:

At the risk of jinxing it, we have a winner! 2 reboots and counting…

Good news! Let me know if the multiple processes reappear. I will wait to hear from some of the other folks who had the problem and volunteered to test. If it works for everyone with no side effects, then I will submit a Pull Request on GitHub for the pfSense developers.

Bill

pfSenseRocks

I have rebooted about 7 times and I haven't been able to reproduce the issue. I have even snort running on 3 interfaces and I get exactly one process per interface.

Thank you so much for fixing this.

turker

Thank you so much.
The problem has been solved.

Atom D2500
4GB RAM
Pfsense 2.1

WAN+LAN+VLAN running smoothly.

Supermule

Does the name SNORT KING sound rude Bill? :D

maverick_slo

Morning all!

Rebooted 3 times no more snort issues.
Running smooth with AC-STD now.

Thanks man!

gogol

Thanks Bill, finally.

I booted a VM and have only 1 snort process now but still with the -E argument. Can you explain a bit what you changed or what was happening?

I am on 2.1 RELEASE.

/root(1): ps auxwwww | grep snort
root   57580  0.0 12.6 527956 260680  ??  Ss    1:13PM   0:11.61 /usr/pbi/snort-i386/bin/snort -R 4082 -E -q -l /var/log/snort/snort_em04082 --pid-path /var/run --nolock-pidfile -G 4082 -c /usr/pbi/snort-i386/etc/snort/snort_4082_em0/snort.conf -i em0
root   77957  0.0  0.1  3468  1240   0  S+    1:14PM   0:00.00 grep snort

bmeeks

Thanks for the testing and quick feedback guys. I will bundle this fix up with a few other bug hotfixes I've been holding and submit a GitHub Pull Request for the pfSense Core Team to review and hopefully approve and merge. The new package version will be 2.6.0 when it gets pushed to the Packages Repository. It will still use the 2.9.4.6 Snort binary.

I have a fully functional package based on the Snort 2.9.5.3 binary with a lot of new GUI features in the works. One of these new features is support for multi-engine configurations for the frag3, stream5_tcp and http_inspect preprocessors. There are also some new GUI configurable options for the SDF Sensitive Data preprocessor and a few others. I still need to do a bit more testing with this version before I release it.

Bill