New Snort pkg 2.5.5 – Read Before you Update to Understand Changed Defaults
-
Bill since the upgrade the auto rule update has not completed successfully. The update log looks ok with no errors. The only thing that appears in the sys log is snort[43971]: Could not remove pid file /var/run/snort_em21407.pid: No such file or directory. I did completely remove snort and install manually. Did not use reinstall . I did keep settings from previous install though. My next step is to clean reinstall without keeping settings. But before I do is there anything else I can check/change/remove? I really don't want to have to set up snort again from scratch.
-
Bill since the upgrade the auto rule update has not completed successfully. The update log looks ok with no errors. The only thing that appears in the sys log is snort[43971]: Could not remove pid file /var/run/snort_em21407.pid: No such file or directory. I did completely remove snort and install manually. Did not use reinstall . I did keep settings from previous install though. My next step is to clean reinstall without keeping settings. But before I do is there anything else I can check/change/remove? I really don't want to have to set up snort again from scratch.
This error:
snort[43971]: Could not remove pid file /var/run/snort_em21407.pid: No such file or directory.is harmless and is a warning only. It's a byproduct of the restart process deleting the PID file during stopping of Snort before the pkill command gets to it. This is done to be 100% sure an existing leftover PID file does not interfere with subsequent starts of Snort. So long explanation to say that error has nothing to do with the rules updating successfully or not.
Look at the Update Log using the new button on the UPDATES tab. If you see nothing alarming in there, things are probably OK. You can double-check by looking at the timestamp on the file snort.rules in the rules directory for your interface.
If you like, post the contents from the View Update Log window and I will review them. You get there by clicking the VIEW LOG button on the UPDATES tab.
Bill
-
Thanks for the explanation Bill.
I have checked the update log within Snort and all appears fine there. Everything shows successful. However, Snort does not restart. Even in the system log it says that its restarting on my wan interface. There is no fatal error or anything else displayed showing that it could not restart. When i got into snort and click enable it starts up just fine. So I am at a loss on why its not restarting. In the previous versions if it failed to restart usually there was a fatal error displayed in the system log saying why it errored out.
I just vpned in and grabbed the update log as well as the same time frame on the sys log. Maybe something fatal with a rule? Not sure… I have not changed any other settings to make this fail. Not sure if you need any other settings that are enabled.
System log
Apr 17 00:04:02 php: : The Rules update has finished.
Apr 17 00:04:02 php: : Snort has restarted with your new set of rules…
Apr 17 00:04:00 SnortStartup[36366]: Snort SOFT START For Wan(1407_em2)…
Apr 17 00:04:00 snort[43971]: Could not remove pid file /var/run/snort_em21407.pid: No such file or directory
Apr 17 00:04:00 snort[43971]: Could not remove pid file /var/run/snort_em21407.pid: No such file or directory
Apr 17 00:03:59 kernel: em2: promiscuous mode disabled
Apr 17 00:03:59 snort[43971]: *** Caught Term-Signal
Apr 17 00:03:59 snort[43971]: *** Caught Term-Signal
Apr 17 00:03:58 SnortStartup[26068]: Snort STOP For Wan(1407_em2)…
Apr 17 00:03:55 php: : Building new sig-msg.map file for WAN...
Apr 17 00:03:43 php: : Resolving and auto-enabling any flowbit-required rules for WAN...
Apr 17 00:03:38 php: : Updating rules configuration for: WAN ...
Apr 17 00:03:38 php: : EmergingThreats rules file update downloaded succsesfully
Apr 17 00:03:36 php: : There is a new set of EmergingThreats rules posted. Downloading...
Apr 17 00:03:35 php: : Snort VRT rules are up to date...
Apr 17 00:03:35 php: : Snort MD5 Attempts: 1Snort update log
Starting rules update… Time: 2013-04-16 00:03:01
Downloading Snort VRT md5 file...
Checking Snort VRT md5 file...
Snort VRT rules are up to date.
Downloading EmergingThreats md5 file...
Checking EmergingThreats md5.
There is a new set of EmergingThreats rules posted. Downloading...
Done downloading EmergingThreats rules file.
Extracting and installing EmergingThreats.org rules...
Installation of EmergingThreats.org rules completed.
Copying new config and map files...
Updating rules configuration for: WAN ...
Restarting Snort to activate the new set of rules...
Snort has restarted with your new set of rules.
The Rules update has finished. Time: 2013-04-16 00:05:27Starting rules update... Time: 2013-04-17 00:03:01
Downloading Snort VRT md5 file...
Checking Snort VRT md5 file...
Snort VRT rules are up to date.
Downloading EmergingThreats md5 file...
Checking EmergingThreats md5.
There is a new set of EmergingThreats rules posted. Downloading...
Done downloading EmergingThreats rules file.
Extracting and installing EmergingThreats.org rules...
Installation of EmergingThreats.org rules completed.
Copying new config and map files...
Updating rules configuration for: WAN ...
Restarting Snort to activate the new set of rules...
Snort has restarted with your new set of rules.
The Rules update has finished. Time: 2013-04-17 00:04:02 -
I have checked the update log within Snort and all appears fine there. Everything shows successful. However, Snort does not restart. Even in the system log it says that its restarting on my wan interface. There is no fatal error or anything else displayed showing that it could not restart. When i got into snort and click enable it starts up just fine. So I am at a loss on why its not restarting. In the previous versions if it failed to restart usually there was a fatal error displayed in the system log saying why it errored out.
That's really weird that it starts manually but not after an auto-update. Give me your pfSense version (2.0.x or 2.1-BETA) and which rule sets you have enabled for download. I assume from the log info that you have Snort VRT and Emerging Threats enabled. Do you have the paid VRT subscription or the free registered user account?
Just so I'm clear on your manual start method, do you go to the Snort Interfaces tab and click the icon there, or are you using the Services option from the pfSense menu?
Bill
-
That's really weird that it starts manually but not after an auto-update. Give me your pfSense version (2.0.x or 2.1-BETA) and which rule sets you have enabled for download. I assume from the log info that you have Snort VRT and Emerging Threats enabled. Do you have the paid VRT subscription or the free registered user account?
Just so I'm clear on your manual start method, do you go to the Snort Interfaces tab and click the icon there, or are you using the Services option from the pfSense menu?
Bill
Bill I am currently on pfsense 2.0.2-release (i386) built on Friday Dec 7 (I have not upgraded to the newer version released within the past week)
I have a paid subscription to Snort VRT. I do have Snort VRT and Emerging Threats enabled. I do not have the Snort GPLv2 enabled.
All rule categories are enabled with the exclusion of the snort_p2p.rules and the snort_p2p.so.rules.I do have a list of suppression ids that I do not want to kick up as an alert.
When I do the manual start I go to the Service menu up top > Snort. Then on the interface tab click the Green Arrow. Then snort starts just fine with no issue turning the green arrow to a red box with an X in it. Nothing else has been touched/changed in between the failed start to starting snort successfully.
I really wish there was a retry option or monitor option. If snort fails to start or when a check is done lets say every 30 min and it finds that snort is not running that it will try to restart it.
-
Bill I am currently on pfsense 2.0.2-release (i386) built on Friday Dec 7 (I have not upgraded to the newer version released within the past week)
…When I do the manual start I go to the Service menu up top > Snort. Then on the interface tab click the Green Arrow. Then snort starts just fine with no issue turning the green arrow to a red box with an X in it. Nothing else has been touched/changed in between the failed start to starting snort successfully.
I really wish there was a retry option or monitor option. If snort fails to start or when a check is done lets say every 30 min and it finds that snort is not running that it will try to restart it.
You are not the first to request some sort of Snort monitor. That's something I can ponder on incorporating for a future update.
As for your more immediate problem, first make sure that you do a package GUI components reinstall to be sure you have the very latest update. Go to Available Packages and click either the pkg or xml icons to reinstall the GUI components.
Here's another thing you can try to help me troubleshoot. It requires editing a file. Here are the steps:
1. From the menu, choose Diagnostics and then Edit File.
2. Browse to /usr/local/etc/rc.d/snort.sh and click it to open the file.
3. Scroll down and find this section:restart) rc_stop rc_start ;;Now edit that section so it looks like this:
restart) rc_stop sleep 3 rc_start ;;4. Click Save to save the edited file.
Let the next auto-update happen and see if Snort restarts. You will need to wait for an actual rule update to happen (that is, some new rules are posted). You can force this by simply deleting the two MD5 files in /usr/local/etc/snort. This will cause a re-download and update of both VRT and ET rules the next time the rule update job runs.
Let me know how this works for you.
Bill
-
For Snort monitoring you could use a script like that
#!/bin/sh SERVICE=snort if P=$(/usr/bin/pgrep $SERVICE) then /bin/echo "$SERVICE is running, PID is $P" else /usr/local/etc/rc.d/snort.sh start fiand let it run every minute via cron. I used that a long time ago, when Snort shut down from time to time.
These days I just have shut downs after upgrades and no solution yet (the logs say snort was started after upgrade and nothing else, but I find it stopped). That is why I am reading here. (2.0.3 amd64, Snort PVer. 2.5.5)Best wishes, Judex
-
As for your more immediate problem, first make sure that you do a package GUI components reinstall to be sure you have the very latest update. Go to Available Packages and click either the pkg or xml icons to reinstall the GUI components.
Here's another thing you can try to help me troubleshoot. It requires editing a file. Here are the steps:
1. From the menu, choose Diagnostics and then Edit File.
2. Browse to /usr/local/etc/rc.d/snort.sh and click it to open the file.
3. Scroll down and find this section:restart) rc_stop rc_start ;;Now edit that section so it looks like this:
restart) rc_stop sleep 3 rc_start ;;4. Click Save to save the edited file.
Let the next auto-update happen and see if Snort restarts. You will need to wait for an actual rule update to happen (that is, some new rules are posted). You can force this by simply deleting the two MD5 files in /usr/local/etc/snort. This will cause a re-download and update of both VRT and ET rules the next time the rule update job runs.
Let me know how this works for you.
Bill
Bill for the reinstall/install piece, I have completely removed Snort (uninstalled it with just leaving settings within snort there) then went into available packages tab and installed it. I have done this twice with the same effect. Wouldn't this make sure all components are installed?
I changed the file via telnet. Its easier from my phone to do that then messing with the browser. :-D
I will post back after the auto update runs and let you know if it works or not.
Thanks for your help.
-
OK. Let me know if the time delay helps out any. The restart of Snort has been a sore spot for quite a while even before I started modifying the code a bit. I know Ermal tried a few different tricks about this time last year when he was making some significant updates to the Snort code.
It seems quite sensitive to the number of active rules users have. At least I've noticed that on my test virtual machines. The more rules, the longer the startup time for sure. I think that may be playing into the "randomness" of this issue among users. By that I mean it impacts some, but not others. Several things could be at play here. Number of active rules, capability (speed) of the underlying hardware, etc.
Bill
-
Bill,
The auto update ran this morning at 12am and the same thing happened and Snort did not restart properly. I went and started it manually and it started up just fine. I pasted the logs again just incase you wanted to check em out.
SYS log
Apr 18 00:06:08 php: : The Rules update has finished.
Apr 18 00:06:04 php: : Building new sig-msg.map file for WAN…
Apr 18 00:05:52 php: : Resolving and auto-enabling any flowbit-required rules for WAN...
Apr 18 00:05:48 kernel: em2: promiscuous mode disabled
Apr 18 00:05:48 kernel: pid 44849 (snort), uid 0: exited on signal 4
Apr 18 00:05:48 php: : Updating rules configuration for: WAN ...
Apr 18 00:05:41 php: : EmergingThreats rules file update downloaded succsesfully
Apr 18 00:05:39 php: : There is a new set of EmergingThreats rules posted. Downloading...
Apr 18 00:05:39 php: : Snort Rules Attempts: 1
Apr 18 00:04:48 php: : There is a new set of Snort VRT rules posted. Downloading...
Apr 18 00:04:48 php: : Snort MD5 Attempts: 2update log
Starting rules update… Time: 2013-04-18 00:03:01
Downloading Snort VRT md5 file...
Checking Snort VRT md5 file...
There is a new set of Snort VRT rules posted. Downloading...
Done downloading rules file.
Downloading EmergingThreats md5 file...
Checking EmergingThreats md5.
There is a new set of EmergingThreats rules posted. Downloading...
Done downloading EmergingThreats rules file.
Extracting and installing EmergingThreats.org rules...
Installation of EmergingThreats.org rules completed.
Extracting and installing Snort VRT rules...
Using Snort VRT precompiled SO rules for FreeBSD-8-1 ...
Installation of Snort VRT rules completed.
Copying new config and map files...
Updating rules configuration for: WAN ...
The Rules update has finished. Time: 2013-04-18 00:06:08This is what the log looks like when i toggle manually I never see the promiscuous mode enabled line when it attempts the auto update and then restarts.
Apr 18 04:58:22 kernel: em2: promiscuous mode enabled
Apr 18 04:58:22 php: /snort/snort_interfaces.php: Interface Rule START for Wan(em2)...
Apr 18 04:57:47 php: /snort/snort_interfaces.php: Toggle(snort starting) for WAN(Wan)... -
Bill,
The auto update ran this morning at 12am and the same thing happened and Snort did not restart properly. I went and started it manually and it started up just fine. I pasted the logs again just incase you wanted to check em out.
OK. I discovered a clue in another user's post on the Snort 2.5.5 Issues Thread about the possible cause of this. I'm researching to be sure about the cause, then can issue a fix.
Bill
-
Bill for what its worth, this problem also happens with the manual rules update. I disabled auto update so it wouldnt run last night. Then I manually did the rules update this morning and same behavior happened.
-
Good catch! I posted some logs from last night regarding the soft start issue in Snort. Both FW's killed Snort after update despite one of them had the fix from Bill.
-
Bill for what its worth, this problem also happens with the manual rules update. I disabled auto update so it wouldnt run last night. Then I manually did the rules update this morning and same behavior happened.
Did you have the change in the snort.sh file that removed the call to rc_stop() in the restart section of the file? The restart of Snort is called from within the GUI code that does the rules update, and it will call the restart script if the update is done manually or automatically.
Ermal reminded me of something else when I consulted him on this. A SOFT START of Snort will fail when a rule update contains updates to SO (Shared Objects) rules. That is one set of changes Snort cannot "refresh" without shutting down and restarting.
Bill
-
Bill for what its worth, this problem also happens with the manual rules update. I disabled auto update so it wouldnt run last night. Then I manually did the rules update this morning and same behavior happened.
Did you have the change in the snort.sh file that removed the call to rc_stop() in the restart section of the file? The restart of Snort is called from within the GUI code that does the rules update, and it will call the restart script if the update is done manually or automatically.
Ermal reminded me of something else when I consulted him on this. A SOFT START of Snort will fail when a rule update contains updates to SO (Shared Objects) rules. That is one set of changes Snort cannot "refresh" without shutting down and restarting.
Bill
The only thing I have done is edit the sleep as you requested.. I didnt see any thing mentioned removing the rc_stop… Did I miss that in a post?
Yea I know about the failed starts every now and then with the so rules.. Even prior to updating it would fail every once and a while and it was due to that.. not often but it would happen..
So what exactly do i need to remove and I will give that a shot? Is this being discussed in another thread too?
-
So what exactly do i need to remove and I will give that a shot? Is this being discussed in another thread too?
It's from this post: http://forum.pfsense.org/index.php/topic,60994.msg330447.html#msg330447
That version of an interim fix may still get tripped up with the SO rules update issue. That one is just endemic to Snort. Snort is supposed to restart itself under those circumstances, though.
I am working on a permanent fix based on what I currently think is happening. Plan on revamping how the shell script works for these restarts (especially the stops). More feedback from folks having the problem is always welcome!
Bill
-
So what exactly do i need to remove and I will give that a shot? Is this being discussed in another thread too?
It's from this post: http://forum.pfsense.org/index.php/topic,60994.msg330447.html#msg330447
That version of an interim fix may still get tripped up with the SO rules update issue. That one is just endemic to Snort. Snort is supposed to restart itself under those circumstances, though.
I am working on a permanent fix based on what I currently think is happening. Plan on revamping how the shell script works for these restarts (especially the stops). More feedback from folks having the problem is always welcome!
Bill
Ahh.. let me make those changes and I will see if that fixes it. well as much as it can. Thanks
-
Bill,
I am going to keep my replies going forward in the topic you linked so everything is together. As you can see in that post there I am having to redo the tests for you.
