Snort down for a couple of days for no obvious reason. Up now. Any idea why?
-
This has happened to me when one of the rules is bad…for example, the openappid file_transfer rule is broken, it is missing the ";)" at the end of the rules line so if I select that rule snort will stop but it does complain in the logs about it. I have manually fixed the line in the rules but when it updates it will get the bad rule again and stop. Have you checked your logs?
-
Also, I would definitely avoid running the update at midnight. Way to many things going on at that time that can interfere.
-
Also, I would definitely avoid running the update at midnight. Way to many things going on at that time that can interfere.
I'll try it with a change in update time although it's always midnight every hour somewhere isn't it?
Edit: just did forced update. Interface said complete. Yellow icon came back. Going to disable snort on WAN for now.
-
Uhm, I meant the midnight local time and the local cronjobs (though, running update exactly on the hour is more likely to hit update server issues as well).
-
Uhm, I meant the midnight local time and the local cronjobs (though, running update exactly on the hour is more likely to hit update server issues as well).
Thanks. I got it, just having fun. Changed to 00:45.
Even weirder, I stopped WAN interface but checked alerts a few minutes later. Snort WAN Disabled but blocks and alerts still collecting. I thought they would stop. Cleared all saved blocks and alerts but they still came on coming. Started interface again and immediately showed green. Looks odd to me.
-
Uhm, I meant the midnight local time and the local cronjobs (though, running update exactly on the hour is more likely to hit update server issues as well).
Thanks. I got it, just having fun. Changed to 00:45.
Even weirder, I stopped WAN interface but checked alerts a few minutes later. Snort WAN Disabled but blocks and alerts still collecting. I thought they would stop. Cleared all saved blocks and alerts but they still came on coming. Started interface again and immediately showed green. Looks odd to me.
That sounds like you have two Snort processes running on the same interface. This can randomly happen if several "restart all packages" commands get executed by pfSense for various reasons (DHCP renew on WAN is one way). To see, check the output of this command:
ps -ax |snort
If you have Snort stopped, then you should see no processes listed. I'm betting you will see one. If you do, kill it by PID or else reboot the firewall if that is easier.
Bill
-
bill,
thanks for your attention on this. I will reboot tomorrow when nobody's on it. Easy and fast.
-
reboot did not help. Uninstall / reinstall appears to work. Green icon after an auto-update, not yellow.
Upon uninstall, one message was 'removed old package update leftovers' or something like that. I wonder if it was cluttered and that caused it to get flaky? I wonder this also because I performed an update to the package and the problems started. Perhaps the current version was interacting with some ancient leftover flotsam?
Again, thanks for the replies above.
Edit:
Spoke too soon. Yellow icon back after suppressing an entry that blocked simple downloads on port 80.
Removing package until new version released for next pfSense update.
FYI to developer: simple router - only pfBlockerNG and OpenVPN client download packages loaded. J1900 based with 120GB SSD and 8GB RAM. Intel ports.
-
Played around with it and figured it out - I think. Problems with reliability are consistent with snort openapppid feature beings turned on and off. I really don't know what it does but I thought I would turn it on a few days ago. A package update coincided with it being turn on, more or less. When on, problems developed. When off, snort worked. After turning it back on, problems came back. Checked off in snort now and snort works as always before these issues developed.
openappid appears to be a new feature. I don't understand it and no documentation exists that tells a non-expert how to use it or even what it does. Problems are gone with it off.
-
The OpenAppID feaure was added by the Snort VRT about 2 or 3 years ago if I recall correctly. Shortly after it was introduced I incorporated support for configuring it within the pfSense Snort package. However, there is much more to using OpenAppID than simply checking the box in the GUI. You must create your own custom rules to actually implement Application ID inspection. There are a critical set of OpenAppID stems that come from the Snort VRT via the updates, but they are not all that you need to actually implement OpenAppID. So if you enabled the feature without also creating the necessary custom rules for traffic inspection, it is actually doing nothing.
There have been several reports of errors within the OpenAppID stems that are packaged in the Snort VRT signature updates. Unfortunately with Snort, when it encounters any kind of syntax error in rules or other items it is loading, it will error out and quit. Suricata will log an error, but then skip the errant rule and continue loading the others. So what is likely happening with OpenAppID enabled is Snort hits one of those random errors that seem to get into the OpenAppID stems update and quits. Because Snort is so terribly chatty and fills the system log with essentially every action it takes when you enable normal logging, the pfSense package always starts Snort with the "quiet" switch to cut down on all the log noise as Snort starts. You can disable this feature and turn on the verbose logging by toggling a parameter on the GLOBAL SETTINGS tab.
Here is how I think this might be happening to you. Enabling the OpenAppID preprocessor will cause Snort to load that piece of code and to download the OpenAppID stem updates along with the regular VRT rules update. Snort will then start to load and process the updated files. If OpenAppID is enabled, and the OpenAppID stem files have any errors in them, Snort will log the error and die. The error will only show up in the system log on pfSense if you have turned on verbose Snort logging (that GLOBAL SETTINGS parameter I mentioned earlier). So if Snort encounters an error in the rules or OpenAppID updates, it will just seemingly die for no reason when the "quiet" switch is enabled. As I mentioned, using the "quiet" switch is the default on pfSense otherwise you get several hundred lines of Snort start-up text in the system log.
Bill