Finding Snort stopped

jeffhammett

I run Snort on a few different pfSenses on different networks.

Occasionally I will login and find that Snort is not running. On most pfSense routers I am running Snort on multiple interfaces and it is usually stopped on all interfaces when I find it stopped. When I do find them this way Snort starts up on all interfaces without problems.

Does anyone have any solutions on how to fix this? I found the thread about Service Watchdog not working properly with Snort on multiple interfaces, so I haven't tried that yet, wanted to see if others had suggestions on what they are doing.

jeffhammett

I was thinking of making a script to run via a cron job, something simple like:

pgrep snort | wc -l

and then if the output doesn't match the number of Snort interfaces that should be running generate some sort of notification.

Unfortunately pfSense doesn't have a command line mail client installed, so email is out of the question. Anyone have any other ideas of how to notify myself?

I have experienced the same problem multiple times, snort goes down randomly on one of the 5 interfaces and you have to manually start it again.

Another odd thing is that you have no indication that a snort interface is down on the dashboard as the service is still green there.
It would be really nice to to indicate it at least for example in yellow.

Regards,

Emanuel

ian_kenefick

@jeffh:

I run Snort on a few different pfSenses on different networks.

Occasionally I will login and find that Snort is not running. On most pfSense routers I am running Snort on multiple interfaces and it is usually stopped on all interfaces when I find it stopped. When I do find them this way Snort starts up on all interfaces without problems.

Does anyone have any solutions on how to fix this? I found the thread about Service Watchdog not working properly with Snort on multiple interfaces, so I haven't tried that yet, wanted to see if others had suggestions on what they are doing.

Better late than never :-\

There are two things you probably want to do here.

[1] Figure out why SNORT is crashing by checking the logs in /var/log/system.log and /var/log/snort/
[2] Deploy a Watchdog Service to restart the service and notify you by email when it goes down. The timestamp for the message is helpful for searching logs for issue. This Watchdog Service is available as a package for PFSense now.

Hope this helps.

BBcan177

@ian_kenefick:

[2] Deploy a Watchdog Service to restart the service and notify you by email when it goes down. The timestamp for the message is helpful for searching logs for issue. This Watchdog Service is available as a package for PFSense now.

Using the Service Watchdog for Snort/Suricata is not recommended. It may work with one interface, but there might be other issues with using that service.

bmeeks

Service Watchdog has problems with Snort in several areas. First up, if you have more than one Snort interface, then you have multiple Snort instances and Service Watchdog gets fooled (it will see one Snort service running and thinks all is well when in fact every interface but one might be down). Second, Snort is stopped by the rules update process to load new rules. The Service Watchdog sees Snort down and restarts it quickly. If the rules update is also trying to restart Snort, then you can wind up with multiple duplicate instances (two Snort processes on the same interface, for example).

Have you seen this random stopping since the last Snort binary update? I seem to recall a bug fix by the Snort guys to address a segfault error (or maybe that was Suricata… I get the release notes confused sometimes :().

Bill