Snort crashed roughly shortly after startup

McFuzz

snort -V output:


   ,,_     -*> Snort! <*-
  o"  )~   Version 2.9.7.2 GRE (Build 177) FreeBSD
   ''''    By Martin Roesch & The Snort Team: http://www.snort.org/contact#team
           Copyright (C) 2014 Cisco and/or its affiliates. All rights reserved.
           Copyright (C) 1998-2013 Sourcefire, Inc., et al.
           Using libpcap version 1.6.2
           Using PCRE version: 8.35 2014-04-04
           Using ZLIB version: 1.2.8

The NICs I have are Intel 82574L; this is on a Supermicro MBD-X7SPE-H-D525-O motherboard (embedded dual core Atom D525).

bmeeks

Well, nothing unusual or unexpected in the version information. It's all like it should be.

This is especially vexing since it appears to die even with no rules selected. It's also weird that the death seems to happen on radically different time intervals. You reported a 2-minute run time and then an almost 16-minute run time.

Have you run an extensive test on your system RAM to rule out a potential memory problem?

Bill

McFuzz

@bmeeks:

Well, nothing unusual or unexpected in the version information. It's all like it should be.

This is especially vexing since it appears to die even with no rules selected. It's also weird that the death seems to happen on radically different time intervals. You reported a 2-minute run time and then an almost 16-minute run time.

Have you run an extensive test on your system RAM to rule out a potential memory problem?

Bill

I haven't done that, no; didn't really see a reason to do that considering the box has been up and running for roughly 3 years now with about 4 days of total downtime… I have pfBlockerNG and OpenVPN running on it without any issues if that means anything. I suppose I can test the RAM but I'd rather not take the box down if I can help it.

bmeeks

I use the OpenVPN, Snort and apcuspd packages on my production box with no issues. I only mentioned a RAM test because Snort can use a lot of RAM and might be the only application "tickling" a particular bank of high RAM (just a guess, though).

Do you have any messages in your system log about packages restarting? You don't mention it, so I assume you don't, but are you using the Service Watchdog package with Snort?

Bill

McFuzz

The only other package that restarts, as I mentioned in earlier posts, is NUT - but that is per design due to the customization that I've set up. The package restarts every 15 minutes through a cron job.

I do not have Service Watchdog installed; I'd imagine it will wreak havoc with the snort package :(

McFuzz

Swapped RAM for kicks - same thing :'(

bmeeks

The RAM guess was a longshot. You are correct that Snort and the Service Watchdog package don't play well together.

I am really stumped by your problem. Have you tried the Suricata package? It will do essentially the same thing as Snort. The look and feel of the two packages are identical. They share tons of the same PHP code for the GUI.

Bill

McFuzz

@bmeeks:

The RAM guess was a longshot. You are correct that Snort and the Service Watchdog package don't play well together.

I am really stumped by your problem. Have you tried the Suricata package? It will do essentially the same thing as Snort. The look and feel of the two packages are identical. They share tons of the same PHP code for the GUI.

Bill

Hrm… I guess I can give that a shot; see what happens. I'll try and report back.

McFuzz

Update: Odd things are happening…!

So I decided to install Suricata to try it out. After installing it, I meant to start configuring it but accidentally clicked on snort as opposed to suricata from the services menu. Imagine my surprise when I noticed that snort was actually running...! I tried looking at system logs but considering the log gets rotated quickly, I was not able to find when it started.

I decided to enable categories/rules and let it run - 20 minutes later, it was still running. I then decided to uninstall suricata figuring snort fixed itself however about 10 minutes after uninstalling, snort crapped out!

So - I re-enabled snort, but it yet again crapped out about 5 minutes later. This got me thinking - I installed Suricata but did not configure it (just as last time), enabled snort and - it has been running for the past two hours. I've then uninstalled Suricata - but snort has not crashed as of yet. Will let it run and see how it behaves.

Insanely odd behavior!

McFuzz

Perhaps my happiness was pre-emptive: I enabled AppID, restarted snort and now the interface died relatively quickly.

I will try the suricata trick.

edit: did not work… let's see what happens if OpenAppID is off...

edit2: nope, disabling OpenAppID did not work either.

Back to square one - how is it that things worked fine for several hours and once OpenAppID was enabled, broke, and not that it is disabled, still broken? I guess suricata was just a coincidence.

Edit 3: Decided to disable openappID, reinstall suricata and then enable snort interface - works! Enabled openappID - works! Forced update (cleared MD5 hashes) - works! So far it has been running for the past 2 hours with no issues... but Suricata is still installed (albeit not running).

McFuzz

Welps - with openappid, snort crapped out about 2 hours after being fired up. Will try a lengthy test with AppID off.