Snort 2.9.2.3 pkg v. 2.5.1 service fails overnight, unable to restart

Fesoj

caustic386,

more could be broken. If you have a saved configuration from your latest working setup, you could reinstall the entire environment via Diagnostics: Backup/restore.

caustic386

Unfortunately I'm in production, so that seems like a bad idea. Is there a way to clean it all out by hand?

Cino

failed on my box when after uninstall and reinstalling snort yesterday… had the normal sensitive data error this morning... I'll do it again and see what happens later today or tomorrow morning

dwood

On AMD64, 2.0.1, Snort is working without issues so far, and updating (auto updating) providing I toggle the sensitive data preprocessor off. This is the latest install as of Ermal's last change to fix the gettet typo. Ccaustic…point 3 will clean up old snort bits after you uninstall.

If you are installing Snort make sure you do the following:

1. Uninstall (if you have an older version, suggest you not toggle "save settings" to on. In other words, start fresh.
2. Create an alias in pfsense to reflect your old whitelist. You will select this alias in the snort whitelist tab later.
3. Run this command using Diagnostics -> Command Prompt: find /* | grep -i snort | xargs rm -rv (removes old snort references)
4. Install latest. Likely for this version you'll want to make sure that senstive data preprocessor is not selected. I've got all others on.
5. Monitor blocking and prepare to add quite a few exclusions! This is the set I'm using pretty much copy/pasted from the suppression tab:

# HTTP Inspect Errors
suppress gen_id 120, sig_id 3
suppress gen_id 120, sig_id 6
suppress gen_id 120, sig_id 8
suppress gen_id 120, sig_id 10
#
suppress gen_id 1, sig_id 2014819
#
# This event indicates that a portable executable file has been downloaded. 
suppress gen_id 1, sig_id 15306
#
# This event indicates that Email Addresses have been observed in traffic on the protected network.
suppress gen_id 138, sig_id 5
#
# This set of instructions can be used as a NOOP to pad buffers on an x86 architecture machines. 
suppress gen_id 1, sig_id 1390
#
# FILE-IDENTIFY download of executable content - x-header  -> stops windows download
suppress gen_id 1, sig_id 16313
#
# FILE-IDENTIFY download of executable content -> stops file downloads
suppress gen_id 1, sig_id 11192
#
#"GPL SHELLCODE x86 NOOP"
suppress gen_id 1, sig_id 648

caustic386

That's exactly the process i've been using, but so far fails every time during nightly updates. I'll try one more time and report back tomorrow.

Cino

@dwood:

Snort is working without issues so far, and updating (auto updating) providing I toggle the sensitive data preprocessor off.

i'll give it a try.. wanted it on to know when CC were being used…

mschiek01

@Cino:

@dwood:

Snort is working without issues so far, and updating (auto updating) providing I toggle the sensitive data preprocessor off.

i'll give it a try.. wanted it on to know when CC were being used…

I am not having any problems with snort stopping on auto updates on 6 different boxes. 3 i386 and 3 amd64.

The sensitive data preprocessor is set to on, on all the boxes.

1 thing I should note is that I am still using the "snort_check_for_rule_updates.php" That Fesoj posted. I have not updated since then.
https://github.com/bsdperimeter/pfsense-packages/pull/291/files

Cino

@mschiek01:

@Cino:

@dwood:

Snort is working without issues so far, and updating (auto updating) providing I toggle the sensitive data preprocessor off.

i'll give it a try.. wanted it on to know when CC were being used…

I am not having any problems with snort stopping on auto updates on 6 different boxes. 3 i386 and 3 amd64.

The sensitive data preprocessor is set to on, on all the boxes.

1 thing I should note is that I am still using the "snort_check_for_rule_updates.php" That Fesoj posted. I have not updated since then.
https://github.com/bsdperimeter/pfsense-packages/pull/291/files

mine is fresh install using both snort and et rules… If that works, can we have ermal pull it in?

Fesoj

Maybe yes, maybe no.

What I've suggested is a q&d patch, not a solution. The problem is that alert types depend on the preprocessors invoked and the rule sets, and different rule sets may need different declarations (actually you also need to look at the rules that are enabled). It boils down to do some kind of resource management, or you go into the details, which requires to understand how Snort works.

If you enable Snort.org AND ET rules, no patching should be required (I have 3 machines running with this config), even if you activate only a few ET rules.

I am currently thinking about a rather general solution to this type of problem, but this would take some time and I don't know whether I'd like to discuss this in public now as it would separate the published rules from what would go into the configuration of an interface (and I currently cannot present any code).

kilthro

I have both ET and Snort rules enabled and it has been auto updating without any problems for me. I don't have the sensitive data preprocesser on.. The only thing I am noticing is snort memory usage keeps increasing. generally out of 2 gig pfsense generally with snort in the past would sit around 25% used on ram.. Now it starts out at 25% and keeps climbing. Its currently at 40%. If I stop snort and restart it manually it resets back down the starting point. Then it starts climbing daily and during high usage. It seems like whatever cache its using isn't released once done. Any reason why this would be happening with the latest version of snort? I never had this issue before. I have wiped and cleanly installed snort and same problem.

Cino

@kilthro:

I have both ET and Snort rules enabled and it has been auto updating without any problems for me. I don't have the sensitive data preprocesser on.. The only thing I am noticing is snort memory usage keeps increasing. generally out of 2 gig pfsense generally with snort in the past would sit around 25% used on ram.. Now it starts out at 25% and keeps climbing. Its currently at 40%. If I stop snort and restart it manually it resets back down the starting point. Then it starts climbing daily and during high usage. It seems like whatever cache its using isn't released once done. Any reason why this would be happening with the latest version of snort? I never had this issue before. I have wiped and cleanly installed snort and same problem.

I've noticed the same thing this morning. My norm is 24% with snort and 11% without snort. This morning it was up to 28%..

I haven't looked at the rule update code since i've been away for the last week. But does it stop and stop snort or is it doing a reload?

miles267

Same issue as others have reported. On latest code.

Snort and ET rules enabled
Sensitive data option enabled under preprocessors on both WAN and LAN interface
if I uninstall and reinstall snort from scratch and update rules, both my WAN and LAN interface start MANUALLY without issue
overnight when the auto-update of rules (snort and ET) occurs, snort on both WAN and LAN interfaces stop and cannot be started manually

Returns the following FATAL error in log:

snort[33033]: FATAL ERROR: /usr/local/etc/snort/snort_60243_em3/preproc_rules/sensitive-data.rules(1) Unknown ClassType: sdf
snort[33033]: FATAL ERROR: /usr/local/etc/snort/snort_60243_em3/preproc_rules/sensitive-data.rules(1) Unknown ClassType: sdf

once sensitive data option is disabled on preprocessors (either WAN or LAN interface), am able to MANUALLY start snort without issue

mschiek01

@Cino:

@kilthro:

I have both ET and Snort rules enabled and it has been auto updating without any problems for me. I don't have the sensitive data preprocesser on.. The only thing I am noticing is snort memory usage keeps increasing. generally out of 2 gig pfsense generally with snort in the past would sit around 25% used on ram.. Now it starts out at 25% and keeps climbing. Its currently at 40%. If I stop snort and restart it manually it resets back down the starting point. Then it starts climbing daily and during high usage. It seems like whatever cache its using isn't released once done. Any reason why this would be happening with the latest version of snort? I never had this issue before. I have wiped and cleanly installed snort and same problem.

I've noticed the same thing this morning. My norm is 24% with snort and 11% without snort. This morning it was up to 28%..

I haven't looked at the rule update code since i've been away for the last week. But does it stop and stop snort or is it doing a reload?

Not seeing any memory problems on my boxes. What is your snort memory settings. I am running A/C. Snort is reloading not starting and stopping.

Jul 28 00:32:33 snort[25620]:
Jul 28 00:32:33 snort[25620]:
Jul 28 00:32:33 snort[25620]: –== Reload Complete ==--
Jul 28 00:32:33 snort[25620]: –== Reload Complete ==--
Jul 28 00:32:33 snort[25620]:
Jul 28 00:32:33 snort[25620]:
Jul 28 00:32:31 snort[25620]: +–--------------------------------------------------------------
Jul 28 00:32:31 snort[25620]: +–--------------------------------------------------------------

Again I am using Fesoj "snort_check_for_rule_updates.php" That Fesoj posted. I have not updated since then.
https://github.com/bsdperimeter/pfsense-packages/pull/291/files

Sensitive data preproc is on and using both rule sets. Not using any SO. rules

kilthro

@Cino:

@kilthro:

I have both ET and Snort rules enabled and it has been auto updating without any problems for me. I don't have the sensitive data preprocesser on.. The only thing I am noticing is snort memory usage keeps increasing. generally out of 2 gig pfsense generally with snort in the past would sit around 25% used on ram.. Now it starts out at 25% and keeps climbing. Its currently at 40%. If I stop snort and restart it manually it resets back down the starting point. Then it starts climbing daily and during high usage. It seems like whatever cache its using isn't released once done. Any reason why this would be happening with the latest version of snort? I never had this issue before. I have wiped and cleanly installed snort and same problem.

I've noticed the same thing this morning. My norm is 24% with snort and 11% without snort. This morning it was up to 28%..

I haven't looked at the rule update code since i've been away for the last week. But does it stop and stop snort or is it doing a reload?

From what i can tell, a reload. Not a stop and start.

kilthro

@mschiek01:

@Cino:

@kilthro:

I have both ET and Snort rules enabled and it has been auto updating without any problems for me. I don't have the sensitive data preprocesser on.. The only thing I am noticing is snort memory usage keeps increasing. generally out of 2 gig pfsense generally with snort in the past would sit around 25% used on ram.. Now it starts out at 25% and keeps climbing. Its currently at 40%. If I stop snort and restart it manually it resets back down the starting point. Then it starts climbing daily and during high usage. It seems like whatever cache its using isn't released once done. Any reason why this would be happening with the latest version of snort? I never had this issue before. I have wiped and cleanly installed snort and same problem.

I've noticed the same thing this morning. My norm is 24% with snort and 11% without snort. This morning it was up to 28%..

I haven't looked at the rule update code since i've been away for the last week. But does it stop and stop snort or is it doing a reload?

Not seeing any memory problems on my boxes. What is your snort memory settings. I am running A/C. Snort is reloading not starting and stopping.

Jul 28 00:32:33 snort[25620]:
Jul 28 00:32:33 snort[25620]:
Jul 28 00:32:33 snort[25620]: –== Reload Complete ==--
Jul 28 00:32:33 snort[25620]: –== Reload Complete ==--
Jul 28 00:32:33 snort[25620]:
Jul 28 00:32:33 snort[25620]:
Jul 28 00:32:31 snort[25620]: +–--------------------------------------------------------------
Jul 28 00:32:31 snort[25620]: +–--------------------------------------------------------------

Again I am using Fesoj "snort_check_for_rule_updates.php" That Fesoj posted. I have not updated since then.
https://github.com/bsdperimeter/pfsense-packages/pull/291/files

Sensitive data preproc is on and using both rule sets. Not using any SO. rules

I have always used AC-BNFA

mschiek01

I used to use ac/bnfa but since the new binary the memory use seems to be under control and I switched to a/c a week or so ago, snort has not stopped since. BTW I have 4 gig memory in i386 and 8 gig in amd63

MediocreFred

Here's a different error in the log… same symptom though.

Jul 27 12:04:37 kernel: pid 44861 (snort), uid 0: exited on signal 11
Jul 27 12:04:37 snort[44861]: FATAL ERROR: Character value out of range, try a binary buffer.

Happens after a rules update. Has anybody seen this one?

Cino

I use AC-BNFA with ET and Snort rules, also a few .SO rules on 2.1-Dev i386. Using only what is provided by the package only.. This is really the only way to test and to let the developer know what is going on with the code… Now if we can get Fesoj's code updated with what is currently being offered from the package manager, i'll be willing to test it also.

PS i386 can't take 4gigs, its a little over 3 gigs. Limitation of 32-bit with memory address allocation.

With it reloading, that kinda makes sense. Just have to figure out why is it increasing its usage.

Fesoj

I am currently using a fresh package install on 2 (virtual) machines and I have NOT applied my latest patch. Both systems are running fine for a couple of hours now. My latest patch is not needed if you load the Snort.org rules (unless my second last patch has been removed meanwhile, but a quick check of the sources show that that is not the case). The rule sets will update during the night, so I'll see tomorrow morning if there is an issue (but I doubt that).

Alerting, blocking, blacklisting (with squid/squidGuard), reporting (with Sarg), and freeRADIUS (hooked up to a MySQL server) is working smoothly. I couldn't be happier.

If others are still seeing problems, I could strip personal info from the images and publish an ova file somewhere. Then we'll see whether the problems persist (maybe some problems could be due to subtle hardware failures ;D).

miles267

Confirming the latest snort code restarts automatically after the update of Snort and ET rules but ONY if the 'Enable Sensitive Data' pre-processor is disabled.

Is anyone else having this issue and is there a way to correct even with a patch until the next code update?