SNORT: promiscuous mode disabled + promiscuous mode enabled error + exited on signal 11 (core dumped)
-
Perhaps it's SSH Guard...?
Nov 14 14:53:00 sshguard 62136 Now monitoring attacks. Nov 14 14:53:00 sshguard 28621 Exiting on signal. Nov 14 14:52:02 kernel igb2: promiscuous mode disabled Nov 14 14:52:02 kernel pid 38492 (snort), jid 0, uid 0: exited on signal 11 (core dumped)
SNORT is still restarting but now after the dump sshguard is logged as "Now monitoring attacks". I hadn't noticed that before, the first post also mentions sshguard. I had just assumed it was snort since it was "core dumped".
Thoughts?
-
@wolfsden3 said in SNORT: promiscuous mode disabled + promiscuous mode enabled error + exited on signal 11 (core dumped):
Perhaps it's SSH Guard...?
Nov 14 14:53:00 sshguard 62136 Now monitoring attacks. Nov 14 14:53:00 sshguard 28621 Exiting on signal. Nov 14 14:52:02 kernel igb2: promiscuous mode disabled Nov 14 14:52:02 kernel pid 38492 (snort), jid 0, uid 0: exited on signal 11 (core dumped)
SNORT is still restarting but now after the dump sshguard is logged as "Now monitoring attacks". I hadn't noticed that before, the first post also mentions sshguard. I had just assumed it was snort since it was "core dumped".
Thoughts?
No,
sshguard
has nothing to do with this. That is restarting due to system log rotations most likely.Snort is faulting for some other reason. In one of your posts up above you intimated this had been happening before you updated to the latest package version. Is that correct?
-
@bmeeks Correct. In the prior version I didn't see this. In the prior version there was yet another bug where I had to add a \ in front of a $ somewhere on line 42 I think it was for the interface to even start in some snort config file. I've since lost that link but it fixed it until the upgrade which is now in a perpetual restart.
I just tried reinstalling snort > no worky. I also tried an uninstall > reinstall, still the same behavior on both boxes.
I'm surprised nobody else has this issue. Maybe it's specific to something I have configured. I'm in legacy mode.
Did more digging online...looks like you replied to a similar thread with the core dump problem here:
https://forum.netgate.com/topic/166076/snort-exited-on-signal-11-core-dumped/7
Still looking for a fix.
-
Try this for me to help focus the troubleshooting:
Go to the INTERFACE SETTINGS tab in Snort and uncheck the Kill States option. Save that change and manually start or restart Snort.
Let me know how that works.
-
Done. I also ticked an option somewhere in the config for more robust logging. I saw this and tried to untick "Resolve Flowbits" for giggles but it still bombed later so I turned resolving flowbits back on.
Nov 14 16:54:59 snort 87355 WARNING: flowbits key 'file.pkp' is set but not ever checked. Nov 14 16:54:59 snort 87355 WARNING: flowbits key 'blackhole.jar' is set but not ever checked. Nov 14 16:54:59 snort 87355 WARNING: flowbits key 'MQTT_Connect' is set but not ever checked. Nov 14 16:54:59 snort 87355 WARNING: flowbits key 'ET.000webhostpost' is set but not ever checked. Nov 14 16:54:59 snort 87355 WARNING: flowbits key 'FraudLoad' is set but not ever checked. Nov 14 16:54:59 snort 87355 WARNING: flowbits key 'smb.neoteris' is set but not ever checked. Nov 14 16:54:59 snort 87355 WARNING: flowbits key 'DigiWatcher232_detection' is set but not ever checked. Nov 14 16:54:59 snort 87355 WARNING: flowbits key 'tlsv1.1_handshake' is set but not ever checked.
-
So far...after unticking kill states, no crash. What's that mean? :P
-
@wolfsden3 said in SNORT: promiscuous mode disabled + promiscuous mode enabled error + exited on signal 11 (core dumped):
So far...after unticking kill states, no crash. What's that mean? :P
Sorry to be late replying. Was away at an appreciation dinner getting stuffed .
This tells me where the problem is in the code. This area was modified about two weeks ago by a Netgate kernel developer who manages
pf
in FreeBSD. The kill states section of kernel system calls changed back in late August in FreeBSD and thus in the latest pfSense. That developer made some fixes in the custom module for Legacy Blocking specifically in the area that calls "kill states" in FreeBSD. Since you no longer get the crash when kill states is disabled, that means that the area he changed is almost certainly where the crash is happening now in Snort. So, at least we have the area narrowed down.This will not be as quick of a fix as a GUI PHP file. I will need to get that developer's assistance again to help track down the failure point and fix it.
-
I've opened Redmine Issue #14986 to track this. I have also exchanged emails with the Netgate developer team and engaged their kernel experts to help out tracking this bug down and fixing it.
-
-
We have same problem with Snort - exited on signal 11 (core dumped) after updated to 4.1.6_13. After untick kill states it is not crashing.
-
@DD said in SNORT: promiscuous mode disabled + promiscuous mode enabled error + exited on signal 11 (core dumped):
We have same problem with Snort - exited on signal 11 (core dumped) after updated to 4.1.6_13. After untick kill states it is not crashing.
Thanks for the report. Can you please confirm to me what pfSense version and family you are using?
Are you running pfSense CE or pfSense Plus, and what is the specific version?
-
@wolfsden3 said in SNORT: promiscuous mode disabled + promiscuous mode enabled error + exited on signal 11 (core dumped):
I saw this and tried to untick "Resolve Flowbits" for giggles but it still bombed later
Flowbits warnings are not a problem and could not cause a Signal 11 segfault.
There is an issue somewhere in the custom blocking plugin and/or in the new FreeBSD
libpfctl
library. The right guy is looking into it. He is one of the main guys maintaining thepf
firewall engine code in FreeBSD. -
@bmeeks We are using pfSense CE 2.7.0-RELEASE.
-
@DD said in SNORT: promiscuous mode disabled + promiscuous mode enabled error + exited on signal 11 (core dumped):
@bmeeks We are using pfSense CE 2.7.0-RELEASE.
Hmm... okay. Was not expecting that because I have thus far been unable to reproduce the issue in my CE 2.7.0-RELEASE virtual machine. Obviously I need to try harder .
Thank you for the additional information.
-
@bmeeks If I enable "kill states", then Snort is working for sometime (sometime 5 minutes, sometime 2 minutes, sometime 30 minutes.....) a then it will stop. After start, it is working again for sometime. When I disabled kill states, snort is working without stoping.
-
@DD said in SNORT: promiscuous mode disabled + promiscuous mode enabled error + exited on signal 11 (core dumped):
@bmeeks If I enable "kill states", then Snort is working for sometime (sometime 5 minutes, sometime 2 minutes, sometime 30 minutes.....) a then it will stop. After start, it is working again for sometime. When I disabled kill states, snort is working without stoping.
Same here - i guess this fault is present only on very "busy" interfaces where frecvent "kill state" are necessary.
-
@DD said in SNORT: promiscuous mode disabled + promiscuous mode enabled error + exited on signal 11 (core dumped):
@bmeeks If I enable "kill states", then Snort is working for sometime (sometime 5 minutes, sometime 2 minutes, sometime 30 minutes.....) a then it will stop. After start, it is working again for sometime. When I disabled kill states, snort is working without stoping.
Yes, that behavior correlates with the current theory of the bug. My suspicion is that when Snort blocks an IP and then attempts the follow-up action of clearing any open firewall states for that IP, it encounters the bug and crashes. But apparently this is not "always", because if it was "always and everytime", I could reproduce it easily and all users would be seeing the crash. So, something else is also likely at play. Still searching for the actual root cause. For example, it may be that two differnet events have to occur together to trigger the bug ???
-
@fireodo said in SNORT: promiscuous mode disabled + promiscuous mode enabled error + exited on signal 11 (core dumped):
Same here - i guess this fault is present only on very "busy" interfaces where frecvent "kill state" are necessary.
Yes, it does seem to require some extra "something" to trigger it because I have thus far not been successful. But my puny test environment with VMware Workstation virtual machines can't generate a ton of traffic.
-
@bmeeks Yes, because if I looking to alerts and blocked tab then alerts and blocked items are increasing and Snort is working ok. But after sometime snort will stop and log only one row: exited on signal 11 (core dumped).
-
Hey guys / gals / they / thems, etc...
I did the 2.7.1 update on one of the boxes I had this issue with > re-enabled kill states > haven't had a problem.
Did 2.7.1 resolve this SNORT issue is some magical way? I think SNORT is the same version but I didn't confirm this.
Many Thanks!
-
@wolfsden3 thereโs an update coming any time now, see https://forum.netgate.com/topic/184112/important-snort-and-suricata-package-announcement-probable-bug-in-legacy-blocking-module/