“snort *** Caught Term-Signal” errors
-
What kind of hardware are you running on?
Signal 4 is an "illegal instruction" error. Typically that happens if you compiled something against the wrong hardware type or severe memory corruption occurred and the CPU attempted to "execute" random data.
-
I run on custom box QOTOM-Q355G4
Not sure if I had this error before 2.5.0 or notOverall performance and work is normal
-
@chudak said in “snort *** Caught Term-Signal” errors:
I run on custom box QOTOM-Q355G4
Not sure if I had this error before 2.5.0 or notOverall performance and work is normal
A Signal 4 error is most definitely not "normal". Something severe happened in that case to trigger that error. That means the CPU tried to execute binary code that is incompatbile with the CPU platform.
That error is coming from the Snort binary and not from the PHP GUI package code.
-
Anything I can do to troubleshoot or maybe collect more info and log a bug?
-
@chudak said in “snort *** Caught Term-Signal” errors:
Anything I can do to troubleshoot or maybe collect more info and log a bug?
This is highly likely something peculiar to your configuration. Otherwise, there would be lots of reports on here. Yours is the only Signal 4 error I've seen posted. Compare that to all the SG-3100 folks with Signal 11 issues, for example. So that strongly indicates it's something in your configuration or it could be a hardware issue. As I've said, a Signal 4 should essentially never happen. It can only happen in running software if the memory is corrupted. One way for that to happen is for hardware failures such that on a memory read from an address the CPU gets back bogus data.
If you want to try some things on your end, the first one I would do is a hardware memory diagnostic test with something like
memtestto thoroughly shake-out your RAM and be sure no issue exists there. If that finds no issues, I would dump your Snort configuration and recreate it from scratch. However, my first bet is on faulty hardware. -
I am doubtful about h/w, otherwise would see something else. And it's been in used for several years. But see your point.
@bmeeks said in “snort *** Caught Term-Signal” errors:
I would dump your Snort configuration and recreate it from scratch.
Any way to dump it and not waste all settings ?
(I did do reinstall today will see maybe it gone)
-
I isolated the time for this issue => 12:45 am
It corresponds to the crontab entry :45 0,6,12,18 * * * root /usr/bin/nice -n20 /usr/local/bin/php -f /usr/local/pkg/snort/snort_check_for_rule_updates.phpFunny part is that it happens only at 12:45 !
I am lost and don't even know what to say :(
Interesting sequences in the log:
Mar 8 11:45:17 php-fpm 35099 /rc.newwanip: rc.newwanip: Info: starting on igb1. Mar 8 11:45:16 check_reload_status 378 Reloading filter Mar 8 11:45:16 check_reload_status 378 rc.newwanip starting igb1 Mar 8 11:45:16 php-fpm 35099 /rc.linkup: Hotplug event detected for LAN(lan) static IP (192.168.90.1 ) Mar 8 11:45:15 kernel igb1: link state changed to UP Mar 8 11:45:15 check_reload_status 378 Linkup starting igb1 Mar 8 11:45:14 php 83972 [Snort] The Rules update has finished. Mar 8 11:45:14 php 83972 [Snort] Snort has restarted with your new set of rules... Mar 8 11:45:12 check_reload_status 378 Reloading filter Mar 8 11:45:12 php-fpm 9076 /rc.linkup: Hotplug event detected for LAN(lan) static IP (192.168.90.1 ) Mar 8 11:45:11 kernel igb1: link state changed to DOWN Mar 8 11:45:11 check_reload_status 378 Linkup starting igb1 Mar 8 11:45:11 php 83972 [Snort] Snort START for WAN(igb0)... Mar 8 11:45:08 snort 93761 *** Caught Term-Signal Mar 8 11:45:07 php 83972 [Snort] Snort STOP for WAN(igb0)... Mar 8 11:45:07 php 83972 [Snort] Building new sid-msg.map file for WAN... Mar 8 11:45:07 php 83972 [Snort] Enabling any flowbit-required rules for: WAN... -
Still on every snort stop/restart see
Apr 2 12:48:08 php 81121 [Snort] Snort START for WAN(igb0)... Apr 2 12:48:08 php 81121 [Snort] Building new sid-msg.map file for WAN... Apr 2 12:48:08 php 81121 [Snort] Enabling any flowbit-required rules for: WAN... Apr 2 12:48:08 php 81121 [Snort] Enabling any flowbit-required rules for: WAN... Apr 2 12:48:07 php 81121 [Snort] Updating rules configuration for: WAN ... Apr 2 12:48:06 snort 1634 *** Caught Term-Signal Apr 2 12:48:05 php-fpm 73757 [Snort] Snort STOP for WAN(igb0)...removed and installed again with no luck :(
-
@chudak said in “snort *** Caught Term-Signal” errors:
Still on every snort stop/restart see
Apr 2 12:48:08 php 81121 [Snort] Snort START for WAN(igb0)... Apr 2 12:48:08 php 81121 [Snort] Building new sid-msg.map file for WAN... Apr 2 12:48:08 php 81121 [Snort] Enabling any flowbit-required rules for: WAN... Apr 2 12:48:08 php 81121 [Snort] Enabling any flowbit-required rules for: WAN... Apr 2 12:48:07 php 81121 [Snort] Updating rules configuration for: WAN ... Apr 2 12:48:06 snort 1634 *** Caught Term-Signal Apr 2 12:48:05 php-fpm 73757 [Snort] Snort STOP for WAN(igb0)...removed and installed again with no luck :(
That's perfectly normal. Snort stops, updates the rules, then starts itself again. That is all expected behavior. The cron task that checks for, and updates, new rules does this.
And if you are using the Inline IPS Mode, then this cycling of Snort will also cycle the physical interface due to the way the netmap kernel device operates. So that will show as a "down" then "up" cycle of the interface.
This automatic stop then restart behavior is exactly why I tell users to NEVER use the Service Watchdog service with either of the two IDS/IPS packages. Service Watchdog would see the "stop", and then immediately issue a restart command of its own while Snort was down updating the rules.
-
@bmeeks said in
This automatic stop then restart behavior is exactly why I tell users to NEVER use the Service Watchdog service with either of the two IDS/IPS packages. Service Watchdog would see the "stop", and then immediately issue a restart command of its own while Snort was down updating the rules.
That sounds good !
Are you saying that if snort is removed from Service Watchdog that ***** Caught Term-Signal** will disappear from the logs ? -
@chudak said in “snort *** Caught Term-Signal” errors:
Are you saying that if snort is removed from Service Watchdog that ***** Caught Term-Signal** will disappear from the logs ?
No, go back and read again what I said. The "Caught Term-Signal" is simply Snort's way of saying "I was told to shutdown, so I am". It is a completely harmless message. It is just logging the fact it is shutting down in a controlled fashion. It is the periodic cron task that actually tells Snort to shutdown, then restarts it.
-
I am happy to accept that this is normal
I was confused by “A Signal 4 error is most definitely not "normal". Something severe happened in that case to trigger that error. That means the CPU tried to execute binary code that is incompatbile with the CPU platform.”
Thx!
-
@chudak said in “snort *** Caught Term-Signal” errors:
I am happy to accept that this is normal
I was confused by “A Signal 4 error is most definitely not "normal". Something severe happened in that case to trigger that error. That means the CPU tried to execute binary code that is incompatbile with the CPU platform.”
Thx!
A Signal 4 error is indeed not normal. A Term-Signal is perfectly normal. They are not at all related.
