Suricata repeatedfails to start on lan interface because of recurring stale file
-
Suricata repeatedly stops working on my lan interface, the log indicates that there is a problem with a stale file.
19/9/2017 – 10:43:24 - <error>-- [ERRCODE: SC_ERR_INITIALIZATION(45)] - pid file '/var/run/suricata_re152829.pid' exists but appears stale. Make sure Suricata is not running and then remove /var/run/suricata_re152829.pid. Aborting!</error>
Yes, removing the file solves the problem and suricata starts again on the interface. However after some time the problem appears again.
-
Does this happen after the restart from the daily rules update process? What other messages are in the firewall's system log around the same time? My first suspicion is two instances of Suricata trying to run and/or start on the same interface. That can happen if the firewall issues a "restart all packages" command during the same time Suricata is self-restarting after a rules update.
Also, are you using a RAMDISK? If so, check and make sure it is not running out of space.
Bill
-
The stale log file may just be a symptom that it didn't shut down properly, or didn't fully start up on a previous attempt.
My experience from today - Suricata 3.x was running fine, but then I updated to Suricata to 4.0.0_1, and then it failed to restart, even after removing the /var/run/suricata*.pid files, it would immediately fail again, so I checked the log:
(Suricata –-> Logs View -- suricata.log)
The log showed the well known STREAMS memory allocation error:
19/9/2017 -- 08:56:07 - <error>-- [ERRCODE: SC_ERR_POOL_INIT(66)] - alloc error
19/9/2017 – 08:56:07 - <error>-- [ERRCODE: SC_ERR_POOL_INIT(66)] - pool grow failedSo, I doubled stream memory cap and halved preallocated sessions as a starting point (note - this may not be an optimal configuration)
Stream Memory Cap 134217728
Preallocated Sessions 16384The result was a successful startup (PFSense XG-1540):
Intel(R) Xeon(R) CPU D-1541 @ 2.10GHz
Current: 2100 MHz, Max: 2101 MHz
16 CPUs: 1 package(s) x 8 core(s) x 2 SMT threads19/9/2017 – 09:03:26 - <notice>-- all 17 packet processing threads, 2 management threads initialized, engine started.
So...after removing the .pid files, check the logs to see if there may be an underlying issue if it fails again, and there might have been something that changed in the 4.0 update</notice></error></error>
-
I don't recall off the top of my head, but there is a forumula that uses the number of cores and memory to compute optimal stream memory settings. I think some adjustments to that area of the binary also happened between the 3.2.x and 4.0.0 versions of the code, so that might account for the fact of not having that exact error using 3.2.x.
Bill
-
I don't recall off the top of my head, but there is a forumula that uses the number of cores and memory to compute optimal stream memory settings. I think some adjustments to that area of the binary also happened between the 3.2.x and 4.0.0 versions of the code, so that might account for the fact of not having that exact error using 3.2.x.
Bill
Seems like I am now (after fixing other issues) having the same issue as op.
my supermicro a1sri-2758f has 8 cores and no threads, its non-standard core count I guess. Might explain the buggy multi threading issues I am experiencing.
-
I don't recall off the top of my head, but there is a forumula that uses the number of cores and memory to compute optimal stream memory settings. I think some adjustments to that area of the binary also happened between the 3.2.x and 4.0.0 versions of the code, so that might account for the fact of not having that exact error using 3.2.x.
Bill
Seems like I am now (after fixing other issues) having the same issue as op.
my supermicro a1sri-2758f has 8 cores and no threads, its non-standard core count I guess. Might explain the buggy multi threading issues I am experiencing.
See my reply in the other thread you linked. You need to at least double the Stream Mem Cap setting, and possible increase it even more.
Bill
