Suricata Cron job wiping logs & alerts every 5 mins
-
Found the problem, the Services, Suricata, Logs Mgmt tab, log directory size limit was enabled. Disabling resolved the problem.
-
Found the problem, the Services, Suricata, Logs Mgmt tab, log directory size limit was enabled. Disabling resolved the problem.
This is a default setting to help with folks shooting themselves in the foot on disk-limited systems. You can either disable this protection (as you did), or you can significantly bump up the values to much larger limits.
Bill
-
Its in part linked to the Ram disk option where /tmp and /var can run in ram.
I set the ram disk to 100mb and set the suricata to max (10MB/3yr) logging options to see how it would react last night.
-
Its in part linked to the Ram disk option where /tmp and /var can run in ram.
I set the ram disk to 100mb and set the suricata to max (10MB/3yr) logging options to see how it would react last night.
Most certainly a RAM disk optioned system can run out of space on those partitions when Suricata logs traffic on a busy network. Be aware that some of the log limit options are handled via 5-minute cron tasks and are not done directly by Suricata itself. It's a mixed bag between what is cleaned up by the cron task and what is done automatically by the Suricata binary. The implication here is if you log a ton of stuff in the middle of one of those 5-minute cron job intervals, you can still run out of disk.
Bill
-
I saw the 5 minute cron job because it was removing the alerts so I like you say, I think it was a combination of the 5min cron job running whilst stuff still going over the network.
I noticed theres a new version of suricata available in the package manager, so I'll give that a go and see how things go.
-
I noticed theres a new version of suricata available in the package manager, so I'll give that a go and see how things go.
Well, there's nothing changed except for XMLRPC sync fixes.
-
Well despite reboots, the alerts logs were wiping every 5mins. Downloaded and testing latest version now, the alerts seem to remain visible for longer than 5mins so far, but I'm now not seeing the suricata_check_cron_misc.inc job being run in the system logs every 5mins.
Will see if a reboot makes a difference.
-
Well so far after a reboot, using latest version (2.1.7) the alerts are still removed when the 5 min cron job suricata_check_cron_misc.inc runs.
Suricata, Logs Management is configured as follows:
Remove Suricata Log Files during package uninstall: Ticked
Auto Log Management: Ticked
Log Directory Size Limit: Enabled
Size in MB: 100MB
Text Log Settings: all 9 options are set to 10 MB and 3 Years.
Unified2 Log Limit: 32
Unified2 Archived Log Retention Period: 7 Days
Captured Files Retention Period: 3 Years
Captured TLS Certs Retention Period: 3 Years.Have I setup the Logs Management incorrectly some how?
Incidentally, in Suricate: Global Settings, I have the option Remove Blocked Hosts Interval:Never, but I'm not blocking at the moment, just observing. I also have Log to System Log ticked which is how I know alerts are being removed the 5 min cron job runs.
I like the extra facilities Suricata offers, but its looking to me like Snort might be a bit more reliable and tested, thus perhaps worth jumping back to Suricata. Opinions?
TIA.
-
Here's a wonderful idea. Stop logging to extremely limited ramdisk when you expect logs to be persistent and not fill up the space? Sigh. Really, people…
-
I'm not logging to ramdisk, its logging to disk, I had in the past (several reboots ago) logged to ramdisk but abandoned that idea due to the fact the ramdisk at the time couldnt do a ramdisk for /tmp and normal disk for /var.
I also figured as theres 9 options each with a max of 10Mb, I figured a 100Mb directory limit should be enough, I could disable that 100MB directory limit option so theres a 100GB plus for the logs to use, just to be on the safe side and see what it does?
Edit. And I havent pulled this yet https://forum.pfsense.org/index.php?topic=101441.0 as I'm still trying to get a secure email server working.
