System Crash Report

mariusm

I got notified of this crash last night:

Crash report begins.  Anonymous machine information:

amd64
12.3-STABLE
FreeBSD 12.3-STABLE RELENG_2_6_0-n226742-1285d6d205f pfSense

Crash report details:

PHP Errors:
[16-May-2022 23:23:32 America/Los_Angeles] PHP Fatal error:  Allowed memory size of 536870912 bytes exhausted (tried to allocate 2914856448 bytes) in /usr/local/www/suricata/suricata_logs_browser.php on line 54



No FreeBSD crash data found.

The system didn't reboot. I believe I was browsing the Suricata logs at the time. Please let me know if you need more information or if I should report this somewhere else.

cmcdonald

@mariusm

Either you're hitting a bug that is causing an infinite loop or your logs are enormous, both could exhaust PHPs memory allocator and cause a premature request termination

mariusm

@cmcdonald The logs are indeed huge.

-rw-r--r--  1 root  wheel  3589175583 May 17 20:22 alerts.log
-rw-r--r--  1 root  wheel   154226440 May 17 20:22 http.log
-rw-r--r--  1 root  wheel        1394 May 17 12:15 suricata.log

I'm working on disabling the rules that produce many bogus alerts.
Meanwhile is there a place where I can configure the logs to rotate and expire?

mariusm

I found the settings for max log size and retention period; they are at their default settings (alert 500KB, http 1MB) which look like they are somehow not being obeyed.

jimp

@mariusm said in System Crash Report:

I found the settings for max log size and retention period; they are at their default settings (alert 500KB, http 1MB) which look like they are somehow not being obeyed.

Did you look at the settings in the system logs for that or in Suricata? The files you showed appear to be specific to suricata and thus are managed separately.

bmeeks

Are you running the latest version of the Suricata package? There were some fixes a few versions back that addressed a problem with log rotation.

bmeeks

@mariusm said in System Crash Report:

I found the settings for max log size and retention period; they are at their default settings (alert 500KB, http 1MB) which look like they are somehow not being obeyed.

Also make sure the check box at the top is ticked (and saved if you have to tick it).

One way that runaway logs can happen is if you get a duplicate Suricata instance running on an interface. That duplicate instance will not always respond to log rotation commands and thus it will continue to write to the same log file forever (until the firewall is rebooted or that rogue process is manually killed by the user).

mariusm

Thanks all for the help. Manually clearing the logs and then restarting suricata seems to have helped; I now see the logs rotating as they should.