After update to 23.09 revewing log files in Suricata produces a PHP memory allocation error

ronv42

I did the update over last weekend and was doing my normal log file reviews in Suricata and it wouldn't load the alerts.log file and then an error shows up on the status bar:

Crash report begins.  Anonymous machine information:

amd64
14.0-CURRENT
FreeBSD 14.0-CURRENT amd64 1400094 #0 plus-RELENG_23_09-n256163-2763857e770: Wed Nov  1 21:18:24 UTC 2023     root@freebsd:/var/jenkins/workspace/pfSense-Plus-snapshots-23_09-main/obj/amd64/WrS3lKLo/var/jenkins/workspace/pfSense-Plus-snapshots-23_09-main/

Crash report details:

PHP Errors:
[16-Nov-2023 06:05:04 US/Central] PHP Fatal error:  Allowed memory size of 536870912 bytes exhausted (tried to allocate 547363440 bytes) in /usr/local/www/suricata/suricata_logs_browser.php on line 54
[16-Nov-2023 06:05:13 US/Central] PHP Fatal error:  Allowed memory size of 536870912 bytes exhausted (tried to allocate 547363440 bytes) in /usr/local/www/suricata/suricata_logs_browser.php on line 54

Memory isn't an issue on this device only 20% of 8 GB of RAM is currently in use. Looks like a PHP configuration issue for the log modules.

bmeeks

This is not unexpected if the log file (or any text file) you are trying to view in the GUI is large. PHP is not a great text file viewer/editor subsystem because it can't easily maintain state across web transactions. Currently it needs to first load the entire file into memory as a huge string and then dump that entire string out as HTML text.

The amount of memory reserved for PHP processes is much less than the amount of physical RAM in the machine. You could have 128 GB of RAM and it still would crash out because the reserved PHP process memory will be exhausted. You will need to copy that file off to a remote machine for viewing or else try the command-line vi editor while logged into a shell prompt.

To prevent this issue in the future, make sure automatic Log Managment is enabled under the LOGS MGMT tab. Choose reasonable max file sizes under that tab. The defaults should be fine.

ronv42

@bmeeks I have automatic log management enabled and kept the defaults. One may think that the logs aren't being pruned. Looking at the log settings for alerts it's set to 500kb.

bmeeks

A 500K log file should be no problem. Nor any file up to even a few megabytes, although it will load and display rather sluggishly.

Perhaps you do have a file that failed to rotate or something. There are many things that can interfere with the rotation as unfortunately that is not a native feature in the Suricata binary.

The GUI package on pfSense has to "force" log file rotation with some code tricks, but occasionally the rotation can fail and the Suricata binary daemon will continue writing to the same log file growing it very large. If this happens, you will need to manage that log via the command-line interface in a console or SSH shell session.

Log files reside under /var/log/suricata/. There are subdirectories underneath that path for each configured Suricata interface.

ronv42

@bmeeks Thanks I am exploring that directory right now. If I delete the logs there would that cause any issues?

bmeeks

@ronv42 said in After update to 23.09 revewing log files in Suricata produces a PHP memory allocation error:

@bmeeks Thanks I am exploring that directory right now. If I delete the logs there would that cause any issues?

No, deleting files there is not a problem with one caveat. If you delete a current file (meaning one of the files without a UNIX timestamp appended), then the currently running Suricata instance on that interface may cease logging until it is restarted or the proper SIGHUP is issued to signal it to re-initialize the log files it was using.

Rotated files will have a UNIX timestamp appended to their filename. "Active" files will not have a timestamp appended.