Does this look like a possible buffer overflow attempt?

NightlyShark

I ask about a possible buffer overflow attempt because of the null chars (\x00).

NightlyShark

Update: After rebooting pfSense, everything was back to normal, eventually... Immediately after rebooting, CPE gateway/router/VoIP combo went down, too. I think I don't need to say that it keeps no logs whatsoever.

NightlyShark

I know that at least this already exists, but I had no crashes or problems prior to this.

NightlyShark

Got it again:

NightlyShark

So, I hope this is the final UPDATE:
I went to view the logs, and no logs existed. It said that either they were not enabled, or that the files didn't exist.
I then went to 'Logs Management' tab, where also no logs existed. Upon saving the existing configuration, without changing anything, log files magically appeared.
I'm thinking, a bug, maybe?
When logs reach maximum size and are deleted, new files are not immediately created?
IDK.

bmeeks

Are you by chance using a RAM disk? If not, what hardware do you have (a spinning hard drive or some type of eMMC device)?

Check your pfSense system log to see if anything was recorded there relative to running out of space.

To answer your question about log rotation, the Suricata PHP package will attempt to rotate the logs (and delete those older than the minimum retention age) and then perform a "touch" operation to create new empty logs. However, the Suricata binary does not natively offer a log rotation option by size. It only offers rotation on a fixed time interval (and does not even allow that for all the logs). So, the PHP package code has to shuffle the physical log files and then send the Suricata binary daemon a SIGHUP telling it to restart logging. Sometimes the binary seems to ignore that request. Usually when this happens, the old log files continue to grow and grow until the disk space is consumed.

The original error you see there is a result of the code on the ALERTS tab (or in the Dashboard Widget) reading a corrupted alerts log file. Zeros are never expected there as Suricata will not write such an entry normally. Only a corrupted file read would result in that.

NightlyShark

@bmeeks Hi! Not using RAM disk, I have an HDD, but its monitored by TrueNAS, not running out of space.

NightlyShark

Somewhere here is the reason this happens:

$buf = "";

while (($buf = fgets($fd)) !== FALSE) {
    $fields = array();
    $tmp = array();

    // Parse alert log entry to find the parts we want to display
    $fields[0] = substr($buf, 0, strpos($buf, '  '));

    // The regular expression match below returns an array as follows:
    // [2] => GID, [3] => SID, [4] => REV, [5] => MSG, [6] => CLASSIFICATION, [7] = PRIORITY
    preg_match('/\[\*{2}\]\s\[((\d+):(\d+):(\d+))\]\s(.*)\[\*{2}\]\s\[Classification:\s(.*)\]\s\[Priority:\s(\d+)\]\s/', $buf, $tmp);
    $fields['gid'] = trim($tmp[2]);
    $fields['sid'] = trim($tmp[3]);
    $fields['rev'] = trim($tmp[4]);
    $fields['msg'] = trim($tmp[5]);
    $fields['class'] = trim($tmp[6]);
    $fields['priority'] = trim($tmp[7]);

    // The regular expression match below looks for the PROTO, SRC and DST fields
    // and returns an array as follows:
    // [1] = PROTO, [2] => SRC:SPORT [3] => DST:DPORT
    if (preg_match('/\{(.*)\}\s(.*)\s->\s(.*)/', $buf, $tmp)) {
        // Get SRC
        $fields['src'] = trim(substr($tmp[2], 0, strrpos($tmp[2], ':')));
        if (is_ipaddrv6($fields['src']))
            $fields['src'] = inet_ntop(inet_pton($fields['src']));

        // Get SPORT
        $fields['sport'] = trim(substr($tmp[2], strrpos($tmp[2], ':') + 1));

        // Get DST
        $fields['dst'] = trim(substr($tmp[3], 0, strrpos($tmp[3], ':')));
        if (is_ipaddrv6($fields['dst']))
            $fields['dst'] = inet_ntop(inet_pton($fields['dst']));

        // Get DPORT
        $fields['dport'] = trim(substr($tmp[3], strrpos($tmp[3], ':') + 1));
    }
    else {
        // If no PROTO and IP ADDR, then this is a DECODER EVENT
        $fields['src'] = gettext("Decoder Event");
        $fields['sport'] = "";
        $fields['dst'] = "";
        $fields['dport'] = "";
    }

    // Create a DateTime object from the event timestamp that
    // we can use to easily manipulate output formats.
    if (($event_tm = date_create_from_format("m/d/Y-H:i:s.u", $fields[0])) !== FALSE) {
        $suricata_alerts[$counter]['timestamp'] = strval(date_timestamp_get($event_tm));
        $suricata_alerts[$counter]['timeonly'] = date_format($event_tm, "H:i:s");
        $suricata_alerts[$counter]['dateonly'] = date_format($event_tm, "M d");
    }
    else {
        // For some reason the event timestamp was invalid, 
        // set some default empty values for the fields.
        $suricata_alerts[$counter]['timestamp'] = 0;
        $suricata_alerts[$counter]['timeonly'] = ' ';
        $suricata_alerts[$counter]['dateonly'] = ' ';
    }

bmeeks

@NightlyShark
I'm thinking the root cause here is a corrupted alerts log file. The PHP code is parsing the alerts log file in /var/log/suricata/suricata_xxxx/ to grab the individual fields and store them into a multidimensional array.

The "time" field is the first one parsed, and it happens with this line of code:

// Parse alert log entry to find the parts we want to display
$fields[0] = substr($buf, 0, strpos($buf, '  '));

Then later on in the code, this line executes to translate the parsed text into a PHP date object for later formatting.

// Create a DateTime object from the event timestamp that
// we can use to easily manipulate output formats.
if (($event_tm = date_create_from_format("m/d/Y-H:i:s.u", $fields[0])) !== FALSE) {

The line above is triggering the error because for some reason the text field is empty. That should never happen. Can you post the contents of the alerts log when this is happening?

NightlyShark

@bmeeks I am sorry for the delay, being an enthusiast puts a limit on how much time I can devote to my server... I disabled the widget and saw no other errors. I will re-enable it (just for testing) and will pull any logs on the first available chance.