Problems disabling payload in suricata json.eve alert logs?

tc

Hi.

I'm sending Suricata logs over syslog to an ELK-stack, but I get some truncated lines in the syslog stream.

One of the loglines that get truncated are unfortunately the alert-message, so I tried to disable the payload part of the message to shorten it. But the payload is still in the output even if I uncheck the Log a packet dump with alerts checkbox. Looking at the suricata.yaml payload: yes is set in the alert section regardless of the checkbox.

Is this a bug in the config generator from the GUI, or have I misunderstood what the checkbox is used for?

Btw. has anyone looked at extending the syslog message limit?

I'm running Suricata 4.1.2_3.

/TC

bmeeks

@tc said in Problems disabling payload in suricata json.eve alert logs?:

Hi.

I'm sending Suricata logs over syslog to an ELK-stack, but I get some truncated lines in the syslog stream.

One of the loglines that get truncated are unfortunately the alert-message, so I tried to disable the payload part of the message to shorten it. But the payload is still in the output even if I uncheck the Log a packet dump with alerts checkbox. Looking at the suricata.yaml payload: yes is set in the alert section regardless of the checkbox.

Is this a bug in the config generator from the GUI, or have I misunderstood what the checkbox is used for?

Btw. has anyone looked at extending the syslog message limit?

I'm running Suricata 4.1.2_3.

/TC

Yes, this is a bug in the code that generates the YAML configuration file. I have logged it to be fixed in my internal tracker, but feel free to also log a bug on the Redmine Bug site for pfSense if you wish.

bmeeks

I initially thought this was a bug in the code that generates the suricata.yaml file for an interface, but upon further testing I am seeing that the YAML configuration parameter changes from "yes" to "no" as I toggle the checkboxes in the GUI between ON and OFF as expected. I do not have an ELK stack configuration to actually test sending of the log file data, though.

Where specifically on your firewall were you looking at the 'suricata.yaml' file that stayed set to "yes"? If you were looking at the file /usr/local/etc/suricata/suricata.yaml, that is not the file actually used by configured interfaces. That is just the sample template installed with the binary. The actual suricata.yaml file for each interface is found within a subdirectory inside /usr/local/etc/suricata for each interface. The applicable subdirectory for an interface will have the physical NIC interface name and a GUID as part of the subdirectory name.

tc

No, I am looking at the /usr/local/etc/suricata/suricata_39789_re0/suricata.yaml, and the payload: yes is still there even if the toggle in the web-gui is off.
But you don't have to have an ELK-stack to see this, just a syslog server to send the logs to.

This is an example of what I get in my syslog stream for one event:

Mar 21 21:22:10 pfsense suricata[98189]: [1:2011716:3] ET SCAN Sipvicious User-Agent Detected (friendly-scanner) [Classification: Attempted Information Leak] [Priority: 2] {UDP} 185.53.88.2:5224 -> REDACTED:5060
Mar 21 21:22:10 pfsense suricata[98189]: {"timestamp": "2019-03-21T21:22:10.009768+0100", "flow_id": 2126532570129960, "in_iface": "re0", "event_type": "alert", "src_ip": "185.53.88.2", "src_port": 5224, "dest_ip": "REDACTED", "dest_port": 5060, "proto": "UDP", "alert": {"action": "allowed", "gid": 1, "signature_id": 2011716, "rev": 3, "signature": "ET SCAN Sipvicious User-Agent Detected (friendly-scanner)", "category": "Attempted Information Leak", "severity": 2, "metadata": {"updated_at": ["2010_07_30"], "created_at": ["2010_07_30"]}}, "app_proto": "failed", "flow": {"pkts_toserver": 1, "pkts_toclient": 0, "bytes_toserver": 420, "bytes_toclient": 0, "start": "2019-03-21T21:22:10.009768+0100"}, "payload": "UkVHSVNURVIgc2lwOjE1OC4xNzQuMTEzLjQgU0lQLzIuMA0KVmlhOiBTSVAvMi4wL1VEUCAxODUuNTMuODguMjo1MjI0O2JyYW5jaD16OWhHNGJLLTMzNzM1NjYxNTE7cnBvcnQNCkNvbnRlbnQtTGVuZ3RoOiAwDQpGcm9tOiAicHBvYngiPHNpcDoxMDBAMS4xLjEuMT47dGFnPTM5NjU2MTY1MzczMTMwMzQzMTMzNjMzNDAxMzQzMjM0MzAzMzMzMzQzMDM4MzcNCkFjY2VwdDogYXBwbGljYXRpb24vc2Rw

even when the EVE Log Alert details Log a packet dump with alerts. is unchecked for that interface. Also note that there are som missing end quotes and brackets in that last line, so the line is truncated.

bmeeks

Hmm...that's the correct location. I'm working on the 4.1.3 update for the Suricata GUI package right now and I just tested this by toggling the GUI parameters on and off and it correctly wrote "yes" and "no" in the suricata.yaml file.

Let me try all possible combinations to see if I can reproduce the problem.

Does the file /usr/local/pkg/suricata/suricata_generate_yaml.php look like this beginning at line 398 on your system?

if (($suricatacfg['eve_log_alerts'] == 'on')) {
	$eve_out_types .= "\n        - alert:";
	$eve_out_types .= "\n            payload: ".($suricatacfg['eve_log_alerts_payload'] == 'on' || $suricatacfg['eve_log_alerts_payload'] == 'only-base64' ?'yes':'no ')."              # enable dumping payload in Base64";
	$eve_out_types .= "\n            payload-buffer-size: 4kb  # max size of payload buffer to output in eve-log";
	$eve_out_types .= "\n            payload-printable: ".($suricatacfg['eve_log_alerts_payload'] == 'on' || $suricatacfg['eve_log_alerts_payload'] == 'only-printable' ?'yes':'no ')."    # enable dumping payload in printable (lossy) format";
	$eve_out_types .= "\n            packet: ".($suricatacfg['eve_log_alerts_packet'] == 'on'?'yes':'no ')."               # enable dumping of packet (without stream segments)";
	$eve_out_types .= "\n            http-body: ".($suricatacfg['eve_log_alerts_payload'] == 'on'?'yes':'no ' || $suricatacfg['eve_log_alerts_payload'] == 'only-base64' ?'yes':'no ')."            # enable dumping of http body in Base64";
	$eve_out_types .= "\n            http-body-printable: ".($suricatacfg['eve_log_alerts_payload'] == 'on' || $suricatacfg['eve_log_alerts_payload'] == 'only-printable' ?'yes':'no ')."  # enable dumping of http body in printable format";
	$eve_out_types .= "\n            tagged-packets: yes       # enable logging of tagged packets for rules using the 'tag' keyword";
}

tc

@bmeeks Yes, as far as I can see there is no differences in that if-block.

bmeeks

That's weird. I've tested all combinations and it works for me. The settings I am toggling are on the INTERFACE SETTINGS tab down in the EVE section.

Changing the drop-down selector to "NO" turns off payload logging (sets that parameter to "no" in suricata.yaml), and toggling the checkbox turns off and on packet logging by setting that parameter to "no" and "yes" in suricata.yaml.

Here is the suricata.yaml section that results from the settings shown above:

      types: 
        - alert:
            payload: yes              # enable dumping payload in Base64
            payload-buffer-size: 4kb  # max size of payload buffer to output in eve-log
            payload-printable: yes    # enable dumping payload in printable (lossy) format
            packet: yes               # enable dumping of packet (without stream segments)
            http-body: yes            # enable dumping of http body in Base64
            http-body-printable: yes  # enable dumping of http body in printable format
            tagged-packets: yes       # enable logging of tagged packets for rules using the 'tag' keyword

And with this configuration:

this is the resulting suricata.yaml section:

      types: 
        - alert:
            payload: no               # enable dumping payload in Base64
            payload-buffer-size: 4kb  # max size of payload buffer to output in eve-log
            payload-printable: no     # enable dumping payload in printable (lossy) format
            packet: no                # enable dumping of packet (without stream segments)
            http-body: yes            # enable dumping of http body in Base64
            http-body-printable: no   # enable dumping of http body in printable format
            tagged-packets: yes       # enable logging of tagged packets for rules using the 'tag' keyword

tc

@bmeeks Aha.... This is a case of not understanding the difference between the dropdown and the checkbox (Data format vs dump).
I've only toggled the checkbox and assumed that it controlled the dump on/off.

Looks much better now.

bmeeks

Good deal! The confusion is also present in the official Suricata documentation, but the way I understand it the Payload selection lets you choose "NO" to disable payload logging, or choose any of the other three options (BASE64, Only Printable, or BOTH) to turn on payload logging.

The Log a packet dump checkbox controls just that setting.

tc

Some final notes on this, after collecting logs for a day.

Unfortunately this did not remove all my problems with truncated json,
some of the dns-entries in the log are also truncated, and probably other stuff as well that I haven't spotted yet.

The conclusion of all this is that trying to send json through the pfsense syslog is a dead end at the moment. There is just too much trouble and loss of information trying to trim it down to fit the pfsense syslog byte-limitations.

So until the syslog is updated to rsyslog or syslog-ng, I'm going to install and use the filebeat solution that has been suggested in another thread in this forum.
How to send Snort alert logs to Graylog without Barnyard2?

Thanks for helping out.

bmeeks

Something like filebeat is probably the best long-term solution. Snort3 is also starting the migration to JSON-type logging output.

I've thought about investigating what would be involved in creating a Filebeat package for pfSense. So far that has progressed no farther than just an idea.

token

Hello OP, how did your filebeats setup go?

I have run into this great log apocalypse and am searching for a solution. Really wish filebeats had made it into a package by now but I guess not.