Suricata Unix Socket
-
Hi all,
I am trying to get Suricata to use telegraf to store its EVE entries in influxdb. I have done the research and came up with the following as the recommended install:
For all changes that I have done, I always restart the telegraf and suricata services so insure changes take effect. I also go back and check the config files to make sure they were not overwritten.
So first of all versions:
Pfsense Version
Suricata and Telegraf Version:
Telegraf Config:
--- I can confirm in influxdb it is writing information into the db, cpu, mem etc etc -- which means the pfsense appliance can get to influx and the influx < -- > telegraf setup is correct:
[[inputs.suricata]] source = "/tmp/suricata-stats.sock" delimiter = "_" [[outputs.influxdb]] urls = ["http://10.0.0.2:8086"] database = "pfsenselogs" ssl_ca = "/usr/local/etc/telegraf.ca"Suricata Config
Check that the file exists:
file /tmp/suricata-stats.sock /tmp/suricata-stats.sock: socket ls -la /tmp/suricata-stats.sock srwxr-xr-x 1 root wheel 0 Nov 24 14:50 /tmp/suricata-stats.sockEVE log Settings in: Config file: /usr/local/etc/suricata/suricata.yaml
# Extensible Event Format (nicknamed EVE) event log in JSON format - eve-log: enabled: yes filetype: regular #regular|syslog|unix_dgram|unix_stream|redis filename: eve.json pcap-file: false community-id: false community-id-seed: 0 xff: enabled: no mode: extra-data header: X-Forwarded-For <<<<TRUNCATED the rest - happy to provide if needed>>>>>>>>>>>>After all this config, I am not getting any Suricata logs stored InfluxDB. Could anyone provide any insight from what I can see I have tried to check everything and get the setup right but obviously it isnt.
Last question when the test in the Suricata settings says:
Suricata will periodically log statistics for this interface to Telegraf via a Unix socket.What does "periodically" mean? I use to use Filebeat into elastic stack which was near Realtime. So trying to understand the "periodically"
-
I need to see all of your EVE log settings from the
suricata.yamlfile for the interface. Hopefully you have discovered there are separatesuricata.yamlconfiguration files for each active Suricata interface. The complete configuration for each active Suricata interface is stored in a sub-directory under/usr/local/etc/suricata/. Also make sure you understand that you cannot edit any config file manually. Those files are recreated from scratch using a template each time you stop and restart Suricata from the GUI. Any change you make to the file is overwritten at the next start up.You should have a section in the
suricata.yamlfor the interface that looks like this with the execption the string variables enclosed in curly braces "{}" below will have their actual content printed to the file (so more llines of stuff will be showing with each expanded string).- eve-log: enabled: {$enable_eve_log} filetype: {$eve_output_type} filename: eve.json redis: {$eve_redis_output} identity: "suricata" facility: {$eve_systemlog_facility} level: {$eve_systemlog_priority} xff: enabled: {$eve_xff_enabled} mode: {$eve_xff_mode} deployment: {$eve_xff_deployment} header: {$eve_xff_header} types: {$eve_out_types} - eve-log: enabled: {$enable_telegraf_eve} filetype: unix_stream filename: {$telegraf_eve_sockname} types: - stats: threads: yesThe key thing here is that you must have two separate EVE configurations in the YAML file. One for the regular file logging, and a separate one for the UNIX socket that telegraf uses.
"Periodically" means stats are logged at the interval specified in the GUI for stats logging. There is a textbox where you can enter an interval value in seconds.
-
@bmeeks said in Suricata Unix Socket:
/usr/local/etc/suricata/
I didnt know that and thank you for highlighting that!
So for the WAN interface here is the suricata.yaml - when I look at the eve-log sections they look correct? I have truncated it to the section around eve-log
%YAML 1.1 --- max-pending-packets: 1024 # Runmode the engine should use. runmode: autofp # If set to auto, the variable is internally switched to 'router' in IPS # mode and 'sniffer-only' in IDS mode. host-mode: auto - eve-log: enabled: yes filetype: regular filename: eve.json redis: server: 127.0.0.1 port: 6379 mode: list key: "suricata" identity: "suricata" facility: local1 level: notice xff: enabled: yes mode: extra-data deployment: reverse header: X-Forwarded-For types: - alert: payload: yes # enable dumping payload in Base64 payload-buffer-size: 4kb # max size of payload buffer to output in eve-log payload-printable: yes # enable dumping payload in printable (lossy) format packet: yes # enable dumping of packet (without stream segments) http-body: yes # enable dumping of http body in Base64 http-body-printable: yes # enable dumping of http body in printable format metadata: yes # enable inclusion of app layer metadata with alert tagged-packets: yes # enable logging of tagged packets for rules using the 'tag' keyword - anomaly: enabled: yes types: decode: yes stream: yes applayer: yes packethdr: yes - http: extended: yes custom: [accept, accept-charset, accept-datetime, accept-encoding, accept-language, accept-range, age, allow, authorization, cache-control, connection, content-encoding, content-language, content-length, content-location, content-md5, content-range, content-type, cookie, date, dnt, etags, from, last-modified, link, location, max-forwards, origin, pragma, proxy-authenticate, proxy-authorization, range, referrer, refresh, retry-after, server, set-cookie, te, trailer, transfer-encoding, upgrade, vary, via, warning, www-authenticate, x-authenticated-user, x-flash-version, x-forwarded-proto, x-requested-with] - dns: version: 2 query: yes answer: yes - tls: extended: yes - dhcp: extended: yes - files: force-magic: no - ssh - nfs - smb - krb5 - ikev2 - tftp - rdp - sip - snmp - ftp - http2 - rfb - mqtt - smtp: extended: yes custom: [bcc, received, reply-to, x-mailer, x-originating-ip] md5: [subject] - stats: totals: yes deltas: no threads: no - flow # Bi-directional flows - netflow # Uni-directional flows - eve-log: enabled: yes filetype: unix_stream filename: /tmp/suricata-stats.sock types: - stats: threads: yes # Suricata user pass through configurationIn addition monitoring the suricata log /var/log/suricata/<interface>/suricata.log
25/11/2021 -- 11:50:01 - <Info> -- eve-log output device (unix_stream) initialized: /tmp/suricata-stats.sockIf I look at the sockets listed they have been established:
sockstat | grep suricata root suricata 97963 6 dgram -> /var/run/logpriv root suricata 97963 8 stream -> /tmp/suricata-stats.sock root suricata 97963 11 stream /var/run/php-fpm.socket root suricata 97963 12 stream /var/run/php-fpm.socket root telegraf 35603 8 stream /tmp/suricata-stats.sock root telegraf 35603 9 stream /tmp/suricata-stats.sock"Periodically" means stats are logged at the interval specified in the GUI for stats logging. There is a textbox where you can enter an interval value in seconds.Thank you that makes sense.
One additional question: What should I be expecting in Influx as in what measurement will it create under the DB I specified in telegraf?
-
@mynetworkrocks said in Suricata Unix Socket:
One additional question: What should I be expecting in Influx as in what measurement will it create under the DB I specified in telegraf?
Sorry, but I don't have an answer for that question. I have not used the telegraf feature. I believe others have, though. You might get more responses by posting in either the Packages or the IDS/IPS sub-forums here.
-
@bmeeks A quick question - looking at the config I posted do you see anything i need to adjust to get this to work?
-
@mynetworkrocks said in Suricata Unix Socket:
@bmeeks A quick question - looking at the config I posted do you see anything i need to adjust to get this to work?
Sorry, but I don't use telegraf. The configuration coding for that option was provided by a Suricata package user, and I just incorporated it into the next release of the GUI package.
There are some older telegraf threads in the Packages sub-forum here. You might find some answers by searching in those.
