Snort: Alert log format
-
Hi all
Trying to create the correct grok pattern for logstash to process my snort logs. All the grok patterns on any of the examples on the web dont match the pfsense alert log format. Seems like the snort package in pfsense uses its own format.
Questions:
- Is the log format documented anywhere so I can use it as a reference to create my own pattern?
- Is the log format stored in a config somewherr and can one change the format or even what it logs?
Thank you for the guidence
-
You may have some luck searching the Graylog "Marketplace" for Snort "extractors" -- there's a relatively active community that uses Snort and pfSense.
You may find this like helpful, though it's a regex example: https://github.com/Graylog2/graylog-guide-snort
Is pfSense doing something especially different with Snort alerts?
-
@carlos-magalhaes said in Snort: Alert log format:
Hi all
Trying to create the correct grok pattern for logstash to process my snort logs. All the grok patterns on any of the examples on the web dont match the pfsense alert log format. Seems like the snort package in pfsense uses its own format.
Questions:
- Is the log format documented anywhere so I can use it as a reference to create my own pattern?
- Is the log format stored in a config somewherr and can one change the format or even what it logs?
Thank you for the guidence
The alert log format for the pfSense package consists of these fields in a CSV file (comma-separated values):
timestamp,sig_generator,sig_id,sig_rev,msg,proto,src,srcport,dst,dstport,id,classification,priorityThese field tags are straight from the Snort documentation. The pfSense package adds one custom patch to enable output of Classification and Priority as CSV fields.
It should be a relatively easy task to create parsing templates for various alert aggregation systems.
You can't change the format of the output without breaking the display of alerts on the ALERTS tab. The PHP code for that tab expects the CSV fields to be present in a specific order.
-
@boobletins said in Snort: Alert log format:
You may have some luck searching the Graylog "Marketplace" for Snort "extractors" -- there's a relatively active community that uses Snort and pfSense.
You may find this like helpful, though it's a regex example: https://github.com/Graylog2/graylog-guide-snort
Is pfSense doing something especially different with Snort alerts?
Thank you for the response. I don’t think it is doing anything necessarily different. There are many grok patterns if you do a google search for the snort alerts. For some reason every single one I have tested does not extract the fields.
Now as the post below states there is one additional field that pfsense adds at the end. I need to go read the documentation for grok patterns. I am not sure of the pattern has to match the whole string exactly or it can just extract parts it finds in the string.
If it is an all or nothing then it might be thay last field that pfsense adds.
-
@bmeeks said in Snort: Alert log format:
@carlos-magalhaes said in Snort: Alert log format:
Hi all
Trying to create the correct grok pattern for logstash to process my snort logs. All the grok patterns on any of the examples on the web dont match the pfsense alert log format. Seems like the snort package in pfsense uses its own format.
Questions:
- Is the log format documented anywhere so I can use it as a reference to create my own pattern?
- Is the log format stored in a config somewherr and can one change the format or even what it logs?
Thank you for the guidence
The alert log format for the pfSense package consists of these fields in a CSV file (comma-separated values):
timestamp,sig_generator,sig_id,sig_rev,msg,proto,src,srcport,dst,dstport,id,classification,priorityThese field tags are straight from the Snort documentation. The pfSense package adds one custom patch to enable output of Classification and Priority as CSV fields.
It should be a relatively easy task to create parsing templates for various alert aggregation systems.
You can't change the format of the output without breaking the display of alerts on the ALERTS tab. The PHP code for that tab expects the CSV fields to be present in a specific order.
Thank you exactly the information I was looking for. As per my response above, just need to check the grok documentation. Not sure if the last field pfsense adds for priority breaks my grok pattern match.
Example: does grok have to find an exact complete string with the fields one has specified. Or it will extract whatever fields it matches in the string and ignore ones it doesnt have a pattern for.
If it ignores the data it does not have a pattern for then I have a different error in my code and need to go figure it out.
Once again thank you for the response.
-
Just an addition question, the msg portion of the csv format: for example -
11/10/18-07:59:50.444058 ,119,4,1,"(http_inspect) BARE BYTE UNICODE ENCODING",TCP,127.0.0.1,1186,127.0.0.2,80,58371,Not Suspicious Traffic,3- Time stamp for the log entry
- Not sure what 119,4,1 is or means?
- So understand the first part its the message/rule explanation
- Get the TCP - Protocol
- Get the source IP (just changed it obviously)
- Think the next item is source port?
- Get the destination IP (just changed it obviously)
- Think the next item is destination port?
- Item Not sure what this item is?
- Rule type ?
- Rule priority?
Just trying to understand the structure to build out the grok for this specific part of the log (msg). Other question - is it always in this format?
Thanks for the help so long.
-
@carlos-magalhaes
The 119,4,1 fields are GID (Generator ID), SID (Signature ID) and Revision, respectively.
The msg field is the text description of what the rule does or detects. It will have quotation marks around it in the log file to make sure any embedded commas in text are not interpreted as field delimiters.
Protocol is self-explanatory.
You have Source IP and port, then destination IP and port.
The item field escapes my memory at the moment. I will have to look it up.
Next are two custom fields added by pfSense: Classification and Priority.
-
Hi,
@bmeeks
(pfsense 2.6, snort 4.1.6)
can you add the "action" and "disposition" fields to the syslog logs.
Currently, pfsense does not send these fields to a remote syslog server.
Regards. -
@johnnybee I have the same question.
Please share with me if you have the answer.
Thanks in advance.
