Suricata Log Parser - Python 3 Script
-
I'm currently using Suricata in Inline Mode and if you go to Services > Suricata > Alerts, on the Save or Remove Logs line, pfSense gives you the ability to Download the Suricata logs. I currently have 1237 entries in this log. Going through each log file individually would be a real chore and I'm not looking to do that. I'm no developer by any means and I'm not looking to reproduce the wheel but I'm wondering if anyone knows of a Python 3 script that would parse through all of the logs in bulk, deduplicate all of the entries, and then display the remaining descriptions? If no one knows of a script, are there any developers out there that would be interested? It would be nice if the script would work on Mac and Linux. That would be most helpful. Thanks.
-
With some trepidation (the setup for this isn't simple), I suggest you look into setting up a Graylog server to receive EVE JSON from Suricata on pfSense and then using Grafana to interact with the data in a useful way.
I'm not an expert on either, and won't be much help should you run into issues. I know that Graylog has an OVA image that can be used and I have a Grafana dashboard I've configured to my liking that I can share. It looks like this (modified from an example found online):
This type of setup can use an enormous amount of disk space depending on what you log. If you just want Suricata Alerts, it won't be too bad. But if you enable all of the EVE logging from Suricata you can easily end up storing multiple GB of log data per day...