pfSense Suricata and Snort logs -> Elastic: Huge logs > 100Gb / Day

[kylarem]

I've setup a filebeat to collect snort, suricata and zeek. ATM zeek doesn't seem to work. Snort's been running great for years on this machine without any issue.

Now I added suricata and a filebeat to collect logs for Elastic SIEM. But I get insane amount of information, it's about 100 Gigabyte per day. The issue doesn't appear on pfSense itself, just inside elasticsearch and kibana.

Also the amount of stuff, DNS, TLS, HTTP, is just ridiculous, I got like 1 million DNS requests per day, which can't be right, on top I "see" over 200 Terrabyte of data transfer. My line isn't capable of doing that. :D
This is not a large enterprise network, just one PC and a few other devices like smartphone, smarttv, just one person...

I'm 100% sure this is not caused by malware, I'm not part of a botnet. Lots of stuff being generated while I'm asleep and most devices are offline and I check pretty thorough for that kind of stuff.

My best guess is that the same things are collected over and over again, cause when I click through in elastic things seem to evaporate into nothing, which is the weirdest thing I've ever seen.

Any tips you guys can give to a tech lady?

Cheers, Kyla.

[NollipfSense]

@kylarem Did you select every rule on the planet for Suricata, Snort, and Zeek? I can understand and appreciate running both Suricata and Snort; however, for a home network, why on Earth do you need three IDS/IPS?

I suggest searching the forum for best practice IDS/IPS.