Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    pfSense Suricata and Snort logs -> Elastic: Huge logs > 100Gb / Day

    Scheduled Pinned Locked Moved IDS/IPS
    2 Posts 2 Posters 831 Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • K
      kylarem
      last edited by kylarem

      I've setup a filebeat to collect snort, suricata and zeek. ATM zeek doesn't seem to work. Snort's been running great for years on this machine without any issue.

      Now I added suricata and a filebeat to collect logs for Elastic SIEM. But I get insane amount of information, it's about 100 Gigabyte per day. The issue doesn't appear on pfSense itself, just inside elasticsearch and kibana.

      Also the amount of stuff, DNS, TLS, HTTP, is just ridiculous, I got like 1 million DNS requests per day, which can't be right, on top I "see" over 200 Terrabyte of data transfer. My line isn't capable of doing that. :D
      This is not a large enterprise network, just one PC and a few other devices like smartphone, smarttv, just one person...

      I'm 100% sure this is not caused by malware, I'm not part of a botnet. Lots of stuff being generated while I'm asleep and most devices are offline and I check pretty thorough for that kind of stuff.

      My best guess is that the same things are collected over and over again, cause when I click through in elastic things seem to evaporate into nothing, which is the weirdest thing I've ever seen.

      Any tips you guys can give to a tech lady?

      Cheers, Kyla.

      NollipfSenseN 1 Reply Last reply Reply Quote 0
      • NollipfSenseN
        NollipfSense @kylarem
        last edited by NollipfSense

        @kylarem Did you select every rule on the planet for Suricata, Snort, and Zeek? I can understand and appreciate running both Suricata and Snort; however, for a home network, why on Earth do you need three IDS/IPS?

        I suggest searching the forum for best practice IDS/IPS.

        pfSense+ 23.09 Lenovo Thinkcentre M93P SFF Quadcore i7 dual Raid-ZFS 128GB-SSD 32GB-RAM PCI-Intel i350-t4 NIC, -Intel QAT 8950.
        pfSense+ 23.09 VM-Proxmox, Dell Precision Xeon-W2155 Nvme 500GB-ZFS 128GB-RAM PCIe-Intel i350-t4, Intel QAT-8950, P-cloud.

        1 Reply Last reply Reply Quote 0
        • First post
          Last post
        Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.