Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    eve.json log not exported

    IDS/IPS
    2
    4
    314
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • M
      michmoor LAYER 8 Rebel Alliance
      last edited by michmoor

      Ive noticed that i am not receiving any eve.json log files in my syslog server but i am getting every other system log dump from the firewall.
      When i go check the contents of the eve.json log file in the GUI i receive the following message

      f4813f16-25ed-4fb1-8ae0-eb8c2f25477b-image.png

      Strangely after attempting to view the log file i get the following system alert

      22:01:37 PHP ERROR: Type: 1, File: /usr/local/www/suricata/suricata_logs_browser.php, Line: 50, Message: Allowed memory size of 536870912 bytes exhausted (tried to allocate 738074848 bytes)
      
      
      

      edit:

      The eve.json log file exists on the system and is being populated

      0c87bad0-a464-4fc6-82f2-da98756b2249-image.png

      Firewall: NetGate,Palo Alto-VM,Juniper SRX
      Routing: Juniper, Arista, Cisco
      Switching: Juniper, Arista, Cisco
      Wireless: Unifi, Aruba IAP
      JNCIP,CCNP Enterprise

      1 Reply Last reply Reply Quote 0
      • bmeeksB
        bmeeks
        last edited by bmeeks

        The file will be way too large to view in the pfSense GUI. The text file handling capability of PHP is very low-tech. It must read the entire file into RAM, then stream the content line-by-line to the browser. So large files exceed the amount of free PHP memory (not related to how much other memory pfSense may be showing you as "free"). When initialized, PHP sets a limit on available PHP memory to prevent various types of DoS attacks.

        So, the short answer is you can't usually view most of the Suricata log files in the GUI because they get so huge so fast and outrun available PHP memory.

        Suricata has one huge shortcoming compared to Snort (and I'm talking at the binary level). Snort lets you set maximum log sizes, and when that size is reached the binary itself (the one doing the logging) will automatically rotate the file. Suricata has no such built-in feature. The best you have available in the binary is an option to rotate some of the log files (but not all) on a timed basis. That's not terribly useful since the files may grow to different sizes at different rates.

        The GUI package attempts to emulate Snort's behavior by forcibly closing the log files, rotating them, and then "touching" a new file and telling Suricata to reload.

        Most times that kluge works, but not always,. Sometimes the Suricata binary stubbornly refuses to start using the "new" log file and continues to write to the old one. That may be what is wrong in your case. To fix it, stop Suricata on all the interfaces, manually rotate and/or move the logs in question, then restart Suricata.

        Suricata upstream could really make things easier and better by offering more log rotation options. The EVE JSON log is a particularly bad offender because SO much data is logged there.

        M 1 Reply Last reply Reply Quote 0
        • M
          michmoor LAYER 8 Rebel Alliance @bmeeks
          last edited by michmoor

          @bmeeks thanks bill I’ll give it a whirl.
          What’s interesting is that this is on a low throughput interface and yet the log file grows to almost a gig.
          Is there a way to tame the eve log file? Should it even be tamed considering it contains important meta data.

          Firewall: NetGate,Palo Alto-VM,Juniper SRX
          Routing: Juniper, Arista, Cisco
          Switching: Juniper, Arista, Cisco
          Wireless: Unifi, Aruba IAP
          JNCIP,CCNP Enterprise

          bmeeksB 1 Reply Last reply Reply Quote 0
          • bmeeksB
            bmeeks @michmoor
            last edited by

            @michmoor said in eve.json log not exported:

            @bmeeks thanks bill I’ll give it a whirl.
            What’s interesting is that this is on a low throughput interface and yet the log file grows to almost a gig.
            Is there a way to tame the eve log file? Should it even be tamed considering it contains important meta data.

            The only way to tame is to reduce the options enabled on the INTERFACE SETTINGS tab for EVE Logs and perhaps reduce the rules. But I would start with reducing some of the logging depending on circumstances.

            1 Reply Last reply Reply Quote 0
            • First post
              Last post
            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.