Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Snort - limiting log directory size

    Scheduled Pinned Locked Moved pfSense Packages
    13 Posts 3 Posters 7.2k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • D
      demco
      last edited by

      Hi,

      Has anyone encounter issue with the log directory limit feature?

      Enabled the feature with 8MB size limit; But the alert file size has grown to more than 40MB and /var usage is > 100%. Tried manually run the command in crontab, but the alert file size remain the same.

      Running pfsense 2.1 amd64 nanobsd, with snort 2.9.5.5 pkg v3.0.1.

      1 Reply Last reply Reply Quote 0
      • bmeeksB
        bmeeks
        last edited by

        @demco:

        Hi,

        Has anyone encounter issue with the log directory limit feature?

        Enabled the feature with 8MB size limit; But the alert file size has grown to more than 40MB and /var usage is > 100%. Tried manually run the command in crontab, but the alert file size remain the same.

        Running pfsense 2.1 amd64 nanobsd, with snort 2.9.5.5 pkg v3.0.1.

        To be honest I've never tested that code since I sort of inherited maintenance of the Snort package last year.  I just assume it worked.  I have plenty of disk space on my firewall and just never paid this setting any attention.  I will take a look at the code.

        Bill

        1 Reply Last reply Reply Quote 0
        • bmeeksB
          bmeeks
          last edited by

          I need a little feedback on what you are seeing to confirm what I think I found in the code.  In your /var/log/snort directories, do you see both an alert file and then one or more files with .u2. in the name?

          If you do, and it's the alert file that is too big, then I found the issue.  The automatic delete code is only cleaning up the .u2. unified2 log files.  It is not touching the alert file.  I can fix that easily in the next update of the Snort package.

          Bill

          1 Reply Last reply Reply Quote 0
          • D
            demco
            last edited by

            Inside the directory /var/log/snort/snort_<instance>/ there are only

            • alert
            • em1.stats
            • snort.log. <timestamp>- barnyard2 folder

            There are no .u2. files. bardyard2 folder is empty as I didn't enable it.

            alert file will grow in size. There will be a lot of snort.log with different timestamp.

            I did add '-K none' to the snort startup command, so that snort.log.* are not generated. Previously these used up the /var /log space. Look like now is the alert file.</timestamp></instance>

            1 Reply Last reply Reply Quote 0
            • bmeeksB
              bmeeks
              last edited by

              @demco:

              Inside the directory /var/log/snort/snort_<instance>/ there are only

              • alert
              • em1.stats
              • snort.log. <timestamp>- barnyard2 folder

              There are no .u2. files. bardyard2 folder is empty as I didn't enable it.

              alert file will grow in size. There will be a lot of snort.log with different timestamp.

              I did add '-K none' to the snort startup command, so that snort.log.* are not generated. Previously these used up the /var /log space. Look like now is the alert file.</timestamp></instance>

              OK.  Thanks for the information.  I believe I know what the problem is and will correct it in the next package update.  For now, a workaround for you will be to stop Snort, remove the large alerts file, then restart Snort.  Don't tinker with the file while Snort is running.  It will lose sync and stop writing to the file.

              Bill

              1 Reply Last reply Reply Quote 0
              • D
                demco
                last edited by

                Thanks Bill.

                Would this update also remove pcap files, for my case they are snort.log.*. Or just trim alert file.

                1 Reply Last reply Reply Quote 0
                • bmeeksB
                  bmeeks
                  last edited by

                  @demco:

                  Thanks Bill.

                  Would this update also remove pcap files, for my case they are snort.log.*. Or just trim alert file.

                  Yes, I will make sure it cleans up all the files in the log directory for the interface.  I noticed that the pattern-matcher that grabs up the log files for deletion was also missing the pattern for those files.  It was only looking for the .u2. files used for Barnyard2.

                  Bill

                  1 Reply Last reply Reply Quote 0
                  • M
                    mallison01
                    last edited by

                    i am also seeing this problem on:

                    2.1-RELEASE (i386)
                    built on Wed Sep 11 18:16:44 EDT 2013
                    FreeBSD 8.3-RELEASE-p11

                    Platform nanobsd (4g)

                    Snort 2.9.5.5 pkg v3.0.1

                    i am on a watchguard x-e platform embedded, with basic vrt rules and cant run snort for more than maybe an hour and it fills the /var directory to 109% so quickly i can watch it fill up, a restart of pfsense will reset everything, i did have the default /var directory at 60m and even tried to up it to 200m and it then filled that up and ran out of swap and killed the process.

                    on the snort package i have barnyard2 disabled, snort is only on a single wan iface and under global settings-general settings; i have the setting:

                    enable directory log limit size and have it set to 5mb

                    when /var fills up i can go to the alerts tab and click on clear logs and it empties several megs of space, way more than the 5mb i have the limit set to.

                    However, with a 200m meg /var size, i can restart pfsense with snort disable and watch the /var size hang right about 25%, but when i start snort it runs up to 109% very quickly, i can stop snort and clear the log through the gui, but i do not drop back down to the 25% i was seeing until a restart.

                    so the question is where/what is all the space coming from and going, looking in winscp i do not see any other large files outside of snort.

                    the only other packages i have is pfblocker, bandwithd, ovpn export, rrd summary, bind (disabled), snort widget, lcdproc, service watchdog.

                    i have a single instance of captive portal, 2 iface with dhcp and everything else is pretty much basic settings.

                    everything runs great and works great until /var fills which is to quick to leave snort running.

                    id really like to run snort if at all possible, is there a fix for this for me or when do you think there will be an update to the snort package?

                    I have a mid size virtual environment and was looking into moving all logs to a virtual syslog server, is it possible to have snort log to an external syslog to help with this issue?

                    1 Reply Last reply Reply Quote 0
                    • bmeeksB
                      bmeeks
                      last edited by

                      @mallison01:

                      i am also seeing this problem on:

                      2.1-RELEASE (i386)
                      built on Wed Sep 11 18:16:44 EDT 2013
                      FreeBSD 8.3-RELEASE-p11

                      Platform nanobsd (4g)

                      Snort 2.9.5.5 pkg v3.0.1

                      i am on a watchguard x-e platform embedded, with basic vrt rules and cant run snort for more than maybe an hour and it fills the /var directory to 109% so quickly i can watch it fill up, a restart of pfsense will reset everything, i did have the default /var directory at 60m and even tried to up it to 200m and it then filled that up and ran out of swap and killed the process.

                      on the snort package i have barnyard2 disabled, snort is only on a single wan iface and under global settings-general settings; i have the setting:

                      enable directory log limit size and have it set to 5mb

                      when /var fills up i can go to the alerts tab and click on clear logs and it empties several megs of space, way more than the 5mb i have the limit set to.

                      However, with a 200m meg /var size, i can restart pfsense with snort disable and watch the /var size hang right about 25%, but when i start snort it runs up to 109% very quickly, i can stop snort and clear the log through the gui, but i do not drop back down to the 25% i was seeing until a restart.

                      so the question is where/what is all the space coming from and going, looking in winscp i do not see any other large files outside of snort.

                      the only other packages i have is pfblocker, bandwithd, ovpn export, rrd summary, bind (disabled), snort widget, lcdproc, service watchdog.

                      i have a single instance of captive portal, 2 iface with dhcp and everything else is pretty much basic settings.

                      everything runs great and works great until /var fills which is to quick to leave snort running.

                      id really like to run snort if at all possible, is there a fix for this for me or when do you think there will be an update to the snort package?

                      I have a mid size virtual environment and was looking into moving all logs to a virtual syslog server, is it possible to have snort log to an external syslog to help with this issue?

                      Snort needs its logs local to be able to read the alert file for displaying alerts on the ALERTS tab.  I will try and get an update out for this in a few days.

                      Bill

                      1 Reply Last reply Reply Quote 0
                      • M
                        mallison01
                        last edited by

                        Ok, i was kinda figuring this, its sounding like your on top of the problem and im assuming from what the other guy explained and from what i read that this seems to be my problem?

                        Once your fix is done, i should be able to run snort on the embedded image? minimal rule set of course?

                        Is there any way to run a carp setup with snort on only one of the firewalls, i can run it in my virtual environment however if it was to go down, i would want the hardware pfsense to take over so i can remotely access, any idea on it that possible? basically a typical carp HA setup but with snort only on the primary pfsense?

                        1 Reply Last reply Reply Quote 0
                        • bmeeksB
                          bmeeks
                          last edited by

                          @mallison01:

                          Once your fix is done, i should be able to run snort on the embedded image? minimal rule set of course?

                          Yes, you should be able to run it on the embedded image provided you give it enough RAM for the rules sets chosen.

                          @mallison01:

                          Is there any way to run a carp setup with snort on only one of the firewalls, i can run it in my virtual environment however if it was to go down, i would want the hardware pfsense to take over so i can remotely access, any idea on it that possible? basically a typical carp HA setup but with snort only on the primary pfsense?

                          I'm no CARP expert, but I suppose you can run Snort on only one of the members.

                          Bill

                          1 Reply Last reply Reply Quote 0
                          • M
                            mallison01
                            last edited by

                            ok sounds good, thanks.

                            if you need some one to test your new code id be interested.

                            1 Reply Last reply Reply Quote 0
                            • bmeeksB
                              bmeeks
                              last edited by

                              @mallison01:

                              ok sounds good, thanks.

                              if you need some one to test your new code id be interested.

                              Thank for the offer, but I was able to test the fix in my VMware environment.  The fix for auto-log rotation is ready and should be posted in a few days.  I found several issues in that old code.  Thank you for pointing out the problem.  I'm glad I looked… :)

                              Bill

                              1 Reply Last reply Reply Quote 0
                              • First post
                                Last post
                              Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.