Snort - limiting log directory size
-
Hi,
Has anyone encounter issue with the log directory limit feature?
Enabled the feature with 8MB size limit; But the alert file size has grown to more than 40MB and /var usage is > 100%. Tried manually run the command in crontab, but the alert file size remain the same.
Running pfsense 2.1 amd64 nanobsd, with snort 2.9.5.5 pkg v3.0.1.
-
Hi,
Has anyone encounter issue with the log directory limit feature?
Enabled the feature with 8MB size limit; But the alert file size has grown to more than 40MB and /var usage is > 100%. Tried manually run the command in crontab, but the alert file size remain the same.
Running pfsense 2.1 amd64 nanobsd, with snort 2.9.5.5 pkg v3.0.1.
To be honest I've never tested that code since I sort of inherited maintenance of the Snort package last year. I just assume it worked. I have plenty of disk space on my firewall and just never paid this setting any attention. I will take a look at the code.
Bill
-
I need a little feedback on what you are seeing to confirm what I think I found in the code. In your /var/log/snort directories, do you see both an alert file and then one or more files with .u2. in the name?
If you do, and it's the alert file that is too big, then I found the issue. The automatic delete code is only cleaning up the .u2. unified2 log files. It is not touching the alert file. I can fix that easily in the next update of the Snort package.
Bill
-
Inside the directory /var/log/snort/snort_<instance>/ there are only
- alert
- em1.stats
- snort.log. <timestamp>- barnyard2 folder
There are no .u2. files. bardyard2 folder is empty as I didn't enable it.
alert file will grow in size. There will be a lot of snort.log with different timestamp.
I did add '-K none' to the snort startup command, so that snort.log.* are not generated. Previously these used up the /var /log space. Look like now is the alert file.</timestamp></instance>
-
Inside the directory /var/log/snort/snort_<instance>/ there are only
- alert
- em1.stats
- snort.log. <timestamp>- barnyard2 folder
There are no .u2. files. bardyard2 folder is empty as I didn't enable it.
alert file will grow in size. There will be a lot of snort.log with different timestamp.
I did add '-K none' to the snort startup command, so that snort.log.* are not generated. Previously these used up the /var /log space. Look like now is the alert file.</timestamp></instance>
OK. Thanks for the information. I believe I know what the problem is and will correct it in the next package update. For now, a workaround for you will be to stop Snort, remove the large alerts file, then restart Snort. Don't tinker with the file while Snort is running. It will lose sync and stop writing to the file.
Bill
-
Thanks Bill.
Would this update also remove pcap files, for my case they are snort.log.*. Or just trim alert file.
-
Thanks Bill.
Would this update also remove pcap files, for my case they are snort.log.*. Or just trim alert file.
Yes, I will make sure it cleans up all the files in the log directory for the interface. I noticed that the pattern-matcher that grabs up the log files for deletion was also missing the pattern for those files. It was only looking for the .u2. files used for Barnyard2.
Bill
-
i am also seeing this problem on:
2.1-RELEASE (i386)
built on Wed Sep 11 18:16:44 EDT 2013
FreeBSD 8.3-RELEASE-p11Platform nanobsd (4g)
Snort 2.9.5.5 pkg v3.0.1
i am on a watchguard x-e platform embedded, with basic vrt rules and cant run snort for more than maybe an hour and it fills the /var directory to 109% so quickly i can watch it fill up, a restart of pfsense will reset everything, i did have the default /var directory at 60m and even tried to up it to 200m and it then filled that up and ran out of swap and killed the process.
on the snort package i have barnyard2 disabled, snort is only on a single wan iface and under global settings-general settings; i have the setting:
enable directory log limit size and have it set to 5mb
when /var fills up i can go to the alerts tab and click on clear logs and it empties several megs of space, way more than the 5mb i have the limit set to.
However, with a 200m meg /var size, i can restart pfsense with snort disable and watch the /var size hang right about 25%, but when i start snort it runs up to 109% very quickly, i can stop snort and clear the log through the gui, but i do not drop back down to the 25% i was seeing until a restart.
so the question is where/what is all the space coming from and going, looking in winscp i do not see any other large files outside of snort.
the only other packages i have is pfblocker, bandwithd, ovpn export, rrd summary, bind (disabled), snort widget, lcdproc, service watchdog.
i have a single instance of captive portal, 2 iface with dhcp and everything else is pretty much basic settings.
everything runs great and works great until /var fills which is to quick to leave snort running.
id really like to run snort if at all possible, is there a fix for this for me or when do you think there will be an update to the snort package?
I have a mid size virtual environment and was looking into moving all logs to a virtual syslog server, is it possible to have snort log to an external syslog to help with this issue?
-
i am also seeing this problem on:
2.1-RELEASE (i386)
built on Wed Sep 11 18:16:44 EDT 2013
FreeBSD 8.3-RELEASE-p11Platform nanobsd (4g)
Snort 2.9.5.5 pkg v3.0.1
i am on a watchguard x-e platform embedded, with basic vrt rules and cant run snort for more than maybe an hour and it fills the /var directory to 109% so quickly i can watch it fill up, a restart of pfsense will reset everything, i did have the default /var directory at 60m and even tried to up it to 200m and it then filled that up and ran out of swap and killed the process.
on the snort package i have barnyard2 disabled, snort is only on a single wan iface and under global settings-general settings; i have the setting:
enable directory log limit size and have it set to 5mb
when /var fills up i can go to the alerts tab and click on clear logs and it empties several megs of space, way more than the 5mb i have the limit set to.
However, with a 200m meg /var size, i can restart pfsense with snort disable and watch the /var size hang right about 25%, but when i start snort it runs up to 109% very quickly, i can stop snort and clear the log through the gui, but i do not drop back down to the 25% i was seeing until a restart.
so the question is where/what is all the space coming from and going, looking in winscp i do not see any other large files outside of snort.
the only other packages i have is pfblocker, bandwithd, ovpn export, rrd summary, bind (disabled), snort widget, lcdproc, service watchdog.
i have a single instance of captive portal, 2 iface with dhcp and everything else is pretty much basic settings.
everything runs great and works great until /var fills which is to quick to leave snort running.
id really like to run snort if at all possible, is there a fix for this for me or when do you think there will be an update to the snort package?
I have a mid size virtual environment and was looking into moving all logs to a virtual syslog server, is it possible to have snort log to an external syslog to help with this issue?
Snort needs its logs local to be able to read the alert file for displaying alerts on the ALERTS tab. I will try and get an update out for this in a few days.
Bill
-
Ok, i was kinda figuring this, its sounding like your on top of the problem and im assuming from what the other guy explained and from what i read that this seems to be my problem?
Once your fix is done, i should be able to run snort on the embedded image? minimal rule set of course?
Is there any way to run a carp setup with snort on only one of the firewalls, i can run it in my virtual environment however if it was to go down, i would want the hardware pfsense to take over so i can remotely access, any idea on it that possible? basically a typical carp HA setup but with snort only on the primary pfsense?
-
Once your fix is done, i should be able to run snort on the embedded image? minimal rule set of course?
Yes, you should be able to run it on the embedded image provided you give it enough RAM for the rules sets chosen.
Is there any way to run a carp setup with snort on only one of the firewalls, i can run it in my virtual environment however if it was to go down, i would want the hardware pfsense to take over so i can remotely access, any idea on it that possible? basically a typical carp HA setup but with snort only on the primary pfsense?
I'm no CARP expert, but I suppose you can run Snort on only one of the members.
Bill
-
ok sounds good, thanks.
if you need some one to test your new code id be interested.
-
ok sounds good, thanks.
if you need some one to test your new code id be interested.
Thank for the offer, but I was able to test the fix in my VMware environment. The fix for auto-log rotation is ready and should be posted in a few days. I found several issues in that old code. Thank you for pointing out the problem. I'm glad I looked… :)
Bill
