Suricata 2.0.3 Package Preview

bmeeks

After updating to Suricata 2.0.3 pkg v2.0.1 I got a lot of errors in the Widget and the Alert-tab.
Clearing the alert-logs and blocked-tab logs took care of it.

Thanks for the quick "avink" bug-fix!

Edit: Found another bug, after updating and editing an interface, the Barnyard tab gives the following error:
Fatal error: Can't use function return value in write context in /usr/local/www/suricata/suricata_barnyard.php on line 99

Edit: the "avink" bug was patched by Renato shortly after the initial merge to production and the pkg version bumped to 2.0.1

I will fix this as well. It was a last-minute patch to fix a problem another user reported with Snort that also impacted Suricata. I tested it on 2.2 pfSense with no issues. If you have 2.1 instead, maybe it's related to the difference in PHP versions. No matter, I know how to fix it and will submit a patch for this ~~and the Avink bug~~.

Bill

digdug3

Thanks Bill!

dotOne

As some people already discovered suricata seems to have problems with PPPoE connections and the log is filling up with these messages:

6/9/2014 – 20:58:10 - <error>-- [ERRCODE: SC_ERR_DATALINK_UNIMPLEMENTED(38)] - Error: datalink type 0 not yet supported in module DecodePcap
6/9/2014 – 20:58:10 - <error>-- [ERRCODE: SC_ERR_DATALINK_UNIMPLEMENTED(38)] - Error: datalink type 0 not yet supported in module DecodePcap
6/9/2014 – 20:58:10 - <error>-- [ERRCODE: SC_ERR_DATALINK_UNIMPLEMENTED(38)] - Error: datalink type 0 not yet supported in module DecodePcap</error></error></error>

According to the suricata website PPPoE should be supported so I did some searching and playing around on my test firewall.
What I did is instead of listening on the PPPoE interface I changed the listening interface on the configuration file to the underlying physical interface (em0) in my case and restarded suricata by hand

suricata.yaml:
pcap:
- interface: em0
checksum-checks: auto
promisc: yes

/usr/pbi/suricata-amd64/bin/suricata -i em0 -D -c /usr/pbi/suricata-amd64/etc/suricata/suricata_2925_pppoe0/suricata.yaml –pidfile /var/run/suricata_pppoe02925.pid

It looks like suricata can perfectly handle this and strips the PPPoE part looking at the actual data.
There are few messages reporting unrecognized ppp frames, but there are only a few of them. Will do some more checking but it looks suricata is running fine now.
It's logging and blocking as it should

So far so good.

Raul Ramos

Thanks @avink, i really appreciate this information.

I will try this (maybe tomorrow) on my real pfsense and report back.

bmeeks

@avink:

As some people already discovered suricata seems to have problems with PPPoE connections and the log is filling up with these messages:

6/9/2014 – 20:58:10 - <error>-- [ERRCODE: SC_ERR_DATALINK_UNIMPLEMENTED(38)] - Error: datalink type 0 not yet supported in module DecodePcap
6/9/2014 – 20:58:10 - <error>-- [ERRCODE: SC_ERR_DATALINK_UNIMPLEMENTED(38)] - Error: datalink type 0 not yet supported in module DecodePcap
6/9/2014 – 20:58:10 - <error>-- [ERRCODE: SC_ERR_DATALINK_UNIMPLEMENTED(38)] - Error: datalink type 0 not yet supported in module DecodePcap</error></error></error>

According to the suricata website PPPoE should be supported so I did some searching and playing around on my test firewall.
What I did is instead of listening on the PPPoE interface I changed the listening interface on the configuration file to the underlying physical interface (em0) in my case and restarded suricata by hand

suricata.yaml:
pcap:
- interface: em0
checksum-checks: auto
promisc: yes

/usr/pbi/suricata-amd64/bin/suricata -i em0 -D -c /usr/pbi/suricata-amd64/etc/suricata/suricata_2925_pppoe0/suricata.yaml –pidfile /var/run/suricata_pppoe02925.pid

It looks like suricata can perfectly handle this and strips the PPPoE part looking at the actual data.
There are few messages reporting unrecognized ppp frames, but there are only a few of them. Will do some more checking but it looks suricata is running fine now.
It's logging and blocking as it should

So far so good.

Thank you for the extra research. This is good information that maybe I can use to fix PPPoE with Suricata. I do not have a PPPoE connection to test with anymore since I switched from DSL to cable modem a couple of years ago. So the trick appears to be maybe capturing the actual physical interface behind the PPPoE interface.

I am sending you a PM so we can communicate about this some more offline.

Bill

Raul Ramos

@mais_um:

Thanks @avink, i really appreciate this information.

I will try this (maybe tomorrow) on my real pfsense and report back.

Tried on a em0 interface and it looks cool. Thanks again.

binaryjay

Thank you for the package… I was having stability issues with snort crashing when under heavy load repeatedly and decided to give suricata a try the DAY before the 2.0 package went up... :) Anyway I upgraded of course... and I have not had the service go down a single time yet. I thought snort was supposed to be the more stable of the two? Anyway...

The only issue I've run across was what I suspect to be suricata filling my /var partition to 100% with... something... causing everything else on the box to go haywire. I couldn't do much else but force a reboot and it has been fine since and will need to monitor growth of /var more carefully and see if it happens again. Logs were not persisted after reboot (thanks nanoBSD ram disk).

The reason I blame suricata was because it was the first time I've seen this happen and the only thing that changed was adding suricata and removing snort. I do have most of the logging turned OFF and the automatic log management turned ON with a very small size limit.

If I can come up with some actual proof of what happened (if it occurs again) I will post back.

This is on nanoBSD.

bmeeks

@binaryjay:

Thank you for the package… I was having stability issues with snort crashing when under heavy load repeatedly and decided to give suricata a try the DAY before the 2.0 package went up... :) Anyway I upgraded of course... and I have not had the service go down a single time yet. I thought snort was supposed to be the more stable of the two? Anyway...

The only issue I've run across was what I suspect to be suricata filling my /var partition to 100% with... something... causing everything else on the box to go haywire. I couldn't do much else but force a reboot and it has been fine since and will need to monitor growth of /var more carefully and see if it happens again. Logs were not persisted after reboot (thanks nanoBSD ram disk).

The reason I blame suricata was because it was the first time I've seen this happen and the only thing that changed was adding suricata and removing snort. I do have most of the logging turned OFF and the automatic log management turned ON with a very small size limit.

If I can come up with some actual proof of what happened (if it occurs again) I will post back.

This is on nanoBSD.

Suricata will produce a LOT of log output if some of the logging options are enabled. I highly suspect it is filling up the /var partition. I more or less left the defaults at the "out-of-the-box" values. Go to the LOGS MGMT tab and check the box to enable automatic management of logs. This will prune and rotate the logs. For nanoBSD, you should probably reduce both the LOG LIMIT sizes and RETENTION PERIODS to numbers somewhat smaller than the defaults. If keeping the logs is a big deal, then you would want to export them off the box somehow. Using Barnyard2 and outputting to a remote syslog server is one way of doing that.

Suricata's log files are all in /var/log/suricata and sub-directories underneath.

Oops…went back and read your original post more carefully and saw where you had enabled the management. How busy is your network? The logs management task only runs once every 5 minutes, and it is possible on extremely busy networks for some of the log files (say EVE, http or dns) to get quite large in 5 minutes when compared to the amount of free space that is likely to be on a nanoBSD partition.

Bill

binaryjay

@bmeeks:

@binaryjay:

Thank you for the package… I was having stability issues with snort crashing when under heavy load repeatedly and decided to give suricata a try the DAY before the 2.0 package went up... :) Anyway I upgraded of course... and I have not had the service go down a single time yet. I thought snort was supposed to be the more stable of the two? Anyway...

The only issue I've run across was what I suspect to be suricata filling my /var partition to 100% with... something... causing everything else on the box to go haywire. I couldn't do much else but force a reboot and it has been fine since and will need to monitor growth of /var more carefully and see if it happens again. Logs were not persisted after reboot (thanks nanoBSD ram disk).

The reason I blame suricata was because it was the first time I've seen this happen and the only thing that changed was adding suricata and removing snort. I do have most of the logging turned OFF and the automatic log management turned ON with a very small size limit.

If I can come up with some actual proof of what happened (if it occurs again) I will post back.

This is on nanoBSD.

Suricata will produce a LOT of log output if some of the logging options are enabled. I highly suspect it is filling up the /var partition. I more or less left the defaults at the "out-of-the-box" values. Go to the LOGS MGMT tab and check the box to enable automatic management of logs. This will prune and rotate the logs. For nanoBSD, you should probably reduce both the LOG LIMIT sizes and RETENTION PERIODS to numbers somewhat smaller than the defaults. If keeping the logs is a big deal, then you would want to export them off the box somehow. Using Barnyard2 and outputting to a remote syslog server is one way of doing that.

Suricata's log files are all in /var/log/suricata and sub-directories underneath.

Oops…went back and read your original post more carefully and saw where you had enabled the management. How busy is your network? The logs management task only runs once every 5 minutes, and it is possible on extremely busy networks for some of the log files (say EVE, http or dns) to get quite large in 5 minutes when compared to the amount of free space that is likely to be on a nanoBSD partition.

Bill

Not busy at all, it is a home network. The most workout it ever gets is bittorrent on a client being forced through openvpn in pfSense. I didn't notice any of the suricata logs ballooning unreasonably during the time I was actually monitoring an tweaking it after install. I did further reduce all of the log sizes and retention periods and it hasn't happened since but I don't feel like what I had them set to before was unreasonable.

My /var partition has not grown at all since I last wrote so I guess it was some fluke occurrence.

Anyway all is smooth running for now, thanks!

bmeeks

@mais_um:

@mais_um:

Thanks @avink, i really appreciate this information.

I will try this (maybe tomorrow) on my real pfsense and report back.

Tried on a em0 interface and it looks cool. Thanks again.

Just be aware that with this setup you are not getting the pure PPPoE traffic. What Suricata sees will include some PPPoE header info and stuff that may occasionally confuse it.

I did some research offline with avink and also swapped e-mails with some of the pfSense developers. The underlying issue with PPPoE using Suricata is that Suricata has no handler for the data link type DLT_NULL that FreeBSD returns for a PPPoE interface. The Suricata binary expects other values, but not the DLT_NULL (which is zero). It will take a patch to the Suricata binary in order for it to support PPPoE on FreeBSD. Snort seems to work with the DLT_NULL data link type that FreeBSD returns.

Bill

geudrik

Bill,

Sincerely appreciate your insight here re: why Suricata is choking on PPPoE links. It looks like a separate sensor hanging off a span is in my future :)

Pat

bmeeks

@geudrik:

Bill,

Sincerely appreciate your insight here re: why Suricata is choking on PPPoE links. It looks like a separate sensor hanging off a span is in my future :)

Pat

I'm glad I finally figured out the root cause of PPPoE not working in Suricata (at least on pfSense). I think I can patch it so that it works, and I intend to attempt that in the near future. I'm finishing up a big update to the Snort package at the moment, then I will look further into a Suricata patch to let PPPoE connections work properly.

Bill

dotOne

You know how to find me if you want to have it tested.

geudrik

@avink:

You know how to find me if you want to have it tested.

Ditto. Would be more than happy to test :)

bmeeks

@avink and @geudrik:

Thanks for offering to help test. It will be a few weeks before I can get to this. I will contact you via PM when I have something to test.

Bill

binaryjay

@bmeeks:

@binaryjay:

Thank you for the package… I was having stability issues with snort crashing when under heavy load repeatedly and decided to give suricata a try the DAY before the 2.0 package went up... :) Anyway I upgraded of course... and I have not had the service go down a single time yet. I thought snort was supposed to be the more stable of the two? Anyway...

The only issue I've run across was what I suspect to be suricata filling my /var partition to 100% with... something... causing everything else on the box to go haywire. I couldn't do much else but force a reboot and it has been fine since and will need to monitor growth of /var more carefully and see if it happens again. Logs were not persisted after reboot (thanks nanoBSD ram disk).

The reason I blame suricata was because it was the first time I've seen this happen and the only thing that changed was adding suricata and removing snort. I do have most of the logging turned OFF and the automatic log management turned ON with a very small size limit.

If I can come up with some actual proof of what happened (if it occurs again) I will post back.

This is on nanoBSD.

Suricata will produce a LOT of log output if some of the logging options are enabled. I highly suspect it is filling up the /var partition. I more or less left the defaults at the "out-of-the-box" values. Go to the LOGS MGMT tab and check the box to enable automatic management of logs. This will prune and rotate the logs. For nanoBSD, you should probably reduce both the LOG LIMIT sizes and RETENTION PERIODS to numbers somewhat smaller than the defaults. If keeping the logs is a big deal, then you would want to export them off the box somehow. Using Barnyard2 and outputting to a remote syslog server is one way of doing that.

Suricata's log files are all in /var/log/suricata and sub-directories underneath.

Oops…went back and read your original post more carefully and saw where you had enabled the management. How busy is your network? The logs management task only runs once every 5 minutes, and it is possible on extremely busy networks for some of the log files (say EVE, http or dns) to get quite large in 5 minutes when compared to the amount of free space that is likely to be on a nanoBSD partition.

Bill

Bill,

It has happened again, I was having clients refuse to get assigned an ip address. I was able to log into the box this time to see what is going on, I found that dhcpd was complaining about /var being out of space. A df -h confirmed that it was overprovisioned.

This installation is a nanoBSD one, and my /var partition is only 120MB (which I believe is more than default, /var on nanoBSD is a ramdisk. I narrowed it down to the suricata alert logs:

This is a small example of the list for the one interface suricata is running on:


-rw-r--r--  1 root  wheel  1737215 Sep 28 15:15 alerts.log.2014_0928_1515
-rw-r--r--  1 root  wheel  6789234 Sep 28 15:20 alerts.log.2014_0928_1520
-rw-r--r--  1 root  wheel  6462547 Sep 28 15:25 alerts.log.2014_0928_1525
-rw-r--r--  1 root  wheel  2383926 Sep 28 15:30 alerts.log.2014_0928_1530
-rw-r--r--  1 root  wheel  1195393 Sep 28 15:35 alerts.log.2014_0928_1535
-rw-r--r--  1 root  wheel  2762914 Sep 28 16:30 alerts.log.2014_0928_1630
-rw-r--r--  1 root  wheel  1308605 Sep 28 16:35 alerts.log.2014_0928_1635
-rw-r--r--  1 root  wheel  3320858 Sep 28 16:50 alerts.log.2014_0928_1650
-rw-r--r--  1 root  wheel  5628798 Sep 28 16:55 alerts.log.2014_0928_1655
-rw-r--r--  1 root  wheel   674842 Sep 28 17:00 alerts.log.2014_0928_1700
-rw-r--r--  1 root  wheel   540330 Sep 28 17:15 alerts.log.2014_0928_1715
-rw-r--r--  1 root  wheel   758454 Sep 28 17:35 alerts.log.2014_0928_1735
-rw-r--r--  1 root  wheel  3895902 Sep 28 17:40 alerts.log.2014_0928_1740
-rw-r--r--  1 root  wheel  4773497 Sep 28 17:55 alerts.log.2014_0928_1755
-rw-r--r--  1 root  wheel     8192 Sep 28 18:00 alerts.log.2014_0928_1800
-rw-r--r--  1 root  wheel     8192 Sep 28 18:05 alerts.log.2014_0928_1805
-rw-r--r--  1 root  wheel     8192 Sep 28 18:15 alerts.log.2014_0928_1815
-rw-r--r--  1 root  wheel     8192 Sep 28 18:20 alerts.log.2014_0928_1820
-rw-r--r--  1 root  wheel   704512 Sep 28 18:25 alerts.log.2014_0928_1825
-rw-r--r--  1 root  wheel  1200180 Sep 28 18:30 alerts.log.2014_0928_1830

These logs, after removal, ended up taking about 100MB.

In logs management:


Enable directory size limit (Default): Yes
Size in MB: 3
alerts: 500KB, 1 Day Retention

Given these settings, I would not expect to see so many rotated alert logs and never see the suricata log folder reach 100MB.

And this is the real kicker fromthe pfSense system log:


Sep 28 18:50:04	kernel: pid 88242 (dhcpd), uid 1002 inumber 12529 on /var: filesystem full
Sep 28 18:50:01	php: suricata_check_cron_misc.inc: [Suricata] Automatic clean-up of Suricata logs completed.
Sep 28 18:50:01	php: suricata_check_cron_misc.inc: [Suricata] Truncating logs for WAN (bce1)...
Sep 28 18:50:01	php: suricata_check_cron_misc.inc: [Suricata] Truncating the Rules Update Log file...
Sep 28 18:50:01	php: suricata_check_cron_misc.inc: [Suricata] Log directory size exceeds configured limit of 3 MB set on Global Settings tab. All Suricata log files will be truncated.
[b]Sep 28 18:49:56	kernel: pid 88242 (dhcpd), uid 1002 inumber 12519 on /var: filesystem full[/b]

You can see it recognizes it needs to truncate, says it has done so but only for the rules update log? I see it has been trying to truncate all day long, the above was just the last instance of it in the logs. Is it because of the 1 day retention, of which there is no shorter period of time to choose? It seems like 1 day is too long with nanoBSD in some cases where something causes your network to get hammered with a lot of alerts one day.

Is it something I'm doing wrong? I'm afraid that this is causing the whole network to nearly go down in flames when it happens so I might have to go without suricata until a solution can be reached.

I suppose in the meantime I can run a script with cron to just blow away the logs myself if it gets too big but it does kind of seem like there is a bug here and I don't think this is an ideal solution. In the meantime I have also doubled the size of the ramdisk holding /var in the hopes it will provide some relief but this is kind of a waste of memory.

Thanks!

Bonus question: When rebooting pfsense, I find suricata not started and there are rafts of errors like this. When I then manually start the service, it starts up fine after the usual long delay.


28/9/2014 -- 19:55:55 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - Can't use file_data with flow:to_server or from_client with http.
28/9/2014 -- 19:55:55 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $EXTERNAL_NET any -> $SMTP_SERVERS 25 (msg:"BROWSER-FIREFOX Mozilla Firefox iframe and xul element reload crash attempt"; flow:to_server,established; file_data; content:"document.createElement|28 27|iframe|27 29|"; fast_pattern:only; content:"<frame"; content:".xul";="" content:".contentdocument.location.reload|28="" 29|";="" metadata:policy="" balanced-ips="" drop,="" policy="" connectivity-ips="" security-ips="" service="" smtp;="" reference:cve,2011-2982;="" classtype:attempted-user;="" sid:25228;="" rev:4;)"="" from="" file="" usr="" pbi="" suricata-i386="" etc="" suricata="" suricata_48468_bce1="" rules="" suricata.rules="" at="" line="" 14217<br="">28/9/2014 -- 19:55:55 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - Can't use file_data with flow:to_server or from_client with http.
28/9/2014 -- 19:55:55 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $EXTERNAL_NET any -> $SMTP_SERVERS 25 (msg:"BROWSER-FIREFOX Mozilla Firefox IDB use-after-free attempt"; flow:established,to_server; file_data; content:"IDBKeyRange"; fast_pattern:only; pcre:"/IDBKeyRange\x2e(only|lowerBound|upperBound|bound)\x28.*?\x29.{0,100}\x2e(lower|upper|lowerOpen|upperOpen)/smi"; metadata:policy balanced-ips drop, policy connectivity-ips drop, policy security-ips drop, service smtp; reference:cve,2012-0469; reference:url,bugzilla.mozilla.org/show_bug.cgi?id=738985; classtype:attempted-user; sid:24574; rev:4;)" from file /usr/pbi/suricata-i386/etc/suricata/suricata_48468_bce1/rules/suricata.rules at line 14220
28/9/2014 -- 19:55:55 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - Can't use file_data with flow:to_server or from_client with http.
28/9/2014 -- 19:55:55 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $EXTERNAL_NET any -> $SMTP_SERVERS 25 (msg:"BROWSER-FIREFOX Mozilla Firefox IDB use-after-free attempt"; flow:established,to_server; file_data; content:"IDBKeyRange.lowerBound("; content:".upper"; within:20; metadata:policy balanced-ips drop, policy connectivity-ips drop, policy security-ips drop, service smtp; reference:cve,2012-0469; reference:url,bugzilla.mozilla.org/show_bug.cgi?id=738985; classtype:attempted-user; sid:24573; rev:3;)" from file /usr/pbi/suricata-i386/etc/suricata/suricata_48468_bce1/rules/suricata.rules at line 14221
28/9/2014 -- 19:55:55 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - Can't use file_data with flow:to_server or from_client with http.
28/9/2014 -- 19:55:55 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $EXTERNAL_NET any -> $SMTP_SERVERS 25 (msg:"BROWSER-FIREFOX Mozilla Firefox IDB use-after-free attempt"; flow:established,to_server; file_data; content:"IDBKeyRange.only("; content:").lower"; within:20; metadata:policy balanced-ips drop, policy connectivity-ips drop, policy security-ips drop, service smtp; reference:cve,2012-0469; reference:url,bugzilla.mozilla.org/show_bug.cgi?id=738985; classtype:attempted-user; sid:24570; rev:3;)" from file /usr/pbi/suricata-i386/etc/suricata/suricata_48468_bce1/rules/suricata.rules at line 14224
28/9/2014 -- 19:55:55 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - Can't use file_data with flow:to_server or from_client with http.
28/9/2014 -- 19:55:55 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $HOME_NET any -> $SMTP_SERVERS 25 (msg:"BROWSER-FIREFOX Mozilla Firefox nsTreeRange Use After Free attempt"; flow:to_server,established; file_data; content:"|2E|view|2E|selection"; nocase; content:"|2E|invalidateSelection"; distance:0; nocase; pcre:"/\x2Eview\x2Eselection.*?\x2Etree\s*\x3D\s*null.*?\x2Einvalidate/smi"; metadata:policy balanced-ips drop, policy connectivity-ips drop, policy security-ips drop, service smtp; reference:cve,2011-0073; reference:url,www.mozilla.org/security/announce/2011/mfsa2011-13.html; classtype:attempted-user; sid:29617; rev:1;)" from file /usr/pbi/suricata-i386/etc/suricata/suricata_48468_bce1/rules/suricata.rules at line 14234
28/9/2014 -- 19:55:55 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - Can't use file_data with flow:to_server or from_client with http.
28/9/2014 -- 19:55:55 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $EXTERNAL_NET any -> $SMTP_SERVERS 25 (msg:"BROWSER-IE Microsoft Internet Explorer execCommand CTreePos memory corruption attempt"; flow:to_server,established; file_data; content:".execCommand"; nocase; content:"undo"; within:15; nocase; content:".execCommand"; within:100; nocase; content:"redo"; within:15; nocase; content:".execCommand"; within:100; nocase; content:"undo"; within:15; nocase; pcre:"/\x2eexecCommand\s*\x28\s*[\x22\x27]\s*undo\s*[\x22\x27].*?\x2eexecCommand\s*\x28\s*[\x22\x27]\s*redo\s*[\x22\x27].*?\x2eexecCommand\s*\x28\s*[\x22\x27]\s*undo\s*[\x22\x27]/smi"; metadata:policy balanced-ips drop, policy connectivity-ips drop, policy security-ips drop, service smtp; reference:cve,2013-3914; reference:url,technet.microsoft.com/en-us/security/bulletin/MS13-088; classtype:attempted-user; sid:28495; rev:1;)" from file /usr/pbi/suricata-i386/etc/suricata/suricata_48468_bce1/rules/suricata.rules at line 14235
28/9/2014 -- 19:55:55 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - Can't use file_data with flow:to_server or from_client with http.
28/9/2014 -- 19:55:55 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $EXTERNAL_NET any -> $SMTP_SERVERS 25 (msg:"BROWSER-IE Microsoft Internet Explorer fontFamily attribute deleted object access memory corruption attempt"; flow:to_server,established; file_data; content:"</error></error></error></error></error></error></error></error></error></error></error></error></frame";></error></error>

bmeeks

Yes, I found that bug with Suricata's log management routine forgetting to delete the "rotated" log files when the logging directory size exceeds the set limit. That fix is coming in the next update. It cleans up the current alert log, but does not cleanup the older rotated logs under some conditions.

The interim fix is to periodically go in and delete those files (the ones with alert.log.xxxxx where xxxxx is a timestamp).

Suricata can barf out a lot of log traffic, and running it on a Nano install is risky (in my opinion) due to the potential for exhausting disk space unexpectedly).

Bill

binaryjay

@bmeeks:

Yes, I found that bug with Suricata's log management routine forgetting to delete the "rotated" log files when the logging directory size exceeds the set limit. That fix is coming in the next update. It cleans up the current alert log, but does not cleanup the older rotated logs under some conditions.

The interim fix is to periodically go in and delete those files (the ones with alert.log.xxxxx where xxxxx is a timestamp).

Suricata can barf out a lot of log traffic, and running it on a Nano install is risky (in my opinion) due to the potential for exhausting disk space unexpectedly).

Bill

Awesome, thank you! Curious, is it an upstream problem or pfsense package specific?

bmeeks

@binaryjay:

Awesome, thank you! Curious, is it an upstream problem or pfsense package specific?

The log management bug I was referring to is pfSense-specific. It is a bug in the package GUI code that I created. My code looks for the file alert.log, but fails to then also do a pattern search for .log. so it can catch the rotated files.

It will be fixed in the next Suricata package update.

The Log Management code in the pfSense Suricata package was added to cope with the accumulation of log files that can occur. It is not a perfect solution, though. For one, the routine runs once every 5 minutes via a cron job. If you have a busy network and a very noisy rule, then a lot of megabytes of log traffic could be created in the five-minute window between runs of the cron job.

I also realized I did not answer your question about the "file_data" keyword errors. Suricata does not currently process all of the same rule keywords and options that Snort does. The "file_data" option is one of those. This is an upstream issue. There are some feature requests posted on the Suricata Redmine site asking that support be added for the new VRT keywords and options.

Bill

demco

Bill, is it possible to use clog? Or there are unintentional consequences?

Regarding the "file_data" keyword and signature parsing errors raised by binaryjay. If Suricata start up with these errors, can we assume it is working and just skipped these signatures?

Thanks, appreciate your work on Suricata and Snort.