Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Suricata 2.0.3 Package Preview

    Scheduled Pinned Locked Moved pfSense Packages
    121 Posts 17 Posters 35.8k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Raul RamosR
      Raul Ramos
      last edited by

      Thanks @avink, i really appreciate this information.

      I will try this (maybe tomorrow) on my real pfsense and report back.

      pfSense:
      ASRock -> Wolfdale1333-D667 (2GB TeamElite Ram)
      Marvell 88SA8040 Sata to CF(Sandisk 4GB) Controller
      NIC's: RTL8100E (Internal ) and Intel® PRO/1000 PT Dual (Intel 82571GB)

      1 Reply Last reply Reply Quote 0
      • bmeeksB
        bmeeks
        last edited by

        @avink:

        As some people already discovered suricata seems to have problems with PPPoE connections and the log is filling up with these messages:

        6/9/2014 – 20:58:10 - <error>-- [ERRCODE: SC_ERR_DATALINK_UNIMPLEMENTED(38)] - Error: datalink type 0 not yet supported in module DecodePcap
        6/9/2014 – 20:58:10 - <error>-- [ERRCODE: SC_ERR_DATALINK_UNIMPLEMENTED(38)] - Error: datalink type 0 not yet supported in module DecodePcap
        6/9/2014 – 20:58:10 - <error>-- [ERRCODE: SC_ERR_DATALINK_UNIMPLEMENTED(38)] - Error: datalink type 0 not yet supported in module DecodePcap</error></error></error>

        According to the suricata website PPPoE should be supported so I did some searching and playing around on my test firewall.
        What I did is instead of listening on the PPPoE interface I changed the listening interface on the configuration file to the underlying physical interface (em0) in my case and restarded suricata by hand

        suricata.yaml:
        pcap:
        - interface: em0  
            checksum-checks: auto
            promisc: yes

        /usr/pbi/suricata-amd64/bin/suricata -i em0 -D -c /usr/pbi/suricata-amd64/etc/suricata/suricata_2925_pppoe0/suricata.yaml –pidfile /var/run/suricata_pppoe02925.pid

        It looks like suricata can perfectly handle this and strips the PPPoE part looking at the actual data.
        There are few messages reporting unrecognized ppp frames, but there are only a few of them. Will do some more checking but it looks suricata is running fine now.
        It's logging and blocking as it should

        So far so good.

        Thank you for the extra research.  This is good information that maybe I can use to fix PPPoE with Suricata.  I do not have a PPPoE connection to test with anymore since I switched from DSL to cable modem a couple of years ago.  So the trick appears to be maybe capturing the actual physical interface behind the PPPoE interface.

        I am sending you a PM so we can communicate about this some more offline.

        Bill

        1 Reply Last reply Reply Quote 0
        • Raul RamosR
          Raul Ramos
          last edited by

          @mais_um:

          Thanks @avink, i really appreciate this information.

          I will try this (maybe tomorrow) on my real pfsense and report back.

          Tried on a em0 interface and it looks cool. Thanks again.

          pfSense:
          ASRock -> Wolfdale1333-D667 (2GB TeamElite Ram)
          Marvell 88SA8040 Sata to CF(Sandisk 4GB) Controller
          NIC's: RTL8100E (Internal ) and Intel® PRO/1000 PT Dual (Intel 82571GB)

          1 Reply Last reply Reply Quote 0
          • B
            binaryjay
            last edited by

            Thank you for the package… I was having stability issues with snort crashing when under heavy load repeatedly and decided to give suricata a try the DAY before the 2.0 package went up... :)  Anyway I upgraded of course...  and I have not had the service go down a single time yet.  I thought snort was supposed to be the more stable of the two?  Anyway...

            The only issue I've run across was what I suspect to be suricata filling my /var partition to 100% with... something... causing everything else on the box to go haywire.  I couldn't do much else but force a reboot and it has been fine since and will need to monitor growth of /var more carefully and see if it happens again.  Logs were not persisted after reboot (thanks nanoBSD ram disk).

            The reason I blame suricata was because it was the first time I've seen this happen and the only thing that changed was adding suricata and removing snort.  I do have most of the logging turned OFF and the automatic log management turned ON with a very small size limit.

            If I can come up with some actual proof of what happened (if it occurs again) I will post back.

            This is on nanoBSD.

            1 Reply Last reply Reply Quote 0
            • bmeeksB
              bmeeks
              last edited by

              @binaryjay:

              Thank you for the package… I was having stability issues with snort crashing when under heavy load repeatedly and decided to give suricata a try the DAY before the 2.0 package went up... :)  Anyway I upgraded of course...  and I have not had the service go down a single time yet.  I thought snort was supposed to be the more stable of the two?  Anyway...

              The only issue I've run across was what I suspect to be suricata filling my /var partition to 100% with... something... causing everything else on the box to go haywire.  I couldn't do much else but force a reboot and it has been fine since and will need to monitor growth of /var more carefully and see if it happens again.  Logs were not persisted after reboot (thanks nanoBSD ram disk).

              The reason I blame suricata was because it was the first time I've seen this happen and the only thing that changed was adding suricata and removing snort.  I do have most of the logging turned OFF and the automatic log management turned ON with a very small size limit.

              If I can come up with some actual proof of what happened (if it occurs again) I will post back.

              This is on nanoBSD.

              Suricata will produce a LOT of log output if some of the logging options are enabled.  I highly suspect it is filling up the /var partition. I more or less left the defaults at the "out-of-the-box" values.  Go to the LOGS MGMT tab and check the box to enable automatic management of logs.  This will prune and rotate the logs.  For nanoBSD, you should probably reduce both the LOG LIMIT sizes and RETENTION PERIODS to numbers somewhat smaller than the defaults.  If keeping the logs is a big deal, then you would want to export them off the box somehow. Using Barnyard2 and outputting to a remote syslog server is one way of doing that.

              Suricata's log files are all in /var/log/suricata and sub-directories underneath.

              Oops…went back and read your original post more carefully and saw where you had enabled the management.  How busy is your network?  The logs management task only runs once every 5 minutes, and it is possible on extremely busy networks for some of the log files (say EVE, http or dns) to get quite large in 5 minutes when compared to the amount of free space that is likely to be on a nanoBSD partition.

              Bill

              1 Reply Last reply Reply Quote 0
              • B
                binaryjay
                last edited by

                @bmeeks:

                @binaryjay:

                Thank you for the package… I was having stability issues with snort crashing when under heavy load repeatedly and decided to give suricata a try the DAY before the 2.0 package went up... :)  Anyway I upgraded of course...  and I have not had the service go down a single time yet.  I thought snort was supposed to be the more stable of the two?  Anyway...

                The only issue I've run across was what I suspect to be suricata filling my /var partition to 100% with... something... causing everything else on the box to go haywire.  I couldn't do much else but force a reboot and it has been fine since and will need to monitor growth of /var more carefully and see if it happens again.  Logs were not persisted after reboot (thanks nanoBSD ram disk).

                The reason I blame suricata was because it was the first time I've seen this happen and the only thing that changed was adding suricata and removing snort.  I do have most of the logging turned OFF and the automatic log management turned ON with a very small size limit.

                If I can come up with some actual proof of what happened (if it occurs again) I will post back.

                This is on nanoBSD.

                Suricata will produce a LOT of log output if some of the logging options are enabled.  I highly suspect it is filling up the /var partition. I more or less left the defaults at the "out-of-the-box" values.  Go to the LOGS MGMT tab and check the box to enable automatic management of logs.  This will prune and rotate the logs.  For nanoBSD, you should probably reduce both the LOG LIMIT sizes and RETENTION PERIODS to numbers somewhat smaller than the defaults.  If keeping the logs is a big deal, then you would want to export them off the box somehow. Using Barnyard2 and outputting to a remote syslog server is one way of doing that.

                Suricata's log files are all in /var/log/suricata and sub-directories underneath.

                Oops…went back and read your original post more carefully and saw where you had enabled the management.  How busy is your network?  The logs management task only runs once every 5 minutes, and it is possible on extremely busy networks for some of the log files (say EVE, http or dns) to get quite large in 5 minutes when compared to the amount of free space that is likely to be on a nanoBSD partition.

                Bill

                Not busy at all, it is a home network.  The most workout it ever gets is bittorrent on a client being forced through openvpn in pfSense.  I didn't notice any of the suricata logs ballooning unreasonably during the time I was actually monitoring an tweaking it after install.  I did further reduce all of the log sizes and retention periods and it hasn't happened since but I don't feel like what I had them set to before was unreasonable.

                My /var partition has not grown at all since I last wrote so I guess it was some fluke occurrence.

                Anyway all is smooth running for now, thanks!

                1 Reply Last reply Reply Quote 0
                • bmeeksB
                  bmeeks
                  last edited by

                  @mais_um:

                  @mais_um:

                  Thanks @avink, i really appreciate this information.

                  I will try this (maybe tomorrow) on my real pfsense and report back.

                  Tried on a em0 interface and it looks cool. Thanks again.

                  Just be aware that with this setup you are not getting the pure PPPoE traffic.  What Suricata sees will include some PPPoE header info and stuff that may occasionally confuse it.

                  I did some research offline with avink and also swapped e-mails with some of the pfSense developers.  The underlying issue with PPPoE using Suricata is that Suricata has no handler for the data link type DLT_NULL that FreeBSD returns for a PPPoE interface.  The Suricata binary expects other values, but not the DLT_NULL (which is zero).  It will take a patch to the Suricata binary in order for it to support PPPoE on FreeBSD.  Snort seems to work with the DLT_NULL data link type that FreeBSD returns.

                  Bill

                  1 Reply Last reply Reply Quote 0
                  • G
                    geudrik
                    last edited by

                    Bill,

                    Sincerely appreciate your insight here re: why Suricata is choking on PPPoE links. It looks like a separate sensor hanging off a span is in my future :)

                    Pat

                    1 Reply Last reply Reply Quote 0
                    • bmeeksB
                      bmeeks
                      last edited by

                      @geudrik:

                      Bill,

                      Sincerely appreciate your insight here re: why Suricata is choking on PPPoE links. It looks like a separate sensor hanging off a span is in my future :)

                      Pat

                      I'm glad I finally figured out the root cause of PPPoE not working in Suricata (at least on pfSense).  I think I can patch it so that it works, and I intend to attempt that in the near future.  I'm finishing up a big update to the Snort package at the moment, then I will look further into a Suricata patch to let PPPoE connections work properly.

                      Bill

                      1 Reply Last reply Reply Quote 0
                      • dotOneD
                        dotOne
                        last edited by

                        You know how to find me if you want to have it tested.

                        1 Reply Last reply Reply Quote 0
                        • G
                          geudrik
                          last edited by

                          @avink:

                          You know how to find me if you want to have it tested.

                          Ditto. Would be more than happy to test :)

                          1 Reply Last reply Reply Quote 0
                          • bmeeksB
                            bmeeks
                            last edited by

                            @avink and @geudrik:

                            Thanks for offering to help test.  It will be a few weeks before I can get to this.  I will contact you via PM when I have something to test.

                            Bill

                            1 Reply Last reply Reply Quote 0
                            • B
                              binaryjay
                              last edited by

                              @bmeeks:

                              @binaryjay:

                              Thank you for the package… I was having stability issues with snort crashing when under heavy load repeatedly and decided to give suricata a try the DAY before the 2.0 package went up... :)  Anyway I upgraded of course...  and I have not had the service go down a single time yet.  I thought snort was supposed to be the more stable of the two?  Anyway...

                              The only issue I've run across was what I suspect to be suricata filling my /var partition to 100% with... something... causing everything else on the box to go haywire.  I couldn't do much else but force a reboot and it has been fine since and will need to monitor growth of /var more carefully and see if it happens again.  Logs were not persisted after reboot (thanks nanoBSD ram disk).

                              The reason I blame suricata was because it was the first time I've seen this happen and the only thing that changed was adding suricata and removing snort.  I do have most of the logging turned OFF and the automatic log management turned ON with a very small size limit.

                              If I can come up with some actual proof of what happened (if it occurs again) I will post back.

                              This is on nanoBSD.

                              Suricata will produce a LOT of log output if some of the logging options are enabled.  I highly suspect it is filling up the /var partition. I more or less left the defaults at the "out-of-the-box" values.  Go to the LOGS MGMT tab and check the box to enable automatic management of logs.  This will prune and rotate the logs.  For nanoBSD, you should probably reduce both the LOG LIMIT sizes and RETENTION PERIODS to numbers somewhat smaller than the defaults.  If keeping the logs is a big deal, then you would want to export them off the box somehow. Using Barnyard2 and outputting to a remote syslog server is one way of doing that.

                              Suricata's log files are all in /var/log/suricata and sub-directories underneath.

                              Oops…went back and read your original post more carefully and saw where you had enabled the management.  How busy is your network?  The logs management task only runs once every 5 minutes, and it is possible on extremely busy networks for some of the log files (say EVE, http or dns) to get quite large in 5 minutes when compared to the amount of free space that is likely to be on a nanoBSD partition.

                              Bill

                              Bill,

                              It has happened again, I was having clients refuse to get assigned an ip address.  I was able to log into the box this time to see what is going on, I found that dhcpd was complaining about /var being out of space.  A df -h confirmed that it was overprovisioned.

                              This installation is a nanoBSD one, and my /var partition is only 120MB (which I believe is more than default, /var  on nanoBSD is a ramdisk.  I narrowed it down to the suricata alert logs:

                              This is a small example of the list for the one interface suricata is running on:

                              
                              -rw-r--r--  1 root  wheel  1737215 Sep 28 15:15 alerts.log.2014_0928_1515
                              -rw-r--r--  1 root  wheel  6789234 Sep 28 15:20 alerts.log.2014_0928_1520
                              -rw-r--r--  1 root  wheel  6462547 Sep 28 15:25 alerts.log.2014_0928_1525
                              -rw-r--r--  1 root  wheel  2383926 Sep 28 15:30 alerts.log.2014_0928_1530
                              -rw-r--r--  1 root  wheel  1195393 Sep 28 15:35 alerts.log.2014_0928_1535
                              -rw-r--r--  1 root  wheel  2762914 Sep 28 16:30 alerts.log.2014_0928_1630
                              -rw-r--r--  1 root  wheel  1308605 Sep 28 16:35 alerts.log.2014_0928_1635
                              -rw-r--r--  1 root  wheel  3320858 Sep 28 16:50 alerts.log.2014_0928_1650
                              -rw-r--r--  1 root  wheel  5628798 Sep 28 16:55 alerts.log.2014_0928_1655
                              -rw-r--r--  1 root  wheel   674842 Sep 28 17:00 alerts.log.2014_0928_1700
                              -rw-r--r--  1 root  wheel   540330 Sep 28 17:15 alerts.log.2014_0928_1715
                              -rw-r--r--  1 root  wheel   758454 Sep 28 17:35 alerts.log.2014_0928_1735
                              -rw-r--r--  1 root  wheel  3895902 Sep 28 17:40 alerts.log.2014_0928_1740
                              -rw-r--r--  1 root  wheel  4773497 Sep 28 17:55 alerts.log.2014_0928_1755
                              -rw-r--r--  1 root  wheel     8192 Sep 28 18:00 alerts.log.2014_0928_1800
                              -rw-r--r--  1 root  wheel     8192 Sep 28 18:05 alerts.log.2014_0928_1805
                              -rw-r--r--  1 root  wheel     8192 Sep 28 18:15 alerts.log.2014_0928_1815
                              -rw-r--r--  1 root  wheel     8192 Sep 28 18:20 alerts.log.2014_0928_1820
                              -rw-r--r--  1 root  wheel   704512 Sep 28 18:25 alerts.log.2014_0928_1825
                              -rw-r--r--  1 root  wheel  1200180 Sep 28 18:30 alerts.log.2014_0928_1830
                              
                              

                              These logs, after removal, ended up taking about 100MB.

                              In logs management:

                              
                              Enable directory size limit (Default): Yes
                              Size in MB: 3
                              alerts: 500KB, 1 Day Retention
                              
                              

                              Given these settings, I would not expect to see so many rotated alert logs and never see the suricata log folder reach 100MB.

                              And this is the real kicker fromthe pfSense system log:

                              
                              Sep 28 18:50:04	kernel: pid 88242 (dhcpd), uid 1002 inumber 12529 on /var: filesystem full
                              Sep 28 18:50:01	php: suricata_check_cron_misc.inc: [Suricata] Automatic clean-up of Suricata logs completed.
                              Sep 28 18:50:01	php: suricata_check_cron_misc.inc: [Suricata] Truncating logs for WAN (bce1)...
                              Sep 28 18:50:01	php: suricata_check_cron_misc.inc: [Suricata] Truncating the Rules Update Log file...
                              Sep 28 18:50:01	php: suricata_check_cron_misc.inc: [Suricata] Log directory size exceeds configured limit of 3 MB set on Global Settings tab. All Suricata log files will be truncated.
                              [b]Sep 28 18:49:56	kernel: pid 88242 (dhcpd), uid 1002 inumber 12519 on /var: filesystem full[/b]
                              
                              

                              You can see it recognizes it needs to truncate, says it has done so but only for the rules update log?  I see it has been trying to truncate all day long, the above was just the last instance of it in the logs.  Is it because of the 1 day retention, of which there is no shorter period of time to choose?  It seems like 1 day is too long with nanoBSD in some cases where something causes your network to get hammered with a lot of alerts one day.

                              Is it something I'm doing wrong?  I'm afraid that this is causing the whole network to nearly go down in flames when it happens so I might have to go without suricata until a solution can be reached.

                              I suppose in the meantime I can run a script with cron to just blow away the logs myself if it gets too big but it does kind of seem like there is a bug here and I don't think this is an ideal solution.  In the meantime I have also doubled the size of the ramdisk holding /var in the hopes it will provide some relief but this is kind of a waste of memory.

                              Thanks!

                              Bonus question:  When rebooting pfsense, I find suricata not started and there are rafts of errors like this.  When I then manually start the service, it starts up fine after the usual long delay.

                              
                              28/9/2014 -- 19:55:55 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - Can't use file_data with flow:to_server or from_client with http.
                              28/9/2014 -- 19:55:55 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $EXTERNAL_NET any -> $SMTP_SERVERS 25 (msg:"BROWSER-FIREFOX Mozilla Firefox iframe and xul element reload crash attempt"; flow:to_server,established; file_data; content:"document.createElement|28 27|iframe|27 29|"; fast_pattern:only; content:"<frame"; content:".xul";="" content:".contentdocument.location.reload|28="" 29|";="" metadata:policy="" balanced-ips="" drop,="" policy="" connectivity-ips="" security-ips="" service="" smtp;="" reference:cve,2011-2982;="" classtype:attempted-user;="" sid:25228;="" rev:4;)"="" from="" file="" usr="" pbi="" suricata-i386="" etc="" suricata="" suricata_48468_bce1="" rules="" suricata.rules="" at="" line="" 14217<br="">28/9/2014 -- 19:55:55 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - Can't use file_data with flow:to_server or from_client with http.
                              28/9/2014 -- 19:55:55 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $EXTERNAL_NET any -> $SMTP_SERVERS 25 (msg:"BROWSER-FIREFOX Mozilla Firefox IDB use-after-free attempt"; flow:established,to_server; file_data; content:"IDBKeyRange"; fast_pattern:only; pcre:"/IDBKeyRange\x2e(only|lowerBound|upperBound|bound)\x28.*?\x29.{0,100}\x2e(lower|upper|lowerOpen|upperOpen)/smi"; metadata:policy balanced-ips drop, policy connectivity-ips drop, policy security-ips drop, service smtp; reference:cve,2012-0469; reference:url,bugzilla.mozilla.org/show_bug.cgi?id=738985; classtype:attempted-user; sid:24574; rev:4;)" from file /usr/pbi/suricata-i386/etc/suricata/suricata_48468_bce1/rules/suricata.rules at line 14220
                              28/9/2014 -- 19:55:55 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - Can't use file_data with flow:to_server or from_client with http.
                              28/9/2014 -- 19:55:55 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $EXTERNAL_NET any -> $SMTP_SERVERS 25 (msg:"BROWSER-FIREFOX Mozilla Firefox IDB use-after-free attempt"; flow:established,to_server; file_data; content:"IDBKeyRange.lowerBound("; content:".upper"; within:20; metadata:policy balanced-ips drop, policy connectivity-ips drop, policy security-ips drop, service smtp; reference:cve,2012-0469; reference:url,bugzilla.mozilla.org/show_bug.cgi?id=738985; classtype:attempted-user; sid:24573; rev:3;)" from file /usr/pbi/suricata-i386/etc/suricata/suricata_48468_bce1/rules/suricata.rules at line 14221
                              28/9/2014 -- 19:55:55 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - Can't use file_data with flow:to_server or from_client with http.
                              28/9/2014 -- 19:55:55 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $EXTERNAL_NET any -> $SMTP_SERVERS 25 (msg:"BROWSER-FIREFOX Mozilla Firefox IDB use-after-free attempt"; flow:established,to_server; file_data; content:"IDBKeyRange.only("; content:").lower"; within:20; metadata:policy balanced-ips drop, policy connectivity-ips drop, policy security-ips drop, service smtp; reference:cve,2012-0469; reference:url,bugzilla.mozilla.org/show_bug.cgi?id=738985; classtype:attempted-user; sid:24570; rev:3;)" from file /usr/pbi/suricata-i386/etc/suricata/suricata_48468_bce1/rules/suricata.rules at line 14224
                              28/9/2014 -- 19:55:55 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - Can't use file_data with flow:to_server or from_client with http.
                              28/9/2014 -- 19:55:55 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $HOME_NET any -> $SMTP_SERVERS 25 (msg:"BROWSER-FIREFOX Mozilla Firefox nsTreeRange Use After Free attempt"; flow:to_server,established; file_data; content:"|2E|view|2E|selection"; nocase; content:"|2E|invalidateSelection"; distance:0; nocase; pcre:"/\x2Eview\x2Eselection.*?\x2Etree\s*\x3D\s*null.*?\x2Einvalidate/smi"; metadata:policy balanced-ips drop, policy connectivity-ips drop, policy security-ips drop, service smtp; reference:cve,2011-0073; reference:url,www.mozilla.org/security/announce/2011/mfsa2011-13.html; classtype:attempted-user; sid:29617; rev:1;)" from file /usr/pbi/suricata-i386/etc/suricata/suricata_48468_bce1/rules/suricata.rules at line 14234
                              28/9/2014 -- 19:55:55 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - Can't use file_data with flow:to_server or from_client with http.
                              28/9/2014 -- 19:55:55 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $EXTERNAL_NET any -> $SMTP_SERVERS 25 (msg:"BROWSER-IE Microsoft Internet Explorer execCommand CTreePos memory corruption attempt"; flow:to_server,established; file_data; content:".execCommand"; nocase; content:"undo"; within:15; nocase; content:".execCommand"; within:100; nocase; content:"redo"; within:15; nocase; content:".execCommand"; within:100; nocase; content:"undo"; within:15; nocase; pcre:"/\x2eexecCommand\s*\x28\s*[\x22\x27]\s*undo\s*[\x22\x27].*?\x2eexecCommand\s*\x28\s*[\x22\x27]\s*redo\s*[\x22\x27].*?\x2eexecCommand\s*\x28\s*[\x22\x27]\s*undo\s*[\x22\x27]/smi"; metadata:policy balanced-ips drop, policy connectivity-ips drop, policy security-ips drop, service smtp; reference:cve,2013-3914; reference:url,technet.microsoft.com/en-us/security/bulletin/MS13-088; classtype:attempted-user; sid:28495; rev:1;)" from file /usr/pbi/suricata-i386/etc/suricata/suricata_48468_bce1/rules/suricata.rules at line 14235
                              28/9/2014 -- 19:55:55 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - Can't use file_data with flow:to_server or from_client with http.
                              28/9/2014 -- 19:55:55 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $EXTERNAL_NET any -> $SMTP_SERVERS 25 (msg:"BROWSER-IE Microsoft Internet Explorer fontFamily attribute deleted object access memory corruption attempt"; flow:to_server,established; file_data; content:"</error></error></error></error></error></error></error></error></error></error></error></error></frame";></error></error> 
                              
                              1 Reply Last reply Reply Quote 0
                              • bmeeksB
                                bmeeks
                                last edited by

                                Yes, I found that bug with Suricata's log management routine forgetting to delete the "rotated" log files when the logging directory size exceeds the set limit.  That fix is coming in the next update.  It cleans up the current alert log, but does not cleanup the older rotated logs under some conditions.

                                The interim fix is to periodically go in and delete those files (the ones with alert.log.xxxxx where xxxxx is a timestamp).

                                Suricata can barf out a lot of log traffic, and running it on a Nano install is risky (in my opinion) due to the potential for exhausting disk space unexpectedly).

                                Bill

                                1 Reply Last reply Reply Quote 0
                                • B
                                  binaryjay
                                  last edited by

                                  @bmeeks:

                                  Yes, I found that bug with Suricata's log management routine forgetting to delete the "rotated" log files when the logging directory size exceeds the set limit.  That fix is coming in the next update.  It cleans up the current alert log, but does not cleanup the older rotated logs under some conditions.

                                  The interim fix is to periodically go in and delete those files (the ones with alert.log.xxxxx where xxxxx is a timestamp).

                                  Suricata can barf out a lot of log traffic, and running it on a Nano install is risky (in my opinion) due to the potential for exhausting disk space unexpectedly).

                                  Bill

                                  Awesome, thank you!  Curious, is it an upstream problem or pfsense package specific?

                                  1 Reply Last reply Reply Quote 0
                                  • bmeeksB
                                    bmeeks
                                    last edited by

                                    @binaryjay:

                                    Awesome, thank you!  Curious, is it an upstream problem or pfsense package specific?

                                    The log management bug I was referring to is pfSense-specific.  It is a bug in the package GUI code that I created.  My code looks for the file alert.log, but fails to then also do a pattern search for .log. so it can catch the rotated files.

                                    It will be fixed in the next Suricata package update.

                                    The Log Management code in the pfSense Suricata package was added to cope with the accumulation of log files that can occur.  It is not a perfect solution, though.  For one, the routine runs once every 5 minutes via a cron job.  If you have a busy network and a very noisy rule, then a lot of megabytes of log traffic could be created in the five-minute window between runs of the cron job.

                                    I also realized I did not answer your question about the "file_data" keyword errors.  Suricata does not currently process all of the same rule keywords and options that Snort does.  The "file_data" option is one of those.  This is an upstream issue.  There are some feature requests posted on the Suricata Redmine site asking that support be added for the new VRT keywords and options.

                                    Bill

                                    1 Reply Last reply Reply Quote 0
                                    • D
                                      demco
                                      last edited by

                                      Bill, is it possible to use clog? Or there are unintentional consequences?

                                      Regarding the "file_data" keyword and signature parsing errors raised by binaryjay. If Suricata start up with these errors, can we assume it is working and just skipped these signatures?

                                      Thanks, appreciate your work on Suricata and Snort.

                                      1 Reply Last reply Reply Quote 0
                                      • bmeeksB
                                        bmeeks
                                        last edited by

                                        @demco:

                                        Bill, is it possible to use clog? Or there are unintentional consequences?

                                        Regarding the "file_data" keyword and signature parsing errors raised by binaryjay. If Suricata start up with these errors, can we assume it is working and just skipped these signatures?

                                        Thanks, appreciate your work on Suricata and Snort.

                                        Yes, Suricata will just skip signatures it can't parse. It prints the error you see and goes to the next signature.  Snort, on the other hand, will print an error and quit on signature parsing errors.

                                        No, the logging method of Suricata is fixed by the underlying binary.  It does its own thing.  It would be a rather substantial rewrite/customization of the binary to have it use clog.  You may already know this, but I will repeat it anyway.  The Suricata and Snort packages on pfSense each consist of two separate but related parts.  What you see and interact with in the GUI is simply PHP code that creates the suricata.yaml (or snort.conf) configuration files used by a separate binary process.  So the GUI components simply help you create the text configuration file that is read and processed by the binary that does the actual network inspection.  That binary runs as a service.  The GUI code is only active during the time you have a menu page or tab open and are actually interacting with it.

                                        Bill

                                        1 Reply Last reply Reply Quote 0
                                        • ?
                                          A Former User
                                          last edited by

                                          Hey Bill,
                                          As far as I can tell, the bug with the logs not getting properly removed happens with the packet captures as well. Dunno if it happens with other logs, those are the first that went over the limits.

                                          1 Reply Last reply Reply Quote 0
                                          • bmeeksB
                                            bmeeks
                                            last edited by

                                            @jflsakfja:

                                            Hey Bill,
                                            As far as I can tell, the bug with the logs not getting properly removed happens with the packet captures as well. Dunno if it happens with other logs, those are the first that went over the limits.

                                            Yeah, it's actually with all the rotated logs.  Dumb mistake I made in the routine that looks for files to remove.  It will be fixed in the next update which I hope to post soon.  Mulling over whether to bump the binary version to 2.0.4 as well to stay in sync with upstream releases.

                                            Bill

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.