Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Suricata 2.1.5 Update – Release Notes

    IDS/IPS
    10
    23
    6.6k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • D
      DigitalDeviant
      last edited by

      @gsiemon:

      Seems that someone else had this problem with Suricata 2.07 back in March.

      https://lists.openinfosecfoundation.org/pipermail/oisf-users/2015-March/004600.html

      Issue appears to be related to a bug fix in 2.07:

      https://redmine.openinfosecfoundation.org/issues/1318

      Recommendation in the mailing list was to reduce the stream.prealloc-sessions variable.  I think this is set under:

      Interfaces - Lan/Flow Stream (Tab) - Flow Manager Settings - Preallocated Sessions. 

      The previous poster appears to have worked around it by increasing the Stream Memory Cap.  The current default settings seems to be not allocating enough Stream Memory for the number of Preallocated Sessions.  Either decreasing the Preallocated Sessions or increasing Stream Memory Cap should resolve the issue.

      The final post in the mailing list provides some guidance for memory/preallocated session settings:

      Also, how can I calculate the highest value that I can use?

      TcpSession structure is 192 bytes, PoolBucket 24. So it should be:

      (192 + 24) * prealloc_sessions * number of threads = memory use in bytes

      For my setup, I seem to have 7 packet processing threads and 1 management thread.

      For the default preallocated sessions (32768) this would require either 56623104 or 49545216 bytes of memory depending on whether the management thread is included in the calculation or not.

      This would explain why increasing the Stream Memory Cap to 64MB has fixed the previous poster's (and my) problem.

      Hope this helps.

      This worked for me.

      1 Reply Last reply Reply Quote 0
      • bmeeksB
        bmeeks
        last edited by

        Thanks for the research.  I will adjust the default value in the next package update.

        Bill

        1 Reply Last reply Reply Quote 0
        • C
          Cino
          last edited by

          @avink:

          I saw the same error during startup, SC_ERR_POOL_INIT and SC_ERR_THREAT_INIT error.
          Could be IPv6 related, I'm using IPv6 on all interfaces. about 45% of all traffic is IPv6.

          To solve I increased 'Stream Memory Cap' to 64Mb.

          André

          I changed my settings based on this post and all my interfaces are back online

          1 Reply Last reply Reply Quote 0
          • V
            viragomann
            last edited by

            At my installation suricata also doesn't start with 64 MB at 'Stream Memory Cap', however, with 96 it does.

            1 Reply Last reply Reply Quote 0
            • C
              ccb056
              last edited by

              @Cino:

              @avink:

              I saw the same error during startup, SC_ERR_POOL_INIT and SC_ERR_THREAT_INIT error.
              Could be IPv6 related, I'm using IPv6 on all interfaces. about 45% of all traffic is IPv6.

              To solve I increased 'Stream Memory Cap' to 64Mb.

              André

              I changed my settings based on this post and all my interfaces are back online

              64 MB worked for me also.

              1 Reply Last reply Reply Quote 0
              • S
                stewgoin
                last edited by

                96 worked here as well, 64 borked.

                1 Reply Last reply Reply Quote 0
                • G
                  gsiemon
                  last edited by

                  My guess is that 96MB worked for you as you have more processors than me.  I have 4 active processors on my setup so I get 7 packet processing threads and a management thread.  If you have more processors then you end up with more threads and therefore need more memory.

                  1 Reply Last reply Reply Quote 0
                  • V
                    viragomann
                    last edited by

                    Quad core with HT

                    1 Reply Last reply Reply Quote 0
                    • G
                      gsiemon
                      last edited by

                      @viragomann:

                      Quad core with HT

                      So you have double the number of logical processors that I have.  So you'd need a minimum of 94.5MB to make it work.

                      1 Reply Last reply Reply Quote 0
                      • V
                        viragomann
                        last edited by

                        I see. Thanks for explanation.

                        1 Reply Last reply Reply Quote 0
                        • M
                          mcentirefj
                          last edited by

                          How do you increase the stream memory cap?

                          Edit: Nevermind, found it in the config.xml

                          Edit2: Got it working. Underestimated how many threads I had. Needed to bump my stream mem cap to 168.75mb according to the formula gsiemon provided:

                          Also, how can I calculate the highest value that I can use?

                          TcpSession structure is 192 bytes, PoolBucket 24. So it should be:

                          (192 + 24) * prealloc_sessions * number of threads = memory use in bytes

                          1 Reply Last reply Reply Quote 0
                          • V
                            viragomann
                            last edited by

                            In the GUI, Suricata interface settings:

                            Suricata > Interface > Flow and Stream
                            "Stream Memory Cap"

                            Suricata2.png
                            Suricata2.png_thumb

                            1 Reply Last reply Reply Quote 1
                            • S
                              SixXxShooTeR
                              last edited by

                              Neither interface will start for me. I know in your Snort write up you mentioned if snort didn't start it was likely because preprocessors werent turned on. Am I missing something like that in the Suricata package?

                              1 Reply Last reply Reply Quote 0
                              • bmeeksB
                                bmeeks
                                last edited by

                                @SixXxShooTeR:

                                Neither interface will start for me. I know in your Snort write up you mentioned if snort didn't start it was likely because preprocessors werent turned on. Am I missing something like that in the Suricata package?

                                No, Suricata does not have preprocessors like Snort does.  Have you looked at the log files?  There is the system log and there are log files for each Suricata interface (look on the LOGS tab in Suricata).

                                Bill

                                1 Reply Last reply Reply Quote 0
                                • S
                                  SixXxShooTeR
                                  last edited by

                                  These are the errors from the WAN suricata.log

                                  6/6/2015 – 02:28:04 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - Can't use file_data with flow:to_server or from_client with http.
                                  6/6/2015 – 02:28:04 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $EXTERNAL_NET any -> $SMTP_SERVERS 25 (msg:"FILE-IDENTIFY CIS file magic detected"; flow:to_server,established; file_data; content:"|43 49 53 00 00 00 00 00|"; fast_pattern:only; flowbits:set,file.cis; flowbits:noalert; metadata:service smtp; classtype:misc-activity; sid:28367; rev:1;)" from file /usr/pbi/suricata-amd64/etc/suricata/suricata_54713_em0/rules/flowbit-required.rules at line 19
                                  6/6/2015 – 02:28:04 - <info>-- 2 rule files processed. 223 rules successfully loaded, 1 rules failed
                                  6/6/2015 -- 02:28:04 - <info>-- 223 signatures processed. 34 are IP-only rules, 4 are inspecting packet payload, 63 inspect application layer, 72 are decoder event only
                                  6/6/2015 -- 02:28:04 - <info>-- building signature grouping structure, stage 1: preprocessing rules... complete
                                  6/6/2015 -- 02:28:04 - <info>-- building signature grouping structure, stage 2: building source address list... complete
                                  6/6/2015 -- 02:28:04 - <info>-- building signature grouping structure, stage 3: building destination address lists... complete
                                  6/6/2015 -- 02:28:04 - <info>-- Threshold config parsed: 0 rule(s) found
                                  6/6/2015 -- 02:28:04 - <info>-- Core dump size is unlimited.
                                  6/6/2015 -- 02:28:04 - <info>-- fast output device (regular) initialized: alerts.log
                                  6/6/2015 -- 02:28:04 - <info>-- http-log output device (regular) initialized: http.log
                                  6/6/2015 -- 02:28:04 - <info>-- Using 1 live device(s).
                                  6/6/2015 -- 02:28:04 - <info>-- using interface em0
                                  6/6/2015 -- 02:28:04 - <info>-- Running in 'auto' checksum mode. Detection of interface state will require 1000 packets.
                                  6/6/2015 -- 02:28:04 - <info>-- Found an MTU of 1500 for 'em0'
                                  6/6/2015 -- 02:28:04 - <info>-- Set snaplen to 1516 for 'em0'
                                  6/6/2015 -- 02:28:04 - <error>-- [ERRCODE: SC_ERR_POOL_INIT(66)] - alloc error
                                  6/6/2015 – 02:28:04 - <error>-- [ERRCODE: SC_ERR_POOL_INIT(66)] - pool grow failed
                                  6/6/2015 – 02:28:04 - <info>-- RunModeIdsPcapAutoFp initialised
                                  6/6/2015 -- 02:28:04 - <error>-- [ERRCODE: SC_ERR_THREAD_INIT(49)] - thread "Detect6" closed on initialization.
                                  6/6/2015 – 02:28:04 - <error>-- [ERRCODE: SC_ERR_INITIALIZATION(45)] - Engine initialization failed, aborting…</error></error></info></error></error></info></info></info></info></info></info></info></info></info></info></info></info></info></info></error></error>

                                  1 Reply Last reply Reply Quote 0
                                  • S
                                    SixXxShooTeR
                                    last edited by

                                    increasing the stream memory cap from 32MB to 64MB fixed the issue.

                                    1 Reply Last reply Reply Quote 0
                                    • bmeeksB
                                      bmeeks
                                      last edited by

                                      @SixXxShooTeR:

                                      increasing the stream memory cap from 32MB to 64MB fixed the issue.

                                      Yes, the old default stream memory setting is too small as of the 2.0.7 release of Suricata.  I will update the default size and make it some larger in the next package update.

                                      Bill

                                      1 Reply Last reply Reply Quote 0
                                      • A atafm2 referenced this topic on
                                      • A atafm2 referenced this topic on
                                      • A atafm2 referenced this topic on
                                      • First post
                                        Last post