Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    2.3.b.20160324.0516 + Suricata IPS eventually kills internet traffic issue…

    Scheduled Pinned Locked Moved 2.3-RC Snapshot Feedback and Issues - ARCHIVED
    13 Posts 4 Posters 4.9k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • T
      Tantamount
      last edited by

      Okay, discovered that igb0 goes down the moment this happens in Suricata:
      28/3/2016 – 22:43:35 - <info>-- building signature grouping structure, stage 3: building destination address lists... complete

      So it's not the driver crashing -- it's just Suricata finishing loading and then restarting the network stuff.

      In troubleshooting, I updated pfsense to the latest version, disabled the hardware acceleration options for the nic, and rebooted.

      Everything seems stable now, although I do very occasionally see these still:
      Mar 28 22:44:52 pfSense kernel: 292.482947 [1162] netmap_grab_packets      bad pkt at 59 len 2159</info>

      1 Reply Last reply Reply Quote 0
      • Raul RamosR
        Raul Ramos
        last edited by

        Suricata have some big changes. Read, at least, the first topic on https://forum.pfsense.org/index.php?topic=107847.0.

        pfSense:
        ASRock -> Wolfdale1333-D667 (2GB TeamElite Ram)
        Marvell 88SA8040 Sata to CF(Sandisk 4GB) Controller
        NIC's: RTL8100E (Internal ) and Intel® PRO/1000 PT Dual (Intel 82571GB)

        1 Reply Last reply Reply Quote 0
        • G
          gsiemon
          last edited by

          You may want to review this post from Bill as he says you need to disable all hardware acceleration on nics running with Suricata IPS.

          https://forum.pfsense.org/index.php?topic=108068.msg601891#msg601891

          1 Reply Last reply Reply Quote 0
          • bmeeksB
            bmeeks
            last edited by

            This –

            @gsiemon:

            You may want to review this post from Bill as he says you need to disable all hardware acceleration on nics running with Suricata IPS.

            https://forum.pfsense.org/index.php?topic=108068.msg601891#msg601891

            Netmap (used for inline IPS mode) requires both TCP segmentation offloading and hardware checksum calcs be disabled as illustrated in the linked post.

            Bill

            1 Reply Last reply Reply Quote 0
            • T
              Tantamount
              last edited by

              Thanks for the confirmation.

              Were the warnings that already exist for hardware acceleration supposed to give fair warning that dragons be here?

              I'm just trying to figure out if I should feature request some type of settings conflict check for the user or not.  Does pfsense do any type of sanity check elsewhere for the user?

              1 Reply Last reply Reply Quote 0
              • bmeeksB
                bmeeks
                last edited by

                @Tantamount:

                Thanks for the confirmation.

                Were the warnings that already exist for hardware acceleration supposed to give fair warning that dragons be here?

                I'm just trying to figure out if I should feature request some type of settings conflict check for the user or not.  Does pfsense do any type of sanity check elsewhere for the user?

                No, the package does not currently check those settings.  But it is a valid suggestion that it should.  I will add that in a coming update.  Working on a few other things, too, so give me a little while to get everything done.  A small update to the package was posted today, but that only fixes a Barnyard2 run dependency that was missing and causing the Barnyard2 binary to not get installed along with Suricata.

                Bill

                1 Reply Last reply Reply Quote 0
                • T
                  Tantamount
                  last edited by

                  I added Suricata to my LAN interface, and am running into the same problem as above:

                  It works for about half a minute, and then all communication on my LAN interface goes dead.

                  /var/log/system.log is mostly silent when this happens:

                  Apr  1 16:03:17 pfSense kernel: em0: link state changed to DOWN
                  Apr  1 16:03:17 pfSense check_reload_status: Linkup starting em0

                  The hardware acceleration settings under System->Advanced->Networking are supposed to be global, correct?  All the hardware choices are disabled.

                  The nic hardware is fairly recently supported I think? Perhaps there are still some bugs to work out?

                  em0@pci0:0:31:6:        class=0x020000 card=0xe0001458 chip=0x15b88086 rev=0x31 hdr=0x00
                      vendor    = 'Intel Corporation'
                      device    = 'Ethernet Connection (2) I219-V'
                      class      = network
                      subclass  = ethernet

                  Oddly I had enabled Suricata on the LAN a number of days back, but forgot to enable the ET Pro classifications.  Once I did that, about a few hours later I started running into this trouble. Hopefully I just need to allocate more memory somewhere?

                  Edit:

                  Am running the latest version of 2.3 beta:

                  2.3-RC (amd64)
                  built on Thu Mar 31 23:48:37 CDT 2016

                  1 Reply Last reply Reply Quote 0
                  • T
                    Tantamount
                    last edited by

                    Apparently Suricata 3.0.1 was released today.  There's mention of lots of stability and memory fixes… perhaps this will help my situation?

                    http://suricata-ids.org/news/

                    1 Reply Last reply Reply Quote 0
                    • bmeeksB
                      bmeeks
                      last edited by

                      @Tantamount:

                      I added Suricata to my LAN interface, and am running into the same problem as above:

                      It works for about half a minute, and then all communication on my LAN interface goes dead.

                      /var/log/system.log is mostly silent when this happens:

                      Apr  1 16:03:17 pfSense kernel: em0: link state changed to DOWN
                      Apr  1 16:03:17 pfSense check_reload_status: Linkup starting em0

                      The hardware acceleration settings under System->Advanced->Networking are supposed to be global, correct?  All the hardware choices are disabled.

                      The nic hardware is fairly recently supported I think? Perhaps there are still some bugs to work out?

                      em0@pci0:0:31:6:        class=0x020000 card=0xe0001458 chip=0x15b88086 rev=0x31 hdr=0x00
                          vendor    = 'Intel Corporation'
                          device    = 'Ethernet Connection (2) I219-V'
                          class      = network
                          subclass  = ethernet

                      Oddly I had enabled Suricata on the LAN a number of days back, but forgot to enable the ET Pro classifications.  Once I did that, about a few hours later I started running into this trouble. Hopefully I just need to allocate more memory somewhere?

                      Edit:

                      Am running the latest version of 2.3 beta:

                      2.3-RC (amd64)
                      built on Thu Mar 31 23:48:37 CDT 2016

                      Were you running Suricata on just the LAN interface or both LAN and WAN at the same time?

                      Bill

                      1 Reply Last reply Reply Quote 0
                      • T
                        Tantamount
                        last edited by

                        Both.

                        1 Reply Last reply Reply Quote 0
                        • bmeeksB
                          bmeeks
                          last edited by

                          @Tantamount:

                          Both.

                          Thanks for the info.  Just looking for some baseline data to start with troubleshooting/reproducing the problem.  I admit to not having run Suricata for an extended period with inline IPS mode enabled.

                          Bill

                          1 Reply Last reply Reply Quote 0
                          • T
                            Tantamount
                            last edited by

                            Well, I'm not certain if it was due to prolonged use, or the more recent enabling of the et pro rules.

                            Also, bad things happen even when I disable the "block" setting (Which I assume disables the inline stuff?).

                            I've just unchecked all of the LAN categories and attempted to re-enable, but the same problem happened.

                            If I use the console and kill the suricata process attached to the lan nic, the lan traffic immediately starts to work again.

                            1 Reply Last reply Reply Quote 0
                            • First post
                              Last post
                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.