Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Taming the beasts… aka suricata blueprint

    Scheduled Pinned Locked Moved IDS/IPS
    504 Posts 64 Posters 297.5k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • C
      Cino
      last edited by

      great work jflsakfja!

      You may want to add decoder-events - 2200013 IPv6 truncated packet to the list for noise. My alert file is mostly this alert. What is strange, none of these show up in the GUI, only in the Alert file itself… Because there is no IP, i'm thinking the GUI doesn't display it. But that's for another thread after I confirm my findings.

      
06/25/2014-06:27:46.251740  [**] [1:2200013:1] SURICATA IPv6 truncated packet [**] [Classification: (null)] [Priority: 3] [**] [Raw pkt: 00 25 90 02 31 65 00 01 5C 33 54 41 86 DD 60 01 EA 40 05 B4 06 34 26 10 01 60 00 11 00 11 00 00 ]
06/25/2014-06:27:46.250371  [**] [1:2200013:1] SURICATA IPv6 truncated packet [**] [Classification: (null)] [Priority: 3] [**] [Raw pkt: 00 25 90 02 31 65 00 01 5C 33 54 41 86 DD 60 01 EA 40 05 B4 06 34 26 10 01 60 00 11 00 11 00 00 ]
06/25/2014-06:27:46.318718  [**] [1:2200013:1] SURICATA IPv6 truncated packet [**] [Classification: (null)] [Priority: 3] [**] [Raw pkt: 00 25 90 02 31 65 00 01 5C 33 54 41 86 DD 60 01 EA 40 05 B4 06 34 26 10 01 60 00 11 00 11 00 00 ]
06/25/2014-06:27:46.251489  [**] [1:2200013:1] SURICATA IPv6 truncated packet [**] [Classification: (null)] [Priority: 3] [**] [Raw pkt: 00 25 90 02 31 65 00 01 5C 33 54 41 86 DD 60 01 EA 40 05 B4 06 34 26 10 01 60 00 11 00 11 00 00 ]
06/25/2014-06:27:46.319106  [**] [1:2200013:1] SURICATA IPv6 truncated packet [**] [Classification: (null)] [Priority: 3] [**] [Raw pkt: 00 25 90 02 31 65 00 01 5C 33 54 41 86 DD 60 01 EA 40 05 B4 06 34 26 10 01 60 00 11 00 11 00 00 ]
06/25/2014-06:27:46.319352  [**] [1:2200013:1] SURICATA IPv6 truncated packet [**] [Classification: (null)] [Priority: 3] [**] [Raw pkt: 00 25 90 02 31 65 00 01 5C 33 54 41 86 DD 60 01 EA 40 05 B4 06 34 26 10 01 60 00 11 00 11 00 00 ]
06/25/2014-06:27:46.543947  [**] [1:2200013:1] SURICATA IPv6 truncated packet [**] [Classification: (null)] [Priority: 3] [**] [Raw pkt: 00 25 90 02 31 65 00 01 5C 33 54 41 86 DD 60 0A B8 B0 05 B4 06 34 26 10 01 60 00 11 00 11 00 00 ]
06/25/2014-06:27:46.319496  [**] [1:2200013:1] SURICATA IPv6 truncated packet [**] [Classification: (null)] [Priority: 3] [**] [Raw pkt: 00 25 90 02 31 65 00 01 5C 33 54 41 86 DD 60 01 EA 40 05 B4 06 34 26 10 01 60 00 11 00 11 00 00 ]
06/25/2014-06:27:46.543837  [**] [1:2200013:1] SURICATA IPv6 truncated packet [**] [Classification: (null)] [Priority: 3] [**] [Raw pkt: 00 25 90 02 31 65 00 01 5C 33 54 41 86 DD 60 0A B8 B0 05 B4 06 34 26 10 01 60 00 11 00 11 00 00 ]
06/25/2014-06:27:46.986163  [**] [1:2200013:1] SURICATA IPv6 truncated packet [**] [Classification: (null)] [Priority: 3] [**] [Raw pkt: 00 25 90 02 31 65 00 01 5C 33 54 41 86 DD 60 05 E7 64 05 B4 06 34 26 10 01 60 00 11 00 11 00 00 ]
06/25/2014-06:27:46.986289  [**] [1:2200013:1] SURICATA IPv6 truncated packet [**] [Classification: (null)] [Priority: 3] [**] [Raw pkt: 00 25 90 02 31 65 00 01 5C 33 54 41 86 DD 60 05 E7 64 05 B4 06 34 26 10 01 60 00 11 00 11 00 00 ]
      1 Reply Last reply Reply Quote 0
      • ?
        A Former User
        last edited by

        Getting the suricata logs on a remote syslog, and I'm running IPv6 tunnels because my upstream provider has vowed not to support IPv6 until AFTER they are completely cut off from the Internet (it will be the last ISP on earth that moves to IPv6). It might be due to the fact that their "3rd level" support technicians have no clue what an IP (4 or 6) address looks like. This is the same company that has to pay a daily fine for "excessive profits".

        I've not seen that alert so far, but I'll keep an eye out for it, thanks for pointing it out.

        So far the list seems to have stabilized a LOT, and I'm thinking about clearing up the list (# and no # rules). What gets on my nerves is that although the list originates from the snort topic (which is almost a year old!) most of the rules are still there.

        1 Reply Last reply Reply Quote 0
        • C
          Cino
          last edited by

          I'm wondering since i'm dual stack IPv4/IPv6(DHCP for both on the WAN), that is why i'm seeing this alert.. My ISP(TWC) doesn't officially support IPv6 but its enabled and working pretty well. Better then IPv4 traffic at times…

          1 Reply Last reply Reply Quote 0
          • ?
            A Former User
            last edited by

            I also dual stack, but have not seen that alert yet.  Same here, I get better speeds over the tunnel, than the native IPv4.

            On a side note, I'm updating the list. Mostly cleaning it up, moving the DO NOT USE! categories to the top so it's easier to find them, removing the # and no # rules…a line added here, a line taken away there...deleting some rules, updating the counts. I said I'm rarely mistaken (mathematically proven by the world's top universities, and the possibility of me being mistaken comes out to 0.000000000001%) but ffs guys. 5 rules on top of a count that says 4 rules are disabled should be noticed by someone by now :D.

            EDIT: Finished cleaning up. The list is now starting fresh for suricata (old rules coming over from snort that did not FP within a reasonable timeframe were removed), cleaned up and ready to go. Enjoy!

            1 Reply Last reply Reply Quote 0
            • ?
              A Former User
              last edited by

              whitelists were renamed to passlists not long ago. That is a definite bug in the suricata package. please post a link pointing to that post, on the most recent version(ed?) topic on suricata or re-post the same thing there. I'm not the suricata package maintainer, bmeeks is  ;D

              bmeeks got a tunnel working a few days back, so IPv6 support for both packages should improve.

              1 Reply Last reply Reply Quote 0
              • C
                Cino
                last edited by

                @jflsakfja:

                I also dual stack, but have not seen that alert yet.  Same here, I get better speeds over the tunnel, than the native IPv4.

                On a side note, I'm updating the list. Mostly cleaning it up, moving the DO NOT USE! categories to the top so it's easier to find them, removing the # and no # rules…a line added here, a line taken away there...deleting some rules, updating the counts. I said I'm rarely mistaken (mathematically proven by the world's top universities, and the possibility of me being mistaken comes out to 0.000000000001%) but ffs guys. 5 rules on top of a count that says 4 rules are disabled should be noticed by someone by now :D.

                EDIT: Finished cleaning up. The list is now starting fresh for suricata (old rules coming over from snort that did not FP within a reasonable timeframe were removed), cleaned up and ready to go. Enjoy!

                You're dual stack and using a Tunnel? When I mean dual stack; I mean receiving IPv4 IPv6 addresses on the same WAN interface.. The alerts are on that interface… Once I get my stuff fine tune, i'm going to copy it over to LAN and see if there is a difference.  I should had started with my LAN interface first but oh well...

                I noticed the counts were wrong, figured your eyes were getting crossed or something from the long post  :o Thanks again for a great thread!

                @avink use this topic https://forum.pfsense.org/index.php?topic=73906.msg428032#msg428032 to report the issue

                1 Reply Last reply Reply Quote 0
                • ?
                  A Former User
                  last edited by

                  Statistics on suricata's system usage

                  While idling, the system is using about 2% CPU and 23% RAM (4GB,32bit so not all available).

                  Under normal load (not extreme "ZOMG! multiple 10Gbps duplex laser links into space!") the CPU occasionally spikes to <10% for a brief moment, and the RAM stays about the same.

                  While reloading rules (when hitting save on the rules page or disabling a rule from the alerts tab) the CPU spikes to 50% with it occasionally going higher (seen up to 92% for a few seconds). CPU is a dual core, so 50% means 100% of one core. RAM usage drops at the start of the reload, and continues to slowly climb to ~23% while the rules are reloaded into RAM.

                  System has been set up according to this topic, with about 50 custom rules. It's the network gateway to a small datacenter, which also provides Internet connectivity for hosts outside the datacenter.

                  Matcher is AC, suricata interface is WAN.

                  @Cino : Having a tunnel means somehow getting the IPv6 addresses onto pfsense. Unless you are doing some pretty complicated stuff (routing packets to Mars, then Venus, then the end of the galaxy :D) all other interfaces that need to use those addresses (except the tunnel's "terminating" WAN interface) ARE dual stacked ;). Clients on the LAN side can connect either using an IPv4 address or completely stop using IPv4 and use an IPv6 address. That is the definition of dual stacked :D

                  1 Reply Last reply Reply Quote 0
                  • dotOneD
                    dotOne
                    last edited by

                    @Cino, Thanks.  I removed my message from this thread.

                    1 Reply Last reply Reply Quote 0
                    • C
                      Cino
                      last edited by

                      @jflsakfja:

                      @Cino : Having a tunnel means somehow getting the IPv6 addresses onto pfsense. Unless you are doing some pretty complicated stuff (routing packets to Mars, then Venus, then the end of the galaxy :D) all other interfaces that need to use those addresses (except the tunnel's "terminating" WAN interface) ARE dual stacked ;). Clients on the LAN side can connect either using an IPv4 address or completely stop using IPv4 and use an IPv6 address. That is the definition of dual stacked :D

                      Maybe I don't understand but then again I think I do  ;)  My WAN has both an IPv4 and a IPv6 address, and so does my LAN/Clients.. Before my ISP provided native IPv6; the WAN was IPv4 and I had a separate WAN(Tunneled) Interface for IPv6.  That being said, my current WAN/LAN/Clients are all dual stacked. When I only had IPv6 via a tunnel, the LAN and Clients were dual stacked while my WANs were not. Does that make sense?

                      Question: With a IPv6 Tunnel setup, do you run suricata on that interface also? Or just the IPv4 WAN Interface?

                      Edit: Noticed emerging-rbn-malvertisers.rules emerging-rbn.rules were not in the DO NOT USE list

                      1 Reply Last reply Reply Quote 0
                      • ?
                        A Former User
                        last edited by

                        They were supposed to remove those categories since they are no longer maintained. I'll add them in the do not use.

                        Yes that's what I meant with dual stack.

                        No need to run suricata on the IPv6 tunnel. It still alerts as is.

                        1 Reply Last reply Reply Quote 0
                        • C
                          Cino
                          last edited by

                          @jflsakfja:

                          They were supposed to remove those categories since they are no longer maintained. I'll add them in the do not use.

                          Do you think they will? after reading the comments in your enable rule list; hell will have to freeze over…

                          1 Reply Last reply Reply Quote 0
                          • ?
                            A Former User
                            last edited by

                            Supposed is the key word  :D

                            1 Reply Last reply Reply Quote 0
                            • T
                              ToxIcon
                              last edited by

                              Thank you for taking the time to write up and
                              giving us this great suricata blueprint,

                              maybe you can help me with this setup

                              I would like to put pfsense in front of an
                              existing firewall, but just as an
                              Suricata IDS and also use the IP Reputation Manager script
                              is this possible, if it is how would i go about doing
                              so. The system that i am using has 3 ports, 1 port will be for managing
                              pfs as a Suricata IDS  and the other 2 ports 1 will be connected to the wan of the other firewall
                              and the other port would be connect to the isp modem.

                              I would like the modem to not see the pfsense Suricata IDS box to act like its not even their and still pass the external address to the
                              other firewall behind the pfsense Suricata IDS  while the pfsense Suricata IDS is still catching the nasty stuff and blocking them.

                              1 Reply Last reply Reply Quote 0
                              • bmeeksB
                                bmeeks
                                last edited by

                                @Cino:

                                great work jflsakfja!

                                You may want to add decoder-events - 2200013 IPv6 truncated packet to the list for noise. My alert file is mostly this alert. What is strange, none of these show up in the GUI, only in the Alert file itself… Because there is no IP, i'm thinking the GUI doesn't display it. But that's for another thread after I confirm my findings.

                                
06/25/2014-06:27:46.251740  [**] [1:2200013:1] SURICATA IPv6 truncated packet [**] [Classification: (null)] [Priority: 3] [**] [Raw pkt: 00 25 90 02 31 65 00 01 5C 33 54 41 86 DD 60 01 EA 40 05 B4 06 34 26 10 01 60 00 11 00 11 00 00 ]
06/25/2014-06:27:46.250371  [**] [1:2200013:1] SURICATA IPv6 truncated packet [**] [Classification: (null)] [Priority: 3] [**] [Raw pkt: 00 25 90 02 31 65 00 01 5C 33 54 41 86 DD 60 01 EA 40 05 B4 06 34 26 10 01 60 00 11 00 11 00 00 ]
06/25/2014-06:27:46.318718  [**] [1:2200013:1] SURICATA IPv6 truncated packet [**] [Classification: (null)] [Priority: 3] [**] [Raw pkt: 00 25 90 02 31 65 00 01 5C 33 54 41 86 DD 60 01 EA 40 05 B4 06 34 26 10 01 60 00 11 00 11 00 00 ]
06/25/2014-06:27:46.251489  [**] [1:2200013:1] SURICATA IPv6 truncated packet [**] [Classification: (null)] [Priority: 3] [**] [Raw pkt: 00 25 90 02 31 65 00 01 5C 33 54 41 86 DD 60 01 EA 40 05 B4 06 34 26 10 01 60 00 11 00 11 00 00 ]
06/25/2014-06:27:46.319106  [**] [1:2200013:1] SURICATA IPv6 truncated packet [**] [Classification: (null)] [Priority: 3] [**] [Raw pkt: 00 25 90 02 31 65 00 01 5C 33 54 41 86 DD 60 01 EA 40 05 B4 06 34 26 10 01 60 00 11 00 11 00 00 ]
06/25/2014-06:27:46.319352  [**] [1:2200013:1] SURICATA IPv6 truncated packet [**] [Classification: (null)] [Priority: 3] [**] [Raw pkt: 00 25 90 02 31 65 00 01 5C 33 54 41 86 DD 60 01 EA 40 05 B4 06 34 26 10 01 60 00 11 00 11 00 00 ]
06/25/2014-06:27:46.543947  [**] [1:2200013:1] SURICATA IPv6 truncated packet [**] [Classification: (null)] [Priority: 3] [**] [Raw pkt: 00 25 90 02 31 65 00 01 5C 33 54 41 86 DD 60 0A B8 B0 05 B4 06 34 26 10 01 60 00 11 00 11 00 00 ]
06/25/2014-06:27:46.319496  [**] [1:2200013:1] SURICATA IPv6 truncated packet [**] [Classification: (null)] [Priority: 3] [**] [Raw pkt: 00 25 90 02 31 65 00 01 5C 33 54 41 86 DD 60 01 EA 40 05 B4 06 34 26 10 01 60 00 11 00 11 00 00 ]
06/25/2014-06:27:46.543837  [**] [1:2200013:1] SURICATA IPv6 truncated packet [**] [Classification: (null)] [Priority: 3] [**] [Raw pkt: 00 25 90 02 31 65 00 01 5C 33 54 41 86 DD 60 0A B8 B0 05 B4 06 34 26 10 01 60 00 11 00 11 00 00 ]
06/25/2014-06:27:46.986163  [**] [1:2200013:1] SURICATA IPv6 truncated packet [**] [Classification: (null)] [Priority: 3] [**] [Raw pkt: 00 25 90 02 31 65 00 01 5C 33 54 41 86 DD 60 05 E7 64 05 B4 06 34 26 10 01 60 00 11 00 11 00 00 ]
06/25/2014-06:27:46.986289  [**] [1:2200013:1] SURICATA IPv6 truncated packet [**] [Classification: (null)] [Priority: 3] [**] [Raw pkt: 00 25 90 02 31 65 00 01 5C 33 54 41 86 DD 60 05 E7 64 05 B4 06 34 26 10 01 60 00 11 00 11 00 00 ]

                                The GUI is not currently showing these alerts because it is not properly detecting them.  The Suricata alert log output is modified to produce a CSV (comma separated values) format.  There is a built-in PHP function that can parse a CSV file and split the result into fields.  For now the pfSense PHP code is counting fields and only showing on the ALERTS tab those alerts that parse into 13 (I think it is 13) fields.  The decoder events currently don't output 13 distinct fields, and hence are dropped by the ALERTS tab PHP code.

                                I can fix that in an upcoming update.

                                Bill

                                1 Reply Last reply Reply Quote 0
                                • bmeeksB
                                  bmeeks
                                  last edited by

                                  @jflsakfja:

                                  whitelists were renamed to passlists not long ago. That is a definite bug in the suricata package. please post a link pointing to that post, on the most recent version(ed?) topic on suricata or re-post the same thing there. I'm not the suricata package maintainer, bmeeks is  ;D

                                  bmeeks got a tunnel working a few days back, so IPv6 support for both packages should improve.

                                  I do finally have a working IPv6 tunnel, and I did see the bug report in a separate post and responded. I will fix it.

                                  Bill

                                  1 Reply Last reply Reply Quote 0
                                  • ?
                                    A Former User
                                    last edited by

                                    @Arist:

                                    Thank you for taking the time to write up and
                                    giving us this great suricata blueprint,

                                    maybe you can help me with this setup

                                    I would like to put pfsense in front of an
                                    existing firewall, but just as an
                                    Suricata IDS and also use the IP Reputation Manager script
                                    is this possible, if it is how would i go about doing
                                    so. The system that i am using has 3 ports, 1 port will be for managing
                                    pfs as a Suricata IDS  and the other 2 ports 1 will be connected to the wan of the other firewall
                                    and the other port would be connect to the isp modem.

                                    I would like the modem to not see the pfsense Suricata IDS box to act like its not even their and still pass the external address to the
                                    other firewall behind the pfsense Suricata IDS  while the pfsense Suricata IDS is still catching the nasty stuff and blocking them.

                                    What you are looking for is running pfsense (+suricata) as a transparent bridge in front of the "normal" firewall. Suricata might have trouble deciding its home net values if running as a bridge (snort did) but that's easily corrected by manually entering the home net.

                                    A transparent bridge is not visible on the network. If you send a packet to one interface, and the rules allow it, it will pop up through the other interface. You can actually do some pretty clever stuff with it, provided you are using a single host (no CARP). Think of it as merging the two interfaces into a single interface, with filtering applied.

                                    A couple of years back I was able to access a public server hosted behind a transparent bridge, from a host on a NATed interface on that transparent bridge (let's say the admin interface), using the server's public IP (universally understood as NOT possible to do). Then again I'm the only person on the planet that managed to get IPv6 working through 30 year old switches :)

                                    That said, my personal recommendation is NOT to run pfsense like that. If you are trying to protect a small network, put pfsense directly as the core router (which allows you to move onto CARP if you so wish), which also saves time (money) + space (1 pc instead of 2 daisy chained firewalls) + power. Then set up firewalls on network hosts, along with other security measures (brute force protection for example) working together to protect your hosts.

                                    EDIT: Clarification: I'm not saying don't ever use transparent bridges. If all you need is a single firewall host, it's actually better to run it as a transparent bridge since the host is not visible from the network. Depending on law mandated paranoia (called certifications in the industry), that might actually be exactly what you should use.
                                    There are also downsides, an example is that the firewall hosts themselves don't have internet access (cannot check for updates), as set up in a plain vanilla transparent bridge (permission is hereby granted to correct me if I'm wrong).

                                    1 Reply Last reply Reply Quote 0
                                    • J
                                      justsomeone
                                      last edited by

                                      Awesome post, thanks!

                                      I'm trying to install pfiprep, and am getting an error when installing one of the dependencies.

                                      $ pkg_add -r grepcidr
tar: Failed to set default locale

                                      Can someone please tell me what to do to fix this?

                                      "Bad shit happens to drunk people."

                                      1 Reply Last reply Reply Quote 0
                                      • BBcan177B
                                        BBcan177 Moderator
                                        last edited by

                                        @justsomeone:

                                        $ pkg_add -r grepcidr
tar: Failed to set default locale

                                        First, I would recommend that you make a Full Backup  Diagnostics:Backup/Restore  (Backup Area "all") and Download configuration… :)

                                        I haven't come across this error on any of my installs or with the ones that I have helped to get working.

                                        What version of pfSense are you using? Is it a Full Install or a Nano version?

                                        Are you seeing any other errors? Could you post the full output of that command?

                                        Can you download the file manually? Maybe a Firewall Rule or Snort is blocking it?

                                        Try to [  [b]ping ftp.freebsd.org   ] and see if you get a reply?

                                        [  [b]fetch ftp://ftp.freebsd.org/pub/FreeBSD/ports/amd64/packages-8.3-release/Latest/grepcidr.tbz  ]
                                                                                              (This path is for the amd64 Release)

                                        [  [b]pkg_add grepcidr.tbz  ]

                                        "Experience is something you don't get until just after you need it."

                                        Website: http://pfBlockerNG.com
                                        Twitter: @BBcan177  #pfBlockerNG
                                        Reddit: https://www.reddit.com/r/pfBlockerNG/new/

                                        1 Reply Last reply Reply Quote 0
                                        • F
                                          foetus
                                          last edited by

                                          My cron job is throwing errors.
                                          Since manually running "/usr/bin/nice -n20 /home/badips/pfiprep >> /home/badips/download.log 2>&1"  gives me an error "Ambiguous output redirect." .
                                          Leaving out the log output redirect it works fine.  (aka >> …  or > ...)  So I could just remove the log output, but I'd rather keep it. :)

                                          For the rest, perfect.
                                          I edited the main script to leave out some lists, and also moved to using the main IR_ lists.
                                          Also great work on the widget.

                                          1 Reply Last reply Reply Quote 0
                                          • C
                                            Cino
                                            last edited by

                                            @foetus:

                                            My cron job is throwing errors.
                                            Since manually running "/usr/bin/nice -n20 /home/badips/pfiprep >> /home/badips/download.log 2>&1"  gives me an error "Ambiguous output redirect." .
                                            Leaving out the log output redirect it works fine.  (aka >> …  or > ...)  So I could just remove the log output, but I'd rather keep it. :)

                                            for the cron job "/usr/bin/nice -n20 /home/badips/pfiprep >> /home/badips/download.log 2>&1"  works for me, but if i run it via directly from the shell, I get the same error you're seeing

                                            running from the shell, i type "/usr/bin/nice -n20 /home/badips/pfiprep >> /home/badips/download.log" instead

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.