Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Taming the beasts… aka suricata blueprint

    Scheduled Pinned Locked Moved IDS/IPS
    504 Posts 64 Posters 301.4k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • W
      wcrowder
      last edited by

      Ran into a small problem with bbcan177's script.  Was working wonderfully then quit updating after the last Alpha update. Running pfSense 2.2 alpha.

      Here's the error:

      SSL options: 81004bff
      Peer verification enabled
      Using CA cert file: /etc/ssl/cert.pem
      Certificate verification failed for /O=www.projecthoneypot.org/OU=Domain Control Validated/CN=www.projecthoneypot.org
      675141692:error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed:/usr/pfSensesrc/src/secure/lib/libssl/../../../crypto/openssl/ssl/s3_clnt.c:1169:
      fetch: https://www.projecthoneypot.org/list_of_ips.php?t=d&rss=1: Authentication error
      
      

      Anyone else run into this?

      NEVER MIND, I'm stupid… LOL..

      1 Reply Last reply Reply Quote 0
      • M
        Mr. Jingles
        last edited by

        Well thank you Bill and JFL for boosting my self esteem  ;D ;D ;D

        But now… 8)

        ( ;D )

        I am struggling with a dnsmasq problem (blocking ad servers via dnsmasq, can't get the script to run in cron - although manually it works), and on the site that provides the lists I also noted a format for Snort:

        http://pgl.yoyo.org/adservers/

        (Select the second drop down - list ad servers IP adresses): there is also a Snort format of the list there.

        Would it make sense to use Snort to block ad servers using that list?

        I know there will be reservations; 'noob, shut up'.

        Ok, I will  :P

        6 and a half billion people know that they are stupid, agressive, lower life forms.

        1 Reply Last reply Reply Quote 0
        • bmeeksB
          bmeeks
          last edited by

          @Hollander:

          Well thank you Bill and JFL for boosting my self esteem  ;D ;D ;D

          But now… 8)

          ( ;D )

          I am struggling with a dnsmasq problem (blocking ad servers via dnsmasq, can't get the script to run in cron - although manually it works), and on the site that provides the lists I also noted a format for Snort:

          http://pgl.yoyo.org/adservers/

          (Select the second drop down - list ad servers IP adresses): there is also a Snort format of the list there.

          Would it make sense to use Snort to block ad servers using that list?

          I know there will be reservations; 'noob, shut up'.

          Ok, I will  :P

          When you can't get a script to run via cron, the #1 cause is forgetting to provide the complete and entire path to all files used in the script.  When a cron task executes, it does not inherit the user environment you have when working at a shell prompt (CLI).  Some examine the script and make sure full paths are provided for all referenced files.

          Bill

          1 Reply Last reply Reply Quote 0
          • BBcan177B
            BBcan177 Moderator
            last edited by

            @wcrowder:

            Ran into a small problem with bbcan177's script.  Was working wonderfully then quit updating after the last Alpha update. Running pfSense 2.2 alpha.

            Here's the error:

            SSL options: 81004bff
            Peer verification enabled
            Using CA cert file: /etc/ssl/cert.pem
            Certificate verification failed for /O=www.projecthoneypot.org/OU=Domain Control Validated/CN=www.projecthoneypot.org
            675141692:error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed:/usr/pfSensesrc/src/secure/lib/libssl/../../../crypto/openssl/ssl/s3_clnt.c:1169:
            fetch: https://www.projecthoneypot.org/list_of_ips.php?t=d&rss=1: Authentication error
            
            

            Anyone else run into this?

            NEVER MIND, I'm stupid… LOL..

            SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
            The server certificate is invalid, either because it is signed by an invalid CA (internal CA, self signed,…), doesn't match the server's name or because it is expired.

            Hi wcrowder, is this still an issue for you?

            "Experience is something you don't get until just after you need it."

            Website: http://pfBlockerNG.com
            Twitter: @BBcan177  #pfBlockerNG
            Reddit: https://www.reddit.com/r/pfBlockerNG/new/

            1 Reply Last reply Reply Quote 0
            • BBcan177B
              BBcan177 Moderator
              last edited by

              @bmeeks:

              I do also plan to offer what is a kind of CLI interface for this.  Do a Google search for enablesid.conf, modifysid.conf and disablesid.conf.  You should get some hits showing how these work with packages such as PulledPork.  The idea for the Suricata and Snort packages is to be able to edit these files offline and then upload them to the firewall.

              This will make the Rules Configuration Process a Breeze. All we would have to do, is copy the text based conf file to any other Snort/Suricata Interface or to another Box. Rules can be disabled or enabled by SID, Category or by PCRE (ie : Autodesk or Adobe - Which will disable/enable any rules that has any of those Keywords in them. This way even as new rules are added, the PCRE will disable them at each rule-update automatically.  :)

              "Experience is something you don't get until just after you need it."

              Website: http://pfBlockerNG.com
              Twitter: @BBcan177  #pfBlockerNG
              Reddit: https://www.reddit.com/r/pfBlockerNG/new/

              1 Reply Last reply Reply Quote 0
              • W
                wcrowder
                last edited by

                Sorta, I just removed the S from each HTTP and it updated, not the best practice.  LOL. I've updated the cert.pem with the latest version of Mozilla's certs, no go, assuming it's something with the 2.2 Alpha. Also takes 7 minutes to get through "configuring Firewall" upon boot. Have  "./pfiprep killdb" twice then pfiprep with bypass=YES after update and it works. Still now SSL. Just messing around. Really fine work on your part.

                1 Reply Last reply Reply Quote 0
                • BBcan177B
                  BBcan177 Moderator
                  last edited by

                  @wcrowder:

                  Sorta, I just removed the S from each HTTP and it updated, not the best practice.  LOL. I've updated the cert.pem with the latest version of Mozilla's certs, assuming it's something with the 2.2 Alpha. Also takes 7 minutes to get through "configuring Firewall" upon boot. Have  "./pfiprep killdb" twice. Just messing around. Really fine work on your part.

                  Hi wcrowder,

                  You need to check the pfSense Cert, as that is the machine that is fetching those files.

                  "Experience is something you don't get until just after you need it."

                  Website: http://pfBlockerNG.com
                  Twitter: @BBcan177  #pfBlockerNG
                  Reddit: https://www.reddit.com/r/pfBlockerNG/new/

                  1 Reply Last reply Reply Quote 0
                  • BBcan177B
                    BBcan177 Moderator
                    last edited by

                    https://www.mail-archive.com/freebsd-security@freebsd.org/msg05085.html
                    Looks like FreeBSD 9 doesn't have a default pem file in /etc/ssl/cert.pem ?

                    fetch
                    –ca-cert=file
                    [SSL] Path to certificate bundle containing trusted CA cer-
                    tificates.  If not specified, /etc/ssl/cert.pem is used.  The
                    file may contain multiple CA certificates. The port
                    security/ca_root_nss is a common source of a current CA bun-
                    dle.

                    Do you have a file in that directory?
                    [ [b] ls -lah /etc/ssl/ ]

                    I'm not the best at certs. But I think you should just be able to generate a Self-signed one from the pfSense Cert Manager? I don't want to suggest you disable the cert check in the fetch cmd line.

                    http://www.freebsd.org/cgi/man.cgi?query=fetch(1)
                    My script fetchs with this line:

                    $pathfetch -v -o $workdir$infile -T 20 $addr$infile

                    Try this:

                    cd /tmp
                    fetch -v -o honeypot.txt "https://www.projecthoneypot.org/list_of_ips.php?t=d&rss=1"

                    Or you can switch to using WGET: (Script pfiprep has an option to switch between fetch and wget)

                    Script Command:
                    $pathwget –no-check-certificate -T 20 -O $workdir$infile $addr$infile

                    Try:
                    wget –no-check-certificate -O honeypot.txt "https://www.projecthoneypot.org/list_of_ips.php?t=d&rss=1"

                    I think Foetus had a similar issue with 2.2. Not sure how he made out.

                    "Experience is something you don't get until just after you need it."

                    Website: http://pfBlockerNG.com
                    Twitter: @BBcan177  #pfBlockerNG
                    Reddit: https://www.reddit.com/r/pfBlockerNG/new/

                    1 Reply Last reply Reply Quote 0
                    • S
                      Supermule Banned
                      last edited by

                      Isnt 2.2 FreeBSD 10 based??

                      1 Reply Last reply Reply Quote 0
                      • M
                        Mr. Jingles
                        last edited by

                        @bmeeks:

                        @Hollander:

                        Well thank you Bill and JFL for boosting my self esteem  ;D ;D ;D

                        But now… 8)

                        ( ;D )

                        I am struggling with a dnsmasq problem (blocking ad servers via dnsmasq, can't get the script to run in cron - although manually it works), and on the site that provides the lists I also noted a format for Snort:

                        http://pgl.yoyo.org/adservers/

                        **(Select the second drop down - list ad servers IP adresses): there is also a Snort format of the list there.

                        Would it make sense to use Snort to block ad servers using that list?**

                        I know there will be reservations; 'noob, shut up'.

                        Ok, I will  :P

                        When you can't get a script to run via cron, the #1 cause is forgetting to provide the complete and entire path to all files used in the script.  When a cron task executes, it does not inherit the user environment you have when working at a shell prompt (CLI).  Some examine the script and make sure full paths are provided for all referenced files.

                        Bill

                        Thanks Bill  ;D

                        I didn't mean to bother you with the problem I have to get a cron job running (I did have the full path, btw): I was only meandering away from the point in question: the text in bold in the above  :P

                        6 and a half billion people know that they are stupid, agressive, lower life forms.

                        1 Reply Last reply Reply Quote 0
                        • BBcan177B
                          BBcan177 Moderator
                          last edited by

                          @Supermule:

                          Isnt 2.2 FreeBSD 10 based??

                          Yes it is, but I couldn't find much info on this issue with my google FU…

                          I did find this link:
                          http://smyck.net/2014/01/22/freebsd-authentication-error/

                          I don't have a 2.2 box, so I can't test it myself. If anyone else has 2.2, can you see if these two commands work? Don't need to be using my Script to test if fetch and https work on 2.2 Alpha.

                          cd /tmp

                          fetch -v -o honeypot.txt "https://www.projecthoneypot.org/list_of_ips.php?t=d&rss=1"
                          wget –no-check-certificate -O honeypot.txt "https://www.projecthoneypot.org/list_of_ips.php?t=d&rss=1"

                          "Experience is something you don't get until just after you need it."

                          Website: http://pfBlockerNG.com
                          Twitter: @BBcan177  #pfBlockerNG
                          Reddit: https://www.reddit.com/r/pfBlockerNG/new/

                          1 Reply Last reply Reply Quote 0
                          • W
                            wcrowder
                            last edited by

                            @BBcan177

                            My problem was a PEBKAC, I had a self signed cert installed in pfSense. I removed it and it works just fine.  Thanks for your hard work. I can report that I'm not having any issues with the script or the widget in 2.2, all seems to be working fine except the patches for dns look up report that they can not be installed cleanly. Not important.

                            Thanks,
                            Bill

                            1 Reply Last reply Reply Quote 0
                            • BBcan177B
                              BBcan177 Moderator
                              last edited by

                              @wcrowder:

                              all seems to be working fine except the patches for dns look up report that they can not be installed cleanly. Not important.

                              Good to hear!

                              Is this the 2.2 diag_dns.php file that matches what you have on your box?
                              https://github.com/pfsense/pfsense/blob/master/usr/local/www/diag_dns.php

                              If its the same, I will post an updated version of the Patch.

                              "Experience is something you don't get until just after you need it."

                              Website: http://pfBlockerNG.com
                              Twitter: @BBcan177  #pfBlockerNG
                              Reddit: https://www.reddit.com/r/pfBlockerNG/new/

                              1 Reply Last reply Reply Quote 0
                              • W
                                wcrowder
                                last edited by

                                I believe it is. On the current snapshot.

                                Patch can NOT be applied cleanly (detail)
                                Patch can NOT be reverted cleanly (detail)

                                Thanks,
                                Bill

                                1 Reply Last reply Reply Quote 0
                                • W
                                  wcrowder
                                  last edited by

                                  Also neither widget displays list count or correctly determines failed downloads. It is correctly displaying other correct information. This is on Today's snapshot of 2.2. I didn't look into it because I can read logs… :)

                                  PfIP Reputation2
                                  minimize
                                  close
                                  (All Downloads Successful)

                                  List Count
                                  Alias         CIDRs         Packets Updated Status
                                  IR_IB         60359        0         Jul 29 22:42
                                  IR_PRI1 2418         0         Jul 29 20:42
                                  IR_PRI2 24652 40         Jul 30 15:52

                                  1 Reply Last reply Reply Quote 0
                                  • W
                                    wcrowder
                                    last edited by

                                    I want to think jflsakfja and BBcan177 for this thread. Fantastic work.

                                    1 Reply Last reply Reply Quote 0
                                    • ?
                                      A Former User
                                      last edited by

                                      @wcrowder:

                                      I want to think jflsakfja and BBcan177 for this thread. Fantastic work.

                                      As long as you are not thinking of us while taking a bath  ;)

                                      1 Reply Last reply Reply Quote 0
                                      • M
                                        Mr. Jingles
                                        last edited by

                                        @jflsakfja:

                                        @wcrowder:

                                        I want to think jflsakfja and BBcan177 for this thread. Fantastic work.

                                        As long as you are not thinking of us while taking a bath  ;)

                                        ;D ;D ;D

                                        6 and a half billion people know that they are stupid, agressive, lower life forms.

                                        1 Reply Last reply Reply Quote 0
                                        • M
                                          Mr. Jingles
                                          last edited by

                                          JFL, could I ask: what is your opinion on this:

                                          Why block an attacker that's already been blocked by your firewall? Because they will continue to beat on your door till they find something they can brute force or exploit. So lets use their initial scanning to identify hostile intent and block.

                                          http://doc.emergingthreats.net/bin/view/Main/WhatEveryIDSUserShouldDo

                                          It would make sense to this noob  ;D

                                          6 and a half billion people know that they are stupid, agressive, lower life forms.

                                          1 Reply Last reply Reply Quote 0
                                          • ?
                                            A Former User
                                            last edited by

                                            @Hollander:

                                            JFL, could I ask: what is your opinion on this:

                                            Why block an attacker that's already been blocked by your firewall? Because they will continue to beat on your door till they find something they can brute force or exploit. So lets use their initial scanning to identify hostile intent and block.

                                            http://doc.emergingthreats.net/bin/view/Main/WhatEveryIDSUserShouldDo

                                            It would make sense to this noob  ;D

                                            Nice to see that most things in this topic are also mirrored in that page. Too closely mirrored, if you catch my drift ;). Now if they had the same spirit about deleting their outdated rules, then we would all be happy ;D

                                            And yes, that's what the recommended unused ports rule (in this topic and that page) does. It identifies people that are port scanning you, but maybe managed to get a list of open ports before they were banned. Cutting off their access before them noticing that you have open ports and try to connect to those, is a sane way of preventing them from getting onto your network. If you know that they are malicious, why even let them in the network in the first place?

                                            My recommended rule was detecting ALL traffic to unused ports. His rule detects traffic that is attemping to be established. What if the "attacker" decides to scan in an unconventional way? You should never see traffic to those ports, end of story. If even a single packet arrives at those ports, then it is malicious, since you did not ask for it.

                                            Didn't have time to read the entire page, but from quickly going through it it looks like he is recommending a similar set as to what is described in this topic. Suricata on pfsense belongs in the network gateway category, and as such should be responsible for protecting the network proactively if it can, or reactively if it cannot (encrypted connections terminated to internal servers for example). From suricata's/pfsense's point of view, it should try to make the network conform to specific requirements, like the requirement to contact specific DNS servers.

                                            What I will comment on though is the section "Systems That Should Never Surf the Web". This is the typical wrong approach to using an IDS/IPS system. If you don't want those systems to surf the web, use what the firewall is designed to do: Be a glorrified router that routes according to specific rules, and block those hosts using a solution as close to the network as possible. If I were responsible for systems that shouldn't surf the web, then I would just air-gap them instead. It's the closest to the network you can be, there isn't actually a network (connecting them to the Internet), a separate physical intranet is acceptable, since not everybody has ninjas sworn in blood oaths to get into their networks.

                                            But shouldn't the firewall be used for unused ports as well? From the inside of the network, absolutely yes. From the outside of the network you need the sentry (suricata) as high on the walls as possible. Bonus points if you can give him binoculars as well. The more he can see, the faster he will alert the rest of the town (pfsense) that danger is coming. The faster the town is warned, the faster the gates will be locked (host added to banned hosts). Ask yourself, what use is the sentry on top of the wall 1km away, monitoring the market stall thieves in the center of the town? Why the medieval thinking? If you haven't yet read the Art of War, you don't belong in the network security field.

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.