Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Suricata false postives?

    IDS/IPS
    3
    21
    7.0k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • bmeeksB
      bmeeks
      last edited by

      @firewalluser:

      I spent about 20mins looking at these rules and reading around to get an understanding of what data these rules contain to work.

      Would you say this problem is simply a syntax translation/find & replace issue, ie snort using dsize, but suricata cant use dsize as an example?

      Yes, the problem is Snort recognizes certain rule option keywords that Suricata does not.  Also, there are some options Snort can do that Suricata can't and vice-versa.  If you look in the Suricata log file, it will show a final tally of how many VRT rules were ignored.  It won't specifically identify them as VRT, but just as rule errors.  I can tell you, though, that only Snort VRT rules will generate those errors.  The Emerging Threats rules that Suricata downloads are specifically written/tailored for Suricata.

      Simply doing a find/replace to remove one keyword and insert another is not a global fix.  As I mentioned, there are some things Snort does that Suricata cannot in terms of using those keywords.  And without the keywords/options, the rules would likely not perform their intended task anyway.

      So the short answer is the preferred configuration is to use just Emerging Threats rules with Suricata because the package downloads the Suricata-optimized version of that rules package.  Otherwise, just understand that around 800 Snort VRT rules are incompatible with Suricata and won't load and thus can't generate alerts.  I'm not saying the VRT rules can't be used at all with Suricata, just that ALL of them can't be used with Suricata.

      Bill

      1 Reply Last reply Reply Quote 0
      • F
        firewalluser
        last edited by

        I was thinking how difficult would it be to parse the snort rules into a format suricata can use. Whilst I appreciate some things suricata does is not present in snort, if the snort rules could be made suricata friendly by parsing them properly in an automated fashion, then it would be a step closer, besides with virus total and other online sources I also wondered how hard it would be to draw in the various signatures to build a rule database of my own, copyrights permitting from the various online sources.

        It wouldnt take long to build a windows app probably less than a day knock something up to pulled down data from known sources, then parse this sort of stuff into a database and then rehash it to various formats, and then it could run as a service on a windows box just doing that job keeping itself upto date before posting the output online somewhere for it to be useful for others as well as myself.

        Capitalism, currently The World's best Entertainment Control System and YOU cant buy it! But you can buy this, or some of this or some of these

        Asch Conformity, mainly the blind leading the blind.

        1 Reply Last reply Reply Quote 0
        • F
          firewalluser
          last edited by

          @bmeeks:

          As @doktornotor stated, there are (at last count according to my feeble memory) nearly 800 Snort VRT rules that use syntax incompatible with the current Suricata rules engine.  There are requests over on the Suricata Redmine site to add the necessary syntax support in later Suricata versions.

          If there are around 800 Snort VRT rules that dont work, why doesnt the system log show error 800 ERRCODE: SC_ERR_INVALID_SIGNATURE(39) errors in the system log, or a total of around 800 errors regarding problems with suricata?

          Do you have a link to the redmine issue regarding this as I've been through all 9 pages of issues at this link
          https://redmine.openinfosecfoundation.org/projects/suricata/issues?page=9&set_filter=1&tracker_id=1

          searching for "snort" or "snort vrt" and I cant find any open issues, plenty of feature requests and closed requests but no open issues using the word "snort"

          I have found 6 issues refering to "snort vrt" which are closed and these appear to have been closed over 5 years ago.
          https://redmine.openinfosecfoundation.org/issues/220
          https://redmine.openinfosecfoundation.org/issues/204
          https://redmine.openinfosecfoundation.org/issues/219
          https://redmine.openinfosecfoundation.org/issues/188
          https://redmine.openinfosecfoundation.org/issues/97
          https://redmine.openinfosecfoundation.org/issues/96

          So if you can provide me with a link to the open issue in suricata's redmine that would be useful, thanks.

          Capitalism, currently The World's best Entertainment Control System and YOU cant buy it! But you can buy this, or some of this or some of these

          Asch Conformity, mainly the blind leading the blind.

          1 Reply Last reply Reply Quote 0
          • D
            doktornotor Banned
            last edited by

            Sigh. Please, read the above explanation again. This is NOT a package issue, this is NOT pfSense issue either. It's simply a Suricata limitation with using VRT rules. There is no such thing like 1:1 compatibility with Snort rules, so incompatible rules are simply skipped.

            (And, WHY THE HECK would you want to flood syslog with 800 warnings?!? Are you mad? You have that noise in suricata logs if you are interested.)

            1 Reply Last reply Reply Quote 0
            • F
              firewalluser
              last edited by

              @bmeeks:

              As @doktornotor stated, there are (at last count according to my feeble memory) nearly 800 Snort VRT rules that use syntax incompatible with the current Suricata rules engine.  There are requests over on the Suricata Redmine site to add the necessary syntax support in later Suricata versions. Once that support is incorporated into Suricata upstream and then the FreeBSD port is also updated, it will be available in pfSense.

              Until then, all users of Suricata should note that a significant number of Snort VRT rules will generate syntax errors and not be loaded or used by Suricata.  Suricata will print error messages (as posted by the OP) that show which specific rules failed to load.

              Bill

              I'm just trying to find the link, as Bill mentioned it, I did NOT say it was a package issue, I did NOT say it was a pfsense issue.

              I'm trying to find out why I'm not seeing a significant number of syntax errors in the system log when the option is checked to send the messages to the system log and in the syslog. It would be wise to find out what works and what doesnt wouldnt you say?

              As to flooding the syslog server, if someone is using a syslog server than cant handle a flood of messages, they need to change their setup, besides it would be useful to know what in the VRT works and what doesnt, which is all I'm trying to find out.

              Capitalism, currently The World's best Entertainment Control System and YOU cant buy it! But you can buy this, or some of this or some of these

              Asch Conformity, mainly the blind leading the blind.

              1 Reply Last reply Reply Quote 0
              • D
                doktornotor Banned
                last edited by

                As to flooding, there's a circular log on pfSense that can certainly serve MUCH better purpose than flooding it with absolutely useless duplicate shit that you are too lazy to find in the proper place.

                1 Reply Last reply Reply Quote 0
                • F
                  firewalluser
                  last edited by

                  Well I seem to be having problems with the logs at the moment as you can see from these posts.
                  https://forum.pfsense.org/index.php?topic=101692.0

                  And as you can see from this post, theres messages appearing in the console but not in the logs.
                  https://forum.pfsense.org/index.php?topic=101408.0

                  So with these problems its probably best to get a copy all messages out of the fw for any post mortem analysis.

                  As it happens I've had to reinstall pfsense from scratch again this morning as reseting the system logs and rebooting the fw didnt sort the problem of the disappearing system logs, along with other oddities.

                  With regard to whether this is absolutely useless duplicate shit, thats a matter of opinion, I guess you cant spot the significance and thus importance of logging data, its how you find anomalies with the system, for when systems get hacked.

                  Capitalism, currently The World's best Entertainment Control System and YOU cant buy it! But you can buy this, or some of this or some of these

                  Asch Conformity, mainly the blind leading the blind.

                  1 Reply Last reply Reply Quote 0
                  • bmeeksB
                    bmeeks
                    last edited by

                    The choice of exactly what is logged where is handled by the Suricata binary. I think rule syntax errors always go to the Suricata log.  You can find it under the LOGS VIEW tab and then selecting the Suricata.log file to view.

                    1 Reply Last reply Reply Quote 0
                    • F
                      firewalluser
                      last edited by

                      I know those SC_ERR_INVALID_SIGNATURE messages appear in Services, Suricata, Logs View Tab, Suricata.log, they are the same ones replicated in the system log when you tick the option, Services, Suricata, Global Settings, General Settings section, Log to System Log.

                      As I cant find the 800 odd snort VRT rules as cited by yourself which dont work with suricata, I've been trying to track this down.

                      I do see info messages saying

                      18374 rules successfully loaded, 13 rules failed

                      but no mention of the 26 messages with SC_ERR_INVALID_SIGNATURE.

                      So you see there are some inconsistency in whats being said and what is being seen on the ground, and I'm trying to establish what is actually going on.

                      Likewise in this thread https://forum.pfsense.org/index.php?topic=101682.0
                      Vane makes the statement Suricata is an Intrusion Detection System (IDS) not an Intrusion Protection System (IPS), you state

                      The design simply triggers off any alert and the source and/or destination IP address is inserted into the snort2c table.  Because the blocking mechanism depends on libpcap and getting copies of packets for inspection by Suricata or Snort, there is no real support for DROP or REJECT.  In other words, there is no inline mode available and thus those rule operations make no sense on pfSense at the moment.

                      Can I also make a suggestion that the description for the Suricata package description is an accurate representation of its current capabilities.

                      Currently the Package manager states

                      High Performance Network IDS, IPS and Security Monitoring engine by OISF

                      which is misleading to pfsense users looking for an IDS where the functionality doesnt exist in PFSense.

                      In the long term its not good for business for pfsense in general to claim its got every bell and whistle under the sun, when much of the functionality is yet to be implemented, or is implemented incorrectly so that holes exist in the firewall leaving many users systems vulnerable.

                      And if you cant provide the link to the request in the Suricata redmine, just say so instead of avoiding the question.

                      You guys have now been called out, lets see how you respond as your business future depends on it, because this is not the only thing wrong with pfsense!

                      Capitalism, currently The World's best Entertainment Control System and YOU cant buy it! But you can buy this, or some of this or some of these

                      Asch Conformity, mainly the blind leading the blind.

                      1 Reply Last reply Reply Quote 0
                      • D
                        doktornotor Banned
                        last edited by

                        Dude - you realize this package is maintained by a volunteer maintainer for free? WTF is that business you are talking about? WTF is wrong with you? The only "business" here for bmeeks is spending tons of his free time on the project.

                        You take absolute non-issue and start attacking the volunteer maintainer who's damn one heck of a good job when maintaining these complex Snort/Suricata monsters for users?

                        You don't like it? You say not good enough? How much did you pay for this? Ah, right - $0. So, if you don't like it and it ain't good enough for you, may I suggest that you move your fucking ass and produce some work to make it better!

                        Incredible. Is your head overheating from too much tinfoil applied?

                        :( >:( >:(

                        1 Reply Last reply Reply Quote 0
                        • F
                          firewalluser
                          last edited by

                          You obviously dont value the time volunteers as you like to call them, spend time testing the aforementioned output by volunteers and then you have the audacity to shoot the volunteer messenger down in flames.

                          Dude, you need to re evaluate yourself.

                          Criticism is constructive feedback, if you hadn't worked it out!

                          Capitalism, currently The World's best Entertainment Control System and YOU cant buy it! But you can buy this, or some of this or some of these

                          Asch Conformity, mainly the blind leading the blind.

                          1 Reply Last reply Reply Quote 0
                          • D
                            doktornotor Banned
                            last edited by

                            That incoherent blurb you produced above has nothing to do with constructive criticism. You've already been told what's the "issue" here, but apparently you are just too dumb to understand it, yet apparently "qualified" enough to advise people about their business. Just piss off, really.

                            1 Reply Last reply Reply Quote 0
                            • F
                              firewalluser
                              last edited by

                              Go back to school if you dont understand English, at least then you might be able to learn something.

                              All I'm asking for is links to the requests in Suricata's Redmine so I can look over these 800 Snort VRT rules which are syntactically incompatible.

                              You see, here https://redmine.openinfosecfoundation.org/projects/suricata/wiki/Known_issues it states known issues, one being

                              Signature language support

                              Suricata supports Snort's signature language. We've focused on Snort version 2.9.x, which is not fully complete yet. ET/ETpro is fully supported, VRT support is steadily improving.

                              But as the wiki has not been updated since 2.0.3 and Suricata is now 2.0.9 stable and beta 2.1.beta4 with all references to "snort vrt" found in redmine being closed, that doesnt tie in with the pfsense Suricata 2.1.7 version shown in the package manager so I'm still trying to find out what these 800 odd incompatible snort vrt rules are, hence the request for a link.

                              Instead of wasting so much time getting defensive and abusive why don't you just answer the question by providing a link?

                              Capitalism, currently The World's best Entertainment Control System and YOU cant buy it! But you can buy this, or some of this or some of these

                              Asch Conformity, mainly the blind leading the blind.

                              1 Reply Last reply Reply Quote 0
                              • D
                                doktornotor Banned
                                last edited by

                                If you want to see those ~800 rules incompatible with Suricata to "look over them", then simply go to the damned GUI, enable all categories and all rules and look at the log. Why should I provide you with some links when you behave like an idiot?!

                                1 Reply Last reply Reply Quote 0
                                • F
                                  firewalluser
                                  last edited by

                                  But that only gives my 70 odd errors in total now which is not the 800 odd originally cited, I'd already looked at that before.

                                  But heres another problem if you think you are so clever, only the suricata.log messages for the last interface appears in the system log. So a user has no way of knowing if a problem exists on any other interface.

                                  Or should I quote here? https://forum.pfsense.org/index.php?topic=101668.msg567130#msg567130

                                  Capitalism, currently The World's best Entertainment Control System and YOU cant buy it! But you can buy this, or some of this or some of these

                                  Asch Conformity, mainly the blind leading the blind.

                                  1 Reply Last reply Reply Quote 0
                                  • D
                                    doktornotor Banned
                                    last edited by

                                    Yeah, so what? How the heck does it matter how many of them are incompatible? They simply are incompatible, noone counts them, except for apparently you because you have no better things to do than harassing maintainers with crap, this ain't any bug but well known Suricata limitation with Snort rules. Move on and perhaps try to produce something useful, like submitting patches upstream to make those rules compatible.

                                    Besides, your testing skills miserably suck, with a short look at the log noise (which you'd like to flood syslog with!!!) shows

                                    
3/11/2015 -- 00:47:02 - <info>-- 3 rule files processed. 15947 rules successfully loaded, 1632 rules failed</info>

                                    At minimum, please stop suggesting that everyone's general syslog should be flooded with crap such as:

                                    
3/11/2015 -- 00:47:02 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $EXTERNAL_NET any -> $SMTP_SERVERS 25 (msg:"FILE-IDENTIFY GIF file magic detected"; flow:to_server,established; file_data; content:"GIF8"; depth:4; fast_pattern; content:"a"; within:1; distance:1; flowbits:set,file.gif; flowbits:noalert; metadata:service smtp; classtype:misc-activity; sid:23647; rev:5;)" from file /usr/pbi/suricata-amd64/etc/suricata/suricata_13310_em0/rules/flowbit-required.rules at line 850
3/11/2015 -- 00:47:02 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - Can't use file_data with flow:to_server or from_client with http.
3/11/2015 -- 00:47:02 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $EXTERNAL_NET any -> $SMTP_SERVERS 25 (msg:"FILE-IDENTIFY RealNetworks Real Media file magic detected"; flow:to_server,established; file_data; content:".RMF"; depth:4; flowbits:set,file.realplayer; flowbits:noalert; metadata:service smtp; classtype:misc-activity; sid:23645; rev:6;)" from file /usr/pbi/suricata-amd64/etc/suricata/suricata_13310_em0/rules/flowbit-required.rules at line 853
3/11/2015 -- 00:47:02 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - Can't use file_data with flow:to_server or from_client with http.
3/11/2015 -- 00:47:02 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $EXTERNAL_NET any -> $SMTP_SERVERS 25 (msg:"FILE-IDENTIFY MPEG sys stream file magic detected"; flow:to_server,established; file_data; content:"|00 00 01 BA|"; depth:4; flowbits:set,file.mpeg; flowbits:noalert; metadata:service smtp; classtype:misc-activity; sid:23640; rev:8;)" from file /usr/pbi/suricata-amd64/etc/suricata/suricata_13310_em0/rules/flowbit-required.rules at line 856
3/11/2015 -- 00:47:02 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - Can't use file_data with flow:to_server or from_client with http.
3/11/2015 -- 00:47:02 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $EXTERNAL_NET any -> $SMTP_SERVERS 25 (msg:"FILE-IDENTIFY MPEG video stream file magic detected"; flow:to_server,established; file_data; content:"|00 00 01 B3|"; depth:4; flowbits:set,file.mpeg; flowbits:noalert; metadata:service smtp; classtype:misc-activity; sid:23639; rev:8;)" from file /usr/pbi/suricata-amd64/etc/suricata/suricata_13310_em0/rules/flowbit-required.rules at line 859
3/11/2015 -- 00:47:02 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - Can't use file_data with flow:to_server or from_client with http.
3/11/2015 -- 00:47:02 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $EXTERNAL_NET any -> $SMTP_SERVERS 25 (msg:"FILE-IDENTIFY Adobe LZMA compressed Flash file magic detected"; flow:to_server,established; file_data; content:"ZWS"; depth:3; flowbits:set,file.swf; flowbits:noalert; metadata:service smtp; classtype:misc-activity; sid:35458; rev:1;)" from file /usr/pbi/suricata-amd64/etc/suricata/suricata_13310_em0/rules/flowbit-required.rules at line 1750
3/11/2015 -- 00:47:02 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - Can't use file_data with flow:to_server or from_client with http.
3/11/2015 -- 00:47:02 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $EXTERNAL_NET any -> $SMTP_SERVERS 25 (msg:"FILE-IDENTIFY M4A file magic detected"; flow:to_server,established; file_data; content:"ftypM4A"; depth:7; offset:4; flowbits:set,file.mp4; flowbits:noalert; metadata:service smtp; classtype:misc-activity; sid:35433; rev:2;)" from file /usr/pbi/suricata-amd64/etc/suricata/suricata_13310_em0/rules/flowbit-required.rules at line 1762
3/11/2015 -- 00:47:02 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - Can't use file_data with flow:to_server or from_client with http.
3/11/2015 -- 00:47:02 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $EXTERNAL_NET any -> $HOME_NET $HTTP_PORTS (msg:"FILE-IDENTIFY JPEG file upload detected"; flow:to_server,established; file_data; content:"|FF D8 FF E1|"; depth:4; flowbits:set,file.jpeg; flowbits:noalert; metadata:service http; classtype:misc-activity; sid:35852; rev:1;)" from file /usr/pbi/suricata-amd64/etc/suricata/suricata_13310_em0/rules/flowbit-required.rules at line 1768
3/11/2015 -- 00:47:02 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - Can't use file_data with flow:to_server or from_client with http.</error></error></error></error></error></error></error></error></error></error></error></error></error></error>

                                    FFS. Ktnxbye.  >:(

                                    1 Reply Last reply Reply Quote 0
                                    • First post
                                      Last post