Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Suricata Not logging signature matches | | Suricata 4.0.13_11 with pfsense 2.4.4-RELEASE-p1 (arm)

    Scheduled Pinned Locked Moved IDS/IPS
    14 Posts 2 Posters 1.5k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • bmeeksB
      bmeeks @s0m3f00l
      last edited by bmeeks

      @s0m3f00l, what rule packages and versions are you using? Do you have Snort rules enabled on the GLOBAL SETTINGS tab? If so, what is the filename you are telling Suricata to download?

      You cannot use Snort3 rules with Suricata. That's one thing that will cause the errors you are seeing. Your references.config file is hosed. That is causing those listed rules in the log to fail. That could have happened if you manually edited it, or if you tried to use the Snort3 rules.

      To look for Signal 10 errors, you must look in the firewall system log under STATUS > SYSTEM LOG.

      1 Reply Last reply Reply Quote 0
      • S
        s0m3f00l
        last edited by

        So here are the rules I have enabled and the version.

        0_1544669696654_Screenshot_2018-12-13 pfSense_Edge fool local - Suricata IDS Interface WAN - Categories.png

        0_1544669790808_Screenshot_2018-12-13 pfSense_Edge fool local - System Package Manager Installed Packages.png

        Not sure if thats what you need but let me know if I can get anything else. This device is production but home use.

        bmeeksB 1 Reply Last reply Reply Quote 0
        • bmeeksB
          bmeeks @s0m3f00l
          last edited by

          @s0m3f00l

          In the suricata.log that you posted in your second reply, how many rules does it say it loaded and how many did it reject? There will be a summary line at the end of the rule loading (past where the ERRCODE: SC_ERR lines are shown).

          You installation appears borked somehow since that very first line in the log indicates it can't open the reference.config file. That file should always be present when the rules and configuration for an interface build successfully. You need to find out what happened to that file. It should be there.

          1 Reply Last reply Reply Quote 1
          • S
            s0m3f00l
            last edited by s0m3f00l

            This post is deleted!
            1 Reply Last reply Reply Quote 0
            • S
              s0m3f00l
              last edited by s0m3f00l

              you're right

              12/12/2018 -- 18:44:58 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert http $HOME_NET any -> $EXTERNAL_NET any (msg:"ET POLICY Vulnerable Java Version 1.5.x Detected"; flow:established,to_server; content:" Java/1.5."; nocase; http_header; flowbits:set,ET.http.javaclient.vulnerable; threshold: type limit, count 2, seconds 300, track by_src; metadata: former_category POLICY; reference:url,www.oracle.com/technetwork/java/javase/documentation/index.html; classtype:bad-unknown; sid:2011581; rev:10; metadata:affected_product Java, attack_target Client_Endpoint, deployment Perimeter, deployment Internal, tag EOL, signature_severity Audit, created_at 2010_09_27, performance_impact Low, updated_at 2018_04_19;)" from file /usr/local/etc/suricata/suricata_17044_mvneta2/rules/flowbit-required.rules at line 208
              12/12/2018 -- 18:44:58 - <Error> -- [ERRCODE: SC_ERR_REFERENCE_UNKNOWN(150)] - unknown reference key "url". Supported keys are defined in reference.config file.  Please have a look at the conf param "reference-config-file"
              12/12/2018 -- 18:44:58 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert http $HOME_NET any -> $EXTERNAL_NET any (msg:"ET POLICY Vulnerable Java Version 1.4.x Detected"; flow:established,to_server; content:"Java/1.4."; http_user_agent; flowbits:set,ET.http.javaclient.vulnerable;  threshold: type limit, count 2, seconds 300, track by_src; metadata: former_category POLICY; reference:url,www.oracle.com/technetwork/java/javase/documentation/index.html; classtype:bad-unknown; sid:2011584; rev:12; metadata:affected_product Java, attack_target Client_Endpoint, deployment Perimeter, deployment Internal, tag EOL, signature_severity Audit, created_at 2010_09_27, performance_impact Low, updated_at 2018_04_19;)" from file /usr/local/etc/suricata/suricata_17044_mvneta2/rules/flowbit-required.rules at line 211
              12/12/2018 -- 18:44:58 - <Error> -- [ERRCODE: SC_ERR_REFERENCE_UNKNOWN(150)] - unknown reference key "url". Supported keys are defined in reference.config file.  Please have a look at the conf param "reference-config-file"
              12/12/2018 -- 18:44:58 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert http $HOME_NET any -> $EXTERNAL_NET any (msg:"ET POLICY Vulnerable Java Version 10.0.x Detected"; flow:established,to_server; content:"Java/10.0."; http_user_agent; content:!"2"; within:1; http_user_agent; flowbits:set,ET.http.javaclient.vulnerable; threshold: type limit, count 2, seconds 300, track by_src; metadata: former_category POLICY; reference:url,www.oracle.com/technetwork/java/javase/10u-relnotes-4108739.html; classtype:bad-unknown; sid:2025518; rev:3; metadata:affected_product Java, attack_target Client_Endpoint, deployment Perimeter, deployment Internal, signature_severity Audit, created_at 2018_04_19, performance_impact Low, updated_at 2018_11_20;)" from file /usr/local/etc/suricata/suricata_17044_mvneta2/rules/flowbit-required.rules at line 214
              12/12/2018 -- 18:44:58 - <Error> -- [ERRCODE: SC_ERR_REFERENCE_UNKNOWN(150)] - unknown reference key "url". Supported keys are defined in reference.config file.  Please have a look at the conf param "reference-config-file"
              12/12/2018 -- 18:44:58 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert http $HOME_NET any -> $EXTERNAL_NET any (msg:"ET POLICY Vulnerable Java Version 9.0.x Detected"; flow:established,to_server; content:"Java/9."; http_user_agent; flowbits:set,ET.http.javaclient.vulnerable; threshold: type limit, count 2, seconds 300, track by_src; metadata: former_category POLICY; reference:url,www.oracle.com/technetwork/java/javase/documentation/9u-relnotes-3704429.html; classtype:bad-unknown; sid:2025314; rev:3; metadata:affected_product Java, attack_target Client_Endpoint, deployment Perimeter, deployment Internal, tag EOL, signature_severity Audit, created_at 2018_02_05, performance_impact Low, updated_at 2018_07_17;)" from file /usr/local/etc/suricata/suricata_17044_mvneta2/rules/flowbit-required.rules at line 217
              12/12/2018 -- 18:44:58 - <Error> -- [ERRCODE: SC_ERR_REFERENCE_UNKNOWN(150)] - unknown reference key "url". Supported keys are defined in reference.config file.  Please have a look at the conf param "reference-config-file"
              12/12/2018 -- 18:44:58 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert http $HOME_NET any -> $EXTERNAL_NET any (msg:"ET POLICY Vulnerable Java Version 1.8.x Detected"; flow:established,to_server; content:" Java/1.8.0_"; http_user_agent; content:!"191"; within:3; http_user_agent; content:!"192"; within:3; http_user_agent; flowbits:set,ET.http.javaclient.vulnerable; threshold: type limit, count 2, seconds 300, track by_src; metadata: former_category POLICY; reference:url,www.oracle.com/technetwork/java/javase/8u-relnotes-2225394.html; classtype:bad-unknown; sid:2019401; rev:25; metadata:affected_product Java, attack_target Client_Endpoint, deployment Perimeter, deployment Internal, signature_severity Audit, created_at 2014_10_15, performance_impact Low, updated_at 2018_07_17;)" from file /usr/local/etc/suricata/suricata_17044_mvneta2/rules/flowbit-required.rules at line 220
              12/12/2018 -- 18:44:58 - <Error> -- [ERRCODE: SC_ERR_REFERENCE_UNKNOWN(150)] - unknown reference key "url". Supported keys are defined in reference.config file.  Please have a look at the conf param "reference-config-file"
              12/12/2018 -- 18:44:58 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert http $HOME_NET any -> $EXTERNAL_NET any (msg:"ET POLICY Vulnerable Java Version 1.7.x Detected"; flow:established,to_server; content:"Java/1.7.0_"; http_user_agent; content:!"201"; within:3; http_user_agent; flowbits:set,ET.http.javaclient.vulnerable; threshold: type limit, count 2, seconds 300, track by_src; metadata: former_category POLICY; reference:url,www.oracle.com/technetwork/java/javase/documentation/javase7supportreleasenotes-1601161.html; classtype:bad-unknown; sid:2014297; rev:53; metadata:affected_product Java, attack_target Client_Endpoint, deployment Perimeter, deployment Internal, signature_severity Audit, created_at 2012_03_01, performance_impact Low, updated_at 2018_07_17;)" from file /usr/local/etc/suricata/suricata_17044_mvneta2/rules/flowbit-required.rules at line 223
              12/12/2018 -- 18:44:58 - <Error> -- [ERRCODE: SC_ERR_REFERENCE_UNKNOWN(150)] - unknown reference key "url". Supported keys are defined in reference.config file.  Please have a look at the conf param "reference-config-file"
              12/12/2018 -- 18:44:58 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert http $HOME_NET any -> $EXTERNAL_NET any (msg:"ET POLICY Vulnerable Java Version 1.6.x Detected"; flow:established,to_server; content:"Java/1.6.0_"; http_user_agent; content:!"211"; within:3; http_user_agent; flowbits:set,ET.http.javaclient.vulnerable; threshold: type limit, count 2, seconds 300, track by_src; metadata: former_category POLICY; reference:url,www.oracle.com/technetwork/articles/javase/overview-156328.html; classtype:bad-unknown; sid:2011582; rev:54; metadata:affected_product Java, attack_target Client_Endpoint, deployment Perimeter, deployment Internal, signature_severity Audit, created_at 2010_09_27, performance_impact Low, updated_at 2018_07_17;)" from file /usr/local/etc/suricata/suricata_17044_mvneta2/rules/flowbit-required.rules at line 226
              12/12/2018 -- 18:44:58 - <Info> -- 2 rule files processed. 4194 rules successfully loaded, 14028 rules failed
              12/12/2018 -- 18:44:58 - <Warning> -- [ERRCODE: SC_ERR_EVENT_ENGINE(210)] - can't suppress sid 2018959, gid 1: unknown rule
              12/12/2018 -- 18:44:58 - <Info> -- Threshold config parsed: 1 rule(s) found
              12/12/2018 -- 18:44:58 - <Info> -- 4195 signatures processed. 0 are IP-only rules, 1157 are inspecting packet payload, 3118 inspect application layer, 103 are decoder event only
              12/12/2018 -- 18:44:59 - <Info> -- alert-pf -> Creating automatic firewall interface IP address Pass List.
              12/12/2018 -- 18:44:59 - <Info> -- alert-pf -> adding firewall interface mvneta1 IPv6 address fe80:0000:0000:0000:0208:a2ff:fe0d:f1dc to automatic interface IP Pass List.
              12/12/2018 -- 18:44:59 - <Info> -- alert-pf -> adding firewall interface mvneta1 IPv4 address 172.16.0.240 to automatic interface IP Pass List.
              12/12/2018 -- 18:44:59 - <Info> -- alert-pf -> adding firewall interface mvneta1 IPv4 address 10.10.10.1 to automatic interface IP Pass List.
              12/12/2018 -- 18:44:59 - <Info> -- alert-pf -> adding firewall interface mvneta2 IPv6 address o automatic interface IP Pass List.
              12/12/2018 -- 18:44:59 - <Info> -- alert-pf -> adding firewall interface mvneta2 IPv4 address  to automatic interface IP Pass List.
              12/12/2018 -- 18:44:59 - <Info> -- alert-pf -> adding firewall interface lo0 IPv6 address 0000:0000:0000:0000:0000:0000:0000:0001 to automatic interface IP Pass List.
              12/12/2018 -- 18:44:59 - <Info> -- alert-pf -> adding firewall interface lo0 IPv6 address fe80:0000:0000:0000:0000:0000:0000:0001 to automatic interface IP Pass List.
              12/12/2018 -- 18:44:59 - <Info> -- alert-pf -> adding firewall interface lo0 IPv4 address 127.0.0.1 to automatic interface IP Pass List.
              12/12/2018 -- 18:44:59 - <Info> -- alert-pf output device (regular) initialized: block.log
              12/12/2018 -- 18:44:59 - <Info> -- alert-pf -> Pass List /usr/local/etc/suricata/suricata_17044_mvneta2/passlist parsed: 12 IP addresses loaded.
              12/12/2018 -- 18:44:59 - <Info> -- alert-pf -> Created firewall interface IP change monitor thread for auto-whitelisting of firewall interface IP addresses.
              12/12/2018 -- 18:44:59 - <Info> -- alert-pf output initialized, pf-table=snort2c  block-ip=both  kill-state=on  block-drops-only=off
              12/12/2018 -- 18:44:59 - <Info> -- fast output device (regular) initialized: alerts.log
              12/12/2018 -- 18:44:59 - <Info> -- http-log output device (regular) initialized: http.log
              12/12/2018 -- 18:44:59 - <Info> -- stats output device (regular) initialized: stats.log
              12/12/2018 -- 18:44:59 - <Info> -- dns-log output device (regular) initialized: dns.log
              12/12/2018 -- 18:44:59 - <Info> -- dns-log output device (regular) initialized: dns.log
              12/12/2018 -- 18:44:59 - <Info> -- Using 1 live device(s).
              12/12/2018 -- 18:44:59 - <Info> -- alert-pf -> Firewall interface IP address change notification monitoring thread started.
              12/12/2018 -- 18:44:59 - <Info> -- using interface mvneta2
              12/12/2018 -- 18:44:59 - <Info> -- Running in 'auto' checksum mode. Detection of interface state will require 1000 packets.
              12/12/2018 -- 18:44:59 - <Info> -- Set snaplen to 1518 for 'mvneta2'
              12/12/2018 -- 18:45:00 - <Info> -- RunModeIdsPcapAutoFp initialised
              12/12/2018 -- 18:45:00 - <Notice> -- all 3 packet processing threads, 4 management threads initialized, engine started.
              12/12/2018 -- 18:45:07 - <Info> -- No packets with invalid checksum, assuming checksum offloading is NOT used
              

              I recreated the config file :

              [2.4.4-RELEASE][admin@pfSense.local]/usr/local/etc/suricata: cp reference.config.sample reference.config
              

              And Now it looks like 17 didn't load

              13/12/2018 -- 06:27:07 - <Notice> -- This is Suricata version 4.0.6 RELEASE
              13/12/2018 -- 06:27:07 - <Info> -- CPUs/cores online: 2
              13/12/2018 -- 06:27:07 - <Info> -- HTTP memcap: 67108864
              13/12/2018 -- 06:27:07 - <Notice> -- using flow hash instead of active packets
              13/12/2018 -- 06:27:08 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - Signature combines packet specific matches (like dsize, flags, ttl) with stream / state matching by matching on app layer proto (like using http_* keywords).
              13/12/2018 -- 06:27:08 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $HOME_NET any -> $EXTERNAL_NET 1024:65535 (msg:"MALWARE-CNC Win.Trojan.Fakeavlock variant outbound connection"; flow:to_server,established; dsize:267<>276; content:"User-Agent|3A| Mozilla/5.0 (Windows|3B| U|3B| MSIE 9.0|3B| Windows NT 9.0|3B| en-US)|0D 0A|"; fast_pattern:only; http_header; urilen:159; pcre:"/\x2f[A-F0-9]{158}/U"; metadata:impact_flag red, policy balanced-ips drop, policy security-ips drop, ruleset community, service http; reference:url,www.virustotal.com/file/c49f7dbc036ad0a86df02cbbde00cb3b3fbd651d82f6c9c5a98170644374f64f/analysis/; classtype:trojan-activity; sid:25675; rev:7;)" from file /usr/local/etc/suricata/suricata_17044_mvneta2/rules/suricata.rules at line 186
              13/12/2018 -- 06:27:08 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - previous keyword has a fast_pattern:only; set. Can't have relative keywords around a fast_pattern only content
              13/12/2018 -- 06:27:08 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $EXTERNAL_NET $HTTP_PORTS -> $HOME_NET any (msg:"MALWARE-OTHER Win.Trojan.Zeus Spam 2013 dated zip/exe HTTP Response - potential malware download"; flow:to_client,established; content:"-2013.zip|0D 0A|"; fast_pattern:only; content:"-2013.zip|0D 0A|"; http_header; content:"-"; within:1; distance:-14; http_header; file_data; content:"-2013.exe"; content:"-"; within:1; distance:-14; metadata:impact_flag red, policy balanced-ips drop, policy security-ips drop, ruleset community, service http; reference:url,www.virustotal.com/en/file/2eff3ee6ac7f5bf85e4ebcbe51974d0708cef666581ef1385c628233614b22c0/analysis/; classtype:trojan-activity; sid:26470; rev:1;)" from file /usr/local/etc/suricata/suricata_17044_mvneta2/rules/suricata.rules at line 210
              13/12/2018 -- 06:27:08 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - previous keyword has a fast_pattern:only; set. Can't have relative keywords around a fast_pattern only content
              13/12/2018 -- 06:27:08 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"MALWARE-CNC Bancos fake JPG encrypted config file download"; flow:to_server,established; content:".com.br|0D 0A 0D 0A|"; fast_pattern:only; content:"/imagens/"; depth:9; http_uri; content:".jpg"; distance:0; http_uri; pcre:"/\.jpg\x20HTTP\/1\.[01]\r\nUser\x2dAgent\x3a\x20[a-z]+\r\nHost\x3a\x20[a-z0-9\x2d\x2e]+\.com\.br\r\n\r\n$/"; metadata:impact_flag red, policy balanced-ips drop, policy security-ips drop, ruleset community, service http; classtype:trojan-activity; sid:26722; rev:1;)" from file /usr/local/etc/suricata/suricata_17044_mvneta2/rules/suricata.rules at line 235
              13/12/2018 -- 06:27:08 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - Signature combines packet specific matches (like dsize, flags, ttl) with stream / state matching by matching on app layer proto (like using http_* keywords).
              13/12/2018 -- 06:27:08 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"MALWARE-CNC Win32/Autorun.JN variant outbound connection"; flow:to_server,established; dsize:142; urilen:8; content:"/u5.htm"; fast_pattern:only; http_uri; content:"//u5.htm"; http_raw_uri; metadata:impact_flag red, policy balanced-ips drop, policy security-ips drop, ruleset community, service http; reference:url,www.microsoft.com/security/portal/threat/encyclopedia/Entry.aspx?Name=Worm%3AWin32%2FAutorun.JN; reference:url,www.virustotal.com/en/file/36144738373c665d262bc007fceaeb9613e59ec29ea3d7424dd9f400af2c0f06/analysis/; classtype:trojan-activity; sid:26966; rev:3;)" from file /usr/local/etc/suricata/suricata_17044_mvneta2/rules/suricata.rules at line 280
              13/12/2018 -- 06:27:08 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - Signature combines packet specific matches (like dsize, flags, ttl) with stream / state matching by matching on app layer proto (like using http_* keywords).
              13/12/2018 -- 06:27:08 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $HOME_NET any -> $EXTERNAL_NET 80 (msg:"MALWARE-CNC Win.Trojan.Conficker variant outbound connection"; flow:to_server,established; dsize:146; urilen:1; content:"GET / HTTP/1.1|0D 0A|User-Agent: Mozilla/4.0 (compatible|3B| MSIE 7.0|3B| Windows NT 5.1|3B| Trident/4.0)|0D 0A|Host: checkip.dyndns.org|0D 0A|Cache-Control: no-cache|0D 0A 0D 0A|"; fast_pattern:only; metadata:impact_flag red, policy balanced-ips drop, policy security-ips drop, ruleset community, service http; reference:url,www.sans.org/security-resources/malwarefaq/conficker-worm.php; classtype:trojan-activity; sid:28542; rev:1;)" from file /usr/local/etc/suricata/suricata_17044_mvneta2/rules/suricata.rules at line 357
              13/12/2018 -- 06:27:08 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - Signature combines packet specific matches (like dsize, flags, ttl) with stream / state matching by matching on app layer proto (like using http_* keywords).
              13/12/2018 -- 06:27:08 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $HOME_NET any -> $EXTERNAL_NET 80 (msg:"MALWARE-CNC Win.Trojan.Conficker variant outbound connection"; flow:to_server,established; dsize:139; urilen:1; content:"GET / HTTP/1.1|0D 0A|User-Agent: Mozilla/4.0 (compatible|3B| MSIE 7.0|3B| Windows NT 5.1|3B| Trident/4.0)|0D 0A|Host: www.ask.com|0D 0A|Cache-Control: no-cache|0D 0A 0D 0A|"; fast_pattern:only; metadata:impact_flag red, policy balanced-ips drop, policy security-ips drop, ruleset community, service http; reference:url,www.sans.org/security-resources/malwarefaq/conficker-worm.php; classtype:trojan-activity; sid:28543; rev:1;)" from file /usr/local/etc/suricata/suricata_17044_mvneta2/rules/suricata.rules at line 358
              13/12/2018 -- 06:27:08 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - previous keyword has a fast_pattern:only; set. Can't have relative keywords around a fast_pattern only content
              13/12/2018 -- 06:27:08 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"MALWARE-CNC Win.Trojan.Injector variant outbound connection"; flow:to_server,established; urilen:9; content:"/load.exe HTTP/1.1|0D 0A|User-Agent: Mozilla/"; fast_pattern:only; content:"|3B 20|MSIE|20|"; http_header; content:")|0D 0A|Host: "; distance:0; http_header; content:!"Accept"; http_header; metadata:impact_flag red, policy balanced-ips drop, policy security-ips drop, ruleset community, service http; reference:url,urlquery.net/search.php?q=%5C%2Fload%5C.exe%24&type=regexp&start=2013-08-24&end=2013-11-22&max=400; reference:url,www.virustotal.com/en/file/032572ea1f34a060ecac98a8e2899dc0f2a41dff199e879050481ddd3818b4d0/analysis/; classtype:trojan-activity; sid:28807; rev:2;)" from file /usr/local/etc/suricata/suricata_17044_mvneta2/rules/suricata.rules at line 366
              13/12/2018 -- 06:27:08 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - Signature combines packet specific matches (like dsize, flags, ttl) with stream / state matching by matching on app layer proto (like using http_* keywords).
              13/12/2018 -- 06:27:08 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"MALWARE-CNC Win.Trojan.WEC variant outbound connection"; flow:to_server,established; dsize:69; urilen:1; content:"GET / HTTP/1.1|0D 0A|User-Agent: Mozilla/4.0|0D 0A|Host: checkip.dyndns.org|0D 0A 0D 0A|"; fast_pattern:only; metadata:impact_flag red, policy balanced-ips drop, policy security-ips drop, ruleset community, service http; reference:url,www.virustotal.com/en/file/164c792247b2822ab1dce8271a9498d3c9172ff21d36feccf83265ded1be8d0b/analysis/; classtype:trojan-activity; sid:29882; rev:2;)" from file /usr/local/etc/suricata/suricata_17044_mvneta2/rules/suricata.rules at line 421
              13/12/2018 -- 06:27:08 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - previous keyword has a fast_pattern:only; set. Can't have relative keywords around a fast_pattern only content
              13/12/2018 -- 06:27:08 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"MALWARE-CNC Win.Trojan.Bancos variant outbound connection"; flow:to_server,established; content:"Content-Length: 166"; content:".php HTTP/1.1|0D 0A|Accept: */*|0D 0A|Content-Type: application/x-www-form-urlencoded|0D 0A|User-Agent: Mozilla/5.0 (Windows NT 6.1|3B| Trident/7.0|3B| rv:11.0) like Gecko|0D 0A|Host: "; fast_pattern:only; content:"v="; depth:2; http_client_body; content:"&c="; within:7; http_client_body; pcre:"/\x3d\x3d$/P"; metadata:impact_flag red, policy balanced-ips drop, policy security-ips drop, ruleset community, service http; reference:url,www.virustotal.com/en/file/51540d7c9a4bc2a430bc50c85cf9cec5c6f2bb755e800a3f3575ba34fe5f008c/analysis; classtype:trojan-activity; sid:29895; rev:2;)" from file /usr/local/etc/suricata/suricata_17044_mvneta2/rules/suricata.rules at line 424
              13/12/2018 -- 06:27:08 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - "http_header" keyword seen with a sticky buffer still set.  Reset sticky buffer with pkt_data before using the modifier.
              13/12/2018 -- 06:27:08 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $EXTERNAL_NET $HTTP_PORTS -> $HOME_NET any (msg:"MALWARE-CNC Win.Trojan.Sodebral HTTP Response attempt"; flow:to_client,established; file_data; dsize:<194; content:"INTERNACIONAL"; depth:13; content:!"Content-Length"; http_header; content:"Transfer-Encoding: chunked"; http_header; metadata:impact_flag red, policy balanced-ips drop, policy security-ips drop, ruleset community, service http; reference:url,www.virustotal.com/en/file/e0290c3900445dc00ca24888924e37fa6ac17ecaddc60591e32b81536b9f5ef7/analysis/; classtype:trojan-activity; sid:32607; rev:1;)" from file /usr/local/etc/suricata/suricata_17044_mvneta2/rules/suricata.rules at line 583
              13/12/2018 -- 06:27:08 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - "http_header" keyword seen with a sticky buffer still set.  Reset sticky buffer with pkt_data before using the modifier.
              13/12/2018 -- 06:27:08 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $EXTERNAL_NET $HTTP_PORTS -> $HOME_NET any (msg:"MALWARE-CNC Win.Trojan.Sodebral HTTP Response attempt"; flow:to_client,established; file_data; dsize:<194; content:"BRASIL"; depth:6; content:!"Content-Length"; http_header; content:"Transfer-Encoding: chunked"; http_header; metadata:impact_flag red, policy balanced-ips drop, policy security-ips drop, ruleset community, service http; reference:url,www.virustotal.com/en/file/e0290c3900445dc00ca24888924e37fa6ac17ecaddc60591e32b81536b9f5ef7/analysis/; classtype:trojan-activity; sid:32608; rev:1;)" from file /usr/local/etc/suricata/suricata_17044_mvneta2/rules/suricata.rules at line 584
              13/12/2018 -- 06:27:08 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - Signature combines packet specific matches (like dsize, flags, ttl) with stream / state matching by matching on app layer proto (like using http_* keywords).
              13/12/2018 -- 06:27:08 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"MALWARE-CNC Win.Agent.BHHK variant outbound connection"; flow:to_server,established; dsize:136; urilen:1; content:"GET / HTTP/1.1|0D 0A|User-Agent: Mozilla/4.0 (compatible|3B| MSIE 7.0|3B| Windows NT 6.0)|0D 0A|Host: windowsupdate.microsoft.com|0D 0A|Connection: Close|0D 0A 0D 0A|"; fast_pattern:only; content:!"Accept"; http_header; metadata:impact_flag red, policy balanced-ips drop, policy security-ips drop, ruleset community, service http; reference:url,www.virustotal.com/en/file/cab1fffe7a34b5bb7dab2cacd406cf15628d835ab63502d28df78c2faeaad366/analysis/1421677054/; classtype:trojan-activity; sid:33227; rev:2;)" from file /usr/local/etc/suricata/suricata_17044_mvneta2/rules/suricata.rules at line 615
              13/12/2018 -- 06:27:08 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - Signature combines packet specific matches (like dsize, flags, ttl) with stream / state matching by matching on app layer proto (like using http_* keywords).
              13/12/2018 -- 06:27:08 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"MALWARE-CNC Win.Trojan.FileEncoder IP geolocation checkin attempt"; flow:to_server,established; dsize:214; urilen:1; content:"GET / HTTP/1.1|0D 0A|User-Agent: Mozilla/4.0 (compatible|3B| MSIE 6.0|3B| Windows NT 5.1|3B| SV1|3B| .NET4.0C|3B| .NET4.0E|3B| .NET CLR 2.0.50727|3B| .NET CLR 3.0.4506.2152|3B| .NET CLR 3.5.30729)|0D 0A|Host: ip-addr.es|0D 0A|Cache-Control: no-cache|0D 0A 0D 0A|"; fast_pattern:only; metadata:impact_flag red, policy balanced-ips drop, policy security-ips drop, ruleset community, service http; reference:url,www.virustotal.com/en/file/17edf82c40df6c7268191def7cbff6e60e78d7388018408800d42581567f78cf/analysis/; classtype:trojan-activity; sid:33449; rev:1;)" from file /usr/local/etc/suricata/suricata_17044_mvneta2/rules/suricata.rules at line 620
              13/12/2018 -- 06:27:08 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - previous keyword has a fast_pattern:only; set. Can't have relative keywords around a fast_pattern only content
              13/12/2018 -- 06:27:08 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"MALWARE-CNC Win.Trojan.GateKeylogger initial exfiltration attempt"; flow:to_server,established; content:"/gate.php"; fast_pattern:only; content:"pc="; nocase; http_client_body; content:"&admin="; distance:0; nocase; http_client_body; content:"&os="; distance:0; nocase; http_client_body; content:"&hid="; distance:0; nocase; http_client_body; content:"&arc="; distance:0; nocase; http_client_body; content:"User-Agent|3A 20|"; http_header; pcre:"/User-Agent\x3a\x20[A-F0-9]{32}\x0d\x0a/H"; metadata:impact_flag red, policy balanced-ips drop, policy security-ips drop, ruleset community, service http; reference:url,www.virustotal.com/en/file/77c802db1731fa8dae1b03d978f89b046309adfa1237b1497a69ccb9c2d82c16/analysis/1459520578/; classtype:trojan-activity; sid:38562; rev:2;)" from file /usr/local/etc/suricata/suricata_17044_mvneta2/rules/suricata.rules at line 742
              13/12/2018 -- 06:27:08 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - "http_stat_code" keyword seen with a sticky buffer still set.  Reset sticky buffer with pkt_data before using the modifier.
              13/12/2018 -- 06:27:08 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $EXTERNAL_NET $HTTP_PORTS -> $HOME_NET any (msg:"MALWARE-CNC Win.Trojan.GateKeylogger fake 404 response"; flow:to_client,established; file_data; content:"200"; http_stat_code; content:"OK"; http_stat_msg; content:">404 Not Found<"; fast_pattern:only; content:" requested URL / was not found "; metadata:impact_flag red, policy balanced-ips drop, policy security-ips drop, ruleset community, service http; reference:url,www.virustotal.com/en/file/77c802db1731fa8dae1b03d978f89b046309adfa1237b1497a69ccb9c2d82c16/analysis/1459520578/; classtype:trojan-activity; sid:38563; rev:1;)" from file /usr/local/etc/suricata/suricata_17044_mvneta2/rules/suricata.rules at line 743
              13/12/2018 -- 06:27:09 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - previous keyword has a fast_pattern:only; set. Can't have relative keywords around a fast_pattern only content
              13/12/2018 -- 06:27:09 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"MALWARE-CNC Osx.Trojan.OceanLotus outbound connection attempt"; flow:to_server,established; content:"/sigstore.db?"; fast_pattern:only; content:"k="; http_uri; content:"?q="; distance:0; http_uri; metadata:impact_flag red, policy balanced-ips drop, policy security-ips drop, ruleset community, service http; reference:url,www.alienvault.com/blogs/labs-research/oceanlotus-for-os-x-an-application-bundle-pretending-to-be-an-adobe-flash-update; classtype:trojan-activity; sid:45400; rev:1;)" from file /usr/local/etc/suricata/suricata_17044_mvneta2/rules/suricata.rules at line 902
              13/12/2018 -- 06:27:13 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert http" from file /usr/local/etc/suricata/suricata_17044_mvneta2/rules/suricata.rules at line 3914
              13/12/2018 -- 06:27:14 - <Info> -- 2 rule files processed. 3951 rules successfully loaded, 17 rules failed
              13/12/2018 -- 06:27:14 - <Warning> -- [ERRCODE: SC_ERR_EVENT_ENGINE(210)] - can't suppress sid 2018959, gid 1: unknown rule
              13/12/2018 -- 06:27:14 - <Info> -- Threshold config parsed: 1 rule(s) found
              13/12/2018 -- 06:27:14 - <Info> -- 3953 signatures processed. 330 are IP-only rules, 1019 are inspecting packet payload, 2789 inspect application layer, 103 are decoder event only
              13/12/2018 -- 06:27:15 - <Info> -- alert-pf -> Creating automatic firewall interface IP address Pass List.
              13/12/2018 -- 06:27:15 - <Info> -- alert-pf -> adding firewall interface mvneta1 IPv6 address fe80:0000:0000:0000:0208:a2ff:fe0d:f1dc to automatic interface IP Pass List.
              13/12/2018 -- 06:27:15 - <Info> -- alert-pf -> adding firewall interface mvneta1 IPv4 address 172.16.0.240 to automatic interface IP Pass List.
              13/12/2018 -- 06:27:15 - <Info> -- alert-pf -> adding firewall interface mvneta1 IPv4 address 10.10.10.1 to automatic interface IP Pass List.
              13/12/2018 -- 06:27:15 - <Info> -- alert-pf -> adding firewall interface mvneta2 IPv6 address fe80:0000:0000:0000:0208:a2ff:fe0d:f1dd to automatic interface IP Pass List.
              13/12/2018 -- 06:27:15 - <Info> -- alert-pf -> adding firewall interface mvneta2 IPv4 address  to automatic interface IP Pass List.
              13/12/2018 -- 06:27:15 - <Info> -- alert-pf -> adding firewall interface lo0 IPv6 address 0000:0000:0000:0000:0000:0000:0000:0001 to automatic interface IP Pass List.
              13/12/2018 -- 06:27:15 - <Info> -- alert-pf -> adding firewall interface lo0 IPv6 address fe80:0000:0000:0000:0000:0000:0000:0001 to automatic interface IP Pass List.
              13/12/2018 -- 06:27:15 - <Info> -- alert-pf -> adding firewall interface lo0 IPv4 address 127.0.0.1 to automatic interface IP Pass List.
              13/12/2018 -- 06:27:15 - <Info> -- alert-pf output device (regular) initialized: block.log
              13/12/2018 -- 06:27:15 - <Info> -- alert-pf -> Pass List /usr/local/etc/suricata/suricata_17044_mvneta2/passlist parsed: 12 IP addresses loaded.
              13/12/2018 -- 06:27:15 - <Info> -- alert-pf -> Created firewall interface IP change monitor thread for auto-whitelisting of firewall interface IP addresses.
              13/12/2018 -- 06:27:15 - <Info> -- alert-pf output initialized, pf-table=snort2c  block-ip=both  kill-state=on  block-drops-only=off
              13/12/2018 -- 06:27:15 - <Info> -- fast output device (regular) initialized: alerts.log
              13/12/2018 -- 06:27:15 - <Info> -- http-log output device (regular) initialized: http.log
              13/12/2018 -- 06:27:15 - <Info> -- stats output device (regular) initialized: stats.log
              13/12/2018 -- 06:27:15 - <Info> -- dns-log output device (regular) initialized: dns.log
              13/12/2018 -- 06:27:15 - <Info> -- dns-log output device (regular) initialized: dns.log
              13/12/2018 -- 06:27:15 - <Info> -- Using 1 live device(s).
              13/12/2018 -- 06:27:15 - <Info> -- alert-pf -> Firewall interface IP address change notification monitoring thread started.
              13/12/2018 -- 06:27:15 - <Info> -- using interface mvneta2
              13/12/2018 -- 06:27:15 - <Info> -- Running in 'auto' checksum mode. Detection of interface state will require 1000 packets.
              13/12/2018 -- 06:27:15 - <Info> -- Set snaplen to 1518 for 'mvneta2'
              13/12/2018 -- 06:27:15 - <Info> -- RunModeIdsPcapAutoFp initialised
              13/12/2018 -- 06:27:15 - <Notice> -- all 3 packet processing threads, 4 management threads initialized, engine started.
              13/12/2018 -- 06:28:25 - <Info> -- No packets with invalid checksum, assuming checksum offloading is NOT used
              

              Thanks for your help I will dig through this and find out why these 17 are a problem. It looks like the file reference.config is not being created correctly during the installation and subsequent reinstalls when upgrading to the new version of PFSENSE.

              1 Reply Last reply Reply Quote 0
              • S
                s0m3f00l
                last edited by

                FYI,

                This continues to be a problem. Again this update did not correctly copy the sample config to the folder.

                Is anyone looking into this on the sg3100?

                1 Reply Last reply Reply Quote 0
                • bmeeksB
                  bmeeks
                  last edited by

                  I have been unable to reproduce your issue. I have installed the Suricata package from scratch on several clean virtual machines and it works every time. I have also tested with an existing package configuration installing onto a clean VM (meaning with all previous traces of Suricata wiped from the disk by deleting all the directories and files). I have done this with just the Emerging Threats Open rules enabled, with just the Snort Subscriber rules enabled, and with both sets of rules enabled. The installed package worked each time with no errors (other than the expected errors you get when loading some of the Snort rules on Suricata; remember a number of Snort rules will not work on Suricata because Suricata does not support the same keywords and rule options as Snort).

                  1 Reply Last reply Reply Quote 1
                  • S
                    s0m3f00l
                    last edited by

                    Well what should be my next steps? Should I rebuild The sg3100? Or would you suggest reinstalling the package from scratch? Either way how do I ensure I keep the suricata rules and settings I currently have.

                    bmeeksB 1 Reply Last reply Reply Quote 0
                    • bmeeksB
                      bmeeks @s0m3f00l
                      last edited by bmeeks

                      @s0m3f00l said in Suricata Not logging signature matches | | Suricata 4.0.13_11 with pfsense 2.4.4-RELEASE-p1 (arm):

                      Well what should be my next steps? Should I rebuild The sg3100? Or would you suggest reinstalling the package from scratch? Either way how do I ensure I keep the suricata rules and settings I currently have.

                      There is no need to rebuild your SG-3100 appliance. But there may be a need to be sure the Suricata slate is as clean as possible. We can try some re-install steps that will still preserve your existing settings, but first I need the answer to a question.

                      I've corresponded with several users here in the forums, and I recall at least one and maybe more had tried to use the Snort3 rules with Suricata. That will definitely hose things up. Were you one of those users? Have you tried to use the Snort3 rules? If so, you need to be sure your GLOBAL SETTINGS parameter for the Snort rules is set to this file: snortrules-snapshot-29120.tar.gz and not one with snort3 in the name. You CANNOT use the file snortrules-snapshot-3000.tar.gz. That file will only work with Snort3 (but not Snort 2.9.x and not Suricata!).

                      At the bottom of the GLOBAL SETTINGS tab is a checkbox (that is checked by default) for saving settings between package removals and re-installs. Leave that box checked (or check it if you ever un-checked it in the past). This will preserve your Suricata setup.

                      Now completely remove Suricata package. Click the trash can icon next to the package on the SYSTEM > PACKAGE MANAGER screen on the Installed Packages tab to uninstall Suricata. Wait for the package removal to complete! The progress bar will turn green when the removal is finished.

                      Now open a command prompt on the firewall itself (a shell prompt). Look for any suricata directories in these paths:

                      /usr/local/etc/suricata
                      /usr/local/pkg/suricata
                      /usr/local/www/suricata

                      If you see any of the above directories, remove them with this command:

                      rm -rf /usr/local/xxx/suricata
                      

                      replace the "xxx" with the part of the path from the list above (e.g., "etc", "pkg", or "www").

                      After completing the steps above, install the Suricata package again by going to SYSTEM > PACKAGE MANAGER and finding Suricata on the Available Packages tab. Let the install fully complete before leaving the page or doing anything on the firewall. It will take several seconds, and possibly up to a couple of minutes, because it will download any enabled rules you had. Let this task complete! Wait for the green bar and the "Success" message before leaving the page.

                      Now see if Suricata started properly. You can examine the suricata.log file for the interface to check for errors. You should see no errors related to missing references. You may see other rule parsing errors if you enable and use any of the Snort rules. Those errors are normal because Suricata does not support several of the newer Snort rule options.

                      Report back with your results.

                      S 1 Reply Last reply Reply Quote 1
                      • S
                        s0m3f00l @bmeeks
                        last edited by

                        @bmeeks , It appears to have worked. After deleting the package from the GUI "/usr/local/etc/suricata" was left behind. I removed the directory and its contents and then everything reinstalled correctly without intervention. I will wait for the next update and test again.

                        @bmeeks, thanks for putting up with me sir. You are extraordinarily helpful and supremely knowledgeable about suricata and snort on PFSENSE, and the community would be lost without your input.

                        Thanks, again.

                        bmeeksB 1 Reply Last reply Reply Quote 0
                        • bmeeksB
                          bmeeks @s0m3f00l
                          last edited by bmeeks

                          @s0m3f00l said in Suricata Not logging signature matches | | Suricata 4.0.13_11 with pfsense 2.4.4-RELEASE-p1 (arm):

                          @bmeeks , It appears to have worked. After deleting the package from the GUI "/usr/local/etc/suricata" was left behind. I removed the directory and its contents and then everything reinstalled correctly without intervention. I will wait for the next update and test again.

                          @bmeeks, thanks for putting up with me sir. You are extraordinarily helpful and supremely knowledgeable about suricata and snort on PFSENSE, and the community would be lost without your input.

                          Thanks, again.

                          Glad you got it working. There was likely a file that had been modified and contained "bad content" being left there. When pkg removes software, it compares the md5 hash of the file being removed to the hash the file had when installed. If different, pkg assumes the user modified the file so it leaves it alone. So simply removing and re-installing the package was not removing that modified file. Manually removing the directory and file gets rid of the malformed file. My guess is that it was a different version of the references.config file that was being left behind. That file then would get used with the next installation, so the problem persisted for you.

                          1 Reply Last reply Reply Quote 1
                          • First post
                            Last post
                          Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.