No Alerts using Suricata inline mode.
-
I setup a new PFsense box which seems to function normally except I do not get any alerts using Suricata Inline Mode. When using Legacy mode with 'Set Legacy to Block On Drop Only' checked, I see the proper alerts and blocks being generated. I setup SID mgmt with drop and disable files and did a rebuild. Enabled Auto SID State Mgmt. Using the WAN and LAN interface in Suricata. Firewall logs are normal.
Not using load balancing or traffic shaping. Only other installed package is Cron. Only have 1 WAN and 1 LAN interface active.
System is a Supermicro 5018A-FTN4 with four standard built in Intel NIC's using igb drivers. All offloading is disabled.
Not a NIC issue. Tried an Intel i210T1 card, which worked in another pfsense box using inline.Tried reinstalling Suricata with no change. There are no suppress rules. No errors in the Suricata log. No netmap messages in the console.
Is there some other setting I may be missing?
-
I will ask the obvious questions first –
1. Where are you looking for "alerts and blocks"? When Inline IPS Mode is used, the BLOCKED tab will always be empty. That tab is not part of the Inline IPS Mode of operation. Instead, look on the ALERTS tab for the interface and any alerts that resulted in drops will be highlighted in the "danger" color for the pfSense theme (that will be red using the default theme).
2. Is your configuration exactly the same with the sole exception being the toggling of Blocking Mode from Legacy to Inline?
3. Is Suricata running on the same interface in both modes?
You can easily test Suricata by loading the Emerging Threats Open Rules and enabling the "ET-Scan" category (I forget the exact name, but it has "scan" in it so you can find it). Next, using a Kali Linux virtual machine or any other machine on the the network with Suricata on it and scan the firewall address with nmap. You will get some hits for VNC and I think MS-SQL server if I recall correctly.
Bill
-
1. Yes, I do know about the blocks tab not showing using inline. I have used inline in the past on another system and I am familiar with its operation.
2.Yes, same config
3. Yes same interface. Even tried changing the WAN interface to an i210T1 NIC I have used with inline in the past.
I have done the scan with ET-Scan category enabled and used Nmap/Zenmap from another system. The firewall picks up the scans and so does Suricata when in Legacy mode. No Alerts when using inline.
One difference with this system is it has 8 cores, so I did up the Stream Memory Cap to 128MB. Using the default cap size generated an error and Suricata would not start up. It starts up fine now with no errors.
I am out of ideas. I even duplicated the entire configuration from a working box. What's up?
-
dcol, what pfSense version are you using? I just upgraded a router yesterday from 2.3.4 to 2.4.1, which upgraded the Suricata package, and this morning changed Suricata to Inline mode on WAN (em0), along with setting up the dropsid.conf file. WAN Rules tab shows the action is Drop for those rules. We had been getting an alert every couple minutes but have had none in the last 3 hours. I've stopped and started Suricata, but haven't rebooted the router yet today.
I do see a few SC_ERR_INVALID_SIGNATURE errors in the suricata.log file but those exist yesterday and it was working in legacy mode overnight.
-
1. Yes, I do know about the blocks tab not showing using inline. I have used inline in the past on another system and I am familiar with its operation.
2.Yes, same config
3. Yes same interface. Even tried changing the WAN interface to an i210T1 NIC I have used with inline in the past.
I have done the scan with ET-Scan category enabled and used Nmap/Zenmap from another system. The firewall picks up the scans and so does Suricata when in Legacy mode. No Alerts when using inline.
One difference with this system is it has 8 cores, so I did up the Stream Memory Cap to 128MB. Using the default cap size generated an error and Suricata would not start up. It starts up fine now with no errors.
I am out of ideas. I even duplicated the entire configuration from a working box. What's up?
So with the identical configuration except for the Blocking Mode changing from "Legacy Mode" to "Inline IPS Mode" on the same hardware box, you get alerts with Legacy Mode but no alerts on the same traffic with Inline IPS Mode?
I'm inclined to say that almost can't happen based on how I understand the underlying Suricata binary source code. Not saying you are wrong, but I really can't imagine a scenario where that can happen. The only change between Legacy Mode and Inline IPS Mode is the use of Netmap for Inline. However, Netmap usually either loads and works or it kills the network at startup. I've never seen a report like this where it fails to pass traffic for inspection but does not break the network.
Edit: I will add one more question – what kind of CPU platform do you have? Is it an ARM or Intel CPU?
Bill
-
@teamits:
dcol, what pfSense version are you using? I just upgraded a router yesterday from 2.3.4 to 2.4.1, which upgraded the Suricata package, and this morning changed Suricata to Inline mode on WAN (em0), along with setting up the dropsid.conf file. WAN Rules tab shows the action is Drop for those rules. We had been getting an alert every couple minutes but have had none in the last 3 hours. I've stopped and started Suricata, but haven't rebooted the router yet today.
I do see a few SC_ERR_INVALID_SIGNATURE errors in the suricata.log file but those exist yesterday and it was working in legacy mode overnight.
Those SC_ERR_INVALID_SIGNATURE errors are to be expected if you are using some of the Snort VRT rules with Suricata. Suricata does not recognize some of the newer rule options and keywords that Snort uses. Suricata prints the error for those rules, does not load them, and proceeds to load the next rule.
What hardware are you running? Does it by chance have an ARM CPU (such as the new Netgate SG-3100)? Problems have been discovered within the Snort binary that prevent it from working on the new ARM hardware platforms. I've not had a similar report about Suricata unless yours turns out to be the first incidence.
Bill
-
We do have a client with the new Netgate 3100 and have Suricata running on it in legacy mode.
Our hardware is an older PC:
Intel(R) Core(TM)2 Duo CPU E7500 @ 2.93GHz
2 CPUs: 1 package(s) x 2 core(s)
2 GB RAM but only 37% usedIt's quite possible I missed something but I tried to follow your posts here so I'm not sure what. I'm not seeing errors, just not seeing alerts.
-
@teamits:
We do have a client with the new Netgate 3100 and have Suricata running on it in legacy mode.
Our hardware is an older PC:
Intel(R) Core(TM)2 Duo CPU E7500 @ 2.93GHz
2 CPUs: 1 package(s) x 2 core(s)
2 GB RAM but only 37% usedIt's quite possible I missed something but I tried to follow your posts here so I'm not sure what. I'm not seeing errors, just not seeing alerts.
Need some clarification on details please. There are some subtle differences between your two posts that left me a little confused –
1. This same hardware (SG-3100, I assume) was working fine with Legacy Mode Suricata on pfSense 2.3.x?
2. Do you know what version of Suricata you were running prior to the upgrade? Version 4.0.0 has been out for quite some time.
3. You upgraded the same hardware to 2.4.x and the Suricata package updated automatically (if so, to what version?) and now does not produce alerts.
4. Do you now get no alerts with either mode (Legacy or Inline), or just no alerts with Inline IPS Mode?
Bill
-
Hi Bill,
(to perhaps clarify I'm not the OP, I'm "me too"-ing the thread)
Ignore the SG-3100, that's at a client site.
We have the PC hardware I detailed. It works (alerts and blocks) if I switch to Legacy mode. In inline mode I get no alerts.
I am not sure about the Suricata version. It was from when we upgraded to 2.3.4 in I think early July.
I am seeing no alerts in Inline mode and alerts in Legacy mode.
If netmap wasn't working right in the driver what would be the symptom? No traffic at all? (the wan NIC is detected as em0, the LAN - w/o Suricata is bge0)
-
@teamits:
Hi Bill,
(to perhaps clarify I'm not the OP, I'm "me too"-ing the thread)
Ignore the SG-3100, that's at a client site.
We have the PC hardware I detailed. It works (alerts and blocks) if I switch to Legacy mode. In inline mode I get no alerts.
I am not sure about the Suricata version. It was from when we upgraded to 2.3.4 in I think early July.
I am seeing no alerts in Inline mode and alerts in Legacy mode.
If netmap wasn't working right in the driver what would be the symptom? No traffic at all? (the wan NIC is detected as em0, the LAN - w/o Suricata is bge0)
I would expect that if Netmap was an issue there would be no connectivity (i.e., the network would be broken).
Last time I tested on a VM, Inline IPS mode worked fine. There are also a number of other users here that make use of that mode and are not reporting any problems.
Some things to check – are $HOME_NET and $EXTERNAL_NET properly defined when Inline IPS Mode is enabled. You can view the values on the INTERFACE SETTINGS tab for the interface where Suricata is enabled. Most all rules depend on the HOME_NET and EXTERNAL_NET variables being correctly configured in order for the rules to trigger.
Have you tried throwing some scans at the machine in question from nmap or a simlar tool?
Examine the SID MGMT log file to see what the tally is for rules processed by that logic. Does it show it really changed the rules you think it did? (Go by numbers, of impacted rules since that is what is displays in the log).
You can examine the actual rules file used by an interface here using the DIAGNOSTICS > EDIT FILE menu option in pfSense –
/usr/local/etc/suricata/suricata__xxxxx_/rules/suricata.rules
Note – the xxxxx part of the path will be a GUID along with the physical interface name.
Open that file and check which rules are enabled and what their actions are.
Bill
-
Long delay on my end here but I finally got back to this today. I upgraded the Suricata package 4.0.0_2 -> 4.0.1_1 which has the side effect of wiping out prior alerts and logs, apparently. The sid_changes.log file has:
Processing drop_sid file: dropsid.conf
Parsed 19356 potential SIDs to match from the provided list of tokens.
Found 19356 matching SIDs in the active rules.
Changed state for 19356 SIDs to 'drop'.…so it seems like it's finding all the rules.
Should "Promiscuous Mode" be checked/on for Inline mode? In a quick test that didn't seem to matter. (it was On for Legacy mode)
/usr/local/etc/suricata/suricata_57335_em0/rules/suricata.rules exists...7 MB. /var/log/suricata/suricata_em057335/alerts.log exists, size 0.
I've not tried scanning ourselves, but the flow of alerts in Legacy mode is pretty constant in Legacy mode. Per https://forum.pfsense.org/index.php?topic=108010.0 I read your post that in Inline mode Suricata is before the firewall (Internet->Suricata->firewall) so it should still be seeing all traffic...?
-
Are you 100% positive the Suricata binary continues to run after the initial startup? It's just not really logical that it runs with the same rule set yet fails to alert on the same traffic as it does in Legacy Mode. It could be that something happens during the last stages of startup that kills the Suricata process. If true, then I could make sense of no alerts.
Post up the actual _suricata.lo_g file's contents from a startup in Inline IPS Mode. You can find it under the LOGS VIEW tab.
Bill
-
Hi Bill, here's the log from when I stopped it, set dropsid.conf and enabled inline, and started it:
24/1/2018 – 12:04:48 - <notice>-- Signal Received. Stopping engine.
24/1/2018 -- 12:04:49 - <info>-- time elapsed 20475.473s
24/1/2018 -- 12:04:49 - <info>-- (RX#01-em0) Packets 7219811, bytes 4790942814
24/1/2018 -- 12:04:49 - <info>-- (RX#01-em0) Pcap Total:7219930 Recv:7219930 Drop:0 (0.0%).
24/1/2018 -- 12:04:49 - <info>-- alert-pf output inserted 157 IP address blocks
24/1/2018 -- 12:04:49 - <info>-- alert-pf output processed 216 alerts
24/1/2018 -- 12:04:49 - <info>-- alert-pf output inserted 157 IP address blocks
24/1/2018 -- 12:04:49 - <info>-- alert-pf output processed 216 alerts
24/1/2018 -- 12:04:49 - <info>-- Alerts: 0
24/1/2018 -- 12:04:49 - <info>-- cleaning up signature grouping structure... complete
24/1/2018 -- 12:04:49 - <notice>-- Stats for 'em0': pkts: 7219811, drop: 0 (0.00%), invalid chksum: 0
24/1/2018 -- 12:05:59 - <notice>-- This is Suricata version 4.0.1 RELEASE
24/1/2018 -- 12:05:59 - <info>-- CPUs/cores online: 2
24/1/2018 -- 12:05:59 - <info>-- Netmap: Setting IPS mode
24/1/2018 -- 12:05:59 - <info>-- HTTP memcap: 67108864
24/1/2018 -- 12:05:59 - <notice>-- using flow hash instead of active packets
24/1/2018 -- 12:05:59 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - Signature combines packet specific matches (like dsize, flags, ttl) with stream / state matching by matching on app layer proto (like using http_* keywords).
24/1/2018 – 12:05:59 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "drop tcp $HOME_NET any -> $EXTERNAL_NET 1024:65535 (msg:"MALWARE-CNC Win.Trojan.Fakeavlock variant outbound connection"; flow:to_server,established; dsize:267<>276; content:"User-Agent|3A| Mozilla/5.0 (Windows|3B| U|3B| MSIE 9.0|3B| Windows NT 9.0|3B| en-US)|0D 0A|"; fast_pattern:only; http_header; urilen:159; pcre:"/\x2f[A-F0-9]{158}/U"; metadata:impact_flag red, policy balanced-ips drop, policy security-ips drop, ruleset community, service http; reference:url,www.virustotal.com/file/c49f7dbc036ad0a86df02cbbde00cb3b3fbd651d82f6c9c5a98170644374f64f/analysis/; classtype:trojan-activity; sid:25675; rev:7;)" from file /usr/local/etc/suricata/suricata_57335_em0/rules/suricata.rules at line 70
24/1/2018 – 12:05:59 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - previous keyword has a fast_pattern:only; set. Can't have relative keywords around a fast_pattern only content
24/1/2018 – 12:05:59 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "drop tcp $EXTERNAL_NET $HTTP_PORTS -> $HOME_NET any (msg:"MALWARE-OTHER Win.Trojan.Zeus Spam 2013 dated zip/exe HTTP Response - potential malware download"; flow:to_client,established; content:"-2013.zip|0D 0A|"; fast_pattern:only; content:"-2013.zip|0D 0A|"; http_header; content:"-"; within:1; distance:-14; http_header; file_data; content:"-2013.exe"; content:"-"; within:1; distance:-14; metadata:impact_flag red, policy balanced-ips drop, policy security-ips drop, ruleset community, service http; reference:url,www.virustotal.com/en/file/2eff3ee6ac7f5bf85e4ebcbe51974d0708cef666581ef1385c628233614b22c0/analysis/; classtype:trojan-activity; sid:26470; rev:1;)" from file /usr/local/etc/suricata/suricata_57335_em0/rules/suricata.rules at line 90
24/1/2018 – 12:05:59 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - previous keyword has a fast_pattern:only; set. Can't have relative keywords around a fast_pattern only content
24/1/2018 – 12:05:59 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "drop tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"MALWARE-CNC Bancos fake JPG encrypted config file download"; flow:to_server,established; content:".com.br|0D 0A 0D 0A|"; fast_pattern:only; content:"/imagens/"; depth:9; http_uri; content:".jpg"; distance:0; http_uri; pcre:"/.jpg\x20HTTP/1.[01]\r\nUser\x2dAgent\x3a\x20[a-z]+\r\nHost\x3a\x20[a-z0-9\x2d\x2e]+.com.br\r\n\r\n$/"; metadata:impact_flag red, policy balanced-ips drop, policy security-ips drop, ruleset community, service http; classtype:trojan-activity; sid:26722; rev:1;)" from file /usr/local/etc/suricata/suricata_57335_em0/rules/suricata.rules at line 114
24/1/2018 – 12:05:59 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - Signature combines packet specific matches (like dsize, flags, ttl) with stream / state matching by matching on app layer proto (like using http_* keywords).
24/1/2018 – 12:05:59 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "drop tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"MALWARE-CNC Win32/Autorun.JN variant outbound connection"; flow:to_server,established; dsize:142; urilen:8; content:"/u5.htm"; fast_pattern:only; http_uri; content:"//u5.htm"; http_raw_uri; metadata:impact_flag red, policy balanced-ips drop, policy security-ips drop, ruleset community, service http; reference:url,www.microsoft.com/security/portal/threat/encyclopedia/Entry.aspx?Name=Worm%3AWin32%2FAutorun.JN; reference:url,www.virustotal.com/en/file/36144738373c665d262bc007fceaeb9613e59ec29ea3d7424dd9f400af2c0f06/analysis/; classtype:trojan-activity; sid:26966; rev:3;)" from file /usr/local/etc/suricata/suricata_57335_em0/rules/suricata.rules at line 159
24/1/2018 – 12:05:59 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - Signature combines packet specific matches (like dsize, flags, ttl) with stream / state matching by matching on app layer proto (like using http_* keywords).
24/1/2018 – 12:05:59 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "drop tcp $HOME_NET any -> $EXTERNAL_NET 80 (msg:"MALWARE-CNC Win.Trojan.Conficker variant outbound connection"; flow:to_server,established; dsize:146; urilen:1; content:"GET / HTTP/1.1|0D 0A|User-Agent: Mozilla/4.0 (compatible|3B| MSIE 7.0|3B| Windows NT 5.1|3B| Trident/4.0)|0D 0A|Host: checkip.dyndns.org|0D 0A|Cache-Control: no-cache|0D 0A 0D 0A|"; fast_pattern:only; metadata:impact_flag red, policy balanced-ips drop, policy security-ips drop, ruleset community, service http; reference:url,www.sans.org/security-resources/malwarefaq/conficker-worm.php; classtype:trojan-activity; sid:28542; rev:1;)" from file /usr/local/etc/suricata/suricata_57335_em0/rules/suricata.rules at line 235
24/1/2018 – 12:05:59 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - Signature combines packet specific matches (like dsize, flags, ttl) with stream / state matching by matching on app layer proto (like using http_* keywords).
24/1/2018 – 12:05:59 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "drop tcp $HOME_NET any -> $EXTERNAL_NET 80 (msg:"MALWARE-CNC Win.Trojan.Conficker variant outbound connection"; flow:to_server,established; dsize:139; urilen:1; content:"GET / HTTP/1.1|0D 0A|User-Agent: Mozilla/4.0 (compatible|3B| MSIE 7.0|3B| Windows NT 5.1|3B| Trident/4.0)|0D 0A|Host: www.ask.com|0D 0A|Cache-Control: no-cache|0D 0A 0D 0A|"; fast_pattern:only; metadata:impact_flag red, policy balanced-ips drop, policy security-ips drop, ruleset community, service http; reference:url,www.sans.org/security-resources/malwarefaq/conficker-worm.php; classtype:trojan-activity; sid:28543; rev:1;)" from file /usr/local/etc/suricata/suricata_57335_em0/rules/suricata.rules at line 236
24/1/2018 – 12:05:59 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - previous keyword has a fast_pattern:only; set. Can't have relative keywords around a fast_pattern only content
24/1/2018 – 12:05:59 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "drop tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"MALWARE-CNC Win.Trojan.Injector variant outbound connection"; flow:to_server,established; urilen:9; content:"/load.exe HTTP/1.1|0D 0A|User-Agent: Mozilla/"; fast_pattern:only; content:"|3B 20|MSIE|20|"; http_header; content:")|0D 0A|Host: "; distance:0; http_header; content:!"Accept"; http_header; metadata:impact_flag red, policy balanced-ips drop, policy security-ips drop, ruleset community, service http; reference:url,urlquery.net/search.php?q=%5C%2Fload%5C.exe%24&type=regexp&start=2013-08-24&end=2013-11-22&max=400; reference:url,www.virustotal.com/en/file/032572ea1f34a060ecac98a8e2899dc0f2a41dff199e879050481ddd3818b4d0/analysis/; classtype:trojan-activity; sid:28807; rev:2;)" from file /usr/local/etc/suricata/suricata_57335_em0/rules/suricata.rules at line 242
24/1/2018 – 12:05:59 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - Signature combines packet specific matches (like dsize, flags, ttl) with stream / state matching by matching on app layer proto (like using http_* keywords).
24/1/2018 – 12:05:59 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "drop tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"MALWARE-CNC Win.Trojan.WEC variant outbound connection"; flow:to_server,established; dsize:69; urilen:1; content:"GET / HTTP/1.1|0D 0A|User-Agent: Mozilla/4.0|0D 0A|Host: checkip.dyndns.org|0D 0A 0D 0A|"; fast_pattern:only; metadata:impact_flag red, policy balanced-ips drop, policy security-ips drop, ruleset community, service http; reference:url,www.virustotal.com/en/file/164c792247b2822ab1dce8271a9498d3c9172ff21d36feccf83265ded1be8d0b/analysis/; classtype:trojan-activity; sid:29882; rev:2;)" from file /usr/local/etc/suricata/suricata_57335_em0/rules/suricata.rules at line 290
24/1/2018 – 12:05:59 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - previous keyword has a fast_pattern:only; set. Can't have relative keywords around a fast_pattern only content
24/1/2018 – 12:05:59 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "drop tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"MALWARE-CNC Win.Trojan.Bancos variant outbound connection"; flow:to_server,established; content:"Content-Length: 166"; content:".php HTTP/1.1|0D 0A|Accept: /|0D 0A|Content-Type: application/x-www-form-urlencoded|0D 0A|User-Agent: Mozilla/5.0 (Windows NT 6.1|3B| Trident/7.0|3B| rv:11.0) like Gecko|0D 0A|Host: "; fast_pattern:only; content:"v="; depth:2; http_client_body; content:"&c="; within:7; http_client_body; pcre:"/\x3d\x3d$/P"; metadata:impact_flag red, policy balanced-ips drop, policy security-ips drop, ruleset community, service http; reference:url,www.virustotal.com/en/file/51540d7c9a4bc2a430bc50c85cf9cec5c6f2bb755e800a3f3575ba34fe5f008c/analysis; classtype:trojan-activity; sid:29895; rev:2;)" from file /usr/local/etc/suricata/suricata_57335_em0/rules/suricata.rules at line 293
24/1/2018 – 12:05:59 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - "http_header" keyword seen with a sticky buffer still set. Reset sticky buffer with pkt_data before using the modifier.
24/1/2018 – 12:05:59 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "drop tcp $EXTERNAL_NET $HTTP_PORTS -> $HOME_NET any (msg:"MALWARE-CNC Win.Trojan.Sodebral HTTP Response attempt"; flow:to_client,established; file_data; dsize:<194; content:"INTERNACIONAL"; depth:13; content:!"Content-Length"; http_header; content:"Transfer-Encoding: chunked"; http_header; metadata:impact_flag red, policy balanced-ips drop, policy security-ips drop, ruleset community, service http; reference:url,www.virustotal.com/en/file/e0290c3900445dc00ca24888924e37fa6ac17ecaddc60591e32b81536b9f5ef7/analysis/; classtype:trojan-activity; sid:32607; rev:1;)" from file /usr/local/etc/suricata/suricata_57335_em0/rules/suricata.rules at line 442
24/1/2018 – 12:05:59 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - "http_header" keyword seen with a sticky buffer still set. Reset sticky buffer with pkt_data before using the modifier.
24/1/2018 – 12:05:59 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "drop tcp $EXTERNAL_NET $HTTP_PORTS -> $HOME_NET any (msg:"MALWARE-CNC Win.Trojan.Sodebral HTTP Response attempt"; flow:to_client,established; file_data; dsize:<194; content:"BRASIL"; depth:6; content:!"Content-Length"; http_header; content:"Transfer-Encoding: chunked"; http_header; metadata:impact_flag red, policy balanced-ips drop, policy security-ips drop, ruleset community, service http; reference:url,www.virustotal.com/en/file/e0290c3900445dc00ca24888924e37fa6ac17ecaddc60591e32b81536b9f5ef7/analysis/; classtype:trojan-activity; sid:32608; rev:1;)" from file /usr/local/etc/suricata/suricata_57335_em0/rules/suricata.rules at line 443
24/1/2018 – 12:05:59 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - Signature combines packet specific matches (like dsize, flags, ttl) with stream / state matching by matching on app layer proto (like using http_* keywords).
24/1/2018 – 12:05:59 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "drop tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"MALWARE-CNC Win.Agent.BHHK variant outbound connection"; flow:to_server,established; dsize:136; urilen:1; content:"GET / HTTP/1.1|0D 0A|User-Agent: Mozilla/4.0 (compatible|3B| MSIE 7.0|3B| Windows NT 6.0)|0D 0A|Host: windowsupdate.microsoft.com|0D 0A|Connection: Close|0D 0A 0D 0A|"; fast_pattern:only; content:!"Accept"; http_header; metadata:impact_flag red, policy balanced-ips drop, policy security-ips drop, ruleset community, service http; reference:url,www.virustotal.com/en/file/cab1fffe7a34b5bb7dab2cacd406cf15628d835ab63502d28df78c2faeaad366/analysis/1421677054/; classtype:trojan-activity; sid:33227; rev:2;)" from file /usr/local/etc/suricata/suricata_57335_em0/rules/suricata.rules at line 474
24/1/2018 – 12:05:59 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - Signature combines packet specific matches (like dsize, flags, ttl) with stream / state matching by matching on app layer proto (like using http_* keywords).
24/1/2018 – 12:05:59 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "drop tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"MALWARE-CNC Win.Trojan.FileEncoder IP geolocation checkin attempt"; flow:to_server,established; dsize:214; urilen:1; content:"GET / HTTP/1.1|0D 0A|User-Agent: Mozilla/4.0 (compatible|3B| MSIE 6.0|3B| Windows NT 5.1|3B| SV1|3B| .NET4.0C|3B| .NET4.0E|3B| .NET CLR 2.0.50727|3B| .NET CLR 3.0.4506.2152|3B| .NET CLR 3.5.30729)|0D 0A|Host: ip-addr.es|0D 0A|Cache-Control: no-cache|0D 0A 0D 0A|"; fast_pattern:only; metadata:impact_flag red, policy balanced-ips drop, policy security-ips drop, ruleset community, service http; reference:url,www.virustotal.com/en/file/17edf82c40df6c7268191def7cbff6e60e78d7388018408800d42581567f78cf/analysis/; classtype:trojan-activity; sid:33449; rev:1;)" from file /usr/local/etc/suricata/suricata_57335_em0/rules/suricata.rules at line 479
24/1/2018 – 12:05:59 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - previous keyword has a fast_pattern:only; set. Can't have relative keywords around a fast_pattern only content
24/1/2018 – 12:05:59 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "drop tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"MALWARE-CNC Win.Trojan.GateKeylogger initial exfiltration attempt"; flow:to_server,established; content:"/gate.php"; fast_pattern:only; content:"pc="; nocase; http_client_body; content:"&admin="; distance:0; nocase; http_client_body; content:"&os="; distance:0; nocase; http_client_body; content:"&hid="; distance:0; nocase; http_client_body; content:"&arc="; distance:0; nocase; http_client_body; content:"User-Agent|3A 20|"; http_header; pcre:"/User-Agent\x3a\x20[A-F0-9]{32}\x0d\x0a/H"; metadata:impact_flag red, policy balanced-ips drop, policy security-ips drop, ruleset community, service http; reference:url,www.virustotal.com/en/file/77c802db1731fa8dae1b03d978f89b046309adfa1237b1497a69ccb9c2d82c16/analysis/1459520578/; classtype:trojan-activity; sid:38562; rev:2;)" from file /usr/local/etc/suricata/suricata_57335_em0/rules/suricata.rules at line 598
24/1/2018 – 12:05:59 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - "http_stat_code" keyword seen with a sticky buffer still set. Reset sticky buffer with pkt_data before using the modifier.
24/1/2018 – 12:05:59 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "drop tcp $EXTERNAL_NET $HTTP_PORTS -> $HOME_NET any (msg:"MALWARE-CNC Win.Trojan.GateKeylogger fake 404 response"; flow:to_client,established; file_data; content:"200"; http_stat_code; content:"OK"; http_stat_msg; content:">404 Not Found<"; fast_pattern:only; content:" requested URL / was not found "; metadata:impact_flag red, policy balanced-ips drop, policy security-ips drop, ruleset community, service http; reference:url,www.virustotal.com/en/file/77c802db1731fa8dae1b03d978f89b046309adfa1237b1497a69ccb9c2d82c16/analysis/1459520578/; classtype:trojan-activity; sid:38563; rev:1;)" from file /usr/local/etc/suricata/suricata_57335_em0/rules/suricata.rules at line 599
24/1/2018 – 12:05:59 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - previous keyword has a fast_pattern:only; set. Can't have relative keywords around a fast_pattern only content
24/1/2018 – 12:05:59 - <error>-- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "drop tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"MALWARE-CNC Osx.Trojan.OceanLotus outbound connection attempt"; flow:to_server,established; content:"/sigstore.db?"; fast_pattern:only; content:"k="; http_uri; content:"?q="; distance:0; http_uri; metadata:impact_flag red, policy balanced-ips drop, policy security-ips drop, ruleset community, service http; reference:url,www.alienvault.com/blogs/labs-research/oceanlotus-for-os-x-an-application-bundle-pretending-to-be-an-adobe-flash-update; classtype:trojan-activity; sid:45400; rev:1;)" from file /usr/local/etc/suricata/suricata_57335_em0/rules/suricata.rules at line 752What looks like the same long list of ERRCODE entries happens in legacy startup also so I'm guessing old/bad rules.
The processes are running:
PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU COMMAND
11 root 155 ki31 0K 32K RUN 0 1561.4 100.00% [idle{idle: cpu0}]
11 root 155 ki31 0K 32K CPU1 1 1560.0 98.68% [idle{idle: cpu1}]
32045 root 41 0 282M 41268K piperd 0 0:00 0.78% php-fpm: pool nginx (php-fpm)
69704 root 20 0 668M 260M uwait 0 0:01 0.20% /usr/local/bin/suricata –netmap -D -c /us
12 root -92 - 0K 384K WAIT 0 268:17 0.00% [intr{irq256: em0:rx0}]
0 root -92 - 0K 256K - 1 122:23 0.00% [kernel{bge0 taskq}]
12 root -100 - 0K 384K WAIT 1 31:40 0.00% [intr{irq20: hpet0+}]
12 root -60 - 0K 384K WAIT 1 28:25 0.00% [intr{swi4: clock (0)}]
16 root -16 - 0K 16K pftm 1 26:47 0.00% [pf purge]
17 root -16 - 0K 16K - 0 19:53 0.00% [rand_harvestq]
12 root -92 - 0K 384K WAIT 1 11:05 0.00% [intr{irq257: em0:tx0}]
31545 root 52 20 13084K 2576K wait 1 6:06 0.00% /bin/sh /var/db/rrd/updaterrd.sh
24 root 16 - 0K 16K syncer 1 5:47 0.00% [syncer]
11393 root 20 0 15076K 2384K nanslp 0 5:21 0.00% [dpinger{dpinger}]
23642 root 20 0 24604K 12424K select 0 4:24 0.00% /usr/local/sbin/ntpd -g -c /var/etc/ntpd.c
346 root 20 0 9556K 4920K select 0 3:36 0.00% /sbin/devd -q -f /etc/pfSense-devd.conf
4 root -16 - 0K 32K - 1 3:01 0.00% [cam{doneq0}]
18 root -16 - 0K 48K psleep 0 2:31 0.00% [pagedaemon{pagedaemon}]The system log:
Jan 24 12:05:59 php-fpm 35817 /suricata/suricata_interfaces.php: [Suricata] Suricata START for WAN(em0)…
Jan 24 12:05:59 php-fpm 35817 /suricata/suricata_interfaces.php: Toggle (suricata starting) for WAN(WAN)...
Jan 24 12:05:58 php-fpm 35817 /suricata/suricata_interfaces.php: [Suricata] Building new sid-msg.map file for WAN…
Jan 24 12:05:58 php-fpm 35817 /suricata/suricata_interfaces.php: [Suricata] Enabling any flowbit-required rules for: WAN…
Jan 24 12:05:52 php-fpm 35817 /suricata/suricata_interfaces.php: [Suricata] Updating rules configuration for: WAN …
Jan 24 12:05:40 check_reload_status Syncing firewall
Jan 24 12:05:08 check_reload_status Syncing firewall
Jan 24 12:04:48 php-fpm 90799 /suricata/suricata_interfaces.php: [Suricata] Suricata STOP for WAN(em0)…
Jan 24 12:04:48 php-fpm 90799 /suricata/suricata_interfaces.php: Toggle (suricata stopping) for WAN(WAN)...The Alerts tab's 250 shown entries go back about 6.5 hours so about 40 alerts per hour today, and I would say that is more or less typical.
Edit: I just upgraded to 4.0.3, no joy.</error></error></error></error></error></error></error></error></error></error></error></error></error></error></error></error></error></error></error></error></error></error></error></error></error></error></error></error></error></error></error></error></notice></info></info></info></notice></notice></info></info></info></info></info></info></info></info></info></notice>
-
The errors are normal when running Snort rules. There are a few Snort rule options and keywords that Suricata does not recognize, and the errors are showing you which Snort rules are giving problems and being ignored. Those rules won't be loaded, so they can't be the cause of the "no alerts" issue.
Is there anything different for your configured interfaces such as having VLANs defined on them perhaps? I'm really having a hard time figuring out what could be going on. Is the traffic part of a VPN or something? Are there any shapers or limiters configured on the interfaces?
Bill
-
Yeah, I am too. :)
No limiters, queues or shapers. Technically LAN and WAN show on the limiter tab but are not configured or enabled…pretty sure that's a default. No VPN.
This particular router has a private IP range in its WAN, and a Virtual IP. (we have another router in front of it that we use for other tenants in our building)
The WAN port is connected to a 100Base-T switch, could the em0 driver disable netmap at that speed? Traffic flows just fine in inline mode though. It's an Intel NIC in a Dell PC.
-
Suricata using Inline IPS Mode will automatically generate some PASS rules as it emulates the behavior of the default Pass List used with Legacy Mode. Those rules will be in a file named passlist.rules in this path –
/usr/local/etc/suricata/suricata__xxxxx_/rules where xxxxx will be a random UUID and the physical interface name.
Take a look in that file, or even better, post its contents back here and let me take a look. I wonder if the code is generating an automatic pass list that is too broad.
Bill
-
passlist.rules is empty in legacy mode. In inline mode it has:
pass ip 8.8.4.4/32 any <> any any (msg:"Pass List Entry - allow all traffic from/to 8.8.4.4/32"; sid:1000001;)
pass ip 10.15.55.1/32 any <> any any (msg:"Pass List Entry - allow all traffic from/to 10.15.55.1/32"; sid:1000002;)
pass ip 10.15.55.42/32 any <> any any (msg:"Pass List Entry - allow all traffic from/to 10.15.55.42/32"; sid:1000003;)
pass ip 10.15.55.43 any <> any any (msg:"Pass List Entry - allow all traffic from/to 10.15.55.43"; sid:1000004;)
pass ip 10.15.55.43/32 any <> any any (msg:"Pass List Entry - allow all traffic from/to 10.15.55.43/32"; sid:1000005;)
pass ip 10.99.99.0/24 any <> any any (msg:"Pass List Entry - allow all traffic from/to 10.99.99.0/24"; sid:1000006;)
pass ip 64.79.96.148/29 any <> any any (msg:"Pass List Entry - allow all traffic from/to 64.79.96.148/29"; sid:1000007;)
pass ip 72.35.12.0/24 any <> any any (msg:"Pass List Entry - allow all traffic from/to 72.35.12.0/24"; sid:1000008;)
pass ip 72.35.23.0/24 any <> any any (msg:"Pass List Entry - allow all traffic from/to 72.35.23.0/24"; sid:1000009;)
pass ip 74.122.194.0/25 any <> any any (msg:"Pass List Entry - allow all traffic from/to 74.122.194.0/25"; sid:1000010;)
pass ip 127.0.0.1/32 any <> any any (msg:"Pass List Entry - allow all traffic from/to 127.0.0.1/32"; sid:1000011;)
pass ip 173.165.105.46 any <> any any (msg:"Pass List Entry - allow all traffic from/to 173.165.105.46"; sid:1000012;)
pass ip 192.162.216.0/22 any <> any any (msg:"Pass List Entry - allow all traffic from/to 192.162.216.0/22"; sid:1000013;)
pass ip 208.70.128.0/21 any <> any any (msg:"Pass List Entry - allow all traffic from/to 208.70.128.0/21"; sid:1000014;)
pass ip ::1/128 any <> any any (msg:"Pass List Entry - allow all traffic from/to ::1/128"; sid:1000015;)
pass ip fe80::21b:21ff:fe24:593/128 any <> any any (msg:"Pass List Entry - allow all traffic from/to fe80::21b:21ff:fe24:593/128"; sid:1000016;)
pass ip fe80::225:64ff:feaf:8afd/128 any <> any any (msg:"Pass List Entry - allow all traffic from/to fe80::225:64ff:feaf:8afd/128"; sid:1000017;)10.15.55.0/24 is the WAN side. .42 is the WAN IP of this router and .43 is a virtual IP on WAN. The public IPs and one of the 10.15.55.43 entries are from a passlist configured in Suricata. 8.8.4.4 and 10.15.55.1 I think it picks up as DNS servers for this router. 10.99.99.0/24 is our LAN. 10.15.55.1 is the WAN gateway (building router).
Edit: Our dropsid.conf contains:
emerging-activex,emerging-attack_response,emerging-botcc.portgrouped,emerging-botcc,emerging-ciarmy,emerging-compromised,emerging-current_events,emerging-deleted,emerging-dos,emerging-exploit,emerging-games,emerging-info,emerging-malware,emerging-mobile_malware,emerging-p2p,emerging-policy,emerging-scada,emerging-scan,emerging-shellcode,emerging-tor,emerging-trojan,emerging-user_agents,emerging-web_client,emerging-web_server,emerging-worm,decoder-events,dns-events,GPLv2_community,http-events,smtp-events,tls-events -
@teamits:
passlist.rules is empty in legacy mode. In inline mode it has:
pass ip 8.8.4.4/32 any <> any any (msg:"Pass List Entry - allow all traffic from/to 8.8.4.4/32"; sid:1000001;)
pass ip 10.15.55.1/32 any <> any any (msg:"Pass List Entry - allow all traffic from/to 10.15.55.1/32"; sid:1000002;)
pass ip 10.15.55.42/32 any <> any any (msg:"Pass List Entry - allow all traffic from/to 10.15.55.42/32"; sid:1000003;)
pass ip 10.15.55.43 any <> any any (msg:"Pass List Entry - allow all traffic from/to 10.15.55.43"; sid:1000004;)
pass ip 10.15.55.43/32 any <> any any (msg:"Pass List Entry - allow all traffic from/to 10.15.55.43/32"; sid:1000005;)
pass ip 10.99.99.0/24 any <> any any (msg:"Pass List Entry - allow all traffic from/to 10.99.99.0/24"; sid:1000006;)
pass ip 64.79.96.148/29 any <> any any (msg:"Pass List Entry - allow all traffic from/to 64.79.96.148/29"; sid:1000007;)
pass ip 72.35.12.0/24 any <> any any (msg:"Pass List Entry - allow all traffic from/to 72.35.12.0/24"; sid:1000008;)
pass ip 72.35.23.0/24 any <> any any (msg:"Pass List Entry - allow all traffic from/to 72.35.23.0/24"; sid:1000009;)
pass ip 74.122.194.0/25 any <> any any (msg:"Pass List Entry - allow all traffic from/to 74.122.194.0/25"; sid:1000010;)
pass ip 127.0.0.1/32 any <> any any (msg:"Pass List Entry - allow all traffic from/to 127.0.0.1/32"; sid:1000011;)
pass ip 173.165.105.46 any <> any any (msg:"Pass List Entry - allow all traffic from/to 173.165.105.46"; sid:1000012;)
pass ip 192.162.216.0/22 any <> any any (msg:"Pass List Entry - allow all traffic from/to 192.162.216.0/22"; sid:1000013;)
pass ip 208.70.128.0/21 any <> any any (msg:"Pass List Entry - allow all traffic from/to 208.70.128.0/21"; sid:1000014;)
pass ip ::1/128 any <> any any (msg:"Pass List Entry - allow all traffic from/to ::1/128"; sid:1000015;)
pass ip fe80::21b:21ff:fe24:593/128 any <> any any (msg:"Pass List Entry - allow all traffic from/to fe80::21b:21ff:fe24:593/128"; sid:1000016;)
pass ip fe80::225:64ff:feaf:8afd/128 any <> any any (msg:"Pass List Entry - allow all traffic from/to fe80::225:64ff:feaf:8afd/128"; sid:1000017;)10.15.55.0/24 is the WAN side. .42 is the WAN IP of this router and .43 is a virtual IP on WAN. The public IPs and one of the 10.15.55.43 entries are from a passlist configured in Suricata. 8.8.4.4 and 10.15.55.1 I think it picks up as DNS servers for this router. 10.99.99.0/24 is our LAN. 10.15.55.1 is the WAN gateway (building router).
Edit: Our dropsid.conf contains:
emerging-activex,emerging-attack_response,emerging-botcc.portgrouped,emerging-botcc,emerging-ciarmy,emerging-compromised,emerging-current_events,emerging-deleted,emerging-dos,emerging-exploit,emerging-games,emerging-info,emerging-malware,emerging-mobile_malware,emerging-p2p,emerging-policy,emerging-scada,emerging-scan,emerging-shellcode,emerging-tor,emerging-trojan,emerging-user_agents,emerging-web_client,emerging-web_server,emerging-worm,decoder-events,dns-events,GPLv2_community,http-events,smtp-events,tls-eventsThe passlist.rules file is only generated and used when Inline IPS Mode is active. Legacy Mode has a completely different process.
Looking at the list I can see that my original logic was flawed in some ways. The passlist is "too inclusive". What I was trying to do was re-create the sort of "automatic pass list" process that Legacy Mode has per the request of users. But the effect with Inline IPS Mode is going to be different. This is an overly broad pass list. I should rework it to include maybe only the firewall interface IPs themselves without the network subnets. What is happening now is the pass list is way too broad and winds up telling Suricata to skip looking at a lot of stuff.
I'm going to back this change out or else completely re-think the logic. I will do that in the next update.
Bill
-
So for clarity, does the pass list set under "Networks Suricata Should Inspect and Protect"/Pass List not apply in Inline mode? Or are you saying that passlist.rules incorporates that, but works in a different way?
viewing our pass list under "Networks Suricata Should Inspect and Protect"/Pass List shows the same list:
8.8.4.4/32
10.15.55.1/32
10.15.55.42/32
10.15.55.43
10.15.55.43/32
10.99.99.0/24
64.79.96.148/29
72.35.12.0/24
72.35.23.0/24
74.122.194.0/25
127.0.0.1/32
173.165.105.46
192.162.216.0/22
208.70.128.0/21
::1/128
fe80::21b:21ff:fe24:593/128
fe80::225:64ff:feaf:8afd/128 -
@teamits:
So for clarity, does the pass list set under "Networks Suricata Should Inspect and Protect"/Pass List not apply in Inline mode? Or are you saying that passlist.rules incorporates that, but works in a different way?
viewing our pass list under "Networks Suricata Should Inspect and Protect"/Pass List shows the same list:
8.8.4.4/32
10.15.55.1/32
10.15.55.42/32
10.15.55.43
10.15.55.43/32
10.99.99.0/24
64.79.96.148/29
72.35.12.0/24
72.35.23.0/24
74.122.194.0/25
127.0.0.1/32
173.165.105.46
192.162.216.0/22
208.70.128.0/21
::1/128
fe80::21b:21ff:fe24:593/128
fe80::225:64ff:feaf:8afd/128It's a little "yes" and a little "no" … :)
You can create a Pass List now with Inline IPS Mode but the result is a bit different. With Legacy Mode, you still see alerts on Pass List IP addresses, but they never generate blocks. This is due to how the custom plugin I wrote operates in conjunction with the packet filter firewall in pfSense. Inline IPS Mode is different as it is native Suricata code (no customization). The only way to simulate a pass list like Legacy Mode uses is to generate rules for the IP addresses with PASS as the action. When Suricata is operating in Inline IPS Mode and encounters a rule with PASS as the action, it does just that -- lets the traffic pass with no inspection and no delay. This means no alerts show up for pass list traffic when using Inline IPS Mode.
The automatic pass list rules for Inline IPS Mode are to broad in that they let anything go by where the pass list IP is on either end of the connection (source or destination).
Bill
