Suricata Not logging signature matches | | Suricata 4.0.13_11 with pfsense 2.4.4-RELEASE-p1 (arm)

s0m3f00l

Hello All,

I upgraded to the most recent PFSENSE a few days ago and noticed that most of my WAN alerts simply stopped. At first I thought the internet had finally cleaned itself up but after a few days I realized something was probably wrong. Looking into it I performed the following test and found no hits and got no hits from suricata. I believe with these versions Suricata is no longer logging signature matches. ICMP matches and outright ip bans appear to work fine but I dont get almost any alerts despite checking almost all of the ET rules. The test I performed follows:

[2.4.4-RELEASE][admin@pfSense_Edge.local]/: cat /usr/local/etc/suricata/rules/emerging-user_agents.rules  | grep -i black
alert http $HOME_NET any -> $EXTERNAL_NET any (msg:"ET USER_AGENTS Suspicious User Agent (BlackSun)"; flow:to_server,established; content:"User-Agent|3a| BlackSun"; nocase; http_header; reference:url,www.bitdefender.com/VIRUS-1000328-en--Trojan.Pws.Wow.NCY.html; reference:url,doc.emergingthreats.net/bin/view/Main/2008983; classtype:trojan-activity; sid:2008983; rev:6; metadata:created_at 2010_07_30, updated_at 2010_07_30;)

On a Local PC

s0m3f00l@LOCAL-deb:~$ curl -A "BlackSun" www.google.com
<!doctype html><html itemscope="" itemtype="http://schema.org/WebPag 
(output omitted)

on the PFSENSE 3100

[2.4.4-RELEASE][admin@pfSense_Edge.local]/: cat /var/log/suricata/suricata_mvneta*/alerts.log | grep -i black
[2.4.4-RELEASE][admin@pfSense_Edge.local]/:

As you can see Suricata misses the Blacksun curl and shows no alerts in the log. Is anyone else having these issues? I am on

Netgate SG-3100
2.4.4-RELEASE-p1 (arm)
built on Thu Nov 29 14:06:34 EST 2018
FreeBSD 11.2-RELEASE-p4

Thanks For Any help or testing you can provide.

-s0m3f00l

bmeeks

Are you sure Suricata is continuing to run on the SG-3100. Some other users have reported Signal 10 Bus Error aborts from Suricata on the armv7 hardware used in the SG-3100. I have an open issue with the pfSense developer team about the issue. There is at least one other thread on here about the problem. Snort had the same issue on the SG-3100 and we fixed that by turning off compiler optimizations for the Snort binary. I suspect a similar thing needs to happen for Suricata now on SG-3100 hardware.

I'm guessing Suricata may be running for a while and then later aborting on the Signal 10. Look all through your system log to see if any Signal 10 errors are shown. Please post back with what you find.

s0m3f00l

Hello,

I believe its running because the ICMP alerts still appear to work and block as usual. I found the following errors in the suricata.log:

2/12/2018 -- 18:44:28 - <Notice> -- using flow hash instead of active packets
12/12/2018 -- 18:44:28 - <Error> -- [ERRCODE: SC_ERR_FOPEN(44)] - Error opening file: "/usr/local/etc/suricata/suricata_17044_mvneta2/reference.config": No such file or directory
12/12/2018 -- 18:44:28 - <Error> -- [ERRCODE: SC_ERR_OPENING_FILE(40)] - please check the "reference-config-file" option in your suricata.yaml file
12/12/2018 -- 18:44:28 - <Error> -- [ERRCODE: SC_ERR_REFERENCE_UNKNOWN(150)] - unknown reference key "bugtraq". Supported keys are defined in reference.config file.  Please have a look at the conf param "reference-config-file"
12/12/2018 -- 18:44:28 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $SQL_SERVERS 1433 -> $EXTERNAL_NET any (msg:"SQL sa login failed"; flow:to_client,established; content:"Login failed for user 'sa'"; fast_pattern:only; metadata:policy balanced-ips drop, policy connectivity-ips drop, policy security-ips drop, ruleset community; reference:bugtraq,4797; reference:cve,2000-1209; reference:nessus,10673; classtype:unsuccessful-user; sid:688; rev:16;)" from file /usr/local/etc/suricata/suricata_17044_mvneta2/rules/suricata.rules at line 9
12/12/2018 -- 18:44:28 - <Error> -- [ERRCODE: SC_ERR_REFERENCE_UNKNOWN(150)] - unknown reference key "url". Supported keys are defined in reference.config file.  Please have a look at the conf param "reference-config-file"
12/12/2018 -- 18:44:28 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"FILE-IDENTIFY RealNetworks Realplayer .ram playlist file download request"; flow:to_server,established; content:".ra"; fast_pattern:only; http_uri; pcre:"/\x2eram?([\?\x5c\x2f]|$)/smiU"; flowbits:set,file.realplayer.playlist; flowbits:noalert; metadata:ruleset community, service http; reference:url,en.wikipedia.org/wiki/.ram; classtype:misc-activity; sid:2419; rev:28;)" from file /usr/local/etc/suricata/suricata_17044_mvneta2/rules/suricata.rules at line 10
12/12/2018 -- 18:44:28 - <Error> -- [ERRCODE: SC_ERR_REFERENCE_UNKNOWN(150)] - unknown reference key "url". Supported keys are defined in reference.config file.  Please have a look at the conf param "reference-config-file"
12/12/2018 -- 18:44:28 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"FILE-IDENTIFY RealNetworks Realplayer .rmp playlist file download request"; flow:to_server,established; content:".rmp"; fast_pattern:only; http_uri; pcre:"/\x2ermp([\?\x5c\x2f]|$)/smiU"; flowbits:set,file.rmp; flowbits:set,file.realplayer.playlist; flowbits:noalert; metadata:ruleset community, service http; reference:url,en.wikipedia.org/wiki/.ram; classtype:misc-activity; sid:2420; rev:30;)" from file /usr/local/etc/suricata/suricata_17044_mvneta2/rules/suricata.rules at line 11
12/12/2018 -- 18:44:28 - <Error> -- [ERRCODE: SC_ERR_REFERENCE_UNKNOWN(150)] - unknown reference key "url". Supported keys are defined in reference.config file.  Please have a look at the conf param "reference-config-file"
12/12/2018 -- 18:44:28 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"FILE-IDENTIFY RealNetworks Realplayer .rt playlist file download request"; flow:to_server,established; content:".rt"; fast_pattern:only; http_uri; pcre:"/\x2ert([\?\x5c\x2f]|$)/smiU"; flowbits:set,file.realplayer.playlist; flowbits:noalert; metadata:ruleset community, service http; reference:url,en.wikipedia.org/wiki/.ram; classtype:misc-activity; sid:2422; rev:29;)" from file /usr/local/etc/suricata/suricata_17044_mvneta2/rules/suricata.rules at line 12
12/12/2018 -- 18:44:28 - <Error> -- [ERRCODE: SC_ERR_REFERENCE_UNKNOWN(150)] - unknown reference key "url". Supported keys are defined in reference.config file.  Please have a look at the conf param "reference-config-file"
12/12/2018 -- 18:44:28 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"FILE-IDENTIFY RealNetworks Realplayer .rp playlist file download request"; flow:to_server,established; content:".rp"; fast_pattern:only; http_uri; pcre:"/\x2erp([\?\x5c\x2f]|$)/smiU"; flowbits:set,file.realplayer.playlist; flowbits:noalert; metadata:ruleset community, service http; reference:url,en.wikipedia.org/wiki/.ram; classtype:misc-activity; sid:2423; rev:28;)" from file /usr/local/etc/suricata/suricata_17044_mvneta2/rules/suricata.rules at line 13
12/12/2018 -- 18:44:28 - <Error> -- [ERRCODE: SC_ERR_REFERENCE_UNKNOWN(150)] - unknown reference key "bugtraq". Supported keys are defined in reference.config file.  Please have a look at the conf param "reference-config-file"

It may be related to your error as well. I am not sure how else to troubleshoot. Am I looking at the correct log? I have tried restarting and reinstalling Suricata and associated packages.

Thank you for your quick reply, I do appreciate the help. Maybe we can addon to your ticket with the DEV?

bmeeks

@s0m3f00l, what rule packages and versions are you using? Do you have Snort rules enabled on the GLOBAL SETTINGS tab? If so, what is the filename you are telling Suricata to download?

You cannot use Snort3 rules with Suricata. That's one thing that will cause the errors you are seeing. Your references.config file is hosed. That is causing those listed rules in the log to fail. That could have happened if you manually edited it, or if you tried to use the Snort3 rules.

To look for Signal 10 errors, you must look in the firewall system log under STATUS > SYSTEM LOG.

s0m3f00l

So here are the rules I have enabled and the version.

0_1544669696654_Screenshot_2018-12-13 pfSense_Edge fool local - Suricata IDS Interface WAN - Categories.png

0_1544669790808_Screenshot_2018-12-13 pfSense_Edge fool local - System Package Manager Installed Packages.png

Not sure if thats what you need but let me know if I can get anything else. This device is production but home use.

bmeeks

@s0m3f00l

In the suricata.log that you posted in your second reply, how many rules does it say it loaded and how many did it reject? There will be a summary line at the end of the rule loading (past where the ERRCODE: SC_ERR lines are shown).

You installation appears borked somehow since that very first line in the log indicates it can't open the reference.config file. That file should always be present when the rules and configuration for an interface build successfully. You need to find out what happened to that file. It should be there.

s0m3f00l

This post is deleted!

s0m3f00l

you're right

12/12/2018 -- 18:44:58 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert http $HOME_NET any -> $EXTERNAL_NET any (msg:"ET POLICY Vulnerable Java Version 1.5.x Detected"; flow:established,to_server; content:" Java/1.5."; nocase; http_header; flowbits:set,ET.http.javaclient.vulnerable; threshold: type limit, count 2, seconds 300, track by_src; metadata: former_category POLICY; reference:url,www.oracle.com/technetwork/java/javase/documentation/index.html; classtype:bad-unknown; sid:2011581; rev:10; metadata:affected_product Java, attack_target Client_Endpoint, deployment Perimeter, deployment Internal, tag EOL, signature_severity Audit, created_at 2010_09_27, performance_impact Low, updated_at 2018_04_19;)" from file /usr/local/etc/suricata/suricata_17044_mvneta2/rules/flowbit-required.rules at line 208
12/12/2018 -- 18:44:58 - <Error> -- [ERRCODE: SC_ERR_REFERENCE_UNKNOWN(150)] - unknown reference key "url". Supported keys are defined in reference.config file.  Please have a look at the conf param "reference-config-file"
12/12/2018 -- 18:44:58 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert http $HOME_NET any -> $EXTERNAL_NET any (msg:"ET POLICY Vulnerable Java Version 1.4.x Detected"; flow:established,to_server; content:"Java/1.4."; http_user_agent; flowbits:set,ET.http.javaclient.vulnerable;  threshold: type limit, count 2, seconds 300, track by_src; metadata: former_category POLICY; reference:url,www.oracle.com/technetwork/java/javase/documentation/index.html; classtype:bad-unknown; sid:2011584; rev:12; metadata:affected_product Java, attack_target Client_Endpoint, deployment Perimeter, deployment Internal, tag EOL, signature_severity Audit, created_at 2010_09_27, performance_impact Low, updated_at 2018_04_19;)" from file /usr/local/etc/suricata/suricata_17044_mvneta2/rules/flowbit-required.rules at line 211
12/12/2018 -- 18:44:58 - <Error> -- [ERRCODE: SC_ERR_REFERENCE_UNKNOWN(150)] - unknown reference key "url". Supported keys are defined in reference.config file.  Please have a look at the conf param "reference-config-file"
12/12/2018 -- 18:44:58 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert http $HOME_NET any -> $EXTERNAL_NET any (msg:"ET POLICY Vulnerable Java Version 10.0.x Detected"; flow:established,to_server; content:"Java/10.0."; http_user_agent; content:!"2"; within:1; http_user_agent; flowbits:set,ET.http.javaclient.vulnerable; threshold: type limit, count 2, seconds 300, track by_src; metadata: former_category POLICY; reference:url,www.oracle.com/technetwork/java/javase/10u-relnotes-4108739.html; classtype:bad-unknown; sid:2025518; rev:3; metadata:affected_product Java, attack_target Client_Endpoint, deployment Perimeter, deployment Internal, signature_severity Audit, created_at 2018_04_19, performance_impact Low, updated_at 2018_11_20;)" from file /usr/local/etc/suricata/suricata_17044_mvneta2/rules/flowbit-required.rules at line 214
12/12/2018 -- 18:44:58 - <Error> -- [ERRCODE: SC_ERR_REFERENCE_UNKNOWN(150)] - unknown reference key "url". Supported keys are defined in reference.config file.  Please have a look at the conf param "reference-config-file"
12/12/2018 -- 18:44:58 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert http $HOME_NET any -> $EXTERNAL_NET any (msg:"ET POLICY Vulnerable Java Version 9.0.x Detected"; flow:established,to_server; content:"Java/9."; http_user_agent; flowbits:set,ET.http.javaclient.vulnerable; threshold: type limit, count 2, seconds 300, track by_src; metadata: former_category POLICY; reference:url,www.oracle.com/technetwork/java/javase/documentation/9u-relnotes-3704429.html; classtype:bad-unknown; sid:2025314; rev:3; metadata:affected_product Java, attack_target Client_Endpoint, deployment Perimeter, deployment Internal, tag EOL, signature_severity Audit, created_at 2018_02_05, performance_impact Low, updated_at 2018_07_17;)" from file /usr/local/etc/suricata/suricata_17044_mvneta2/rules/flowbit-required.rules at line 217
12/12/2018 -- 18:44:58 - <Error> -- [ERRCODE: SC_ERR_REFERENCE_UNKNOWN(150)] - unknown reference key "url". Supported keys are defined in reference.config file.  Please have a look at the conf param "reference-config-file"
12/12/2018 -- 18:44:58 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert http $HOME_NET any -> $EXTERNAL_NET any (msg:"ET POLICY Vulnerable Java Version 1.8.x Detected"; flow:established,to_server; content:" Java/1.8.0_"; http_user_agent; content:!"191"; within:3; http_user_agent; content:!"192"; within:3; http_user_agent; flowbits:set,ET.http.javaclient.vulnerable; threshold: type limit, count 2, seconds 300, track by_src; metadata: former_category POLICY; reference:url,www.oracle.com/technetwork/java/javase/8u-relnotes-2225394.html; classtype:bad-unknown; sid:2019401; rev:25; metadata:affected_product Java, attack_target Client_Endpoint, deployment Perimeter, deployment Internal, signature_severity Audit, created_at 2014_10_15, performance_impact Low, updated_at 2018_07_17;)" from file /usr/local/etc/suricata/suricata_17044_mvneta2/rules/flowbit-required.rules at line 220
12/12/2018 -- 18:44:58 - <Error> -- [ERRCODE: SC_ERR_REFERENCE_UNKNOWN(150)] - unknown reference key "url". Supported keys are defined in reference.config file.  Please have a look at the conf param "reference-config-file"
12/12/2018 -- 18:44:58 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert http $HOME_NET any -> $EXTERNAL_NET any (msg:"ET POLICY Vulnerable Java Version 1.7.x Detected"; flow:established,to_server; content:"Java/1.7.0_"; http_user_agent; content:!"201"; within:3; http_user_agent; flowbits:set,ET.http.javaclient.vulnerable; threshold: type limit, count 2, seconds 300, track by_src; metadata: former_category POLICY; reference:url,www.oracle.com/technetwork/java/javase/documentation/javase7supportreleasenotes-1601161.html; classtype:bad-unknown; sid:2014297; rev:53; metadata:affected_product Java, attack_target Client_Endpoint, deployment Perimeter, deployment Internal, signature_severity Audit, created_at 2012_03_01, performance_impact Low, updated_at 2018_07_17;)" from file /usr/local/etc/suricata/suricata_17044_mvneta2/rules/flowbit-required.rules at line 223
12/12/2018 -- 18:44:58 - <Error> -- [ERRCODE: SC_ERR_REFERENCE_UNKNOWN(150)] - unknown reference key "url". Supported keys are defined in reference.config file.  Please have a look at the conf param "reference-config-file"
12/12/2018 -- 18:44:58 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert http $HOME_NET any -> $EXTERNAL_NET any (msg:"ET POLICY Vulnerable Java Version 1.6.x Detected"; flow:established,to_server; content:"Java/1.6.0_"; http_user_agent; content:!"211"; within:3; http_user_agent; flowbits:set,ET.http.javaclient.vulnerable; threshold: type limit, count 2, seconds 300, track by_src; metadata: former_category POLICY; reference:url,www.oracle.com/technetwork/articles/javase/overview-156328.html; classtype:bad-unknown; sid:2011582; rev:54; metadata:affected_product Java, attack_target Client_Endpoint, deployment Perimeter, deployment Internal, signature_severity Audit, created_at 2010_09_27, performance_impact Low, updated_at 2018_07_17;)" from file /usr/local/etc/suricata/suricata_17044_mvneta2/rules/flowbit-required.rules at line 226
12/12/2018 -- 18:44:58 - <Info> -- 2 rule files processed. 4194 rules successfully loaded, 14028 rules failed
12/12/2018 -- 18:44:58 - <Warning> -- [ERRCODE: SC_ERR_EVENT_ENGINE(210)] - can't suppress sid 2018959, gid 1: unknown rule
12/12/2018 -- 18:44:58 - <Info> -- Threshold config parsed: 1 rule(s) found
12/12/2018 -- 18:44:58 - <Info> -- 4195 signatures processed. 0 are IP-only rules, 1157 are inspecting packet payload, 3118 inspect application layer, 103 are decoder event only
12/12/2018 -- 18:44:59 - <Info> -- alert-pf -> Creating automatic firewall interface IP address Pass List.
12/12/2018 -- 18:44:59 - <Info> -- alert-pf -> adding firewall interface mvneta1 IPv6 address fe80:0000:0000:0000:0208:a2ff:fe0d:f1dc to automatic interface IP Pass List.
12/12/2018 -- 18:44:59 - <Info> -- alert-pf -> adding firewall interface mvneta1 IPv4 address 172.16.0.240 to automatic interface IP Pass List.
12/12/2018 -- 18:44:59 - <Info> -- alert-pf -> adding firewall interface mvneta1 IPv4 address 10.10.10.1 to automatic interface IP Pass List.
12/12/2018 -- 18:44:59 - <Info> -- alert-pf -> adding firewall interface mvneta2 IPv6 address o automatic interface IP Pass List.
12/12/2018 -- 18:44:59 - <Info> -- alert-pf -> adding firewall interface mvneta2 IPv4 address  to automatic interface IP Pass List.
12/12/2018 -- 18:44:59 - <Info> -- alert-pf -> adding firewall interface lo0 IPv6 address 0000:0000:0000:0000:0000:0000:0000:0001 to automatic interface IP Pass List.
12/12/2018 -- 18:44:59 - <Info> -- alert-pf -> adding firewall interface lo0 IPv6 address fe80:0000:0000:0000:0000:0000:0000:0001 to automatic interface IP Pass List.
12/12/2018 -- 18:44:59 - <Info> -- alert-pf -> adding firewall interface lo0 IPv4 address 127.0.0.1 to automatic interface IP Pass List.
12/12/2018 -- 18:44:59 - <Info> -- alert-pf output device (regular) initialized: block.log
12/12/2018 -- 18:44:59 - <Info> -- alert-pf -> Pass List /usr/local/etc/suricata/suricata_17044_mvneta2/passlist parsed: 12 IP addresses loaded.
12/12/2018 -- 18:44:59 - <Info> -- alert-pf -> Created firewall interface IP change monitor thread for auto-whitelisting of firewall interface IP addresses.
12/12/2018 -- 18:44:59 - <Info> -- alert-pf output initialized, pf-table=snort2c  block-ip=both  kill-state=on  block-drops-only=off
12/12/2018 -- 18:44:59 - <Info> -- fast output device (regular) initialized: alerts.log
12/12/2018 -- 18:44:59 - <Info> -- http-log output device (regular) initialized: http.log
12/12/2018 -- 18:44:59 - <Info> -- stats output device (regular) initialized: stats.log
12/12/2018 -- 18:44:59 - <Info> -- dns-log output device (regular) initialized: dns.log
12/12/2018 -- 18:44:59 - <Info> -- dns-log output device (regular) initialized: dns.log
12/12/2018 -- 18:44:59 - <Info> -- Using 1 live device(s).
12/12/2018 -- 18:44:59 - <Info> -- alert-pf -> Firewall interface IP address change notification monitoring thread started.
12/12/2018 -- 18:44:59 - <Info> -- using interface mvneta2
12/12/2018 -- 18:44:59 - <Info> -- Running in 'auto' checksum mode. Detection of interface state will require 1000 packets.
12/12/2018 -- 18:44:59 - <Info> -- Set snaplen to 1518 for 'mvneta2'
12/12/2018 -- 18:45:00 - <Info> -- RunModeIdsPcapAutoFp initialised
12/12/2018 -- 18:45:00 - <Notice> -- all 3 packet processing threads, 4 management threads initialized, engine started.
12/12/2018 -- 18:45:07 - <Info> -- No packets with invalid checksum, assuming checksum offloading is NOT used

I recreated the config file :

[2.4.4-RELEASE][admin@pfSense.local]/usr/local/etc/suricata: cp reference.config.sample reference.config

And Now it looks like 17 didn't load

13/12/2018 -- 06:27:07 - <Notice> -- This is Suricata version 4.0.6 RELEASE
13/12/2018 -- 06:27:07 - <Info> -- CPUs/cores online: 2
13/12/2018 -- 06:27:07 - <Info> -- HTTP memcap: 67108864
13/12/2018 -- 06:27:07 - <Notice> -- using flow hash instead of active packets
13/12/2018 -- 06:27:08 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - Signature combines packet specific matches (like dsize, flags, ttl) with stream / state matching by matching on app layer proto (like using http_* keywords).
13/12/2018 -- 06:27:08 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $HOME_NET any -> $EXTERNAL_NET 1024:65535 (msg:"MALWARE-CNC Win.Trojan.Fakeavlock variant outbound connection"; flow:to_server,established; dsize:267<>276; content:"User-Agent|3A| Mozilla/5.0 (Windows|3B| U|3B| MSIE 9.0|3B| Windows NT 9.0|3B| en-US)|0D 0A|"; fast_pattern:only; http_header; urilen:159; pcre:"/\x2f[A-F0-9]{158}/U"; metadata:impact_flag red, policy balanced-ips drop, policy security-ips drop, ruleset community, service http; reference:url,www.virustotal.com/file/c49f7dbc036ad0a86df02cbbde00cb3b3fbd651d82f6c9c5a98170644374f64f/analysis/; classtype:trojan-activity; sid:25675; rev:7;)" from file /usr/local/etc/suricata/suricata_17044_mvneta2/rules/suricata.rules at line 186
13/12/2018 -- 06:27:08 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - previous keyword has a fast_pattern:only; set. Can't have relative keywords around a fast_pattern only content
13/12/2018 -- 06:27:08 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $EXTERNAL_NET $HTTP_PORTS -> $HOME_NET any (msg:"MALWARE-OTHER Win.Trojan.Zeus Spam 2013 dated zip/exe HTTP Response - potential malware download"; flow:to_client,established; content:"-2013.zip|0D 0A|"; fast_pattern:only; content:"-2013.zip|0D 0A|"; http_header; content:"-"; within:1; distance:-14; http_header; file_data; content:"-2013.exe"; content:"-"; within:1; distance:-14; metadata:impact_flag red, policy balanced-ips drop, policy security-ips drop, ruleset community, service http; reference:url,www.virustotal.com/en/file/2eff3ee6ac7f5bf85e4ebcbe51974d0708cef666581ef1385c628233614b22c0/analysis/; classtype:trojan-activity; sid:26470; rev:1;)" from file /usr/local/etc/suricata/suricata_17044_mvneta2/rules/suricata.rules at line 210
13/12/2018 -- 06:27:08 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - previous keyword has a fast_pattern:only; set. Can't have relative keywords around a fast_pattern only content
13/12/2018 -- 06:27:08 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"MALWARE-CNC Bancos fake JPG encrypted config file download"; flow:to_server,established; content:".com.br|0D 0A 0D 0A|"; fast_pattern:only; content:"/imagens/"; depth:9; http_uri; content:".jpg"; distance:0; http_uri; pcre:"/\.jpg\x20HTTP\/1\.[01]\r\nUser\x2dAgent\x3a\x20[a-z]+\r\nHost\x3a\x20[a-z0-9\x2d\x2e]+\.com\.br\r\n\r\n$/"; metadata:impact_flag red, policy balanced-ips drop, policy security-ips drop, ruleset community, service http; classtype:trojan-activity; sid:26722; rev:1;)" from file /usr/local/etc/suricata/suricata_17044_mvneta2/rules/suricata.rules at line 235
13/12/2018 -- 06:27:08 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - Signature combines packet specific matches (like dsize, flags, ttl) with stream / state matching by matching on app layer proto (like using http_* keywords).
13/12/2018 -- 06:27:08 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"MALWARE-CNC Win32/Autorun.JN variant outbound connection"; flow:to_server,established; dsize:142; urilen:8; content:"/u5.htm"; fast_pattern:only; http_uri; content:"//u5.htm"; http_raw_uri; metadata:impact_flag red, policy balanced-ips drop, policy security-ips drop, ruleset community, service http; reference:url,www.microsoft.com/security/portal/threat/encyclopedia/Entry.aspx?Name=Worm%3AWin32%2FAutorun.JN; reference:url,www.virustotal.com/en/file/36144738373c665d262bc007fceaeb9613e59ec29ea3d7424dd9f400af2c0f06/analysis/; classtype:trojan-activity; sid:26966; rev:3;)" from file /usr/local/etc/suricata/suricata_17044_mvneta2/rules/suricata.rules at line 280
13/12/2018 -- 06:27:08 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - Signature combines packet specific matches (like dsize, flags, ttl) with stream / state matching by matching on app layer proto (like using http_* keywords).
13/12/2018 -- 06:27:08 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $HOME_NET any -> $EXTERNAL_NET 80 (msg:"MALWARE-CNC Win.Trojan.Conficker variant outbound connection"; flow:to_server,established; dsize:146; urilen:1; content:"GET / HTTP/1.1|0D 0A|User-Agent: Mozilla/4.0 (compatible|3B| MSIE 7.0|3B| Windows NT 5.1|3B| Trident/4.0)|0D 0A|Host: checkip.dyndns.org|0D 0A|Cache-Control: no-cache|0D 0A 0D 0A|"; fast_pattern:only; metadata:impact_flag red, policy balanced-ips drop, policy security-ips drop, ruleset community, service http; reference:url,www.sans.org/security-resources/malwarefaq/conficker-worm.php; classtype:trojan-activity; sid:28542; rev:1;)" from file /usr/local/etc/suricata/suricata_17044_mvneta2/rules/suricata.rules at line 357
13/12/2018 -- 06:27:08 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - Signature combines packet specific matches (like dsize, flags, ttl) with stream / state matching by matching on app layer proto (like using http_* keywords).
13/12/2018 -- 06:27:08 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $HOME_NET any -> $EXTERNAL_NET 80 (msg:"MALWARE-CNC Win.Trojan.Conficker variant outbound connection"; flow:to_server,established; dsize:139; urilen:1; content:"GET / HTTP/1.1|0D 0A|User-Agent: Mozilla/4.0 (compatible|3B| MSIE 7.0|3B| Windows NT 5.1|3B| Trident/4.0)|0D 0A|Host: www.ask.com|0D 0A|Cache-Control: no-cache|0D 0A 0D 0A|"; fast_pattern:only; metadata:impact_flag red, policy balanced-ips drop, policy security-ips drop, ruleset community, service http; reference:url,www.sans.org/security-resources/malwarefaq/conficker-worm.php; classtype:trojan-activity; sid:28543; rev:1;)" from file /usr/local/etc/suricata/suricata_17044_mvneta2/rules/suricata.rules at line 358
13/12/2018 -- 06:27:08 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - previous keyword has a fast_pattern:only; set. Can't have relative keywords around a fast_pattern only content
13/12/2018 -- 06:27:08 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"MALWARE-CNC Win.Trojan.Injector variant outbound connection"; flow:to_server,established; urilen:9; content:"/load.exe HTTP/1.1|0D 0A|User-Agent: Mozilla/"; fast_pattern:only; content:"|3B 20|MSIE|20|"; http_header; content:")|0D 0A|Host: "; distance:0; http_header; content:!"Accept"; http_header; metadata:impact_flag red, policy balanced-ips drop, policy security-ips drop, ruleset community, service http; reference:url,urlquery.net/search.php?q=%5C%2Fload%5C.exe%24&type=regexp&start=2013-08-24&end=2013-11-22&max=400; reference:url,www.virustotal.com/en/file/032572ea1f34a060ecac98a8e2899dc0f2a41dff199e879050481ddd3818b4d0/analysis/; classtype:trojan-activity; sid:28807; rev:2;)" from file /usr/local/etc/suricata/suricata_17044_mvneta2/rules/suricata.rules at line 366
13/12/2018 -- 06:27:08 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - Signature combines packet specific matches (like dsize, flags, ttl) with stream / state matching by matching on app layer proto (like using http_* keywords).
13/12/2018 -- 06:27:08 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"MALWARE-CNC Win.Trojan.WEC variant outbound connection"; flow:to_server,established; dsize:69; urilen:1; content:"GET / HTTP/1.1|0D 0A|User-Agent: Mozilla/4.0|0D 0A|Host: checkip.dyndns.org|0D 0A 0D 0A|"; fast_pattern:only; metadata:impact_flag red, policy balanced-ips drop, policy security-ips drop, ruleset community, service http; reference:url,www.virustotal.com/en/file/164c792247b2822ab1dce8271a9498d3c9172ff21d36feccf83265ded1be8d0b/analysis/; classtype:trojan-activity; sid:29882; rev:2;)" from file /usr/local/etc/suricata/suricata_17044_mvneta2/rules/suricata.rules at line 421
13/12/2018 -- 06:27:08 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - previous keyword has a fast_pattern:only; set. Can't have relative keywords around a fast_pattern only content
13/12/2018 -- 06:27:08 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"MALWARE-CNC Win.Trojan.Bancos variant outbound connection"; flow:to_server,established; content:"Content-Length: 166"; content:".php HTTP/1.1|0D 0A|Accept: */*|0D 0A|Content-Type: application/x-www-form-urlencoded|0D 0A|User-Agent: Mozilla/5.0 (Windows NT 6.1|3B| Trident/7.0|3B| rv:11.0) like Gecko|0D 0A|Host: "; fast_pattern:only; content:"v="; depth:2; http_client_body; content:"&c="; within:7; http_client_body; pcre:"/\x3d\x3d$/P"; metadata:impact_flag red, policy balanced-ips drop, policy security-ips drop, ruleset community, service http; reference:url,www.virustotal.com/en/file/51540d7c9a4bc2a430bc50c85cf9cec5c6f2bb755e800a3f3575ba34fe5f008c/analysis; classtype:trojan-activity; sid:29895; rev:2;)" from file /usr/local/etc/suricata/suricata_17044_mvneta2/rules/suricata.rules at line 424
13/12/2018 -- 06:27:08 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - "http_header" keyword seen with a sticky buffer still set.  Reset sticky buffer with pkt_data before using the modifier.
13/12/2018 -- 06:27:08 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $EXTERNAL_NET $HTTP_PORTS -> $HOME_NET any (msg:"MALWARE-CNC Win.Trojan.Sodebral HTTP Response attempt"; flow:to_client,established; file_data; dsize:<194; content:"INTERNACIONAL"; depth:13; content:!"Content-Length"; http_header; content:"Transfer-Encoding: chunked"; http_header; metadata:impact_flag red, policy balanced-ips drop, policy security-ips drop, ruleset community, service http; reference:url,www.virustotal.com/en/file/e0290c3900445dc00ca24888924e37fa6ac17ecaddc60591e32b81536b9f5ef7/analysis/; classtype:trojan-activity; sid:32607; rev:1;)" from file /usr/local/etc/suricata/suricata_17044_mvneta2/rules/suricata.rules at line 583
13/12/2018 -- 06:27:08 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - "http_header" keyword seen with a sticky buffer still set.  Reset sticky buffer with pkt_data before using the modifier.
13/12/2018 -- 06:27:08 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $EXTERNAL_NET $HTTP_PORTS -> $HOME_NET any (msg:"MALWARE-CNC Win.Trojan.Sodebral HTTP Response attempt"; flow:to_client,established; file_data; dsize:<194; content:"BRASIL"; depth:6; content:!"Content-Length"; http_header; content:"Transfer-Encoding: chunked"; http_header; metadata:impact_flag red, policy balanced-ips drop, policy security-ips drop, ruleset community, service http; reference:url,www.virustotal.com/en/file/e0290c3900445dc00ca24888924e37fa6ac17ecaddc60591e32b81536b9f5ef7/analysis/; classtype:trojan-activity; sid:32608; rev:1;)" from file /usr/local/etc/suricata/suricata_17044_mvneta2/rules/suricata.rules at line 584
13/12/2018 -- 06:27:08 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - Signature combines packet specific matches (like dsize, flags, ttl) with stream / state matching by matching on app layer proto (like using http_* keywords).
13/12/2018 -- 06:27:08 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"MALWARE-CNC Win.Agent.BHHK variant outbound connection"; flow:to_server,established; dsize:136; urilen:1; content:"GET / HTTP/1.1|0D 0A|User-Agent: Mozilla/4.0 (compatible|3B| MSIE 7.0|3B| Windows NT 6.0)|0D 0A|Host: windowsupdate.microsoft.com|0D 0A|Connection: Close|0D 0A 0D 0A|"; fast_pattern:only; content:!"Accept"; http_header; metadata:impact_flag red, policy balanced-ips drop, policy security-ips drop, ruleset community, service http; reference:url,www.virustotal.com/en/file/cab1fffe7a34b5bb7dab2cacd406cf15628d835ab63502d28df78c2faeaad366/analysis/1421677054/; classtype:trojan-activity; sid:33227; rev:2;)" from file /usr/local/etc/suricata/suricata_17044_mvneta2/rules/suricata.rules at line 615
13/12/2018 -- 06:27:08 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - Signature combines packet specific matches (like dsize, flags, ttl) with stream / state matching by matching on app layer proto (like using http_* keywords).
13/12/2018 -- 06:27:08 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"MALWARE-CNC Win.Trojan.FileEncoder IP geolocation checkin attempt"; flow:to_server,established; dsize:214; urilen:1; content:"GET / HTTP/1.1|0D 0A|User-Agent: Mozilla/4.0 (compatible|3B| MSIE 6.0|3B| Windows NT 5.1|3B| SV1|3B| .NET4.0C|3B| .NET4.0E|3B| .NET CLR 2.0.50727|3B| .NET CLR 3.0.4506.2152|3B| .NET CLR 3.5.30729)|0D 0A|Host: ip-addr.es|0D 0A|Cache-Control: no-cache|0D 0A 0D 0A|"; fast_pattern:only; metadata:impact_flag red, policy balanced-ips drop, policy security-ips drop, ruleset community, service http; reference:url,www.virustotal.com/en/file/17edf82c40df6c7268191def7cbff6e60e78d7388018408800d42581567f78cf/analysis/; classtype:trojan-activity; sid:33449; rev:1;)" from file /usr/local/etc/suricata/suricata_17044_mvneta2/rules/suricata.rules at line 620
13/12/2018 -- 06:27:08 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - previous keyword has a fast_pattern:only; set. Can't have relative keywords around a fast_pattern only content
13/12/2018 -- 06:27:08 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"MALWARE-CNC Win.Trojan.GateKeylogger initial exfiltration attempt"; flow:to_server,established; content:"/gate.php"; fast_pattern:only; content:"pc="; nocase; http_client_body; content:"&admin="; distance:0; nocase; http_client_body; content:"&os="; distance:0; nocase; http_client_body; content:"&hid="; distance:0; nocase; http_client_body; content:"&arc="; distance:0; nocase; http_client_body; content:"User-Agent|3A 20|"; http_header; pcre:"/User-Agent\x3a\x20[A-F0-9]{32}\x0d\x0a/H"; metadata:impact_flag red, policy balanced-ips drop, policy security-ips drop, ruleset community, service http; reference:url,www.virustotal.com/en/file/77c802db1731fa8dae1b03d978f89b046309adfa1237b1497a69ccb9c2d82c16/analysis/1459520578/; classtype:trojan-activity; sid:38562; rev:2;)" from file /usr/local/etc/suricata/suricata_17044_mvneta2/rules/suricata.rules at line 742
13/12/2018 -- 06:27:08 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - "http_stat_code" keyword seen with a sticky buffer still set.  Reset sticky buffer with pkt_data before using the modifier.
13/12/2018 -- 06:27:08 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $EXTERNAL_NET $HTTP_PORTS -> $HOME_NET any (msg:"MALWARE-CNC Win.Trojan.GateKeylogger fake 404 response"; flow:to_client,established; file_data; content:"200"; http_stat_code; content:"OK"; http_stat_msg; content:">404 Not Found<"; fast_pattern:only; content:" requested URL / was not found "; metadata:impact_flag red, policy balanced-ips drop, policy security-ips drop, ruleset community, service http; reference:url,www.virustotal.com/en/file/77c802db1731fa8dae1b03d978f89b046309adfa1237b1497a69ccb9c2d82c16/analysis/1459520578/; classtype:trojan-activity; sid:38563; rev:1;)" from file /usr/local/etc/suricata/suricata_17044_mvneta2/rules/suricata.rules at line 743
13/12/2018 -- 06:27:09 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - previous keyword has a fast_pattern:only; set. Can't have relative keywords around a fast_pattern only content
13/12/2018 -- 06:27:09 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"MALWARE-CNC Osx.Trojan.OceanLotus outbound connection attempt"; flow:to_server,established; content:"/sigstore.db?"; fast_pattern:only; content:"k="; http_uri; content:"?q="; distance:0; http_uri; metadata:impact_flag red, policy balanced-ips drop, policy security-ips drop, ruleset community, service http; reference:url,www.alienvault.com/blogs/labs-research/oceanlotus-for-os-x-an-application-bundle-pretending-to-be-an-adobe-flash-update; classtype:trojan-activity; sid:45400; rev:1;)" from file /usr/local/etc/suricata/suricata_17044_mvneta2/rules/suricata.rules at line 902
13/12/2018 -- 06:27:13 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert http" from file /usr/local/etc/suricata/suricata_17044_mvneta2/rules/suricata.rules at line 3914
13/12/2018 -- 06:27:14 - <Info> -- 2 rule files processed. 3951 rules successfully loaded, 17 rules failed
13/12/2018 -- 06:27:14 - <Warning> -- [ERRCODE: SC_ERR_EVENT_ENGINE(210)] - can't suppress sid 2018959, gid 1: unknown rule
13/12/2018 -- 06:27:14 - <Info> -- Threshold config parsed: 1 rule(s) found
13/12/2018 -- 06:27:14 - <Info> -- 3953 signatures processed. 330 are IP-only rules, 1019 are inspecting packet payload, 2789 inspect application layer, 103 are decoder event only
13/12/2018 -- 06:27:15 - <Info> -- alert-pf -> Creating automatic firewall interface IP address Pass List.
13/12/2018 -- 06:27:15 - <Info> -- alert-pf -> adding firewall interface mvneta1 IPv6 address fe80:0000:0000:0000:0208:a2ff:fe0d:f1dc to automatic interface IP Pass List.
13/12/2018 -- 06:27:15 - <Info> -- alert-pf -> adding firewall interface mvneta1 IPv4 address 172.16.0.240 to automatic interface IP Pass List.
13/12/2018 -- 06:27:15 - <Info> -- alert-pf -> adding firewall interface mvneta1 IPv4 address 10.10.10.1 to automatic interface IP Pass List.
13/12/2018 -- 06:27:15 - <Info> -- alert-pf -> adding firewall interface mvneta2 IPv6 address fe80:0000:0000:0000:0208:a2ff:fe0d:f1dd to automatic interface IP Pass List.
13/12/2018 -- 06:27:15 - <Info> -- alert-pf -> adding firewall interface mvneta2 IPv4 address  to automatic interface IP Pass List.
13/12/2018 -- 06:27:15 - <Info> -- alert-pf -> adding firewall interface lo0 IPv6 address 0000:0000:0000:0000:0000:0000:0000:0001 to automatic interface IP Pass List.
13/12/2018 -- 06:27:15 - <Info> -- alert-pf -> adding firewall interface lo0 IPv6 address fe80:0000:0000:0000:0000:0000:0000:0001 to automatic interface IP Pass List.
13/12/2018 -- 06:27:15 - <Info> -- alert-pf -> adding firewall interface lo0 IPv4 address 127.0.0.1 to automatic interface IP Pass List.
13/12/2018 -- 06:27:15 - <Info> -- alert-pf output device (regular) initialized: block.log
13/12/2018 -- 06:27:15 - <Info> -- alert-pf -> Pass List /usr/local/etc/suricata/suricata_17044_mvneta2/passlist parsed: 12 IP addresses loaded.
13/12/2018 -- 06:27:15 - <Info> -- alert-pf -> Created firewall interface IP change monitor thread for auto-whitelisting of firewall interface IP addresses.
13/12/2018 -- 06:27:15 - <Info> -- alert-pf output initialized, pf-table=snort2c  block-ip=both  kill-state=on  block-drops-only=off
13/12/2018 -- 06:27:15 - <Info> -- fast output device (regular) initialized: alerts.log
13/12/2018 -- 06:27:15 - <Info> -- http-log output device (regular) initialized: http.log
13/12/2018 -- 06:27:15 - <Info> -- stats output device (regular) initialized: stats.log
13/12/2018 -- 06:27:15 - <Info> -- dns-log output device (regular) initialized: dns.log
13/12/2018 -- 06:27:15 - <Info> -- dns-log output device (regular) initialized: dns.log
13/12/2018 -- 06:27:15 - <Info> -- Using 1 live device(s).
13/12/2018 -- 06:27:15 - <Info> -- alert-pf -> Firewall interface IP address change notification monitoring thread started.
13/12/2018 -- 06:27:15 - <Info> -- using interface mvneta2
13/12/2018 -- 06:27:15 - <Info> -- Running in 'auto' checksum mode. Detection of interface state will require 1000 packets.
13/12/2018 -- 06:27:15 - <Info> -- Set snaplen to 1518 for 'mvneta2'
13/12/2018 -- 06:27:15 - <Info> -- RunModeIdsPcapAutoFp initialised
13/12/2018 -- 06:27:15 - <Notice> -- all 3 packet processing threads, 4 management threads initialized, engine started.
13/12/2018 -- 06:28:25 - <Info> -- No packets with invalid checksum, assuming checksum offloading is NOT used

Thanks for your help I will dig through this and find out why these 17 are a problem. It looks like the file reference.config is not being created correctly during the installation and subsequent reinstalls when upgrading to the new version of PFSENSE.

s0m3f00l

FYI,

This continues to be a problem. Again this update did not correctly copy the sample config to the folder.

Is anyone looking into this on the sg3100?

bmeeks

I have been unable to reproduce your issue. I have installed the Suricata package from scratch on several clean virtual machines and it works every time. I have also tested with an existing package configuration installing onto a clean VM (meaning with all previous traces of Suricata wiped from the disk by deleting all the directories and files). I have done this with just the Emerging Threats Open rules enabled, with just the Snort Subscriber rules enabled, and with both sets of rules enabled. The installed package worked each time with no errors (other than the expected errors you get when loading some of the Snort rules on Suricata; remember a number of Snort rules will not work on Suricata because Suricata does not support the same keywords and rule options as Snort).

s0m3f00l

Well what should be my next steps? Should I rebuild The sg3100? Or would you suggest reinstalling the package from scratch? Either way how do I ensure I keep the suricata rules and settings I currently have.

bmeeks

@s0m3f00l said in Suricata Not logging signature matches | | Suricata 4.0.13_11 with pfsense 2.4.4-RELEASE-p1 (arm):

Well what should be my next steps? Should I rebuild The sg3100? Or would you suggest reinstalling the package from scratch? Either way how do I ensure I keep the suricata rules and settings I currently have.

There is no need to rebuild your SG-3100 appliance. But there may be a need to be sure the Suricata slate is as clean as possible. We can try some re-install steps that will still preserve your existing settings, but first I need the answer to a question.

I've corresponded with several users here in the forums, and I recall at least one and maybe more had tried to use the Snort3 rules with Suricata. That will definitely hose things up. Were you one of those users? Have you tried to use the Snort3 rules? If so, you need to be sure your GLOBAL SETTINGS parameter for the Snort rules is set to this file: snortrules-snapshot-29120.tar.gz and not one with snort3 in the name. You CANNOT use the file snortrules-snapshot-3000.tar.gz. That file will only work with Snort3 (but not Snort 2.9.x and not Suricata!).

At the bottom of the GLOBAL SETTINGS tab is a checkbox (that is checked by default) for saving settings between package removals and re-installs. Leave that box checked (or check it if you ever un-checked it in the past). This will preserve your Suricata setup.

Now completely remove Suricata package. Click the trash can icon next to the package on the SYSTEM > PACKAGE MANAGER screen on the Installed Packages tab to uninstall Suricata. Wait for the package removal to complete! The progress bar will turn green when the removal is finished.

Now open a command prompt on the firewall itself (a shell prompt). Look for any suricata directories in these paths:

/usr/local/etc/suricata
/usr/local/pkg/suricata
/usr/local/www/suricata

If you see any of the above directories, remove them with this command:

rm -rf /usr/local/xxx/suricata

replace the "xxx" with the part of the path from the list above (e.g., "etc", "pkg", or "www").

After completing the steps above, install the Suricata package again by going to SYSTEM > PACKAGE MANAGER and finding Suricata on the Available Packages tab. Let the install fully complete before leaving the page or doing anything on the firewall. It will take several seconds, and possibly up to a couple of minutes, because it will download any enabled rules you had. Let this task complete! Wait for the green bar and the "Success" message before leaving the page.

Now see if Suricata started properly. You can examine the suricata.log file for the interface to check for errors. You should see no errors related to missing references. You may see other rule parsing errors if you enable and use any of the Snort rules. Those errors are normal because Suricata does not support several of the newer Snort rule options.

Report back with your results.

s0m3f00l

@bmeeks , It appears to have worked. After deleting the package from the GUI "/usr/local/etc/suricata" was left behind. I removed the directory and its contents and then everything reinstalled correctly without intervention. I will wait for the next update and test again.

@bmeeks, thanks for putting up with me sir. You are extraordinarily helpful and supremely knowledgeable about suricata and snort on PFSENSE, and the community would be lost without your input.

Thanks, again.

bmeeks

@s0m3f00l said in Suricata Not logging signature matches | | Suricata 4.0.13_11 with pfsense 2.4.4-RELEASE-p1 (arm):

@bmeeks , It appears to have worked. After deleting the package from the GUI "/usr/local/etc/suricata" was left behind. I removed the directory and its contents and then everything reinstalled correctly without intervention. I will wait for the next update and test again.

@bmeeks, thanks for putting up with me sir. You are extraordinarily helpful and supremely knowledgeable about suricata and snort on PFSENSE, and the community would be lost without your input.

Thanks, again.

Glad you got it working. There was likely a file that had been modified and contained "bad content" being left there. When pkg removes software, it compares the md5 hash of the file being removed to the hash the file had when installed. If different, pkg assumes the user modified the file so it leaves it alone. So simply removing and re-installing the package was not removing that modified file. Manually removing the directory and file gets rid of the malformed file. My guess is that it was a different version of the references.config file that was being left behind. That file then would get used with the next installation, so the problem persisted for you.