Suricata bug
-
2.2-RC (amd64)
built on Tue Dec 23 05:11:07 CST 2014Still crashing
Need some additional information such as:
(1) was this a working 2.0.4 install and just recently broke with perhaps an overnight rule update?
(2) or was this a working 2.0.3 install that broke upon the upgrade to 2.0.4?
If #1 is true, then a new or recently enabled rule is at fault. You would need to find and disable it until it is fixed by the rule author. Another possibility is you are experiencing the random LibHTP segfault bug that is reported to be fixed in the new 2.0.5 release of the Suricata binary. I am currently working on getting that new version ready for pfSense, but it will be a few more days.
Bill
-
I have tested the latest Suricata update two different ways and cannot reproduce this problem. I first performed an upgrade of the package on a current 2.2-RC VM. Next, I wiped the Suricata configuration completely and performed a clean install with no previous configuration. In both cases Suricata performed as expected.
The tests were performed on a December 23rd snapshot of pfSense 2.2-RC on a virtual machine.
Bill
-
It is definitely working now 2.1.5 bare metal
Unfortunately I know nothing of previous versions -
I am ready for more instructed questions
-
Different domains maybe ? Mine is xxxxxxxxx.xxxxxxxxx.xxxxx
-
It is definitely working now 2.1.5 bare metal
This statement confuses me unless it is a typo. Are you saying it is working now, or did you leave out the word "not" in the statement?
When you say you know nothing about previous versions, does that none were installed, or did you inherit this firewall and don't know what might have been installed previously?
Bill
-
Whoa ! The reports are from test system, production runs 2.1.5
-
24/12/2014 – 20:20:19 - <error>- [ERRCODE: SC_ERR_FATAL(171)] - failed to open file: /usr/local/etc/suricata//suricata.yaml: No such file or directory
Double slash troubles ? /usr/local/etc/suricata//suricata.yaml</error>
-
24/12/2014 – 20:20:19 - <error>- [ERRCODE: SC_ERR_FATAL(171)] - failed to open file: /usr/local/etc/suricata//suricata.yaml: No such file or directory
Double slash troubles ? /usr/local/etc/suricata//suricata.yaml</error>
There is something seriously wrong with the config on the box throwing this error. That is not even the correct path. It should be /usr/pbi/suricata_amd64/…
Have you tried totally wiping this box and reinstalling pfSense 2.2 from scratch on it using the full-install image?
Bill
-
This is exactly what I did 4 days ago. Gonna give it another fresh install.
-
I'm also seeing:
kernel: pid 22127 (suricata), uid 0: exited on signal 4 (core dumped)When I try to start suricata.
Where exactly are you seeing this:
24/12/2014 – 20:20:19 - <error>- [ERRCODE: SC_ERR_FATAL(171)] - failed to open file: /usr/local/etc/suricata//suricata.yaml: No such file or directory</error> -
8 Shell
suricata -T
-
I have confirmed that on some virtual machine installs Suricata will core dump on an illegal instruction. The problem happens due, I think, to some kind of bug in the C compiler on FreeBSD 10.1. I have not confirmed this.
For you folks seeing a Suricata core dump, can you try running this from the command line and post back what you get?
suricata --build-info
Normally that line should print out a series of lines providing the build information and compiled options. If you are experiencing something else, hopefully it will print a little bit of a hint in the error message (like the "illegal instruction" message I see on some VMware virtual machines).
Bill
-
This is Suricata version 2.0.4 RELEASE
Features: IPFW PCAP_SET_BUFF LIBPCAP_VERSION_MAJOR=1 HAVE_PACKET_FANOUT LIBNET1.1 HAVE_HTP_URI_NORMALIZE_HOOK PCRE_JIT HAVE_NSS HAVE_LIBJANSSON
SIMD support: SSE_3
Atomic intrisics: 1 2 4 8 16 byte(s)
64-bits, Little-endian architecture
GCC version 4.2.1 Compatible FreeBSD Clang 3.4 (tags/RELEASE_34/final 197956), C version 199901
compiled with -fstack-protector
compiled with _FORTIFY_SOURCE=2
L1 cache line size (CLS)=64
compiled with LibHTP v0.5.15, linked against LibHTP v0.5.15
Suricata Configuration:
AF_PACKET support: no
PF_RING support: no
NFQueue support: no
NFLOG support: no
IPFW support: yes
DAG enabled: no
Napatech enabled: no
Unix socket enabled: yes
Detection enabled: yeslibnss support: yes
libnspr support: yes
libjansson support: yes
Prelude support: no
PCRE jit: yes
LUA support: no
libluajit: no
libgeoip: yes
Non-bundled htp: no
Old barnyard2 support: no
CUDA enabled: noSuricatasc install: no
Unit tests enabled: no
Debug output enabled: no
Debug validation enabled: no
Profiling enabled: no
Profiling locks enabled: no
Coccinelle / spatch: noGeneric build parameters:
Installation prefix (–prefix): /usr/local
Configuration directory (--sysconfdir): /usr/local/etc/suricata/
Log directory (--localstatedir) : /var/log/suricata/Host: amd64-portbld-freebsd10.0
GCC binary: cc
GCC Protect enabled: yes
GCC march native enabled: yes
GCC Profile enabled: nosuricata -T provides the same thing:
25/12/2014 -- 10:37:22 - <error>- [ERRCODE: SC_ERR_FATAL(171)] - failed to open file: /usr/local/etc/suricata//suricata.yaml: No such file or directory</error> -
suricata -T provides the same thing:
25/12/2014 – 10:37:22 - <error>- [ERRCODE: SC_ERR_FATAL(171)] - failed to open file: /usr/local/etc/suricata//suricata.yaml: No such file or directory</error>On pfSense, you can't just run "suricata -T" without also providing the proper YAML config file path. PBI packages on pfSense are configured to use a special path.
How did you install Suricata on this box? Was is it via System…Packages...Available Packages?
What prints in the system log when you attempt to start the Suricata service from Services…Suricata by clicking the red X icon?
~~Also, the paths for these settings are incorrect:
Installation prefix (–prefix): /usr/local
Configuration directory (--sysconfdir): /usr/local/etc/suricata/They should read /usr/pbi/suricata_amd64/ instead of /usr/local.~~
Last edit to scratch the statements above … the paths are apparently different on 2.2 versus 2.1 (which I was comparing to). The /usr/local prefix is OK on 2.2 as that is what is showing on my currently working 2.2-RC virtual machine.
Bill
-
Yes installed from system packages.
Ack #133 (Req-Sent)
Dec 25 11:59:19 php-fpm[92494]: /suricata/suricata_interfaces.php: Toggle (suricata starting) for WAN(WAN)…
Dec 25 11:59:19 php-fpm[92494]: /suricata/suricata_interfaces.php: [Suricata] Updating rules configuration for: WAN …
Dec 25 11:59:28 php-fpm[92494]: /suricata/suricata_interfaces.php: [Suricata] Enabling any flowbit-required rules for: WAN…
Dec 25 11:59:29 php-fpm[92494]: /suricata/suricata_interfaces.php: [Suricata] Building new sid-msg.map file for WAN…
Dec 25 11:59:31 suricata: 25/12/2014 -- 11:59:31 - <notice>-- This is Suricata version 2.0.4 RELEASE
Dec 25 11:59:31 barnyard2[19148]: Found pid path directive (/var/run)
Dec 25 11:59:31 barnyard2[19148]: Running in Continuous mode
Dec 25 11:59:31 barnyard2[19148]:
Dec 25 11:59:31 barnyard2[19148]: –== Initializing Barnyard2 ==--
Dec 25 11:59:31 barnyard2[19148]: Initializing Input Plugins!
Dec 25 11:59:31 barnyard2[19148]: Initializing Output Plugins!
Dec 25 11:59:31 barnyard2[19148]: Found pid path directive (/var/run)
Dec 25 11:59:31 barnyard2[19148]: +[ Signature Suppress list ]+ –--------------------------
Dec 25 11:59:31 barnyard2[19148]: +[No entry in Signature Suppress List]+
Dec 25 11:59:31 barnyard2[19148]: –-------------------------- +[ Signature Suppress list ]+
Dec 25 11:59:31 kernel: pid 18635 (suricata), uid 0: exited on signal 4 (core dumped)</notice> -
Yes installed from system packages.
Ack #133 (Req-Sent)
Dec 25 11:59:19 php-fpm[92494]: /suricata/suricata_interfaces.php: Toggle (suricata starting) for WAN(WAN)…
Dec 25 11:59:19 php-fpm[92494]: /suricata/suricata_interfaces.php: [Suricata] Updating rules configuration for: WAN …
Dec 25 11:59:28 php-fpm[92494]: /suricata/suricata_interfaces.php: [Suricata] Enabling any flowbit-required rules for: WAN…
Dec 25 11:59:29 php-fpm[92494]: /suricata/suricata_interfaces.php: [Suricata] Building new sid-msg.map file for WAN…
Dec 25 11:59:31 suricata: 25/12/2014 -- 11:59:31 - <notice>-- This is Suricata version 2.0.4 RELEASE
Dec 25 11:59:31 barnyard2[19148]: Found pid path directive (/var/run)
Dec 25 11:59:31 barnyard2[19148]: Running in Continuous mode
Dec 25 11:59:31 barnyard2[19148]:
Dec 25 11:59:31 barnyard2[19148]: –== Initializing Barnyard2 ==--
Dec 25 11:59:31 barnyard2[19148]: Initializing Input Plugins!
Dec 25 11:59:31 barnyard2[19148]: Initializing Output Plugins!
Dec 25 11:59:31 barnyard2[19148]: Found pid path directive (/var/run)
Dec 25 11:59:31 barnyard2[19148]: +[ Signature Suppress list ]+ –--------------------------
Dec 25 11:59:31 barnyard2[19148]: +[No entry in Signature Suppress List]+
Dec 25 11:59:31 barnyard2[19148]: –-------------------------- +[ Signature Suppress list ]+
Dec 25 11:59:31 kernel: pid 18635 (suricata), uid 0: exited on signal 4 (core dumped)</notice>Have you ever installed Suricata on this firewall before? In other words, is there a existing configuration?
Can you provide the output of suricata.log from the LOGS VIEW tab?
Bill
-
Before 2.0.4? Yes, I believe I installed suricata at an earlier release and let it upgrade. Tried uninstalling and re-installing but it made no difference.
25/12/2014 – 11:59:31 - <notice>-- This is Suricata version 2.0.4 RELEASE
25/12/2014 -- 11:59:31 - <info>-- CPUs/cores online: 4
25/12/2014 -- 11:59:31 - <info>-- Live rule reloads enabled
25/12/2014 -- 11:59:31 - <info>-- 'default' server has 'request-body-minimal-inspect-size' set to 33882 and 'request-body-inspect-window' set to 4053 after randomization.
25/12/2014 -- 11:59:31 - <info>-- 'default' server has 'response-body-minimal-inspect-size' set to 33695 and 'response-body-inspect-window' set to 4218 after randomization.
25/12/2014 -- 11:59:31 - <info>-- HTTP memcap: 67108864
25/12/2014 -- 11:59:31 - <info>-- DNS request flood protection level: 500
25/12/2014 -- 11:59:31 - <info>-- DNS per flow memcap (state-memcap): 524288
25/12/2014 -- 11:59:31 - <info>-- DNS global memcap: 16777216</info></info></info></info></info></info></info></info></notice> -
suricata -T provides the same thing:
25/12/2014 – 10:37:22 - <error>- [ERRCODE: SC_ERR_FATAL(171)] - failed to open file: /usr/local/etc/suricata//suricata.yaml: No such file or directory</error>On pfSense, you can't just run "suricata -T" without also providing the proper YAML config file path. PBI packages on pfSense are configured to use a special path.
How did you install Suricata on this box? Was is it via System…Packages...Available Packages?
What prints in the system log when you attempt to start the Suricata service from Services…Suricata by clicking the red X icon?
~~Also, the paths for these settings are incorrect:
Installation prefix (–prefix): /usr/local
Configuration directory (--sysconfdir): /usr/local/etc/suricata/They should read /usr/pbi/suricata_amd64/ instead of /usr/local.~~
Last edit to scratch the statements above … the paths are apparently different on 2.2 versus 2.1 (which I was comparing to). The /usr/local prefix is OK on 2.2 as that is what is showing on my currently working 2.2-RC virtual machine.
Bill
Maybe that's part of the problem? On my box at least, /usr/local/etc/suricata/ doesn't even exist. If I search for suricata.yaml, the only file that I find is:
/usr/pbi/suricata-amd64/local/etc/suricata/suricata_23278_pppoe0/suricata.yaml
I'm guessing that's the problem…?
-
suricata -T provides the same thing:
25/12/2014 – 10:37:22 - <error>- [ERRCODE: SC_ERR_FATAL(171)] - failed to open file: /usr/local/etc/suricata//suricata.yaml: No such file or directory</error>On pfSense, you can't just run "suricata -T" without also providing the proper YAML config file path. PBI packages on pfSense are configured to use a special path.
How did you install Suricata on this box? Was is it via System…Packages...Available Packages?
What prints in the system log when you attempt to start the Suricata service from Services…Suricata by clicking the red X icon?
~~Also, the paths for these settings are incorrect:
Installation prefix (–prefix): /usr/local
Configuration directory (--sysconfdir): /usr/local/etc/suricata/They should read /usr/pbi/suricata_amd64/ instead of /usr/local.~~
Last edit to scratch the statements above … the paths are apparently different on 2.2 versus 2.1 (which I was comparing to). The /usr/local prefix is OK on 2.2 as that is what is showing on my currently working 2.2-RC virtual machine.
Bill
Maybe that's part of the problem? On my box at least, /usr/local/etc/suricata/ doesn't even exist. If I search for suricata.yaml, the only file that I find is:
/usr/pbi/suricata-amd64/local/etc/suricata/suricata_23278_pppoe0/suricata.yaml
I'm guessing that's the problem…?
The PBI wrappers should take care of directing things to the real path. I just noticed that you are trying to use Suricata on a PPPoE connection. That is not currently supported by the underlying binary (it's not a GUI package or pfSense limitation, it is a limitation of Suricata on FreeBSD).
By the way, here is what I would have expected as the remainder of the suricata.log contents …
25/12/2014 -- 14:46:24 - <info>-- allocated 1572864 bytes of memory for the defrag hash... 65536 buckets of size 24 25/12/2014 -- 14:46:24 - <info>-- preallocated 65535 defrag trackers of size 136 25/12/2014 -- 14:46:24 - <info>-- defrag memory usage: 10485624 bytes, maximum: 33554432 25/12/2014 -- 14:46:24 - <info>-- AutoFP mode using "Active Packets" flow load balancer 25/12/2014 -- 14:46:24 - <info>-- preallocated 1024 packets. Total memory 3508224 25/12/2014 -- 14:46:24 - <info>-- allocated 262144 bytes of memory for the host hash... 4096 buckets of size 64 25/12/2014 -- 14:46:24 - <info>-- preallocated 1000 hosts of size 80 25/12/2014 -- 14:46:24 - <info>-- host memory usage: 358144 bytes, maximum: 16777216 25/12/2014 -- 14:46:24 - <info>-- allocated 4194304 bytes of memory for the flow hash... 65536 buckets of size 64 25/12/2014 -- 14:46:24 - <info>-- preallocated 10000 flows of size 216 25/12/2014 -- 14:46:24 - <info>-- flow memory usage: 6434304 bytes, maximum: 33554432 25/12/2014 -- 14:46:24 - <info>-- IP reputation disabled 25/12/2014 -- 14:46:24 - <info>-- using magic-file /usr/share/misc/magic 25/12/2014 -- 14:46:24 - <info>-- Delayed detect disabled 25/12/2014 -- 14:46:31 - <info>-- 2 rule files processed. 16138 rules successfully loaded, 0 rules failed 25/12/2014 -- 14:46:31 - <info>-- 16146 signatures processed. 1074 are IP-only rules, 5578 are inspecting packet payload, 12087 inspect application layer, 72 are decoder event only 25/12/2014 -- 14:46:31 - <info>-- building signature grouping structure, stage 1: preprocessing rules... complete 25/12/2014 -- 14:46:32 - <info>-- building signature grouping structure, stage 2: building source address list... complete 25/12/2014 -- 14:46:39 - <info>-- building signature grouping structure, stage 3: building destination address lists... complete 25/12/2014 -- 14:46:41 - <info>-- Threshold config parsed: 0 rule(s) found 25/12/2014 -- 14:46:41 - <info>-- Core dump size is unlimited. 25/12/2014 -- 14:46:41 - <info>-- alert-pf output device (regular) initialized: block.log 25/12/2014 -- 14:46:41 - <info>-- Pass List /usr/pbi/suricata-amd64/etc/suricata/suricata_26555_em0/passlist parsed: 11 IP addresses loaded. 25/12/2014 -- 14:46:41 - <info>-- alert-pf output initialized, pf-table=snort2c block-ip=both kill-state=on 25/12/2014 -- 14:46:41 - <info>-- fast output device (regular) initialized: alerts.log 25/12/2014 -- 14:46:41 - <info>-- http-log output device (regular) initialized: http.log 25/12/2014 -- 14:46:41 - <info>-- Using 1 live device(s). 25/12/2014 -- 14:46:41 - <info>-- using interface em0 25/12/2014 -- 14:46:41 - <info>-- Running in 'auto' checksum mode. Detection of interface state will require 1000 packets. 25/12/2014 -- 14:46:41 - <info>-- Found an MTU of 1500 for 'em0' 25/12/2014 -- 14:46:41 - <info>-- Set snaplen to 1516 for 'em0' 25/12/2014 -- 14:46:41 - <info>-- RunModeIdsPcapAutoFp initialised 25/12/2014 -- 14:46:41 - <info>-- stream "prealloc-sessions": 32768 (per thread) 25/12/2014 -- 14:46:41 - <info>-- stream "memcap": 33554432 25/12/2014 -- 14:46:41 - <info>-- stream "midstream" session pickups: disabled 25/12/2014 -- 14:46:41 - <info>-- stream "async-oneside": disabled 25/12/2014 -- 14:46:41 - <info>-- stream "checksum-validation": disabled 25/12/2014 -- 14:46:41 - <info>-- stream."inline": disabled 25/12/2014 -- 14:46:41 - <info>-- stream "max-synack-queued": 5 25/12/2014 -- 14:46:41 - <info>-- stream.reassembly "memcap": 67108864 25/12/2014 -- 14:46:41 - <info>-- stream.reassembly "depth": 0 25/12/2014 -- 14:46:41 - <info>-- stream.reassembly "toserver-chunk-size": 2629 25/12/2014 -- 14:46:41 - <info>-- stream.reassembly "toclient-chunk-size": 2511 25/12/2014 -- 14:46:41 - <info>-- stream.reassembly.raw: enabled 25/12/2014 -- 14:46:41 - <info>-- segment pool: pktsize 4, prealloc 256 25/12/2014 -- 14:46:41 - <info>-- segment pool: pktsize 16, prealloc 512 25/12/2014 -- 14:46:41 - <info>-- segment pool: pktsize 112, prealloc 512 25/12/2014 -- 14:46:41 - <info>-- segment pool: pktsize 248, prealloc 512 25/12/2014 -- 14:46:41 - <info>-- segment pool: pktsize 512, prealloc 512 25/12/2014 -- 14:46:41 - <info>-- segment pool: pktsize 768, prealloc 1024 25/12/2014 -- 14:46:41 - <info>-- segment pool: pktsize 1448, prealloc 1024 25/12/2014 -- 14:46:41 - <info>-- segment pool: pktsize 65535, prealloc 128 25/12/2014 -- 14:46:41 - <info>-- stream.reassembly "chunk-prealloc": 250 25/12/2014 -- 14:46:41 - <notice>-- all 4 packet processing threads, 1 management threads initialized, engine started. 25/12/2014 -- 14:47:17 - <info>-- No packets with invalid checksum, assuming checksum offloading is NOT used</info></notice></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info>
Bill