Suricata does not start?

killmasta93

Hi,
I was wondering if someone else has had this issue before, Currently moving from snort to suricata
Unistalled snort first then installed suricata,
After configured something similar to snort but it wont start.
Something odd also is the downloaded of the packages

Starting rules update...  Time: 2020-06-16 17:21:57
	Downloading Emerging Threats Open rules md5 file...
	Emerging Threats Open rules md5 download failed.
	Server returned error code 404.
	Server error message was: 404 Not Found
	Emerging Threats Open rules will not be updated.
	Downloading Snort VRT rules md5 file...
	Checking Snort VRT rules md5 file...
	Snort VRT rules are up to date.
	Downloading Snort GPLv2 Community Rules md5 file...
	Checking Snort GPLv2 Community Rules md5 file...
	Snort GPLv2 Community Rules are up to date.
The Rules update has finished.  Time: 2020-06-16 17:21:58

I checked on the categories and only see this

I tried starting the service and the logs i dont see any errors

Jun 16 17:26:01 	php 		[Suricata] Suricata START for WAN(vtnet1)...
Jun 16 17:26:01 	php 		[Suricata] Building new sid-msg.map file for WAN...
Jun 16 17:26:01 	php 		[Suricata] Enabling any flowbit-required rules for: WAN...
Jun 16 17:26:01 	php 		[Suricata] Updating rules configuration for: WAN ...
Jun 16 17:26:01 	php-fpm 	17503 	Starting Suricata on WAN(vtnet1) per user request...
Jun 16 17:22:38 	php-fpm 	17503 	[Suricata] Building new sid-msg.map file for WAN...
Jun 16 17:22:38 	php-fpm 	17503 	[Suricata] Enabling any flowbit-required rules for: WAN...
Jun 16 17:22:38 	php-fpm 	17503 	[Suricata] Updating rules configuration for: WAN ...

bmeeks

Suricata logs to its own unique file. On the LOGS VIEW tab in Suricata select the interface where Suricata is installed (it seems to be WAN from your screenshots) and then select the suricata.log file in the drop-down selector to view. The reason it is not starting will be in there. Number one most common reason is not enough stream memcap when using a high-core count CPU. The default value is fine for most users, but if you have a high-core count CPU you will need to significantly bump up the memory limit for stream memcap. It can be found on the FLOW/STREAM tab when editing an interface.

By the way, I do not recommend running any IDS/IPS on the WAN for most users. You should generally place it on the LAN for two reasons. First, out of the box pfSense will block all unsolicited inbound traffic anyway, so putting an IPS out in front of the firewall adds pretty much nothing to security and results in it detecting and logging a bunch of Internet noise that the firewall is not passing through to your other networks anyway. Second, when running on the WAN, the IDS/IPS only sees outbound traffic after NAT is applied. So local hosts (from your LAN, for instance) can't be easily identified because all outbound local traffic will show up as having the WAN's public IP address. Putting the IDS/IPS on the LAN solves this problem.

killmasta93

Thank you again for the reply, so i checked in the logs but i got this

as for the stream memcap this is the screen shot i also have this on the general config

As for the LAN your right so i would change from WAN to LAN by the way is there any good list for the whitelist?

Thank you

bmeeks

If the suricata.log file is empty, that means the binary never even attempts to start. The very first thing it does is write some version info to that log, and that step can't really fail if the binary launches.

Based on the empty log I now suspect you are missing a required runtime dependency.

First, what is your currently installed pfSense version? If it is not 2.4.5_p1, then you CANNOT install or update packages because the packages repository has been recompiled with the latest pfSense version update. Trying to install those packages on an older pfSense version leads to broken packages. So if you are not on pfSense-2.4.5_p1, then update pfSense. After that is done, remove the Suricata package and install it again.

If you are current with your pfSense version, then the next step is determining what is missing by attempting to start Suricata from the shell prompt. Get to a shell prompt on your firewall either directly via the local console or remotely using an SSH connection. At the shell prompt, issue this command and see what the output is:

/usr/local/bin/suricata -V

Post the output from that command back here.

killmasta93

Thanks for the reply, currently running

2.4.5-DEVELOPMENT (amd64)
built on Mon Dec 03 19:17:43 EST 2018
FreeBSD 11.2-RELEASE-p4

So i guess i need to update? to a new version?

Thank you

bmeeks

@killmasta93 said in Suricata does not start?:

Thanks for the reply, currently running
2.4.5-DEVELOPMENT (amd64)
built on Mon Dec 03 19:17:43 EST 2018
FreeBSD 11.2-RELEASE-p4
So i guess i need to update? to a new version?

Thank you

Most definitely you need to update pfSense to 2.4.5_p1. However, that alone may not completely repair your packages problem. You may also need to delete the Suricata package first, update pfSense, then after the the update is completed reinstall Suricata. To be safe, I would delete the Suricata package, update pfSense, then reinstall Suricata.

You cannot update/install packages in pfSense when your installed pfSense version is not current. That's because packages are recompiled to be comptabile with the current pfSense version.

killmasta93

Thanks for the reply, you were right had to update it and worked flawless, out of curiosity what are the correct or decent rules to have running on the LAN?

on the ips policy i have it connectivity i also have checked Snort GPLv2
i have not checked any emerging rules because not really sure which was wont give me the false alarm
I also see it has the whitelist option like snort. i tried looking for whitelist for suricata but only found for snort

and on the performance this is what i have

Thank you again

bmeeks

If you run either of the Snort Subscriber Rules (paid or free registered), then there is no need to run the Snort GPLv2 rules. Those rules are already in the other paid or registered Snort packages.

The best setup for a new user is to use one of the IPS policies, but just remember that Suricata is not Snort and thus a number of Snort rules will not load in Suricata. Look in the suricata.log file for the interface and you will likely see some errors related to unrecognized syntax from the Snort rules. Those rules will not be loaded by Suricata and thus any protection they offered is not available.

For this reason, I will usually suggest new users run Snort and then either register for the free Snort rules or pay the $30 annually for the subscription, and then run the IPS Connectivity Policy for starters. That is an excellent policy that will seldom false-positive. Put Snort on the LAN and run that policy. You will still likely need to disable a few of the HTTP_INSPECT preprocessor rules. To see which ones, let Snort run for a couple of weeks in IDS-only (non-blocking mode) and keep an eye on the alerts you receive. See which are false positives and disable those rules. Likely will be several from the HTTP_INSPECT preprocessor that fit that category.

Contrary to popular belief, there is no significant performance advantage to home network users by running Suricata. It's multithreaded model is, to be honest, more hype than substance when running on typical home network hardware and with typical home network traffic loads. And with Suricata you pick up the additional penalty of it failing to load a number of Snort rules because it does not recognize all of the same keywords as Snort. Suricata is optimized for the Emerging Threats rules package, but that package is not marked with IPS Policy metadata, so you can' choose an IPS Policy when using only the ET rules.

killmasta93

Thanks for the reply, i was running snort for a while but could not get it working perfectly as to many false positives, as currently im going to still run it on the homelab then run it on my production server. But out of curiosity ive seen alots of discussions snort vs suricata in your case what have you been using?

Thank you

bmeeks

@killmasta93 said in Suricata does not start?:

Thanks for the reply, i was running snort for a while but could not get it working perfectly as to many false positives, as currently im going to still run it on the homelab then run it on my production server. But out of curiosity ive seen alots of discussions snort vs suricata in your case what have you been using?

Thank you

I have always used Snort.

kvamsi.k143

Suricata has been not running since start of this month.
Is it that the ET rules are not available? Here is the log when I try to update the rules.

Please suggest.

Starting rules update... Time: 2020-04-20 03:00:11
Downloading Emerging Threats Open rules md5 file...
Checking Emerging Threats Open rules md5 file...
Emerging Threats Open rules are up to date.
Downloading Snort VRT rules md5 file...
Checking Snort VRT rules md5 file...
There is a new set of Snort rules posted.

kvamsi.k143

to be precise, ET rules are failing since 7th June.

Starting rules update... Time: 2020-06-06 03:00:29
Downloading Emerging Threats Open rules md5 file...
Checking Emerging Threats Open rules md5 file...
There is a new set of Emerging Threats Open rules posted.
Downloading file 'emerging.rules.tar.gz'...
Done downloading rules file.
Downloading Snort VRT rules md5 file...
Checking Snort VRT rules md5 file...
There is a new set of Snort rules posted.
Downloading file 'community-rules.tar.gz'...
Done downloading rules file.
Downloading Snort GPLv2 Community Rules md5 file...
Checking Snort GPLv2 Community Rules md5 file...
There is a new set of Snort GPLv2 Community Rules posted.
Downloading file 'community-rules.tar.gz'...
Done downloading rules file.
Extracting and installing Emerging Threats Open rules...
Installation of Emerging Threats Open rules completed.
Extracting and installing Snort rules...
Installation of Snort rules completed.
Extracting and installing Snort GPLv2 Community Rules...
Installation of Snort GPLv2 Community Rules completed.
Copying new config and map files...
Updating rules configuration for: INTERNET ...
Updating rules configuration for: LAN ...
Updating rules configuration for: LAN1 ...
Restarting Suricata to activate the new set of rules...
Suricata has restarted with your new set of rules.
The Rules update has finished. Time: 2020-06-06 03:01:11

Starting rules update... Time: 2020-06-07 14:18:04
Downloading Emerging Threats Open rules md5 file...
Emerging Threats Open rules md5 download failed.
Server returned error code 404.
Server error message was: 404 Not Found
Emerging Threats Open rules will not be updated.
Downloading Snort VRT rules md5 file...
Checking Snort VRT rules md5 file...
There is a new set of Snort rules posted.
Downloading file 'community-rules.tar.gz'...
Done downloading rules file.
Downloading Snort GPLv2 Community Rules md5 file...
Checking Snort GPLv2 Community Rules md5 file...
There is a new set of Snort GPLv2 Community Rules posted.
Downloading file 'community-rules.tar.gz'...
Done downloading rules file.
Extracting and installing Snort rules...
Installation of Snort rules completed.
Extracting and installing Snort GPLv2 Community Rules...
Installation of Snort GPLv2 Community Rules completed.
Copying new config and map files...
Updating rules configuration for: INTERNET ...
Updating rules configuration for: LAN ...
Updating rules configuration for: LAN1 ...
The Rules update has finished. Time: 2020-06-07 14:18:24

bmeeks

@kvamsi-k143 said in Suricata does not start?:

to be precise, ET rules are failing since 7th June.

Starting rules update... Time: 2020-06-06 03:00:29
Downloading Emerging Threats Open rules md5 file...
Checking Emerging Threats Open rules md5 file...
There is a new set of Emerging Threats Open rules posted.
Downloading file 'emerging.rules.tar.gz'...
Done downloading rules file.
Downloading Snort VRT rules md5 file...
Checking Snort VRT rules md5 file...
There is a new set of Snort rules posted.
Downloading file 'community-rules.tar.gz'...
Done downloading rules file.
Downloading Snort GPLv2 Community Rules md5 file...
Checking Snort GPLv2 Community Rules md5 file...
There is a new set of Snort GPLv2 Community Rules posted.
Downloading file 'community-rules.tar.gz'...
Done downloading rules file.
Extracting and installing Emerging Threats Open rules...
Installation of Emerging Threats Open rules completed.
Extracting and installing Snort rules...
Installation of Snort rules completed.
Extracting and installing Snort GPLv2 Community Rules...
Installation of Snort GPLv2 Community Rules completed.
Copying new config and map files...
Updating rules configuration for: INTERNET ...
Updating rules configuration for: LAN ...
Updating rules configuration for: LAN1 ...
Restarting Suricata to activate the new set of rules...
Suricata has restarted with your new set of rules.
The Rules update has finished. Time: 2020-06-06 03:01:11

Starting rules update... Time: 2020-06-07 14:18:04
Downloading Emerging Threats Open rules md5 file...
Emerging Threats Open rules md5 download failed.
Server returned error code 404.
Server error message was: 404 Not Found
Emerging Threats Open rules will not be updated.
Downloading Snort VRT rules md5 file...
Checking Snort VRT rules md5 file...
There is a new set of Snort rules posted.
Downloading file 'community-rules.tar.gz'...
Done downloading rules file.
Downloading Snort GPLv2 Community Rules md5 file...
Checking Snort GPLv2 Community Rules md5 file...
There is a new set of Snort GPLv2 Community Rules posted.
Downloading file 'community-rules.tar.gz'...
Done downloading rules file.
Extracting and installing Snort rules...
Installation of Snort rules completed.
Extracting and installing Snort GPLv2 Community Rules...
Installation of Snort GPLv2 Community Rules completed.
Copying new config and map files...
Updating rules configuration for: INTERNET ...
Updating rules configuration for: LAN ...
Updating rules configuration for: LAN1 ...
The Rules update has finished. Time: 2020-06-07 14:18:24

I just installed the Suricata 5.0.2_3 package on a virtual machine this morning with no issues at all. The Emerging Threats Open rules downloaded and installed just fine. Here is the install log from the package installation --

[18/18] Installing pfSense-pkg-suricata-5.0.2_3...
[18/18] Extracting pfSense-pkg-suricata-5.0.2_3: .......... done
Saving updated package information...
done.
Loading package configuration... done.
Configuring package components...
Loading package instructions...
Custom commands...
Executing custom_php_install_command()...Saved settings detected...
Migrating settings to new configuration... done.
Downloading Emerging Threats Open rules md5 file... done.
There is a new set of Emerging Threats Open rules posted. Downloading... done.
Downloading Snort VRT rules md5 file... done.
There is a new set of Snort rules posted. Downloading... done.
Downloading Snort GPLv2 Community Rules md5 file... done.
There is a new set of Snort GPLv2 Community Rules posted. Downloading... done.
Installing Emerging Threats Open rules... done.
Installing Snort rules... done.
Installing Snort GPLv2 Community Rules... done.
Updating rules configuration for: WAN ... done.
Updating rules configuration for: OPT1 ... done.
Updating rules configuration for: LAN ... done.
Cleaning up after rules extraction... done.
The Rules update has finished.
Generating suricata.yaml configuration file from saved settings.
Generating YAML configuration file for WAN... done.
Generating YAML configuration file for OPT1... done.
Generating YAML configuration file for LAN... done.
Finished rebuilding Suricata configuration from saved settings.

And here is the Updates Log entry created during the package installation --

Starting rules update...  Time: 2020-06-29 07:53:45
	Downloading Emerging Threats Open rules md5 file...
	Checking Emerging Threats Open rules md5 file...
	There is a new set of Emerging Threats Open rules posted.
	Downloading file 'emerging.rules.tar.gz'...
	Done downloading rules file.
	Downloading Snort VRT rules md5 file...
	Checking Snort VRT rules md5 file...
	There is a new set of Snort rules posted.
	Downloading file 'snortrules-snapshot-29151.tar.gz'...
	Done downloading rules file.
	Downloading Snort GPLv2 Community Rules md5 file...
	Checking Snort GPLv2 Community Rules md5 file...
	There is a new set of Snort GPLv2 Community Rules posted.
	Downloading file 'community-rules.tar.gz'...
	Done downloading rules file.
	Extracting and installing Emerging Threats Open rules...
	Installation of Emerging Threats Open rules completed.
	Extracting and installing Snort rules...
	Installation of Snort rules completed.
	Extracting and installing Snort GPLv2 Community Rules...
	Installation of Snort GPLv2 Community Rules completed.
	Copying new config and map files...
	Updating rules configuration for: WAN ...
	Updating rules configuration for: OPT1 ...
	Updating rules configuration for: LAN ...
The Rules update has finished.  Time: 2020-06-29 07:54:10

So the package code seems to be working fine and the URL is up and available. Perhaps you have something on your end interfering? Some common issues I've seen in the past are problems with Squid or Squidguard (if installed) and rarely a pfBlockerNG IP list might cause a problem (although that was more common with Snort rules as they are hosted on AWS). The base URL is https://rules.emergingthreats.net/open/. And then depending on which Suricata version branch you are running, either the suricata-4.0 or suricata-5.0 directories.

What version of Suricata are you running and what type of hardware do you have?

kvamsi.k143

@bmeeks Suricata is running on 5.0.2_3.
Yes, I do have Squidguard and pfBlockerNG running on my pfSense box.

Just to be sure added the link in the Global Settings>Custom URL, still the same issue.
I also tried to re-install the Suricata package. Any other log I can provide you to investigate this please?

bmeeks

@kvamsi-k143 said in Suricata does not start?:

@bmeeks Suricata is running on 5.0.2_3.
Yes, I do have Squidguard and pfBlockerNG running on my pfSense box.

Just to be sure added the link in the Global Settings>Custom URL, still the same issue.
I also tried to re-install the Suricata package. Any other log I can provide you to investigate this please?

Remove or disable Squidguard and then see what happens with the download.

There is nothing wrong with the Suricata package code. If there were, thousands of users would be here complaining. The issue is installed packages interfering with each other in your particular setup. My first guess for the culprit is Squidguard because it tries to police web traffic, and the rules download is pure web traffic -- SSL in the case of the Snort and Emerging Threats rules.

When you have a package like Squidguard installed and you experience any kind of problem reaching a URL, the very first thing you should do is disable Squidguard and test again. Does the problem go away? If so, then you found your cause. If not, try the same thing with the next package you have installed (pfBlockerNG or Suricata). At some point you will find which package is blocking the download. This is basic troubleshooting 101. When you have a number of packages installed that all will do blocking in some form or other, it is highly probable that when you have download difficulties one of those "blocking" packages is the cause.

kvamsi.k143

@bmeeks I tried with disabling Squidguard and pfBlockerNG and still no luck.

BTW.. here is what I found if that helps..

[2.4.5-RELEASE][admin@bheema.kandulaz.com]/root: ps -ax | grep suricata
57745 0 S+ 0:00.00 grep suricata
[2.4.5-RELEASE][admin@bheema.kandulaz.com]/root: suricata -V
Shared object "libluajit-5.1.so.2" not found, required by "suricata"
[2.4.5-RELEASE][admin@bheema.kandulaz.com]/root: /usr/local/etc/rc.d
/usr/local/etc/rc.d: Permission denied.
[2.4.5-RELEASE][admin@bheema.kandulaz.com]/root: /usr/local/etc/rc.d/suricata.sh start
ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/graphviz /usr/local/lib/ipsec /usr/local/lib/mysql /usr/local/lib/nss /usr/local/lib/perl5/5.30/mach/CORE
32-bit compatibility ldconfig path:

bmeeks

@kvamsi-k143 said in Suricata does not start?:

@bmeeks I tried with disabling Squidguard and pfBlockerNG and still no luck.

BTW.. here is what I found if that helps..

[2.4.5-RELEASE][admin@bheema.kandulaz.com]/root: ps -ax | grep suricata
57745 0 S+ 0:00.00 grep suricata
[2.4.5-RELEASE][admin@bheema.kandulaz.com]/root: suricata -V
Shared object "libluajit-5.1.so.2" not found, required by "suricata"
[2.4.5-RELEASE][admin@bheema.kandulaz.com]/root: /usr/local/etc/rc.d
/usr/local/etc/rc.d: Permission denied.
[2.4.5-RELEASE][admin@bheema.kandulaz.com]/root: /usr/local/etc/rc.d/suricata.sh start
ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/graphviz /usr/local/lib/ipsec /usr/local/lib/mysql /usr/local/lib/nss /usr/local/lib/perl5/5.30/mach/CORE
32-bit compatibility ldconfig path:

How did we go suddenly from "Suricata will not download Emerging Threats rules" to "Suricata won't start"?

The package configuration on your firewall is broken. What order did you perform the last upgrade? Did you update pfSense first and only after that install package updates, or did you install or update packages AFTER pfSense 2.4.5 came out but BEFORE you upgraded your firewall? If you installed or updated a package before you upgraded pfSense to 2.4.5, then you have a broken package system. I can tell that because the libluajit-5.1.so.2 library is not used by Suricata on pfSense 2.4.5. The library luajit-openresty is used instead. The fact your Suricata install is trying to use the older non-existent library tells me your upgrade sequence was likely not correct.

At this point, my suggestion is to perform a backup of your configuration, wipe pfSense off your hardware and install again from scratch with the current 2.4.5_p1 image file. As part of the install you can import your configuration backup. Instructions are in the Netgate docs.

kvamsi.k143

@bmeeks
Thanks for your time in helping me on this. There are two issues on my pfSense box. 1) ET not updating and 2) Suricata services not running on the interfaces.

BTW, I did update pfSense before Suricata, I am aware of the catch.

I did check the other thread while investigating. That is when I found issue with "libluajit-5.1.so.2" not found, required by "suricata".

Post your suggestion, I took time to flash the pfSense from scratch after taking a backup of the config. Thanks to the documentation. All was up and running in just a couple of hours, including installation of all packages.

I am overwhelmed with your response. Kudos to you..!!
Owe one for you mate...!!!

bmeeks

@kvamsi-k143 said in Suricata does not start?:

@bmeeks
Thanks for your time in helping me on this. There are two issues on my pfSense box. 1) ET not updating and 2) Suricata services not running on the interfaces.

BTW, I did update pfSense before Suricata, I am aware of the catch.

I did check the other thread while investigating. That is when I found issue with "libluajit-5.1.so.2" not found, required by "suricata".

Post your suggestion, I took time to flash the pfSense from scratch after taking a backup of the config. Thanks to the documentation. All was up and running in just a couple of hours, including installation of all packages.

I am overwhelmed with your response. Kudos to you..!!
Owe one for you mate...!!!

Glad you got it sorted out. Something was definitely out of whack with the shared libraries, and that was preventing the start of Suricata. Don't really see how that would have impacted the failure to download the ET Open rules, though.