Suricata does not start?

bmeeks

If you run either of the Snort Subscriber Rules (paid or free registered), then there is no need to run the Snort GPLv2 rules. Those rules are already in the other paid or registered Snort packages.

The best setup for a new user is to use one of the IPS policies, but just remember that Suricata is not Snort and thus a number of Snort rules will not load in Suricata. Look in the suricata.log file for the interface and you will likely see some errors related to unrecognized syntax from the Snort rules. Those rules will not be loaded by Suricata and thus any protection they offered is not available.

For this reason, I will usually suggest new users run Snort and then either register for the free Snort rules or pay the $30 annually for the subscription, and then run the IPS Connectivity Policy for starters. That is an excellent policy that will seldom false-positive. Put Snort on the LAN and run that policy. You will still likely need to disable a few of the HTTP_INSPECT preprocessor rules. To see which ones, let Snort run for a couple of weeks in IDS-only (non-blocking mode) and keep an eye on the alerts you receive. See which are false positives and disable those rules. Likely will be several from the HTTP_INSPECT preprocessor that fit that category.

Contrary to popular belief, there is no significant performance advantage to home network users by running Suricata. It's multithreaded model is, to be honest, more hype than substance when running on typical home network hardware and with typical home network traffic loads. And with Suricata you pick up the additional penalty of it failing to load a number of Snort rules because it does not recognize all of the same keywords as Snort. Suricata is optimized for the Emerging Threats rules package, but that package is not marked with IPS Policy metadata, so you can' choose an IPS Policy when using only the ET rules.

killmasta93

Thanks for the reply, i was running snort for a while but could not get it working perfectly as to many false positives, as currently im going to still run it on the homelab then run it on my production server. But out of curiosity ive seen alots of discussions snort vs suricata in your case what have you been using?

Thank you

bmeeks

@killmasta93 said in Suricata does not start?:

Thanks for the reply, i was running snort for a while but could not get it working perfectly as to many false positives, as currently im going to still run it on the homelab then run it on my production server. But out of curiosity ive seen alots of discussions snort vs suricata in your case what have you been using?

Thank you

I have always used Snort.

kvamsi.k143

Suricata has been not running since start of this month.
Is it that the ET rules are not available? Here is the log when I try to update the rules.

Please suggest.

Starting rules update... Time: 2020-04-20 03:00:11
Downloading Emerging Threats Open rules md5 file...
Checking Emerging Threats Open rules md5 file...
Emerging Threats Open rules are up to date.
Downloading Snort VRT rules md5 file...
Checking Snort VRT rules md5 file...
There is a new set of Snort rules posted.

kvamsi.k143

to be precise, ET rules are failing since 7th June.

Starting rules update... Time: 2020-06-06 03:00:29
Downloading Emerging Threats Open rules md5 file...
Checking Emerging Threats Open rules md5 file...
There is a new set of Emerging Threats Open rules posted.
Downloading file 'emerging.rules.tar.gz'...
Done downloading rules file.
Downloading Snort VRT rules md5 file...
Checking Snort VRT rules md5 file...
There is a new set of Snort rules posted.
Downloading file 'community-rules.tar.gz'...
Done downloading rules file.
Downloading Snort GPLv2 Community Rules md5 file...
Checking Snort GPLv2 Community Rules md5 file...
There is a new set of Snort GPLv2 Community Rules posted.
Downloading file 'community-rules.tar.gz'...
Done downloading rules file.
Extracting and installing Emerging Threats Open rules...
Installation of Emerging Threats Open rules completed.
Extracting and installing Snort rules...
Installation of Snort rules completed.
Extracting and installing Snort GPLv2 Community Rules...
Installation of Snort GPLv2 Community Rules completed.
Copying new config and map files...
Updating rules configuration for: INTERNET ...
Updating rules configuration for: LAN ...
Updating rules configuration for: LAN1 ...
Restarting Suricata to activate the new set of rules...
Suricata has restarted with your new set of rules.
The Rules update has finished. Time: 2020-06-06 03:01:11

Starting rules update... Time: 2020-06-07 14:18:04
Downloading Emerging Threats Open rules md5 file...
Emerging Threats Open rules md5 download failed.
Server returned error code 404.
Server error message was: 404 Not Found
Emerging Threats Open rules will not be updated.
Downloading Snort VRT rules md5 file...
Checking Snort VRT rules md5 file...
There is a new set of Snort rules posted.
Downloading file 'community-rules.tar.gz'...
Done downloading rules file.
Downloading Snort GPLv2 Community Rules md5 file...
Checking Snort GPLv2 Community Rules md5 file...
There is a new set of Snort GPLv2 Community Rules posted.
Downloading file 'community-rules.tar.gz'...
Done downloading rules file.
Extracting and installing Snort rules...
Installation of Snort rules completed.
Extracting and installing Snort GPLv2 Community Rules...
Installation of Snort GPLv2 Community Rules completed.
Copying new config and map files...
Updating rules configuration for: INTERNET ...
Updating rules configuration for: LAN ...
Updating rules configuration for: LAN1 ...
The Rules update has finished. Time: 2020-06-07 14:18:24

bmeeks

@kvamsi-k143 said in Suricata does not start?:

to be precise, ET rules are failing since 7th June.

Starting rules update... Time: 2020-06-06 03:00:29
Downloading Emerging Threats Open rules md5 file...
Checking Emerging Threats Open rules md5 file...
There is a new set of Emerging Threats Open rules posted.
Downloading file 'emerging.rules.tar.gz'...
Done downloading rules file.
Downloading Snort VRT rules md5 file...
Checking Snort VRT rules md5 file...
There is a new set of Snort rules posted.
Downloading file 'community-rules.tar.gz'...
Done downloading rules file.
Downloading Snort GPLv2 Community Rules md5 file...
Checking Snort GPLv2 Community Rules md5 file...
There is a new set of Snort GPLv2 Community Rules posted.
Downloading file 'community-rules.tar.gz'...
Done downloading rules file.
Extracting and installing Emerging Threats Open rules...
Installation of Emerging Threats Open rules completed.
Extracting and installing Snort rules...
Installation of Snort rules completed.
Extracting and installing Snort GPLv2 Community Rules...
Installation of Snort GPLv2 Community Rules completed.
Copying new config and map files...
Updating rules configuration for: INTERNET ...
Updating rules configuration for: LAN ...
Updating rules configuration for: LAN1 ...
Restarting Suricata to activate the new set of rules...
Suricata has restarted with your new set of rules.
The Rules update has finished. Time: 2020-06-06 03:01:11

Starting rules update... Time: 2020-06-07 14:18:04
Downloading Emerging Threats Open rules md5 file...
Emerging Threats Open rules md5 download failed.
Server returned error code 404.
Server error message was: 404 Not Found
Emerging Threats Open rules will not be updated.
Downloading Snort VRT rules md5 file...
Checking Snort VRT rules md5 file...
There is a new set of Snort rules posted.
Downloading file 'community-rules.tar.gz'...
Done downloading rules file.
Downloading Snort GPLv2 Community Rules md5 file...
Checking Snort GPLv2 Community Rules md5 file...
There is a new set of Snort GPLv2 Community Rules posted.
Downloading file 'community-rules.tar.gz'...
Done downloading rules file.
Extracting and installing Snort rules...
Installation of Snort rules completed.
Extracting and installing Snort GPLv2 Community Rules...
Installation of Snort GPLv2 Community Rules completed.
Copying new config and map files...
Updating rules configuration for: INTERNET ...
Updating rules configuration for: LAN ...
Updating rules configuration for: LAN1 ...
The Rules update has finished. Time: 2020-06-07 14:18:24

I just installed the Suricata 5.0.2_3 package on a virtual machine this morning with no issues at all. The Emerging Threats Open rules downloaded and installed just fine. Here is the install log from the package installation --

[18/18] Installing pfSense-pkg-suricata-5.0.2_3...
[18/18] Extracting pfSense-pkg-suricata-5.0.2_3: .......... done
Saving updated package information...
done.
Loading package configuration... done.
Configuring package components...
Loading package instructions...
Custom commands...
Executing custom_php_install_command()...Saved settings detected...
Migrating settings to new configuration... done.
Downloading Emerging Threats Open rules md5 file... done.
There is a new set of Emerging Threats Open rules posted. Downloading... done.
Downloading Snort VRT rules md5 file... done.
There is a new set of Snort rules posted. Downloading... done.
Downloading Snort GPLv2 Community Rules md5 file... done.
There is a new set of Snort GPLv2 Community Rules posted. Downloading... done.
Installing Emerging Threats Open rules... done.
Installing Snort rules... done.
Installing Snort GPLv2 Community Rules... done.
Updating rules configuration for: WAN ... done.
Updating rules configuration for: OPT1 ... done.
Updating rules configuration for: LAN ... done.
Cleaning up after rules extraction... done.
The Rules update has finished.
Generating suricata.yaml configuration file from saved settings.
Generating YAML configuration file for WAN... done.
Generating YAML configuration file for OPT1... done.
Generating YAML configuration file for LAN... done.
Finished rebuilding Suricata configuration from saved settings.

And here is the Updates Log entry created during the package installation --

Starting rules update...  Time: 2020-06-29 07:53:45
	Downloading Emerging Threats Open rules md5 file...
	Checking Emerging Threats Open rules md5 file...
	There is a new set of Emerging Threats Open rules posted.
	Downloading file 'emerging.rules.tar.gz'...
	Done downloading rules file.
	Downloading Snort VRT rules md5 file...
	Checking Snort VRT rules md5 file...
	There is a new set of Snort rules posted.
	Downloading file 'snortrules-snapshot-29151.tar.gz'...
	Done downloading rules file.
	Downloading Snort GPLv2 Community Rules md5 file...
	Checking Snort GPLv2 Community Rules md5 file...
	There is a new set of Snort GPLv2 Community Rules posted.
	Downloading file 'community-rules.tar.gz'...
	Done downloading rules file.
	Extracting and installing Emerging Threats Open rules...
	Installation of Emerging Threats Open rules completed.
	Extracting and installing Snort rules...
	Installation of Snort rules completed.
	Extracting and installing Snort GPLv2 Community Rules...
	Installation of Snort GPLv2 Community Rules completed.
	Copying new config and map files...
	Updating rules configuration for: WAN ...
	Updating rules configuration for: OPT1 ...
	Updating rules configuration for: LAN ...
The Rules update has finished.  Time: 2020-06-29 07:54:10

So the package code seems to be working fine and the URL is up and available. Perhaps you have something on your end interfering? Some common issues I've seen in the past are problems with Squid or Squidguard (if installed) and rarely a pfBlockerNG IP list might cause a problem (although that was more common with Snort rules as they are hosted on AWS). The base URL is https://rules.emergingthreats.net/open/. And then depending on which Suricata version branch you are running, either the suricata-4.0 or suricata-5.0 directories.

What version of Suricata are you running and what type of hardware do you have?

kvamsi.k143

@bmeeks Suricata is running on 5.0.2_3.
Yes, I do have Squidguard and pfBlockerNG running on my pfSense box.

Just to be sure added the link in the Global Settings>Custom URL, still the same issue.
I also tried to re-install the Suricata package. Any other log I can provide you to investigate this please?

bmeeks

@kvamsi-k143 said in Suricata does not start?:

@bmeeks Suricata is running on 5.0.2_3.
Yes, I do have Squidguard and pfBlockerNG running on my pfSense box.

Just to be sure added the link in the Global Settings>Custom URL, still the same issue.
I also tried to re-install the Suricata package. Any other log I can provide you to investigate this please?

Remove or disable Squidguard and then see what happens with the download.

There is nothing wrong with the Suricata package code. If there were, thousands of users would be here complaining. The issue is installed packages interfering with each other in your particular setup. My first guess for the culprit is Squidguard because it tries to police web traffic, and the rules download is pure web traffic -- SSL in the case of the Snort and Emerging Threats rules.

When you have a package like Squidguard installed and you experience any kind of problem reaching a URL, the very first thing you should do is disable Squidguard and test again. Does the problem go away? If so, then you found your cause. If not, try the same thing with the next package you have installed (pfBlockerNG or Suricata). At some point you will find which package is blocking the download. This is basic troubleshooting 101. When you have a number of packages installed that all will do blocking in some form or other, it is highly probable that when you have download difficulties one of those "blocking" packages is the cause.

kvamsi.k143

@bmeeks I tried with disabling Squidguard and pfBlockerNG and still no luck.

BTW.. here is what I found if that helps..

[2.4.5-RELEASE][admin@bheema.kandulaz.com]/root: ps -ax | grep suricata
57745 0 S+ 0:00.00 grep suricata
[2.4.5-RELEASE][admin@bheema.kandulaz.com]/root: suricata -V
Shared object "libluajit-5.1.so.2" not found, required by "suricata"
[2.4.5-RELEASE][admin@bheema.kandulaz.com]/root: /usr/local/etc/rc.d
/usr/local/etc/rc.d: Permission denied.
[2.4.5-RELEASE][admin@bheema.kandulaz.com]/root: /usr/local/etc/rc.d/suricata.sh start
ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/graphviz /usr/local/lib/ipsec /usr/local/lib/mysql /usr/local/lib/nss /usr/local/lib/perl5/5.30/mach/CORE
32-bit compatibility ldconfig path:

bmeeks

@kvamsi-k143 said in Suricata does not start?:

@bmeeks I tried with disabling Squidguard and pfBlockerNG and still no luck.

BTW.. here is what I found if that helps..

[2.4.5-RELEASE][admin@bheema.kandulaz.com]/root: ps -ax | grep suricata
57745 0 S+ 0:00.00 grep suricata
[2.4.5-RELEASE][admin@bheema.kandulaz.com]/root: suricata -V
Shared object "libluajit-5.1.so.2" not found, required by "suricata"
[2.4.5-RELEASE][admin@bheema.kandulaz.com]/root: /usr/local/etc/rc.d
/usr/local/etc/rc.d: Permission denied.
[2.4.5-RELEASE][admin@bheema.kandulaz.com]/root: /usr/local/etc/rc.d/suricata.sh start
ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/graphviz /usr/local/lib/ipsec /usr/local/lib/mysql /usr/local/lib/nss /usr/local/lib/perl5/5.30/mach/CORE
32-bit compatibility ldconfig path:

How did we go suddenly from "Suricata will not download Emerging Threats rules" to "Suricata won't start"?

The package configuration on your firewall is broken. What order did you perform the last upgrade? Did you update pfSense first and only after that install package updates, or did you install or update packages AFTER pfSense 2.4.5 came out but BEFORE you upgraded your firewall? If you installed or updated a package before you upgraded pfSense to 2.4.5, then you have a broken package system. I can tell that because the libluajit-5.1.so.2 library is not used by Suricata on pfSense 2.4.5. The library luajit-openresty is used instead. The fact your Suricata install is trying to use the older non-existent library tells me your upgrade sequence was likely not correct.

At this point, my suggestion is to perform a backup of your configuration, wipe pfSense off your hardware and install again from scratch with the current 2.4.5_p1 image file. As part of the install you can import your configuration backup. Instructions are in the Netgate docs.

kvamsi.k143

@bmeeks
Thanks for your time in helping me on this. There are two issues on my pfSense box. 1) ET not updating and 2) Suricata services not running on the interfaces.

BTW, I did update pfSense before Suricata, I am aware of the catch.

I did check the other thread while investigating. That is when I found issue with "libluajit-5.1.so.2" not found, required by "suricata".

Post your suggestion, I took time to flash the pfSense from scratch after taking a backup of the config. Thanks to the documentation. All was up and running in just a couple of hours, including installation of all packages.

I am overwhelmed with your response. Kudos to you..!!
Owe one for you mate...!!!

bmeeks

@kvamsi-k143 said in Suricata does not start?:

@bmeeks
Thanks for your time in helping me on this. There are two issues on my pfSense box. 1) ET not updating and 2) Suricata services not running on the interfaces.

BTW, I did update pfSense before Suricata, I am aware of the catch.

I did check the other thread while investigating. That is when I found issue with "libluajit-5.1.so.2" not found, required by "suricata".

Post your suggestion, I took time to flash the pfSense from scratch after taking a backup of the config. Thanks to the documentation. All was up and running in just a couple of hours, including installation of all packages.

I am overwhelmed with your response. Kudos to you..!!
Owe one for you mate...!!!

Glad you got it sorted out. Something was definitely out of whack with the shared libraries, and that was preventing the start of Suricata. Don't really see how that would have impacted the failure to download the ET Open rules, though.