Suricata not visible in menu
-
If it's not visible under SERVICES, then the package did not complete installation. You need to examine the pfSense system log to see if anything helpful is printed there. You also need to carefully read and pay attention to any and all messages printed to the status window while the package is installing.
Most likely something is bombing with the download of a rules tarball. But you will need to review the status window of the package installation screen and the pfSense system log to see what clues are there. Post back here with your findings.
And this, hopefully, goes without needing to be said. Do not navigate away from the package installation screen until you see the green bar and status message indicating the installation is complete and succeeded.
-
@teamits I tried different browsers on different PCs. No difference.
Where do I find the option to delete the Suricata data? Could not find it when uninstalling and reinstalling. -
@bmeeks Thank you. I followed your advice. Unfortunately no success yet. I post the log...maybe you will be able to tell more from this?
>>> Installing pfSense-pkg-suricata...
Updating pfSense-core repository catalogue...
pfSense-core repository is up to date.
Updating pfSense repository catalogue...
pfSense repository is up to date.
All repositories are up to date.
Checking integrity... done (0 conflicting)
The following 13 package(s) will be affected (of 0 checked):New packages to be INSTALLED:
pfSense-pkg-suricata: 4.1.6_3 [pfSense]
suricata: 4.1.6 [pfSense]
libyaml: 0.1.6_2 [pfSense]
nss: 3.39 [pfSense]
nspr: 4.20 [pfSense]
libpcap: 1.8.1 [pfSense]
libnet: 1.1.6_5,1 [pfSense]
py27-yaml: 5.1 [pfSense]
jansson: 2.11 [pfSense]
hyperscan: 4.6.0 [pfSense]
hiredis: 0.13.3 [pfSense]
barnyard2: 1.13_1 [pfSense]
broccoli: 1.97,1 [pfSense]Number of packages to be installed: 13
The process will require 37 MiB more space.
[1/13] Installing nspr-4.20...
[1/13] Extracting nspr-4.20: .......... done
[2/13] Installing libyaml-0.1.6_2...
[2/13] Extracting libyaml-0.1.6_2: ......... done
[3/13] Installing nss-3.39...
[3/13] Extracting nss-3.39: .......... done
[4/13] Installing libpcap-1.8.1...
[4/13] Extracting libpcap-1.8.1: .......... done
[5/13] Installing libnet-1.1.6_5,1...
[5/13] Extracting libnet-1.1.6_5,1: .......... done
[6/13] Installing py27-yaml-5.1...
[6/13] Extracting py27-yaml-5.1: .......... done
[7/13] Installing jansson-2.11...
[7/13] Extracting jansson-2.11: .......... done
[8/13] Installing hyperscan-4.6.0...
[8/13] Extracting hyperscan-4.6.0: .......... done
[9/13] Installing hiredis-0.13.3...
[9/13] Extracting hiredis-0.13.3: .......... done
[10/13] Installing broccoli-1.97,1...
[10/13] Extracting broccoli-1.97,1: .......... done
[11/13] Installing suricata-4.1.6...
[11/13] Extracting suricata-4.1.6: .......... done
[12/13] Installing barnyard2-1.13_1...
[12/13] Extracting barnyard2-1.13_1: ...... done
[13/13] Installing pfSense-pkg-suricata-4.1.6_3...
[13/13] Extracting pfSense-pkg-suricata-4.1.6_3: .......... done
Saving updated package information...
done.
Loading package configuration... done.
Configuring package components...
Loading package instructions...
Custom commands...
Executing custom_php_install_command()...Message from suricata-4.1.6:===========================================================================
If you want to run Suricata in IDS mode, add to /etc/rc.conf:
suricata_enable="YES" suricata_interface="<if>"NOTE: Declaring suricata_interface is MANDATORY for Suricata in IDS Mode.
However, if you want to run Suricata in Inline IPS Mode in divert(4) mode,
add to /etc/rc.conf:suricata_enable="YES" suricata_divertport="8000"NOTE:
Suricata won't start in IDS mode without an interface configured.
Therefore if you omit suricata_interface from rc.conf, FreeBSD's
rc.d/suricata will automatically try to start Suricata in IPS Mode
(on divert port 8000, by default).Alternatively, if you want to run Suricata in Inline IPS Mode in high-speed
netmap(4) mode, add to /etc/rc.conf:suricata_enable="YES" suricata_netmap="YES"NOTE:
Suricata requires additional interface settings in the configuration
file to run in netmap(4) mode.RULES: Suricata IDS/IPS Engine comes without rules by default. You should
add rules by yourself and set an updating strategy. To do so, please visit:http://www.openinfosecfoundation.org/documentation/rules.html
http://www.openinfosecfoundation.org/documentation/emerging-threats.htmlYou may want to try BPF in zerocopy mode to test performance improvements:
sysctl -w net.bpf.zerocopy_enable=1Don't forget to add net.bpf.zerocopy_enable=1 to /etc/sysctl.conf
===========================================================================
Message from barnyard2-1.13_1:Read the notes in the barnyard2.conf file for how to configure
/usr/local/etc/barnyard2.conf after installation. For addtional information
see the Securixlive FAQ at http://www.securixlive.com/barnyard2/faq.php.In order to enable barnyard2 to start on boot, you must edit /etc/rc.conf
with the appropriate flags, etc. See the FreeBSD Handbook for syntax:
http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/configtuning-rcng.htmlFor the various options available, type % barnyard2 -h after install or read
the options in the startup script - in /usr/local/etc/rc.d.Barnyard2 can process unified2 files from snort or suricata. It can also
interact with snortsam firewall rules as well as the sguil-sensor. Those
ports must be installed separately if you wish to use them.
Cleaning up cache... done.
SuccessYour last point is clear and I always followed the procedure.
-
what if you try to go to https://pfSense.localhost/suricata/suricata_interfaces.php
? -
@defender110 said in Suricata not visible in menu:
Executing custom_php_install_command()...
I assume this is a green field install of Suricata (as in you have never configured Suricata on this box before)? If that is not the case, then there should be more output after the line above in the package installation status window. If this is a green field install, then the status window output looks normal.
And you are looking under SERVICES in the pfSense menu and you do not see Suricata as a choice? Is that correct?
To see if the actual GUI package files installed, you can browse the file system under DIAGNOSTICS > EDIT and then look for files in these two paths:
/usr/local/pkg/suricata
/usr/local/www/suricataUnder the SERVICES menu, can you see the last Wake-on-LAN option? Perhaps on your device's web browser the SERVICES menu is being truncated ???Edit: never mind, just noticed the menu in your earlier screencap post.I'm rapidly running out of ideas to investigate.
-
@kiokoman Nothing at all. Blank page. :-(
-
Are you by chance using a RAM disk configuration? If so, make sure it is large enough to support downloading and unzipping all the required files.
-
@bmeeks Thanks for all your great ideas. I don't think it helped though. I attached 2 screenshots of the links you provided. the files seem to be there.
The other 2 screenshots are when i uninstalled again and installed. Think I am doing it correctly.
Also kind of out of ideas. I also run another pfSense on a HP T620 (thisone is a HP T730), and it appears and works without any issue.
-
@bmeeks Well...no. Not at the moment. Even though I think I should, as I am running it on a SSD.
I had toyed with this, but I kept running out of space on /var/run after 4-5-6 days, even when I set the max size to 200. I am not sure why. It started at maybe 10% and over the days filled up.
I have not tackeled that issue yet. -
Well, putting the option under the SERVICES menu is done by the core pfSense code and not the Suricata package itself. The package is getting installed, but the final step done by the pfSense package manager code is to write the SERVICES menu entry. That is obviously not happening.
I really have no idea why at this point.
-
@defender110 said in Suricata not visible in menu:
@bmeeks Well...no. Not at the moment. Even though I think I should, as I am running it on a SSD.
I had toyed with this, but I kept running out of space on /var/run after 4-5-6 days, even when I set the max size to 200. I am not sure why. It started at maybe 10% and over the days filled up.
I have not tackeled that issue yet.I asked about the RAM disk because it is generally a very bad idea, especially for packages. So you DO NOT want to use a RAM disk. I was asking just in case you might have been using one. They are bad for precisely the issue you stated -- you run out of space unexpectedly and weird things then happen in the applications.
-
@bmeeks Ok. Clear! I was not aware of that. I thought with an SSD its beneficial to do. But in this case I will simply leave it as it is. I am not logging a lot, so i think the write access is not to crazy.
Thanks loads! -
@defender110 said in Suricata not visible in menu:
@bmeeks Ok. Clear! I was not aware of that. I thought with an SSD its beneficial to do. But in this case I will simply leave it as it is. I am not logging a lot, so i think the write access is not to crazy.
Thanks loads!Modern SSDs don't have the issues with frequent writes that the first generation SSDs had.
-
@bmeeks Right. I guess I am out of luck then. Will have to deal with Snort in this case, because that actually installs just fine.
Thanks for the help though! -
My only remaining theory at this point is that something is maybe weird in the
config.xmlon this problem machine. The SERVICES menu is populated from information contained in a section of theconfig.xmlfile.Look through it (in
/conf) to see if the formatting appears good. It is a standard XML file. It's got to be something specific to this machine since it works for you on another machine. You would expect package or pfSense bugs to show up on both machines. -
I've spent the last 24 hours working a very similar problem. I'm sleep deprived so sorry if this is rambling...
I was trying to restore an encrypted backup from/to 2.4.5-RELEASE-p1. Every time I'd do a restore, the "reinstalling packages in the background..." message would hang. Eventually I'd have a system that appeared to be running normally but "Suricata" was not showing in the Services menu or in the "Service Status" widget. It was showing as an installed package. The system logs showed the WatchDog service was repeatedly trying to restart Suricata and failing. And the message that packages were installing in the background would only go away if forced.
I'd go through the whole clear key, reinstall packages process. I tried restoring to fresh installs of 2.4.5-RELEASE-p1, I tried restoring older backups, more recent backups. Always the same result. I tried manually removing and reinstalling every installed package.
In addition to Suricata acting wacky, sometimes DarkStat, pfBlockerNG, and nut would freak out. By killing locked processes, I'd eventually be able to get everything back with the exception of Suricata which always ended up in the same state described above.
It wasn't until I stumbled across this post and saw the suggestion to try accessing the settings via the URL below.
https://pfSense.localhost/suricata/suricata_interfaces.php
I was able to access the settings and noticed that none of the "INSTALLED RULE SET MD5 SIGNATURES" had downloaded. When I forced the download, the Suricata package reinstall went a step further than it did before but still without any errors. This reminded me of seeing a MaxMind download error on the console or in the logs during one of many re-installs last night. I checked the restored "GeoLite2 DB License Key" and it was still showing my older style (more characters) MaxMind key from before the recent requirement to "register for a free MaxMind user account." When I replaced the older key with my new shorter "GeoLite2 DB License Key", I was able to remove and reinstall the Suricata package on the first try.
For the record, the DarkStat service was also acting really weird. When I finally got things to work, I'd also just removed DarkStat without a reinstall. Maybe unrelated but worth noting.
Anyway, I thought this nugget of information might identify a bug in the backup and restore of Suri and/or help someone else with the problem.
Now I sleep.
Regards - Tim
-
@tim_co said in Suricata not visible in menu:
The system logs showed the WatchDog service was repeatedly trying to restart Suricata and failing. And the message that packages were installing in the background would only go away if forced.
This is very likely the cause of your problems. I've said in these forums dozens of times "DO NOT RUN THE SERVICE WATCHDOG PACKAGE WITH SURICATA OR SNORT!!!".
That package does not know how to properly monitor the IDS packages nor does it understand they stop and restart themselves automatically for several reasons (uninstall, rules updates and rules downloads following fresh installs or updates). Service Watchdog only knows to monitor for an instance of the Suricata or Snort daemon, and when it does not see a copy of the daemon it tries to restart the package. That can cause all manner of chaos when Snort or Suricata is trying to update itself and/or download and update rules and automatically restart.
If you have multiple configured Suricata or Snort interfaces, the Service Watchdog package is even more useless as it only looks for a single running daemon. If it sees one, it is happy. However, when you run multiple interfaces there are multiple copies of the daemon running (one per configured interface). So you could have Suricata or Snort dead on the LAN due to some failure but still running on the WAN. Service Watchdog would be blindly happy because it would see a single copy of the daemon running and think everything is fine. But it really is not because the LAN instance is dead, but Service Watchdog does not know how to determine that.
Remove the Service Watchdog package and I bet all of the problems you described with all of the packages will disappear.
-
@bmeeks - Per my post, "When I replaced the older key with my new shorter "GeoLite2 DB License Key", I was able to remove and reinstall the Suricata package on the first try." was what resolved my issue. Period. I didn't touch the watchdog service. It's been running for years, its still running, and it was running while I applied the fix.
If you'd like to continue to opine about the watchdog service, suricata/snort, and ignorant folks like me who choose run the two together, please start a new post. Somewhere else. No need to reply speculating about how my conclusions were flawed and it was really all the watchdog service. You've made your opinion on the matter crystal clear. I am not a smart man.
I've been up for nearly 24 hours straight working on this. Please understand all caps elitist arrogant condescending posts are why I rarely contribute on highly technical forums anymore.
I rely on pfsense and was pleased to feel I was making a small contribution. I felt I had something that might help the community. I won't be made to feel small. I respectfully reject your hypothesis.
No need to reply. Move along. Thank you.
Edit - @bmeeks you're obviously free to reply but I won't be reading anything you write.
-
@tim_co said in Suricata not visible in menu:
@bmeeks - Per my post, "When I replaced the older key with my new shorter "GeoLite2 DB License Key", I was able to remove and reinstall the Suricata package on the first try." was what resolved my issue. Period. I didn't touch the watchdog service. It's been running for years, its still running, and it was running while I applied the fix.
If you'd like to continue to opine about the watchdog service, suricata/snort, and ignorant folks like me who choose run the two together, please start a new post. Somewhere else. No need to reply speculating about how my conclusions were flawed and it was really all the watchdog service. You've made your opinion on the matter crystal clear. I am not a smart man.
I've been up for nearly 24 hours straight working on this. Please understand all caps elitist arrogant condescending posts are why I rarely contribute on highly technical forums anymore.
I rely on pfsense and was pleased to feel I was making a small contribution. I felt I had something that might help the community. I won't be made to feel small. I respectfully reject your hypothesis.
No need to reply. Move along. Thank you.
Edit - @bmeeks you're obviously free to reply but I won't be reading anything you write.
Just in case you do decide to return and read this post, I am the creator of the Suricata package and the maintainer of the Snort package, so I am quite familiar with how the code in both of them works and how Service Watchdog does not work well with either of them. I once investigated modifying Service Watchdog to make it perform better with the two IDS/IPS packages, but too many changes were required and the effort was abandoned. I opine about the combination of Service Watchdog and the IDS/IPS packages because so many folks inadvertently shoot themselves in the foot with the combination. And it usually takes a while to drag it out of them that they are using Service Watchdog when I am trying to diagnose their issue.
Your change in the MaxMind key would have no bearing on Suricata as it did not use the key until the most recent MaxMind change requiring registration at the end of last year. Suricata never stored a MaxMind key prior to January 2020 as one was not required. So you should never have had a longer MaxMind key for Suricata as it used the old free database with no login requirement. Could you be confusing the MaxMind key with a requirement in pfBlockerNG as that package used the newer database much sooner than Suricata? Or have you changed your Suricata MaxMind key recently (as in after January 2020 when the requirement first appeared in Suricata)?
-
more respect for colleagues
and before you write, find out who you are talking to......
I don't want to defend Bill, it's my own opinionbut now you have compromised yourself highly
