@Cool_Corona said in Suricata Select all and change all in a given category??:

@bmeeks said in Suricata Select all and change all in a given category??:

Sorry, but no. That feature is not something that makes much sense to me. If you want to modify lots and lots of rules, then use the SID MGMT tab features. That tab is tailor-made for the task.

I have been asked about it before and I gave the same answer then. A feature like that is not on my list. The PHP code necessary behind the scenes to keep track of possibly hundreds of checked parameters across a $_POST session call is burdensome and would be prone to errors.

The idea behind the RULES tab is to let you modify just a handful of rules here and there. For largescale rule mods, use the SID MGMT tab and the accompanying custom SID.conf files.

Thanks B. Would SID mgmt be a local issue or download lists available online and therefore be a potential security risk?

SID MGMT makes use of text-based configuration files modelled after those used with PulledPork. Using selection features in the conf files lets you choose rules using several different criteria. You can then modify a rule's content, action or state ('enabled' or 'disabled').

There is a sticky post at the top of this forum describing the feature and how to use it. Here is a link: https://forum.netgate.com/topic/128480/how-automatic-sid-management-and-user-rule-overrides-work-in-snort-and-suricata.

Enable SID MGMT on that tab and then open and read through the provided sample config files. They have comments inside and examples.

@bmeeks

Thank you,

@bmeeks Understood. Thanks!

The term "hardened" does in fact mean some small customizations are done to the baseline FreeBSD operating system. pfSense does the same. Neither firewall distro uses an unmodified FreeBSD underlying OS, but so far as I know the bulk (if not all) of the customization is within the two native firewall engines within FreeBSD: pf and ipfw.

What is not clearly stated in the link you provided is whether the FreeBSD versions quoted are RELEASE or STABLE. See, there is both a FreeBSD 12.1/RELEASE and a FreeBSD 12.1/STABLE version. Each version is different although both say "12.1". The STABLE branch is more "current" as it contains features and enhancements over the RELEASE branch, but those additional changes also entail some amount of additional risk that is more similar to what you expect with development versions of a product.

@bmeeks Thank you

@bmeeks ok, got it

yeah I even went into the trouble of compiling the Intel driver but still same issues, it even rebooted on me ...

@cavaco said in Suricata not Working after Cpu change:

@bmeeks thank you very much after setting the memory from 32 m to almost 1000 m it started ..... , but will try with lower

thank you very much

Yeah, 1024 MB is probably a bit much unless you have tons of free RAM. Try 512 MB or even 256 MB instead.

@bmeeks Ah, sorry for my misunderstanding. I'll do that, as well. I turned off the WAN interface, turned on the LAN one, and set it to Connectivity instead of Balanced.

Here's my system resources with those changes:

alt text

@bmeeks Okay Bill, I'll just disable ... thank you!

Suricata will not restart itself except when it does an automatic rules update. The binary has no mechanism to even accomplish this. The PHP GUI code takes care of restarting Suricata after a rules update download.

pfSense will, under some circumstances, issue a "restart all packages" command. Could that have been going on at the same time you were trying to stop Suricata? Very unusual if that were the case, though.

@interessierter said in SNORT blocking too much:

Guys please stop this.

Not meant to be personal ^^
Didn't know what you know, but now I get the picture.

@interessierter said in SNORT blocking too much:

When I m a company with 100.000 employee s it get hard to "know" my traffic

The easy one : that corner guy exists !!!

But ok, serious : as you said "100 000" means a totally random bit stream is what snort will be seeing.
DPI became already quiet impossible then : TLS killed the access real to the data payload years. A solution that still might pull it off : Use a proxy on/in front of your snort device, have it "unpacked" all traffic, scan, and pack it back into TLS again.
This proxy device would be situated on pfSense, but in front of it. Centralized administration of all devices in the network would be needed to handle the 'cert' issues.

For a small group of persons (small society or even family) it's actually possible to find some common patterns, but a huge group will always trigger some (random) pattern/rule. You're talking of several OS's - thousands of application, that can all have there "network errors", that might trigger some rule.
Keep in mind : snort is somewhat limited to the packet headers, or "headers in headers" and some transport flags. It's like reading a postal envelop, and drawing a conclusion when seeing the destination and sender, and the type of letter, and their occurrence.

Btw : for purely learning how DPI works, I've been using snort and friends for some years in the past. It became a close to 7/7 daily job to maintain traffic flowing. And I was using the rules accessible by subscription.
I'm still using it (DPI) today, on my mail server, as that one still 'sees' the traffic 'in clear' in the mail boxes.

IMHO : I lost somewhat of the faith in the usefulness of DPI as it exists today. It's goes totally against the all TLS hype.

@interessierter said in SNORT blocking too much:

When the snort rules gets sold to enterprise customers, than I was expecting here a bit more.

Rule support should be found there where they are created ;)

That is a lot of Snort processes. To answer your question, there are no tuneables of that sort within Snort itself. And the way Snort works on pfSense results in each interface you run it on being a totally separate process without any awareness of the other Snort instances. So any limitations would be within FreeBSD itself.

To see what is going on, I suggest you do what I mentioned earlier. Go to the GLOBAL SETTINGS tab, and down at the bottom of that page, check the box to turn on verbose logging. Then go and attempt a start of an interface that does not normally start with performance stats enabled. After the failure to start (assuming it does fail), then examine the pfSense system log line-by-line to see all the Snort messages. Something may get logged to help you troubleshoot.

Note that pfSense uses a circular binary logging system called clog for the system log. So in order to have plenty of circular buffer space for logging, go to the System Log > Settings tab under STATUS and set the number of lines to display to a very large value like 1000 or perhaps more. Snort will log a lot of information as it starts up.

@bmeeks Ok. I see MicroUSB port on the back, and two USB-A ports on the front. None are in use, nor have I ever used any.
This unit shipped with pfSense already installed and with base configuration. Although I am aware of the console option, I have never used it.

@TTWE said in Trying to diagnose non starting packages:

Hi

I have just installed PFsense and in the process of setting it up.
However when I went to install additional packages (Suricata) they wont start at all (I have tried other as a test and non of them start).
I don't get any error messages and I have looked in the system logs, however being new to this I have had no luck.

I would greatly appreciate any help I can get trying to diagnose this problem. I will put all the information I have and the system specs underneath.

Many Thanks TTWE

Version 2.4.5-RELEASE (amd64)
built on Tue Mar 24 15:25:50 EDT 2020
FreeBSD 11.3-STABLE

CPU Type Intel(R) Xeon(R) CPU E5630 @ 2.53GHz
16 CPUs: 2 package(s) x 4 core(s) x 2 hardware threads
AES-NI CPU Crypto: Yes (inactive)
Memory 64 GB 1% average use.

suricata security 5.0.2_2 High Performance Network IDS, IPS and Security Monitoring engine by OISF.

You can find out why Suricata is not starting by going to the LOGS VIEW tab, selecting the interface you want to view logs for in the Interface drop-down selector, and then choosing the suricata.log file in the log file drop-down selector.

I can pretty much guarantee you that your problem is going to me a memory allocation error due to an insufficient TCP Stream Memcap setting. For high core-count boxes you need to dramatically increase the stream memcap value on the Flow/Stream tab. Try 256 MB and then work up there since you have so many CPUs and cores.

Here is a link to the Suricata upstream Redmine site where they have a project underway to improve the OOBE (out-of-box experience) by improving some default values: https://redmine.openinfosecfoundation.org/issues/1343. Once they incorporate those into Suricata, I will make some updates to the pfSense package. But in order to not create a memory hog on smaller systems, I may not use values quite as large as mentioned in that thread.

The current default in the pfSense package is fine for dual or quad-core CPUs, but is not enough for high core-count boxes like you have.

Sorry, I don't intend to hijack this threat, but.. 😉 A admin moved my thread from IDS to Development.
I'm not a expert, but I think there is like bmeeks expressed something fishy with netmap and/or Nic drivers. I wrote my experience with a Wireguard Client on Linux Mint on this threat.
https://forum.netgate.com/topic/153255/bug-2-5-0-development-amd64-built-on-sun-may-03-23-56-0-snort-2-9-16-inline-ips-throttles-wireguard-speed.

I noticed a significant speed drop with WG in March as I changed snort IPS to inline IPS mode. Since I changed back to Legacy Mode my speed is back. Everything OK.

I have setup a ovpn client to Mullvad, my speed is > 350Mbit/s regardless if on Legacy or IPS Mode, with my ISP regardless IPS/Legacy Mode ~ 950Mbit/s. When using WG on a Client PC speed in IPS Mode throttles down ~ 70Mbit/s in Legacy ~ 830MBit/s.

Remark: IPS Mode never created problems between March when first time used and changed back to Legacy in 5th of May, no crash or what ever.

Thks :)

@JSmorada said in Cron job for hourly restart of Snort:

I'm running on bare metal; the version of pfSense is 2.4.5-RELEASE (amd64) and Snort is at version 3.2.9.11. I haven't checked the logs but will take a look to see and report on it later today.https://forum.netgate.com/topic/153269/cron-job-for-hourly-restart-of-snort/3#

Have you looked to see if any alerts are occurring from Snort? What IP addresses are shown as being blocked (if you have blocking enabled)?

The fact you say a restart of Snort solves the issue is not consistent with Snort blocking because restarting Snort will NOT remove any existing blocks. Blocks, once inserted, remain until the IP addresses are removed from the snort2c table. That happens only via three things: (1) a manual clearing of the table by the user; (2) the periodic "Remove Blocked Hosts" cron task executes if enabled; (3) or a reboot of pfSense itself.

So based on the above, are you 100% sure Snort is the cause of your issue? Do you have any other package running on your firewall?

What type of NIC do you have? One thing that restarting Snort would do is "tickle" the NIC driver due to the libpcap library tearing down and recreating a subscription for copies of packets and placing the NIC in and out of promiscuous mode. However, that would indicate an issue with the NIC or its driver and not a problem with Snort.

What happens if you leave Snort disabled for say a day? Do you have any connection problems then? That is one way to narrow down the issue. If you leave Snort disabled and still have a network interruption after some period, that would certainly clear Snort of being responsible.

Thanks for the great explanations Bill. I appreciate your time.

@pftdm007 said in Snort intercepts incoming traffic on WAN going to DMZ clients:

@bmeeks said in Snort intercepts incoming traffic on WAN going to DMZ clients:

I don't really understand your question. If a client on the DMZ is speaking with a client on the LAN, and that conversation triggers a Snort rule on the LAN, then it will fire an alert on the LAN. This is expected. I'm not sure what you expect to happen in that case.

Hello @bmeeks

That wasn't really a question more than trying to understand how snort works. Basically you're saying that the alert triggered on LAN occurred during the reply from LAN client to DMZ client? That would make sense...

Request: DMZ client -> DMZ iface -> LAN iface -> LAN client
Response: LAN client -> LAN iface -> TRIGGERS Snort alert & traffic blocked (or alert only)

Yes, although it could also have occured in the other direction when the DMZ client initiated the conversation with the LAN client. Snort would have seen the packet as it left the firewall's filtering/routing engine and was being handed to the NIC driver for transmit. Remember that Snort sits between the NIC driver and the firewall's filter and routing engine. Either way, the alert would be triggered by the LAN instance of Snort and would only show up on the ALERTS tab when you are viewing the LAN alerts.