Suricata: New alerts for rules in the suppress list still visible in Alerts Log

priller

There was already a discussion about this issue here: https://forum.pfsense.org/index.php?topic=74778.0
However, that topic was originally started for an issue with Snort and I started talking about the same/similar problem with Suricata. That caused some confusion in the discussion. So, here is a fresh topic.

Problem: Rules that have already been added to the suppress list continue to be triggered and added to the Alerts log. Once added to the Alerts log, the "Add this alert to the suppress list" is greyed out (see attached image) and the hover tells you that "This alert is already in the Suppress List". Note: These are new alerts, not ones that were there prior to being suppressed.

In my testing, this problem only occurs when multiple interfaces are configured in Suricata. I have not seen the problem when only a single interface is configured.

Recreate

Start clean

Uninstall Suricata without saving the current configuration.
Reboot pfSense
Install Suricata
Reconfigure from scratch

Configure a single interface (I did LAN first)

Configured LAN interface with minimal ET rules
As alerts are recorded, add the undesired to the Suppress List
Clear the Alert log so you don't get confused
Monitor that suppressed rules don't come back.

Results: Everything looks great at this point, working as expected.

Configure another interface ( This time OPT1 )

Configured OPT1 interface with minimal ET rules
As alerts are recorded, add the undesired to the Suppress List
Clear the Alert log so you don't get confused
Monitor that suppressed rules don't come back.

Note: Both interfaces have their own Suppress list, using the auto-generated names.

Results: Once you activate the second interface, rules that were suppressed on the first interface immediately start getting written to the Alerts log with the "Add this alert to the suppress list" grayed out. The second interface also has the same issue.

Package restarts and system reboots do not change the behavior.

I have run through the entire process described above twice, with the same results.

–-

2.1.2-RELEASE (amd64)
Suricata 1.4.6 pkg v0.3
Snort 2.9.6.0 pkg v3.0.6

As I recall, I did not see this problem on 2.1.1. I first installed Suricata with v0.1 and know the issue wasn't there then. Not positive exactly when it started, but it started just recently ... and I think the upgrade to 2.1.2 was the trigger.

Bill ... I know you are working on other items with Snort / Suricata right now, but wanted to put this down while it was still fresh in my head. Nothing urgent.

supress3_good_bad.jpg_thumb

bmeeks

@priller:

–--

Bill ... I know you are working on other items with Snort / Suricata right now, but wanted to put this down while it was still fresh in my head. Nothing urgent.

Thanks for putting this in its own thread. Your additional information about multiple interfaces seeming to trigger the problem is helpful. While I am working on the blocking plugin, I can take a look at this. Since I've not seen any complaints about this suppress list behavior on the Suricata lists on Google, I'm suspicious that I may have a bug in the package code that I have not yet noticed. Wouldn't be the first time… :-[

Bill

priller

I can now trigger this behavior with a single interface configured.

This is with Suricata 1.4.6 pkg v1.0 - 2.1.2-RELEASE (amd64).

I wasn't even trying to break it (at least this time!). :-[

Here is how it happened.

Due to this happening previously, I had uninstalled Suricata with configuration removed and system rebooted.
I configured a single interface (LAN) with a few ET categories, http and stats logging.
As I added rules to the suppress list (mostly stream) everything was working perfectly.
As the number of stream rules added to the suppress list grew, I finally decided to disable them all. This was via selecting stream-event.rules under the LAN Rules tab. Selected "Disable all rules in the current Category" and hit apply.
All appeared to be OK. Then I restarted Suricata from the Status: Services menu.
[color=red]KaBoom! Everything that was in the suppressed list started showing in the alert tab with the "Add this alert to the suppress list" greyed out (see previousy attached image) and the hover tells you that "This alert is already in the Suppress List".

–-

I tried to correct by stop/start the service. Stop/start the interface. No joy.

I then deleted the interface and added it back in. Now it is behaving normally again.

bmeeks

@priller:

I can now trigger this behavior with a single interface configured.

This is with Suricata 1.4.6 pkg v1.0 - 2.1.2-RELEASE (amd64).

I wasn't even trying to break it (at least this time!). :-[

Here is how it happened.

Due to this happening previously, I had uninstalled Suricata with configuration removed and system rebooted.

I configured a single interface (LAN) with a few ET categories, http and stats logging.

As I added rules to the suppress list (mostly stream) everything was working perfectly.

As the number of stream rules added to the suppress list grew, I finally decided to disable them all. This was via selecting stream-event.rules under the LAN Rules tab. Selected "Disable all rules in the current Category" and hit apply.

All appeared to be OK. Then I restarted Suricata from the Status: Services menu.

[color=red]KaBoom! Everything that was in the suppressed list started showing in the alert tab with the "Add this alert to the suppress list" greyed out (see previousy attached image) and the hover tells you that "This alert is already in the Suppress List".

–-

I tried to correct by stop/start the service. Stop/start the interface. No joy.

I then deleted the interface and added it back in. Now it is behaving normally again.

Thanks for the detailed report. I will investigate this. It had fallen between the cracks while I was busy with the new blocking module. From your description, it sure sounds like maybe I have a dumb mistake in the code someplace, but thus far I don't see it. That certainly does not mean it's not there, though… :-[

The icon for adding to the Suppress List being grayed-out is understandable. That keys off the value being saved in the config.xml file section for Suricata. It does not key off the actual file contents Suricata uses to process the suppress list. My guess is something is going wrong with that file (or the path to it). I will take a fresh look.

Bill

bmeeks

I am pretty certain I found the problem with Suricata ignoring Suppress List entries. It was a bug in the Suricata 1.4.6 binary that was fixed in Suricata 1.4.7. See Bug #1000 on https://redmine.openinfosecfoundation.org/issues/1000. I back-ported this fix to the 1.4.6 binary used currently on pfSense and submitted the patch to the Core Team Developers for review.

I also discovered that a setting in the suricata.yaml configuration file can influence this bug. Setting the "delayed-detect" parameter to "no" should prevent the bug from occurring. Right now this value is hard-coded to "yes", but can be manually edited in the file /usr/local/pkg/suricata_yaml_template.inc. Find the "delayed-detect: yes" line in that file and change it to "no". Save the settings for a Suricata interface and then restart it.

I have also added the ability to disable the "delayed-detect" feature to the GUI in the next Suricata update. The Pull Request for that update is posted here: https://github.com/pfsense/pfsense-packages/pull/654.

Based on my testing, either the patched binary or turning off the delayed-detect feature worked. Translated, that means you need the new binary if you want to leave "delayed-detect" set to "on", but if you don't have the new binary yet, you can just set "delayed-detect" to off and should still not see the bug.

Bill

priller

@bmeeks:

Based on my testing, either the patched binary or turning off the delayed-detect feature worked. Translated, that means you need the new binary if you want to leave "delayed-detect" set to "on", but if you don't have the new binary yet, you can just set "delayed-detect" to off and should still not see the bug.

It seems to be OK with that set to "off".

We tried that previously ( https://forum.pfsense.org/index.php?topic=74778.msg410720#msg410720 ) and still had a problem. Other than having multiple interfaces then vs. a single one now, I'm not sure what else is different in the config .. or the sequence of events to trigger it. Anyway, looking much better today.

bmeeks

@priller:

@bmeeks:

Based on my testing, either the patched binary or turning off the delayed-detect feature worked. Translated, that means you need the new binary if you want to leave "delayed-detect" set to "on", but if you don't have the new binary yet, you can just set "delayed-detect" to off and should still not see the bug.

It seems to be OK with that set to "off".

We tried that previously ( https://forum.pfsense.org/index.php?topic=74778.msg410720#msg410720 ) and still had a problem. Other than having multiple interfaces then vs. a single one now, I'm not sure what else is different in the config .. or the sequence of events to trigger it. Anyway, looking much better today.

From my understanding of the bug and the Suricata binary code, the call to load the Suppress List entries was made and the list successfully loaded before the rule signatures had been loaded. So there were, at that moment, no rules in the memory table for the Suppress List to match and thus the flag to "not alert" was not set for any Suppress List rules. Delayed-Detect is an option designed for inline IPS operation that allows Suricata to go ahead and come up and start passing traffic even before all the rule signatures (detection) are loaded and enabled. So in this mode, there is a time where traffic will pass uninspected. In the current quasi-IPS/IDS mode Suricata uses, this parameter has no meaning but I had defaulted it to "on" for the future. When you disable Delayed-Detect, the rules load earlier and thus are in memory when the Suppress List is loaded and parsed. So this means rules can match up.

The reason this worked using the icons on the Alerts tab is those add the entry to the list and then initiate a "live rule swap" within Suricata. This is not the same as a restart, thus the rules are in memory and can be matched up against the Suppress List when it loads as part of the "live rule swap". Restarting the service, however, does trigger the bug when Delayed-Detect is enabled.

I've done two things to address this in the upcoming release. First, I back-ported a patch from 1.4.7 to the 1.4.6 version used on FreeBSD that makes sure the Suppress List is loaded after the rules have been processed. Second, I added a checkbox on the INTERFACES tab to allow the Delayed-Detect feature to be manually enabled or disabled by the user. It will default to disabled.

Bill