Using Suricata SID Mgmt

newUser2pfSense

Ohhhhhh, I see what you mean by "state" now that I see it in the rules I listed, the column with the the green circle and checkmark icon.

It's good to know that I don't have to change anything in the SID State Order dropdown and that I can just leave it as whatever default it's in.

This is going to save me a boat load of time.

I'm wondering though, what will SID Mgmt do about all of the other rules that I've changed to Drop? Will it bypass those rules and use only the rules in the dropRules.conf file or will it take all of the other rules I've dropped into account as well?

bmeeks

@newUser2pfSense said in Using Suricata SID Mgmt:

Ohhhhhh, I see what you mean by "state" now that I see it in the rules I listed, the column with the the green circle and checkmark icon.

It's good to know that I don't have to change anything in the SID State Order dropdown and that I can just leave it as whatever default it's in.

This is going to save me a boat load of time.

Yes, that green icon means a rule is "default enabled" by the creator of the rule. If you see the red X icon, that means the rule is "default disabled" by the creator of the rule meaning the rule text has the comment sign (that "#" symbol) at the beginning of the line of text so that rule is not loaded into memory by Suricata.

As a user, you can force rules to a given state. But don't confuse "state' with "action". State controls whether a rule is even loaded into memory. Action controls what the rule does when it is triggered (alert, drop traffic, send a reject response or pass the traffic).

Read that link I posted up above about how SIG MGMT works. Suricata uses both configurations (user-changed/forced and SID management conf files) to set rule actions.

newUser2pfSense

That's fantastic to know that the other rules I've Dropped will not be overlooked.

bmeeks

@newUser2pfSense said in Using Suricata SID Mgmt:

I'm wondering though, what will SID Mgmt do about all of the other rules that I've changed to Drop? Will it bypass those rules and use only the rules in the dropRules.conf file or will it take all of the other rules I've dropped into account as well?

I suggest concentrating on not using "dropped" when talking about rules management. Most experienced admins will think you are talking about the "state", or changing a rule from enabled to disabled (like I did, initially). You should use the term "changing rule actions to DROP" to describe what you are doing. You are changing the "action" to DROP. You are NOT "dropping" the rule. You would be disabling those rules to "drop" them from the list of active rules. It is a very important distinction if you discuss this with other IDS/IPS admins.

newUser2pfSense

Noted Bill...I think that's why initially I had trouble with the verbiage as well. I wasn't understanding. I'm glad now to know the distinction.

Again, and as always, thanks for all of your help. I definitely appreciate it.

bmeeks

@newUser2pfSense said in Using Suricata SID Mgmt:

Noted Bill...I think that's why initially I had trouble with the verbiage as well. I wasn't understanding. I'm glad now to know the distinction.

Again, and as always, thanks for all of your help. I definitely appreciate it.

You're welcome. Easy way to remember:

State refers to "enabled" or "disabled" and determines whether a given rule is even loaded into memory and processed by the IPS engine;
Action refers to what the rules does when triggered. Does it just alert, does it pass the traffic, does it drop the traffic, or does it drop and send a reject reply to the originator?

newUser2pfSense

As a followup, I downloaded the Alerts to one of my interfaces to check to see if the action to Drop rules that I created in SID Mgmt actually worked.

This is what I have in my dropRules.conf file -
Drop the following from the rules:
ET COMPROMISED Known Compromised or Hostile Host Traffic group
1 thru 247
1:2500000-1:2500492

[sharp in front of the first 3 lines]

From the Alerts log, this is a manual action to Drop rules line where it clearly shows it's dropped:
11/24/2020-23:17:06.276547 [Drop] [] [1:2500154:5627] ET COMPROMISED Known Compromised or Hostile Host Traffic group 78 [] [Classification: Misc Attack] [Priority: 2] {TCP} 139.59.116.115:46910 -> <my ip address:and port>

From the Alerts log, this is a SID Mgmt action to Drop rules line where it clearly shows that it wasn't dropped:
11/24/2020-23:37:01.804877 [] [1:2500452:5627] ET COMPROMISED Known Compromised or Hostile Host Traffic group 227 [] [Classification: Misc Attack] [Priority: 2] {TCP} 67.205.141.165:48618 -> <my ip address:and port>

[sorry, the asterisks are missing in both lines]

Here is my Interface SID Management List Assignments:

Have I done something incorrect? Or does SID Mgmt not show a Drop?

bmeeks

When you make changes on the SID MGMT tab, you need to do one of the following in order for the change to take effect. Nothing within Suricata is dynamic (as in it "just happens" without any user effort).

When you make a cfhange, place a check in the box under the Rebuild column for the interface you are modifying. This instructs Suricata to rebuild its internal rules file and then load the new file.
Go to the INTERFACES tab and physically restart Suricata on the interface you modified on the SID MGMT tab.

There is also a curently open bug on the upstream Suricata Redmine site that says rule action changes made when "live reload" is active are not honored. That could be an issue for you when you check the Rebuild checkbox as that will use the "live reload" feature of the Suricata binary. In that case, saving the SID MGMT tab changes and then going to the INTERFACES tab and cycling the affected interface should work.

All the GUI code in the pfSense package does is create the text-based suricata.yaml configuration file for the binary suricata executable to read. Anytime you make changes to configuration in the GUI you generally need to restart the binary for the changes to be seen. This entails going to the INTERFACES tab and restarting the binary from there. There are a couple of exceptions when "live reload" can work. One is when you manually "force" a rule and the other is when the Rebuild checkbox is clicked on the SID MGMT tab.

newUser2pfSense

I actually did the Rebuild on the interface thinking that Suricata would update/reload the rules. However, I didn't go to the interfaces tab and physically restart Suricata on the interface. I'll give that a try.

bmeeks

@newUser2pfSense said in Using Suricata SID Mgmt:

I actually did the Rebuild on the interface thinking that Suricata would update/reload the rules. However, I didn't go to the interfaces tab and physically restart Suricata on the interface. I'll give that a try.

One other thing to do is to check the actual active rules file to see if the SID MGMT regex is actually finding and modifying the rule or rules you want changed. You can find the file by going to the RULES tab and then choosing "Active Rules" in the drop-down there for viewing. If you have a ton of selected rules, it can take a long time for the page to load after selecting "Active Rules". You would be looking for the specific SIDs you changed and verifying the action was changed to DROP.

You can also go to the LOGS VIEW tab and select the SID Mgmt log for viewing. You will see entries in there detailing whether or not your regex identified rules for changing and how many it changed.

newUser2pfSense

I'm currently running pfSense 2.4.5-RELEASE-p1 (amd64) and Suricata 5.0.4.

The manual action to Drop the rules are currently like this -
ET COMPROMISED Known Compromised or Hostile Host Traffic group 1
thru
ET COMPROMISED Known Compromised or Hostile Host Traffic group 217
[this if from when I was manually setting the action to Drop]

The SID Mgmt entry in the dropRules.conf file are collectively -
ET COMPROMISED Known Compromised or Hostile Host Traffic group 1
thru
ET COMPROMISED Known Compromised or Hostile Host Traffic group 247
[1:2500000-1:2500492]

So, ET COMPROMISED Known Compromised or Hostile Host Traffic group 218
thru
ET COMPROMISED Known Compromised or Hostile Host Traffic group 247
the action is not shown to Drop in the Active Rules. They are still shown to Alert.

In the Logs View tab, the Log File to View dropdown doesn't show a SID Mgmt log to view. The closest is a sid_changes.log which dispalys - Log file does not exist or that logging feature is not enabled.

bmeeks

sid_changes.log is the correct file. I was working from a faulty memory ... .

You must also select the proper interface in the Interface drop-down on that LOGS VIEW tab when viewing logs. It will default to the WAN when initially opened. Perhaps you were trying to find a sid_changes.log file for an interface which does not have any SID MGMT changes configured.

If you had the correct interface selected and it says no sid_changes.log file exists, then you have not saved something correctly and run a rules rebuild. That log file is only created when SID MGMT changes are enabled, a valid configuration file is created and assigned to an interface, and then the interface is cycled to load the change. That's when the log file is written.

newUser2pfSense

Hmmmmm...my WAN and LAN Logs View tab look like this -

I'm not sure what I've done wrong. In SID Mgmt, I checked the Rebuild boxes and clicked Save for the interfaces and then restarted the interfaces before I checked the Logs View tab above.

bmeeks

At the very top of the SID MGMT tab is a checkbox to "Enable" that feature. I assume you placed a check there and saved it?

If no log file is available, that strongly indicates SID MGMT is not enabled for the interface. That log file is written when the rules file for an interface is created. Part of the creation of the rules file is processing SID management changes (when enabled and when rules match the regex) and any user-forced rule changes to action or state. Even if no regex matches are detected, the sig_changes.log is still created showing zero matches when SID Managment is enabled on an interface.

Trust me the process works. I have a standard test that is part of my testing suite for every new Suricata package release. It changes the ET-Scan rules to DROP via SIG MGMT.

newUser2pfSense

Doh...Again, no one has ever accused me of being smart .

I checked the box at the top of SID Mgmt, chose the interfaces to Rebuild and Save, restarted the interfaces and then checked the Active Rules. All are now set on the action to Drop. As well, I can now view the sid_changes.log file.

Nice!!! Whew. I'm so glad this works.

newUser2pfSense

Ok, so now the Suricata Updates are displaying "Not Downloaded".
Installed Rule Set MD5 Signatures.png

So I chose to Force update and a half hour later, the updates are still not downloaded. In the Status > System Logs > System > General, I'm seeing -

[Suricata] There is a new set of Emerging Threats Open rules posted. Downloading emerging.rules.tar.gz...
[Suricata] Emerging Threats Open rules file update downloaded successfully.
[Suricata] There is a new set of Snort rules posted. Downloading snortrules-snapshot-29151.tar.gz...

The Snort rules is where it looks to have stopped.

Interestingly though, more than a half hour later, the updates still appear to be downloading -
Rules Update Task.png

Not sure if this is normal or not.

newUser2pfSense

So a hard restart of pfSense resolved the update issue. I think I'm out of the weeds now.

Thanks Bill.

bmeeks

@newUser2pfSense said in Using Suricata SID Mgmt:

So a hard restart of pfSense resolved the update issue. I think I'm out of the weeds now.

Thanks Bill.

You're not 100% out of the woods. You need to change your configuration to pull down the most current Snort 2.9.x rule set. That 2.9.15.1 version is now outdated. Read the information in this thread to understand why and how you must manually configure Suricata to obtain the most current Snort rules: https://forum.netgate.com/topic/110325/using-snort-vrt-rules-with-suricata-and-keeping-them-updated.

newUser2pfSense

Thanks for the link Bill. Read it all and will keep in mind to check every 30 days for any Snort rule updates. I changed the 2.9.15.1 version of the Snort rules to snortrules-snapshot-29170.tar.gz. Suricata updated with no issues.