Suricata Package Updated to 3.1.2 – Release Notes

bmeeks

Suricata v3.1.2 Release Notes

This update of the Suricata GUI package adds three new features, restores Pass List functionality when running with inline IPS mode enabled, and corrects four bugs. The underlying Suricata binary is also updated to version 3.1.2.

IMPORTANT UPDATING INFORMATION
To be sure you get the new 3.1.2 version of the Suricata binary, you should uninstall the Suricata package and then reinstall instead of just clicking the upgrade button. It appears that pkg will not swap out the existing 3.0 binary for the new 3.1.2 binary on a plain upgrade. Removing the package and installing it again does remove the binary and forces the download of the new 3.1.2 binary. Make sure you have the "Save Settings" box checked on the GLOBAL SETTINGS tab so the uninstall will not cause you to lose your settings. This box is checked by default.

New Features

The RULES tab has an action column with an icon to indicate if the rule action is alert or drop. Clicking the icon allows toggling the rule action via a user override. The GID:SID of the rule along with the user overriden action are both stored in the firewall config for Suricata and used to set the rule action when the enforcing rules file for the interface instance is written (the suricata.rules file located in the /rules directory of the interface).
The list of rules displayed on the RULES tab can now be filtered by state (enabled or disabled) to help with identifying enabled or disabled rules.
A new option has been added when operating with inline IPS mode enabled. When the Snort VRT rules package is enabled and a Snort VRT IPS Policy is chosen on the CATEGORIES tab, a selection is available for automatically setting the rule action for the chosen policy rules. This action is specified in the IPS policy metadata within the rule. When using an IPS Policy, the default action for all the policy rules is alert. But each of the policy rules also contains a suggested action for each policy category (Connectivity, Balanced, Security, etc.). The suggested action may be to only alert on traffic, or it may instead suggest dropping the traffic. The new IPS Policy Mode selector has two settings: Alert or Policy. When set to Alert, all IPS Policy rules will just alert no matter what the rule metadata specifies for the policy. When set to Policy, the rule action will be changed to that specified by the IPS Policy metadata provided with each VRT rule. When this mode is selected, you can see which rules in the policy are set to drop and which to alert by going to the RULES tab and selecting the IPS Policy in the drop-down categories selector. All of the IPS Policy rules will display and icons in the Action column will indicate if the rule action is alert or drop. Note that for the large policy sets like Balanced or Security, it may take several seconds to complete the rules load and display them.

Restored Functionality

When using inline IPS mode the Pass List function was not available due to changes in the blocking architecture from dependence on the pf firewall to the use of Netmap. The Pass List functionality has now been duplicated when inline IPS mode is enabled by automatically generating the required PASS rules in the Suricata rule set. The rules will prevent traffic from hosts on the Pass List from being dropped. You still create and assign Pass Lists the same as in Legacy Mode, but now they will also "just work" when using inline IPS Mode.

Bug Fixes

Auto Log Mgmt is not default checked on a fresh install.
REFRESH checkbox on ALERTS tab is not default checked on a fresh install.
Suricata error "[ERRCODE: SC_ERR_CONF_YAML_ERROR(242)] - Please use 'tls-store' in YAML to configure TLS storage" generated when TLS logging is enabled.
Make sure critical Suricata directories in /var/log and /var/db exist at each package sync in case these are on a RAM disk.

Important Information on Rule State/Action Overrides and Order of Precedent

This discussion is to help administrators understand the mechanics behind the rule state and action overrides available in the package GUI.

Rule signatures used by Suricata to detect malicious traffic must have an action keyword specified. The valid action keywords are: alert, pass, drop or reject. The action keyword is the very first parameter in a rule signature. The action is rather self-explanatory. "Alert" will print an alert in the log and display on the ALERTS tab. "Pass" will simply allow the traffic to pass. "Drop" will block the traffic by simply discarding the packet. The sender will receive no indication of what has happened to the packet. It is a silent drop. "Reject" will send a notification packet back to the sender advising the destination rejected the packet. There is an action order parameter in the Suricata configuration file that tells the engine in what order to process rule signatures. In the pfSense package, this order is the default of Pass, Drop, Reject then Alert. This means all Pass rules are evaluated first, and if none match the packet the inspection then proceeds to all Drop rules, then Reject rules and finally Alert rules. A new feature has been added to allow the user to force a rule action to either "forced to alert" or "forced to drop". This forced action will override the default rule action as received from the rule vendor.

The other critical parameter for a rule signature is its state. The state can be either Enabled or Disabled. There is no actual place in the rule for this parameter. Instead, you disable a rule by simply placing a comment character ("#") at the start of the rule line. So "commented out" rules are not loaded by the Suricata rules engine and thus are considered "disabled". All loaded rules are considered as "enabled". Some new IDS/IPS users have an incorrect assumption about the rules packages available from vendors such as the Snort VRT and Emerging Threats (now Proofpoint). Those rules packages are published with a number of the rules in the categories commented out or "disabled" by default. There are various reasons for this. No matter the reason, never assume all rules in a rule category are actually "active". Suricata helps you in this area by displaying icons next to each rule on the RULES tab showing the state of the rule. There are four possible states to show: (1) default enabled, (2) default disabled, (3) forced enabled by user and (4) forced disabled by user. The first two would indicate the rule's state "as is" from the rule creator. So "default" would indicate the vendor shipped the rule in the current state (which may be enabled or disabled). The other two indications show a user-override of the state. The user has clicked on the rule to force it to always be either enabled or disabled no matter what the default state from the rule vendor might be. This override will survive rule updates because it is stored in the firewall's configuration and will be applied at the end of each rule set generation for the monitored Suricata interface.

Now let's talk about how a collection of rules is generated for the Suricata binary to inspect traffic against. Each time you start Suricata from the GUI, and when saving the configuration from the GUI pages, a process is called to generate the suricata.yaml file that tells the Suricata binary what to do. This file is specific to each Suricata-monitored interface. In addition to creating the configuration file, the GUI code also creates up to four custom rules files for Suricata to read rules from. Those files are named as follows: (1) suricata.rules, (2) passlist.rules, (3) custom.rules and (4) flowbit-required.rules. Note the _passlist.rule_s file is only generated when Suricata is operated with the inline IPS mode enabled. The suricata.rules file contains all of the "enabled" rules you have chosen from the rules categories provided by the vendors. Only rules that match all of your specified selection criteria get written to the suricata.rules file. The custom.rules file obviously contains any custom user rules you create. The flowbit-required.rules file contains any rules you may not have selected as enabled but that are required by one or more of the rules you did enable. Finally, when using inline IPS mode, the passlist.rules file contains the generated Pass rules that allow Pass List hosts to bypass the blocking mechanism.

As a user, you have two mechanisms for "forcing your will" upon the rules. You can force-enable, force-disable, force-alert or force-drop specific rules using the appropriate icons on the RULES tab. You can also accomplish the same thing using the automatic SID management features on the SID MGMT tab. There is one important thing to keep in mind! The SID management changes are processed first, then the specific user overrides for force-enabled, force-disabled, force-alert and force-drop are processed. So clicking the icons to force action or state will be the last word for the rule and will override the default setting from the rule vendor and any setting changes from SID MGMT tab.

Thanks again for the effort Bill. Now I can move Suricata inline to prime time on the WAN instead of snort.
The new features is a real time saver. Much appreciated. Snort on WAN and Suricata Inline on LAN have been working great. Time to switch both interfaces. No conflicts so far with snort2c. ;D

gsiemon

Bill,

Thankyou for this. It sounds like you have been really busy recently. I upgraded my Suricata package (from whatever the previous version was) on pfSense 2.3.2_1 today, by clicking on the update icon next to the suricata package. I noticed after the install had reported success that the Suricata.log file was still showing that it was on 3.0 ie the first line in the logs said


This is Suricata version 3.0 RELEASE
``` 
I stopped the service and restarted and got the same entry in the logs.  Checking "suricata -V" from a command prompt generated the same response. 

I've just uninstalled and reinstalled the package and its now showing

This is Suricata version 3.1.2 RELEASE


I didn't look closely at the first package upgrade log, but it appears that may not have pulled down the updated binary.

Worth others checking to see if this is a bug or just restricted to my system.

"Worth others checking to see if this is a bug or just restricted to my system."
Same here. Cleared up after uninstall then install. Had in General system log ,kernel netmap bad pkg messages. It may have not loaded the binary properly. Good catch. ;)

bmeeks

@gsiemon:

Bill,

I've just uninstalled and reinstalled the package and its now showing
This is Suricata version 3.1.2 RELEASE  
I didn't look closely at the first package upgrade log, but it appears that may not have pulled down the updated binary.

Worth others checking to see if this is a bug or just restricted to my system.

I'm not sure exactly what time today the 2.3.x pfSense package repository updated with the new Suricata binary. It could be you caught it in between the update of the GUI packge and the compilation and post of the updated binary.

Bill

gsiemon

@bmeeks:

I'm not sure exactly what time today the 2.3.x pfSense package repository updated with the new Suricata binary. It could be you caught it in between the update of the GUI packge and the compilation and post of the updated binary.

Bill

Thanks Bill. Its possible. I upgraded the package only a few minutes before I posted, so a good 8 hours after your post.

doktornotor


$ suricata -V
This is Suricata version 3.1.2 RELEASE

Looks just fine on 2.3.3 snapshots.

dhboyd26

I also had to uninstall, then reinstall to get the binary to update. Not sure why, but it appears to not want to update the binary if it is already there. All other bits and pieces of the GUI appeared fine, just no Suricate binary upgrade.

Working now, thanks bmeeks!

n3by

Thank you for pointing this update problem.

I updated Suricata today without any problem, I did not checked until now when it reveal v 3.0.
Uninstalled and reinstalled and now is on v 3.1.2

bmeeks

I'll relay this issue to Renato on the pfSense team. He handles the binary packages stuff. Thanks for the reports.

Edit: added a new caution to the Release Notes post at the start of this thread.

Bill

doktornotor

Hmmm, odd since I did not uninstall before upgrade for sure.

mind12

Hello,

please help me understand the new Inline mode passlist feature.

My questions are:

How can I find the generated PASS rules in the GUI? I found them through the CLI by searching for the passlist.rules file.
By default Suricata uses the default passlist on the running interfaces and it includes the local networks. Does this mean that all the traffic inbound and outbound from those networks will be ignored and passed by Suricata? If yes (I assume because Suricata ALERTS stopped after I upgraded to the new version and appeared after modifying the passlist), I would definitely instruct users to modify the default passlist or Suricata will be non-effective.

Thank you in advance.

bmeeks

@mind12:

Hello,

please help me understand the new Inline mode passlist feature.

My questions are:

How can I find the generated PASS rules in the GUI? I found them through the CLI by searching for the passlist.rules file.

By default Suricata uses the default passlist on the running interfaces and it includes the local networks. Does this mean that all the traffic inbound and outbound from those networks will be ignored and passed by Suricata? If yes (I assume because Suricata ALERTS stopped after I upgraded to the new version and appeared after modifying the passlist), I would definitely instruct users to modify the default passlist or Suricata will be non-effective.

Thank you in advance.

Currently the PASS rules cannot be viewed directly in the GUI. I can see about adding that as a new feature in an upcoming update.

You are correct that Pass List hosts will not generate blocks or alerts currently with the default rule action order being PASS, DROP, REJECT then ALERT. The Pass List can be customized quite easily by creating a list and checking/unchecking the options on the pass list creation page. You can create an alias to hold an essentially unlimited number of other custom addresses. Remember aliases can be nested, so you can create a single alias to reference in the Pass List and then put all other aliases you want included within that single alias.

You can see the addresses and networks that will be in the new auto-PASS rules with inline IPS mode by going to the INTERFACE SETTINGS tab for the interface and scrolling down to the Pass List section. There is a button there for viewing the contents of the pass list (the IP addresses and networks). While it won't show the exact PASS rules, any IP address shown there will get translated into a PASS rule.

You can remove Local Networks and other defaults from the Pass List when you create your own. Just be aware that those hosts can then become blocked, and that may not be what you really want in all cases. Generally speaking it is external traffic you are trying to block. There is likely some room for improvement in how the new auto-Pass List feature is implemented.

Bill

ntct

Hi Bill,

How to donate this package?? :)

mind12

@bmeeks:

@mind12:

Hello,

please help me understand the new Inline mode passlist feature.

My questions are:

How can I find the generated PASS rules in the GUI? I found them through the CLI by searching for the passlist.rules file.

By default Suricata uses the default passlist on the running interfaces and it includes the local networks. Does this mean that all the traffic inbound and outbound from those networks will be ignored and passed by Suricata? If yes (I assume because Suricata ALERTS stopped after I upgraded to the new version and appeared after modifying the passlist), I would definitely instruct users to modify the default passlist or Suricata will be non-effective.

Thank you in advance.

Currently the PASS rules cannot be viewed directly in the GUI. I can see about adding that as a new feature in an upcoming update.

You are correct that Pass List hosts will not generate blocks or alerts currently with the default rule action order being PASS, DROP, REJECT then ALERT. The Pass List can be customized quite easily by creating a list and checking/unchecking the options on the pass list creation page. You can create an alias to hold an essentially unlimited number of other custom addresses. Remember aliases can be nested, so you can create a single alias to reference in the Pass List and then put all other aliases you want included within that single alias.

You can see the addresses and networks that will be in the new auto-PASS rules with inline IPS mode by going to the INTERFACE SETTINGS tab for the interface and scrolling down to the Pass List section. There is a button there for viewing the contents of the pass list (the IP addresses and networks). While it won't show the exact PASS rules, any IP address shown there will get translated into a PASS rule.

You can remove Local Networks and other defaults from the Pass List when you create your own. Just be aware that those hosts can then become blocked, and that may not be what you really want in all cases. Generally speaking it is external traffic you are trying to block. There is likely some room for improvement in how the new auto-Pass List feature is implemented.

Bill

Thank you for the information.
One side note:
Yes we are trying to block usually external traffic but if you leave the local networks in the default passlist you won't block any external traffic at all. The passlist will allow all the traffic FROM and TO the local networks. We can't change this behavior because in the generated rules the sources and the destinations are any any.
ie:```
pass ip 127.0.0.1/32 any <> any any (msg:"Pass List Entry - allow all traffic from/to 127.0.0.1/32"; sid:1000001;)

Redyr

@bmeeks:

@mind12:

Hello,

please help me understand the new Inline mode passlist feature.

My questions are:

How can I find the generated PASS rules in the GUI? I found them through the CLI by searching for the passlist.rules file.

By default Suricata uses the default passlist on the running interfaces and it includes the local networks. Does this mean that all the traffic inbound and outbound from those networks will be ignored and passed by Suricata? If yes (I assume because Suricata ALERTS stopped after I upgraded to the new version and appeared after modifying the passlist), I would definitely instruct users to modify the default passlist or Suricata will be non-effective.

Thank you in advance.

Currently the PASS rules cannot be viewed directly in the GUI. I can see about adding that as a new feature in an upcoming update.

You are correct that Pass List hosts will not generate blocks or alerts currently with the default rule action order being PASS, DROP, REJECT then ALERT. The Pass List can be customized quite easily by creating a list and checking/unchecking the options on the pass list creation page. You can create an alias to hold an essentially unlimited number of other custom addresses. Remember aliases can be nested, so you can create a single alias to reference in the Pass List and then put all other aliases you want included within that single alias.

You can see the addresses and networks that will be in the new auto-PASS rules with inline IPS mode by going to the INTERFACE SETTINGS tab for the interface and scrolling down to the Pass List section. There is a button there for viewing the contents of the pass list (the IP addresses and networks). While it won't show the exact PASS rules, any IP address shown there will get translated into a PASS rule.

You can remove Local Networks and other defaults from the Pass List when you create your own. Just be aware that those hosts can then become blocked, and that may not be what you really want in all cases. Generally speaking it is external traffic you are trying to block. There is likely some room for improvement in how the new auto-Pass List feature is implemented.

Bill

Hello Bill,

Thank you for making Suricata 3.1.2 possible. Some of us, including me wanted this to happen, and as usual with new stuff things can go wrong, I understand. I will give you a feedback in regards to the new version, because right now I keep it disabled, due to issues that @mind12 reported.

First the pass list includes all the network ranges plus 2 external IPs marked in red, which are not mine, not my ISP's, just external hosts…Why Suricata will include those by default? - Please see the attached picture.
The pass list for WAN and for LAN contains the same list of IPs, is it ok?
If I try to manually edit the pass list file, the same IPs will be present next time I restart the service.
If I try to manually create a custom pass list, the GUI will say the alias is not eligible. Please see the attached Screenshot. What kind of alias should I define? I tried all the types.
If Suricata is enabled, no traffic will be analyzed, and I cannot change that default pass list, even if I delete it, it will be re-written.
I tried to uninstall Suricata, and I unchecked the "Settings will not be removed during package deinstallation." option. Why after reinstalling the same configuration is still present? I notice this with other packages for eg. Squid
Also which IPs should I include in the pass list for WAN or for LAN in order to be safe, and not get blocked?
The suppresed user rules from 3.0 version, are now present in Wansurpress and Lansurpress lists, but the Suricata log states that "no such rules are present"

Please take your time, and come back with some hints.

![Services_ Suricata_ Edit Interface Settings - WAN passlist.png](/public/imported_attachments/1/Services_ Suricata_ Edit Interface Settings - WAN passlist.png)
![Services_ Suricata_ Edit Interface Settings - WAN passlist.png_thumb](/public/imported_attachments/1/Services_ Suricata_ Edit Interface Settings - WAN passlist.png_thumb)
![Suricata_ Select alias.png](/public/imported_attachments/1/Suricata_ Select alias.png)
![Suricata_ Select alias.png_thumb](/public/imported_attachments/1/Suricata_ Select alias.png_thumb)

mind12

Hi Redyr,

Regarding creating a custom Passlist:

first create an alias at Firewall/Aliases/IP menu containing the networks that you would like to always pass (ie. I used - firewall interfaces, dnsbl address), don't add the protected local networks to it
go to services/suricata/pass lists, create a new list and add the alias to it
go to the suricata interface settings and pick the created list
save and restart suricata

These steps worked for me.

Redyr

10x @mind12 , I'll try what you suggested

bmeeks

@ntct:

Hi Bill,

How to denote this package?? :)

I don't understand your question. Can you clarify what you mean by "denote"?

Thanks,
Bill

bmeeks

@mind12:

Thank you for the information.
One side note:
Yes we are trying to block usually external traffic but if you leave the local networks in the default passlist you won't block any external traffic at all. The passlist will allow all the traffic FROM and TO the local networks. We can't change this behavior because in the generated rules the sources and the destinations are any any.
ie:```
pass ip 127.0.0.1/32 any <> any any (msg:"Pass List Entry - allow all traffic from/to 127.0.0.1/32"; sid:1000001;)

You are correct. This is my mistake. Should have used the options of SRC, DST or BOTH as part of the logic when generating the PASS rules. Using "any" and "any" for the SRC and DST is not good. Let me think this over in my head and fix it, but I need to think about it carefully. Any suggestions from users are welcomed!

Bill