It is processed first and then those IPs don't hit the other rules, but they will still generate a "blacklist" alert. Are you getting alerts beside the "blacklist" alert?

@DmitryDev said in Suricata Custom rule: error in with content, offset... How it works? INVALID SIGNATURE:

@bmeeks Sorry for my very bad English. You understood me correctly.
I read the documentation from the official Suricata site.

I'll try to see log file.

I do not mean to fault you for your English! I speak and write only a single language, so I am impressed with those who are multilingual. It's just that the differences in sentence structure among the world's languages make translation a bit tricky sometimes ... ☺ .

Post back if you need additional help. User @NogBadTheBad frequents this forum and he is a very good rule author.

Thanks! It is working now.

@bmeeks said in Suricata syslog is truncated!:

You will have much better luck with something like GrayLog or setting up an ELK stack on a separate host. Suricata is logging more and more of its output in JSON format. This is the direction of the upstream developers. So you need to enable the JSON logging options for each interface and then create a system on your own to suck up the JSON log files and export them off to a separate consolidator host. I believe some users here have tried filebeat and had success with that. It can process JSON logs.

Thanks, actually i'm receiving EVE JSON over syslog now (bcoz payload data is needed).
it seems i should try filebeat.

@Simbad said in Flowbit IDs in the current ruleset exceeds the maximum number of IDs that are allowed (1024):

The number of flowbit IDs in the current ruleset exceeds the maximum number of IDs that are allowed

To fix this, edit the following file:

/usr/local/pkg/snort/snort_conf_template.inc

Add this new line immediately above line 38:

# Configure maximum number of flowbit references. For more information, see README.flowbits # config flowbits_size: 2000

The maximum allowed value for flowbits_size is 2048, so you can experiment by lowering or increasing this value for your setup, but you can't exceed 2048.

After making this edit, got to the INTERFACE SETTINGS tab for each configured Snort interface and click Save to re-generate the configuration file for that interface. Then go to INTERFACES and start/restart that interface.

This change will be overwritten if you remove and install Snort again. I will put this on my bug fix list and increase the value in a future Snort package update.

@rizkhan99 said in Suricata/Snort not starting (Resolved):

@bmeeks @JohnSCarter

Guys, even after following all the guidelines, my snort and suricata packages remain disabled even in the "Status" -> "Services" option. Trying to enable them from the "Interfaces" option in "Services" -> "Snort" or "Suricata" is also not working. The log files e.g. suricata.log are also empty. System log file show the following message (for Suricata) which seem to be normal but still these services don't start:

May 3 22:04:56 php /tmp/suricata_bce039898_startcmd.php: [Suricata] Suricata START for WAN(bce0)... May 3 22:04:56 php /tmp/suricata_bce039898_startcmd.php: [Suricata] Building new sid-msg.map file for IPCORE... May 3 22:04:55 php /tmp/suricata_bce039898_startcmd.php: [Suricata] Updating rules configuration for: IPCORE ... May 3 22:04:55 php-fpm 63967 /suricata/suricata_interfaces.php: Starting Suricata on IPCORE(bce0) per user request... May 3 22:04:41 SuricataStartup 20258 Suricata START for WAN(39898_bce0)... May 3 22:04:25 check_reload_status Syncing firewall

I have tried enabling snort and suricata from terminal by the following commands:

/usr/local/etc/rc.d/snort start /usr/local/etc/rc.d/suricata start

The output says the service has started however "ps -ef | grep snort" or suricata doesn't show up anything.

The following commands also say that the service is "not" running:

/usr/local/etc/rc.d/snort status /usr/local/etc/rc.d/suricata status

I have checked all this on both snort and suricata by having installed only one of these packages at a time, to avoid any conflicts between these packages, if any. However, no success.

My pfsense version is: 2.4.4-RELEASE-p2
FreeBSD version is 11.2-RELEASE-p6
Snort version is 3.2.9.8_5
Suricata version is 4.1.2_3

Please help...

Regards,

Rizwan

First of all, you do not start/stop these packages using the command line. You need to do it from the GUI on the INTERFACES tab in either Snort or Suricata (depending on which you have installed at the moment).

Have you done all of the steps outlined in my previous post? If so, then go to SERVICES > SURICATA and the Interfaces tab will be showing. Click the start icon to start the process. You will see a green gear spinning while the process starts up. If it fails to start, then you will find the reason by going to the LOGS VIEW tab and opening and viewing the suricata.log file for the interface you tried to start up.

If the above steps do not either resolve the issue or give you a clue on what's wrong (Suricata is very good about logging any errors during startup), then open a CLI session on the firewall and type this command just to see if Suricata and its dependencies are properly installed:

/usr/local/bin/suricata -v

That should result in a printout to the terminal showing the installed Suricata version and some basic copyright info. If you see any messages about missing libraries or anything else, then Suricata did not properly install. For what it's worth, the only time I've seen an empty suricata.log file for an interface is when the installation did not complete and therefore some dependency library is missing. In that case, Suricata can't even start as the OS will refuse to start it due to the missing libraries. When it isn't allowed to start by the OS, then of course it can't log anything to the suricata.log file for the interface. If that's what is happening in your case, then the CLI command I posted will uncover the problem.

Post back here what you find if you still have problems.

thanks for your detailed information.

Update - I probably should have started my own thread as situation appears unique.

My original idea was to tweak some of the general pfSense IPv6 settings. That didn't work.

On a closer look, I realized my IPv6 alerts were for port scanning. When I turned on AppID, I also turned on Portscan Detection. I could have turned it off PS Detection completely, but I simply changed the sensitivity from 'medium' to 'low' and I haven't had another alert since.

@Simbad said in Snort Package Update to v2.9.13 (binary) and v3.2.9.8_6 (GUI) - Release Notes:

Hi, why don't you update openappid? (https://files.pfsense.org/openappid/).

https://blog.snort.org/2019/04/update-to-snort-openappid-detectors.html

You are confusing the available free OpenAppID rules (written by a third-party and hosted by Netgate) with the OpenAppID rule stubs which are produced by the Snort team. That post on the Snort blog was about the rule stubs. These are two separate things, but you need both for OpenAppID to work. The rule stubs (the portion produced by the Snort team) will automatically update at your next rules update after they are posted to the Snort site. The free OpenAppID rules, on the other hand, only update if and when the third-party author (who was affiliated with a University in Brazil) makes a change. I don't think he has made any changes in quite some time.

The rule stubs are the foundation upon which OpenAppID works, but without the text rules written by that third-party OpenAppID does not work. You are also free to create your own OpenAppID rules using the latest features afforded by the new rule stubs. You can add them as Custom Rules on the RULES tab.

Are you sure you copied that SID correctly? I can't find it in the Snort rules lookup site. I did a quick Google search for "Sitescout" and found this. The site describes itself as a self-serve advertising platform where apparently buyers "bid" for advertising space or something like that. I did not read all the documentation.

What rule category is that rule from? Offhand I would think it's not malicious by itself, but if it is an ad server site, it's certainly possible for someone to compromise a server there and then it could become malicious.

@NRgia said in Suricata 4.1.3 Update (posted for pfSense 2.5 Development) - Release Notes:

Hello @bmeeks
A new port version for Suricata is available 4.1.3_2 at Freshports . It's not a major update, but it includes an update for Rust language, which is used by Suricata.

Maybe when you have time, can you update the package for the guys to test in pfSense-2.5 DEVELOPMENT ?

https://www.freshports.org/security/suricata/

Thank you

This will happen sort of auto-magically next quarter when the pfSense devs update FreeBSD-ports for pfSense to the latest version of the upstream FreeBSD ports tree. They sync the DEVEL tree to FreeBSD ports upstream each quarter. For example, just this past April 1st the DEVEL tree was updated. That means the Rust 1.34 and Suricata 4.1.3_2update was just missed, but it should get picked up with the June 1st sync. At that point Suricata will build with the new Rust version.

Yeah, that's a quirk in the logic. Sort of an edge case in a way. The logic for the DROP SID assignment list first checks that blocking is enabled and then checks if the mode is "IPS Inline" or "Block Drops Only". Only if those conditions are true will the drop-down get populated with the currently selected list. When the conditional evalutates to FALSE, then the list is set to "Not Applicable" on the assumption that without the proper blocking mode enabled there is no point to selecting a DROP SID list.

In your case, by turning off blocking before removing the list, it tripped up the conditional test. I can improve that by not triggering the "you can't delete this list" message when the proper blocking mode is not enabled on the interface. I will put that in my bug list for Suricata to address in a future update. Thanks for the report and especially for the follow-up giving the solution.

Since you did not provide a sample of your rules or a screenshot of your interface configuration all I can do is guess. Here are some guesses:

You did not check the box to "kill states" when blocking an offender; You are testing with IP addresses that are contained within the default Pass List used by Legacy Mode blocking; You used two mutually exclusive terms to describe your setup, "legacy" and "inline" don't go together in Suricata. You must use either Legacy Mode blocking or Inline IPS Mode blocking. There is no such thing as "legacy inline", so which do you actually have configured? The fact you said the "alerts show up in red" indicates you may be using Inline IPS Mode; If you are actually using Inline IPS Mode and your interface is a PPPoE configuration, then it won't block as PPPoE traffic does not pass through a netmap interface. Inline IPS Mode uses netmap; With Inline IPS Mode, you must actually change the rule action to DROP in order to drop or block traffic. If you leave the rule action as ALERT, then that's all you get: just an alert in red.

Read up on Suricata's operation and how to use SID MGMT features here: https://forum.netgate.com/topic/128480/how-automatic-sid-management-and-user-rule-overrides-work-in-snort-and-suricata.

@gbqs
Snort cannot run as a true inline IPS at this time on pfSense. Suricata can providing you have NIC hardware whose drivers fully support Netmap on FreeBSD. Not all do, so beware.

@johnpoz is correct. Having an IDS/IPS or pfBlockerNG is not mandatory to secure your data. They are just two of many different tools that when used in the right context for the right reason can enhance security. But they are not required. It all depends on the specific network that needs protection and what constitutes "normal" traffic on that network.

My personal opinion is that most small home networks really don't need either package. The very best security practice is simply being committed to keeping your software packages updated. This means the firewall itself and of course any client applications on PCs, tablets, phones, etc. That simple practice goes a very long way towards enhancing security.

If you have network users at home that are what I call "free clickers" (meaning they will click on any link anywhere .. 😁 ), then it might be helpful to have some additional tool such as an IDS/IPS or pfBlockerNG to help protect those users from themselves. On the other hand, if you have responsible, alert and careful users (that watch what they click), you very well need nothing else besides maybe the built-in anti-virus that comes with Windows just so you can scan any files you download.

In a business network, there are other considerations where using an IDS/IPS or a tool such as pfBlockerNG with its geo-blocking capability is helpful to security. A great use of an IDS/IPS in a business network is to let it scan outbound traffic using rules that look for malware CNC server and botnet destinations, traffic destined to known untrusted countries, or any other traffic that should not normally be exiting your network. For example, if you have internal DNS servers that clients are configured to use, you could have a rule that would alert on any outbound DNS request that did not originate from your internal DNS server. Another handy thing for business networks would be using Snort's OpenAppID technology to identify non-work related traffic that violates a business policy.

I am not a fan of having a list of say a couple of million IP addresses that my firewall is actively blocking. I would instead turn that around and be much more specific with what I allow in and then let the default deny rule take care of everything else. Your firewall will sweat a lot less and you won't have memory and stability issues caused by having huge IP block lists. Do a quick search here on the forum for users posting about Unbound problems that are frequently the result of having huge DNS blacklists enabled. I know some folks use this feature for ad blocking, but I prefer to do ad blocking at the client level using tools like uBlock Origin in the browser. Between that and AdBlock for YouTube I don't see a single add on any web site I visit or any YouTube video I watch. Granted I'm an old fart and do my web browsing on a PC where the screen is big enough for me to see it ... 😁 . Maybe if all my browsing was on my iPad or iPhone, where ad blockers are not as prolific, I might go for something like Pi-Hole or DNSBL.

Just my two cents worth for the debate ...
Bill

@bmeeks thanks for your great help!!!

@talaverde said in ET Open Ruleset not downloading:

pfBlockerNG is a possiblity

A package such as pfBlockerNG is a very useful tool, but it can be misused or misapplied sometimes leading to frustration. It works essentially as a long list of IP addresses to be blocked. Those lists can be configured from many sources. Not all of the sources are "current", and even those that are can frequently contain errors in the form of a legitimate web site IP address or netblock being lumped into a "bad actor" list.

So when you have a security tool such as an IDS/IPS layered with another security tool such as pfBlockerNG, you have to immediately consider any "failing to connect" issues on your network as being caused by one of those packages. So in your case, if I saw failing ET-Open downloads, my first instinct would be to check my pfBlockerNG blocks to see if the address had gotten inadvertently blocked. The rules vendors use various CDNs (content distribution networks) to host their rules file for worldwide download. Sometimes a pfBlockerNG list might get overly aggressive and block one of those CDNs (or a segment of a CDN) because a bad actor IP lives in the same netblock. This has happened to folks in the past with AWS addresses.

In the same vein, if I had connectivity issues on a client with a web site or other service, I would check both the IDS/IPS alerts to see if the address showed up there as well as the pfBlockerNG alerts to see if something there tagged it. I would do that before I considered anything else on the client itself. Neither of these tools (Snort/Suricata nor pfBlockerNG) is a "click it on and forget it" type of package. They require constant baby sitting by a knowledgeable admin.

So in the future, when you have any kind of connectivity issues outside of something obvious like a hardware failure, look first at your IDS/IPS and pfBlockerNG tools as the source of the connectivity issue. Only after eliminating both packages as the cause of the "block" should you look at potential client issues such as software bugs or something.

@pftdm007 said in Snort intermittently seems to crash, trying to find why:

@bmeeks

Thanks for letting me know that! I did not happen to browse the forums about this specific issue/topic... I think these services should not be available in Watchdog's available services to monitor if this is that critical? Anyways, +1 for implementing a real monitoring feature for such critical processes I think..

Snort and Suricata are unique in how they operate. They are quite different from services such as the ntpd daemon or something like sshd, for example. The two main differences are Snort and Suricata spawn separate executables for each configured interface, and both services restart themselves following a rules update. If you have LAN and WAN instances of Snort or Suricata configured, that means two separate snort or suricata processes are running: one per interface. Service Watchdog simply does the equivalent of a

ps -ax | grep snort

to see if a monitored process is running. So if it finds any snort process with that query, it is happy. But Snort might be running on the LAN yet have crashed on the WAN. Service Watchdog would never know that.

The other thing that trips up Service Watchdog is that it does not understand that Snort and Suricata restart themselves following a rules update. So if Service Watchdog happens to test for the existence of a Snort or Suricata process during that rules update restart, it will find no running instance and immediately try to start one up. That will likely collide with the restarting Snort or Suricata is doing itself following the rules update and can result in a crash.