Supressing an entire ruleset, where the ruleset needs to be enabled

[justme2]

All,

For whatever reason, just not seeing "how" to accomplish this (if it's even readily possible). For example let's say that you wanted to block/drop all the emerging-ciarmy.rules (you'd enable it in the sid management as "emerging-ciarmy") - fairly simple. However, as those are often noisy and you just want those squashed (without seeing the clutter in the logs), how do you equally state same in the suppression list [logically]? (without having to lookup and specify each item in the ruleset or do you have to create a script that will generate the line items for the suppression list?)

Thanks!

[bmeeks]

So you want to enable the rules for dropping traffic, but you don't want to see any alerts from those drops?

That is not a typical configuration, so that's why it is not easily configured in the GUI. Suppress List entries are not designed to be regex pattern matched as are the SID MGMT entries. Suppress List entries are one-for-one matches to either specific rule SIDs or specific host IP addresses.

Where are you running the IDS/IPS? Sounds like maybe on the WAN if you are seeing lots of Emerging-CIARMY drops/alerts. I would only run those rules on the LAN or other internal interface. And in that scenario, traffic to/from a CIARMY host would be of interest and I would want to know which internal host triggered that rule.

Your default WAN firewall rules are already going to block all unsolicited inbound traffic of the sort that normally triggers CIARMY rules. Why not just let the firewall block those and not waste Suricata's time and energy scanning packets the firewall rules are going to drop anyway?

[justme2]

Correct. This is on the external. Not necessarily "normal" and ensures that those addresses can't even reach a service that may be port forwarded/proxied via the firewall on the external interface. eg: the socket [itself] cannot be reached. It also means that the inside doesn't even have to deal with seeing the traffic if the outside instance drops them [at arm's length]. Said differently, if someone were running a proxied NAT/Rule or haproxy (or anything else) that would allow something "past" the firewall rule - why even let "known bad" near those in the first place. Simply improves overall posture and reduce potential for issue.

Agree that for all other ports, it would make sense as to avoid even "looking" at those packets but in this case it puts a protective [and selective] veil over the "outside" of things.

[bmeeks]

Are you using Legacy Blocking Mode or Inline IPS Mode? Take a look at the two diagrams below that shows how the two IDS/IPS packages are plumbed into the network path.

If using Legacy Mode Blocking, then the firewall is doing the blocking anyway so you may as well allow the default deny rule to work in your favor and not waste cycles scanning with the IDS/IPS. Legacy Mode Blocking works by sending copies of packets to the IDS/IPS for processing while the original packet continues straight to the firewall engine. Only if the copied packet triggers an IDS/IPS rule will a block be inserted into the firewall for "future" packets to hit (but not the current packet that caused the initial alert).

If using Inline IPS Mode, then it's true the IPS can drop a packet before the firewall sees it. But I will also offer this bit of info. pfBlockerNG and an IP list matching the CIARMY list will accomplish the same thing more efficiently as it won't be scanning every packet. You lose a significant fraction of throughput when using Inline IPS Mode because every packet is scanned.

[justme2]

Using inline.

[bmeeks]

I've said this many times over the last couple of years when giving IDS/IPS configuration advice. You are not using the IDS/IPS to protect your firewall. You use it to protect the hosts behind the firewall.

When running on an internal interface all local hosts are still perfectly protected as no traffic can reach them without traversing the IDS/IPS anyway. Consider that when running on the LAN, no traffic can get from the firewall engine to a LAN host without traversing the IPS when using Inline IPS mode. That means it would go from Internet through WAN rules and then get routed to LAN rules, then finally through the IPS before reaching an internal host. Refer to the diagrams I posted above.

Putting the IDS/IPS on the LAN side helps relieve pressure on it as it will no longer be scanning those "noise" packets. The pf firewall engine will have dealt with the majority of those.

[justme2]

There are more lists than "just" the ciarmy stuff that are in the rule feeds. Basically any "list" of IPs/networks is applied externally. If there was a 1:1 with pfBlockerNG - would look to go down that path, but (at least time I checked) there simply wasn't sufficient parity.

[bmeeks]

@justme2 said in Supressing an entire ruleset, where the ruleset needs to be enabled:

There are more lists than "just" the ciarmy stuff that are in the rule feeds. Basically any "list" of IPs/networks is applied externally. If there was a 1:1 with pfBlockerNG - would look to go down that path, but (at least time I checked) there simply wasn't sufficient parity.

May be so as I have not tried to find a CIARMY duplicate for pfBlockerNG. But I still stand by my advice to move your sensor to the internal interface (or interfaces) to make better use of CPU cycles -- unless you just have CPU and RAM to burn.

Not saying putting it on the WAN is "wrong", but just offering suggestions to improve efficiency in throughput and resource utilization on the firewall.

A beneficial side effect of putting it on an internal interface is it would eliminate the alerts/blocks you don't want to see (that Internet noise I mentioned). I would want to see if one of my internal hosts communicated with a CIARMY IP address, though. Having the sensor on the inside interface would accomplish both goals.

[justme2]

There's a key element in this which is that means putting 100% trust that something like haproxy/.... won't end up with a flaw that can exploited given the attack access to the firewall. As haproxy runs as root - a flaw opens a broad door. Why not simply "close" the risk [at least] from known nefarious sources?

[bmeeks]

@justme2 said in Supressing an entire ruleset, where the ruleset needs to be enabled:

putting 100% trust that something like haproxy/.... won't end up with a flaw that can exploited given the attack access to the firewall

Well, you have the same vulnerability exposure in the IDS/IPS as well, don't you? If you search the Suricata Redmine site, for example, you will find a few past bugs that allowed bypassing of packet inspection. Who's to say another such bug won't pop up? I would say the risk is about the same.

[bmeeks]

Nothing wrong with putting the IDS/IPS where you have it, but one limitation you will have to live with is you won't be able to easily accomplish what your original post asked about. There is no way short of typing in every single individual SID to suppress an entire category of alerts.

[justme2]

Agreed, but at least having a couple measures in place to "try to" provide protection is better than not. Seems easier to justify consumption of more CPUs and/or faster cores for more potential protection than having to explain the inverse if something were to happen. Nothing is 100% and applying as much "reasonable" protection should be applied where possible - throughout the path of data flow (not "just" the firewall). Making the cost of more/faster CPUs "cheap" even if seemingly burdensome.

[justme2]

That is what I was trying to determine - ?if? there was an easy/existing means. if not, it simply means some effort to create something for purpose. Not the end of the world, but will add "maintenance" to the effort.

[bmeeks]

@justme2 said in Supressing an entire ruleset, where the ruleset needs to be enabled:

Agreed, but at least having a couple measures in place to "try to" provide protection is better than not. Seems easier to justify consumption of more CPUs and/or faster cores for more potential protection than having to explain the inverse if something were to happen. Nothing is 100% and applying as much "reasonable" protection should be applied where possible - throughout the path of data flow (not "just" the firewall). Making the cost of more/faster CPUs "cheap" even if seemingly burdensome.

I understand the "defense in depth" concept for cyber security. Worked in that area of nuclear power for several years before retiring. We were regulated like no tomorrow with a long line of NRC inspectors imagining every conceivable manner of "what if" . I think they must have gotten a larger bonus the more far-out-there their idea of a perceived threat was.

Of course nothing was connected to the Internet, and everything was isolated from other internal company networks via one-way data diode appliances. The only real threat was portable media (USB sticks and CD/DVDs) and a laptop. All of those were heavily policed, controlled, and scanned before use with multiple malware scanning techniques. And all the critical networks were behind multiple layers of physical security including guys with guns.

[bmeeks]

@justme2 said in Supressing an entire ruleset, where the ruleset needs to be enabled:

That is what I was trying to determine - ?if? there was an easy/existing means. if not, it simply means some effort to create something for purpose. Not the end of the world, but will add "maintenance" to the effort.

You could perhaps get creative with sed and regex to pull out the rule SIDs from the rules category file. The master file will be in /usr/local/etc/snort/rules/ if using Snort, and in /usr/local/share/suricata/rules if using Suricata. The rules files are just plain text.

You would write the extracted SIDs, with the correct suppresion list syntax, to a file. I would begin by creating a custom Suppress List and assigning it to the interface. Then using a crontab driven script to update it each time the rules update.

[justme2]

Had a few in-between other activities and came up with this. RE: bonus on ideas - "..things that make you go Hmmmmm....." lol

Including if useful to anyone else - this appears to solve the riddle..... Not "pretty" but addresses the need with a few options that can be applied to satisfy some conditions. Could probably be augmented to enable specification of "interface" to further simplify.

Spoiler

#!/usr/local/bin/perl
#
use strict;
use Getopt::Long;
$| = 1;
#
GetOptions('debug'=>\$PROC::DEBUG,'include=s'=>\$PROC::INCLUDE,'severity=s'=>\$PROC::SEVERITY,'targetfile=s'=>\$PROC::TGTFILE,'mergefile=s'=>\$PROC::MRGFILE,);
if (defined($PROC::DEBUG)) { $PROC::DEBUG=1; } else { $PROC::DEBUG=0; }
%PROC::INCS=();
if (defined($PROC::INCLUDE)) {
    foreach(split(/,/,$PROC::INCLUDE)) { $PROC::INCS{$_}=0; }
    }
#
@PROC::DIRS=('/usr/local/share/suricata/rules','/usr/local/etc/snort/rules',);
%PROC::HASH=();
#
foreach(@PROC::DIRS) {
    my $RDIR=$_;
    opendir(DIR, "$RDIR"); rewinddir(DIR);
    while(my $FILE=readdir(DIR)) {
        if ($FILE=~/\.rules$/) {
            my $CHECK=$FILE; $CHECK=~s/\.rules$//;
            if ((keys(%PROC::INCS)>0) && (exists($PROC::INCS{$CHECK}))) { &procFile("$RDIR/$FILE"); } elsif
               (keys(%PROC::INCS)==0) { &procFile("$RDIR/$FILE"); }
            }
        }
    closedir(DIR);
    }
#
if (defined($PROC::MRGFILE)) {
    open(INF, "<$PROC::MRGFILE");
    while(my $LINE=<INF>) {
        chomp($LINE);
        if (($LINE!~/^#/) && ($LINE!~/^[[:space:]]{0,}$/) && ($LINE=~/^suppress[[:space:]]{1,}/)) {
            my $GID=$LINE; $GID=~s/^.*gen_id[[:space:]]{1,}//; $GID=~s/,.*//;
            my $SID=$LINE; $SID=~s/^.*.sig_id[[:space:]]{1,}//; if ($SID=~/,/) { $SID=~s/,.*//; }
            if (exists($PROC::HASH{$GID}{$SID})) { delete($PROC::HASH{$GID}{$SID}); }
            }
        }
    close(INF);
    }
my $FH;
if (defined($PROC::TGTFILE)) { open $FH, ">", "$PROC::TGTFILE" || die("ERROR: $PROC::TGTFILE $!\n"); select($FH); } elsif
   (defined($PROC::MRGFILE)) { open $FH, ">>", "$PROC::MRGFILE" || die("ERROR: $PROC::MRGFILE $!\n"); select($FH); print $FH ("\n"); }
foreach my $ID (keys %PROC::HASH) {
    foreach my $SID (sort {$a<=>$b} keys %{$PROC::HASH{$ID}}) {
        my $MSG=$PROC::HASH{$ID}{$SID}{msg};
        my $FILE=$PROC::HASH{$ID}{$SID}{file};
        print ("# ($FILE) $MSG\nsuppress gen_id $ID, sig_id $SID\n\n");
        }
    }
if (defined($PROC::TGTFILE) || defined($PROC::MRGFILE)) { close $FH; }
#
if (defined($PROC::MRGFILE)) {
    my $F = do { local $/ = undef; open my $FH, "<", "$PROC::MRGFILE"; <$FH>; };
    $F=~s/\n//g; $F=~s/#/\n\n#/g; $F=~s/suppress[[:space:]]{1,}/\nsuppress /g; $F=~s/^\n{1,}//;
    open(OUF, ">$PROC::MRGFILE"); print OUF ("$F\n"); close(OUF);
    }
#
sub procFile {
my ($FILE)=(shift);
if ($PROC::DEBUG==1) { print ("\tFILE : $FILE\n"); }
open(INF, "<$FILE");
while(my $LINE=<INF>) {
    chomp($LINE);
    if ($LINE=~/^alert ip \[/) {
        my $SID=$LINE; $SID=~s/^.*.sid://; $SID=~s/;.*//;
        my $MSG=$LINE; $MSG=~s/^.*.msg://; $MSG=~s/;.*//; $MSG=~s/"//g;
        my $SEV=$LINE; $SEV=~s/^.*.signature_severity //; $SEV=~s/,.*//;
        my $F=$FILE; $F=~s/\.rules$//; $F=~s/^.*.\///;
        if ($PROC::DEBUG==1) { print ("\t\t1 : $SID : $F : $SEV\n"); }
        if (defined($PROC::SEVERITY) && ($SEV eq $PROC::SEVERITY)) {
            $PROC::HASH{1}{$SID}{msg}=$MSG;
            $PROC::HASH{1}{$SID}{file}=$F;
            } elsif (!defined($PROC::SEVERITY)) {
            $PROC::HASH{1}{$SID}{msg}=$MSG;
            $PROC::HASH{1}{$SID}{file}=$F;
            }
        }
    }
close(INF);
return;
}
#
__END__
## 
## Documentation
##
=head1 NAME

generate-suppress.pl

=head1 SYNOPSIS

generate-suppress.pl --debug --include=<include> --severity=<severity> --targetfile=<targetfile> --mergefile=<mergefile>

=head1 DESCRIPTION

Generates suppression data from source rules for "alert ip" style entries

=head1 FUNCTION

Insert or merge alert ip suppression data for SNORT/Suricata

=head1 OPTIONS

=over

=item <debug>

Enables debugging output.

=item <include>

Specifies which rule(s) to include in the resultant data. Comma separated - B<NO> spaces.

=item <severity>

Filter resultant rules to only those that match severity.
(As of creation, appears to be either "Major" or "Minor")

=item <targetfile>

If the target file exists, it B<WILL> be overwritten with the results.

=item <mergefile>

If the mergefile already exists, it will be read and only [missing] deltas will be added.

=item NOTE
If neither targetfile nor mergefile are specified, the results are printed to STDOUT.

=back
    
=head1 COMMON USAGE

perl generate-suppress.pl --mergefile=/usr/local/etc/suricata/suricata_<ID>_<interface>/threshold.config

=cut