Suricata package blocking mode coming soon



  • The binary patch to implement blocking of offenders in Suricata is complete and has been merged.  The supporting GUI package update Pull Request has been posted and is awaiting approval.  Once approved and merged, the Suricata package on pfSense will have the same blocking functionality as the Snort package.

    Here is a link to the Pull Request for anyone interested in the details:  https://github.com/pfsense/pfsense-packages/pull/652

    When the new package is approved and merged into the repository, I will post some release notes.

    UPDATE:  the new updated Suricata package has been posted to the package repository.  You should now see an update for version 1.0 under System…Packages.

    Bill



  • NOTE – this implementation uses the same libpcap system and alias table as the Snort package and is thus not yet a true inline IPS solution. Additionally, because Suricata now shares the blocking alias table with Snort, DO NOT run both Suricata and Snort in blocking mode on the same machine! It is OK to run them both side-by-side on the same machine, but just make sure only one is configured for blocking operation.

    Thanks Bill.

    Hmmm, which one to run in blocking mode?  Decisions…


  • Banned

    Wasnt Suricata supposed to be inline?



  • @Supermule:

    Wasnt Suricata supposed to be inline?

    Yeah, that's a good question.  I'm wondering why one would choose suricata over snort without that change.  I guess it should be faster since it's multithreaded and can use more than one core but what I'm seeing so far with this 1.0 package is that suricata uses much more CPU than snort with an identical rule set (snort in blocking mode, suricata in alert).  According to top, suricata is in ucond state whereas snort is in nanslp.

    EDIT:  CPU usage data from my system at various points:
    Snort - Suricata @ idle
    0.98% - 8.35%
    1.66% - 12.26%
    1.37% - 13.67%
    0.39% - 13.96%

    Snort - Suricata @ 85Mbit/s load inbound
    7.85% - 30.67%
    10.79% - 30.52%
    7.67% - 27.93%
    11.28% - 37.61%

    Snort - Suricata @ 40Mbit/s load outbound
    15.48% - 17.29%
    15.52% - 18.46%
    14.87% - 16.14%
    15.68% - 18.93%



  • @Jason:

    @Supermule:

    Wasnt Suricata supposed to be inline?

    Yeah, that's a good question.  I'm wondering why one would choose suricata over snort without that change.  I guess it should be faster since it's multithreaded and can use more than one core but what I'm seeing so far with this 1.0 package is that suricata uses much more CPU than snort with an identical rule set (snort in blocking mode, suricata in alert).  According to top, suricata is in ucond state whereas snort is in nanslp.

    EDIT:  CPU usage data from my system at various points:
    Snort - Suricata @ idle
    0.98% - 8.35%
    1.66% - 12.26%
    1.37% - 13.67%
    0.39% - 13.96%

    Snort - Suricata @ 85Mbit/s load inbound
    7.85% - 30.67%
    10.79% - 30.52%
    7.67% - 27.93%
    11.28% - 37.61%

    Snort - Suricata @ 40Mbit/s load outbound
    15.48% - 17.29%
    15.52% - 18.46%
    14.87% - 16.14%
    15.68% - 18.93%

    Interesting results.  Supposedly Suricata can scale better being multi-threaded, but there was a long rebuttal post on a blog a year or so ago from the Snort VRT guys pretty hotly contesting that assertion.  They showed that Snort can easily hold its own.  I honestly don't know which is better.  Obviously there are fanboys for each… ;), but I will stay out of the fight.

    My goal with publishing the Suricata package was to offer a viable alternative to Snort in the event the purchase of Sourcefire by Cisco led to something bad happening to the open source Snort code and rules.

    Bill



  • @Supermule:

    Wasnt Suricata supposed to be inline?

    Yes, and hopefully it will be eventually.  The problem is that it needs ipfw to work a certain way, and ipfw on pfSense has been "customized a bit" to make Layer 7 traffic shaping work.  That customization got us part of the way to having inline mode, but not all the way.  For example, it only works for TCP and UDP and only on IPv4; but even then it doesn't really work correctly.  I tried for several weeks to get it to work, but never was successful.  It goes into a loop and drops the same packet over and over.  Ermal told me earlier this year he would make some changes to help Suricata run in inline drop mode.  But it can't run inline until the pfSense guys make some kernel-level changes (well, in ipfw anyway).

    So using the alias table and libpcap like Snort does is a temp solution.  It may work just fine for the majority of folks.  Another issue with true inline is that it is going to be a performance drag.  There is no good way around that on FreeBSD due to the context swaps necessary to send a packet from the kernel stack up to user land to be inspected, then back from user land to the kernel stack to be passed on.  Ermal told me he thinks the throughput for inline mode might well max out at only 50 megabits/sec or so (maybe a little higher).

    The commercial world attacks this inline problem with customized hardware and highly customized kernels and IPS engines so that the trip back and forth from kernel space to user land is not necessary.  For an open source product like pfSense, that level of customization is a bit much.

    Not saying I'm giving up on inline mode, but it definitely can't happen until there are some changes within pfSense itself.

    Bill



  • @bmeeks:

    Not saying I'm giving up on inline mode, but it definitely can't happen until there are some changes within pfSense itself.

    Bill

    Thanks for the clarification on this Bill.  I agree on the fan boys on both sides of the fence.  There are quite a few groups that have done side by side tests out there and so far it still shows that Snort gets the nod, not by much though.  Given my (simple) needs I'm going to hang with Snort for now until "inline" is possible or Cisco destroys Snort… whichever comes first.

    Rick



  • @Ramosel:

    @bmeeks:

    Not saying I'm giving up on inline mode, but it definitely can't happen until there are some changes within pfSense itself.

    Bill

    Thanks for the clarification on this Bill.  I agree on the fan boys on both sides of the fence.  There are quite a few groups that have done side by side tests out there and so far it still shows that Snort gets the nod, not by much though.  Given my (simple) needs I'm going to hang with Snort for now until "inline" is possible or Cisco destroys Snort… whichever comes first.

    Rick

    I think in terms of "detection and protection" the two are pretty much in a dead heat.  As I said in a different thread, Snort is easier to setup for newbies because of the pre-defined IPS Policies you can select.  Suricata offers more "information" surrounding alerts what with its expanded HTTP and TLS logs (and even DNS and SMB logs in the new 2.0 version).  These extra logs, combined with the file extraction and storage feature, and pcap files provide a rich set of "context" around alerts that you can use for analysis.  Snort is a bit lacking in this area, but they are playing "catch up" very well.

    Bill



  • @bmeeks:

    As I said in a different thread, Snort is easier to setup for newbies because of the pre-defined IPS Policies you can select.

    There are settings in Suricata that you have for IPS Policies, do these not apply to your above statement?  I'm finding Suricata to be an awesome replacement for Snort.

    I've already ditched Snort and am currently running Suricata in Blocking mode and working on force-disabling the false positives.  Kid in a candy store.

    Many thanks Bill for your work.



  • @AhnHEL:

    There are settings in Suricata that you have for IPS Policies, do these not apply to your above statement?  I'm finding Suricata to be an awesome replacement for Snort.

    The only issue with the pre-defined policies in Suricata is that they are dependent on the Snort VRT rules.  The Snort VRT rules are the only ones with that metadata encoded within them.  There are a number of Snort VRT rules that contain Snort-specific keywords.  These rules will fail to compile on Suricata.  Errors will be printed in the suricata.log file, but Suricata will continue to start up.  This is different from Snort where any failed rule will cause a fatal exit on startup.  So look in your suricata.log file for your interfaces to see which, if any, Snort VRT rules might have choked.

    Suricata does best with the Emerging Threats rules because they create a specific package targeted at Suricata.  Either the ET OPEN or ET PRO rules will work.  The Snort rules will work, for the most part, but you do need to check the log to see if any Snort VRT rules you chose failed to compile.

    I probably need to check the GUI page to see if I made this point clear.  I think it's coded so the IPS Policy box is disabled when the Snort VRT rules are not selected (the same behavior as in Snort).

    Bill



  • @AhnHEL:

    I've already ditched Snort and am currently running Suricata in Blocking mode and working on force-disabling the false positives.  Kid in a candy store.

    Many thanks Bill for your work.

    Then you should really like the upcoming 2.0 version of the Suricata binary.  In my admittedly very limited testing thus far, it seems to have zero spurious stream alerts – at least when compared to what I get from 1.4.6 in my VM environment.  I'm not sure how much of that is bugs in the 1.4.6 stream processor and how is due to the strange virtual connections you can have in VMware Workstation.  I do most of my basic testing using Workstation.

    Anyway, I am interested in hearing how Suricata works for you in a real environment and how many persistent false positives you see with things like the stream alerts.

    Bill



  • So far the only Stream Alerts I've seen involve the following:

    SURICATA STREAM 3way handshake SYNACK resend with different seq
    SURICATA TLS invalid record type
    SURICATA STREAM FIN out of window
    SURICATA STREAM FIN invalid ack

    All those alerts resolve back to akamai technologies or 1e100.net.  Doesnt seem to be hindering any browsing, 5 alerts within a 24 hour period.

    I'm gettting a bunch of these SURICATA HTTP response header invalid with a Source from the Private IP address of my Cable Modem Diagnostic page.  I just Suppressed that one.

    Ran a torrent that saturated my 114Mbps Cable line and only saw 17% CPU spike.  Snort would max out one core on my CPU with my old 57Mbps connection so this is an improvement on my heavily used home network.


  • Moderator

    @bmeeks:

    Then you should really like the upcoming 2.0 version of the Suricata binary.  In my admittedly very limited testing thus far, it seems to have zero spurious stream alerts – at least when compared to what I get from 1.4.6 in my VM environment.  I'm not sure how much of that is bugs in the 1.4.6 stream processor and how is due to the strange virtual connections you can have in VMware Workstation.  I do most of my basic testing using Workstation.

    Anyway, I am interested in hearing how Suricata works for you in a real environment and how many persistent false positives you see with things like the stream alerts.

    If we had that "duplicate" Interface function, the testing would be a lot easier as we could test a modified Interface with a clean Suppress list and all the rules enabled and then quickly revert back to the previously configured Interface.

    With the way it is now, its hard to do this testing with a Production setup as you need to setup a new suppress list and enable the previously disabled rules to see if they re-alert with the new binary etc…



  • @BBcan17:

    @bmeeks:

    Then you should really like the upcoming 2.0 version of the Suricata binary.  In my admittedly very limited testing thus far, it seems to have zero spurious stream alerts – at least when compared to what I get from 1.4.6 in my VM environment.  I'm not sure how much of that is bugs in the 1.4.6 stream processor and how is due to the strange virtual connections you can have in VMware Workstation.  I do most of my basic testing using Workstation.

    Anyway, I am interested in hearing how Suricata works for you in a real environment and how many persistent false positives you see with things like the stream alerts.

    If we had that "duplicate" Interface function, the testing would be a lot easier as we could test a modified Interface with a clean Suppress list and all the rules enabled and then quickly revert back to the previously configured Interface.

    With the way it is now, its hard to do this testing with a Production setup as you need to setup a new suppress list and enable the previously disabled rules to see if they re-alert with the new binary etc…

    I will add that functionality to the next release.  It will work pretty much like the same button for firewall rules.  Click the (+) to create a new interface "based on" the one you click.  It will inherit the same settings for everything except the physical NIC interface and the name.

    Bill


  • Moderator

    Great stuff… This will make testing that much easier!!

    EDIT:

    On another note, also having the functionality to create a fresh NEW interface would also be beneficial instead of trying to re-enable all of the previously disabled rules.



  • @BBcan17:

    Great stuff… This will make testing that much easier!!

    EDIT:

    On another note, also having the functionality to create a fresh NEW interface would also be beneficial instead of trying to re-enable all of the previously disabled rules.

    That functionality is there today with the current (+) icon.  I'm not changing that one.  I will be adding new (+) icons beside each configured interface.  Clicking the (+) beside an already configured interface will perform the DUP function.  The layout will be pretty much just like the firewall rules page.  So the (+) icon at the top right of the tab will create a new fresh interface (just like it does today).  The (+) icon beside an existing configured interface will DUP it.

    In terms of the rules, on the RULES tab today is an icon that will remove all forced enable/disable changes for all rules on the interface.  This essentially resets the rules to their default state.  There is a similar icon that will do this for only the currently selected category.

    Bill