NEW - Suricata 1.4.6 IDS pkg. v0.2-BETA Released



  • Suricata 1.4.6 IDS (Intrusion Detection System)

    An update to the new Suricata BETA package has been posted to the Packages repository.  This update corrects a number of minor bugs from the initial 0.1-BETA release and introduces several new features.  Feedback from testers is encouraged.  You can find documentation on Suricata and its configuration settings here:  http://suricata-ids.org/

    IMPORTANT NOTES:

    These initial BETA versions are intended for IDS (intrusion detection mode only) operation and do not offer blocking of offenders.  That feature will be included in a later release.

    The Suricata package is offered only for pfSense versions 2.1 and higher.  If you are running an older version of pfSense, the package will not appear under System…Packages.

    While the Suricata engine can decode and utilize nearly all of the Snort VRT rules, there are a few Snort rules that contain keywords and metadata Suricata does not understand.  Suricata will print an error to the log, and skip those rules when encountered.  The Emerging Threats team offers a Suricata-optimized set of both the free Emerging Threats Open rules and the subscription-based Emerging Threats Pro rules.  When Emerging Threats rules are enabled, the Suricata package will download and use the optimized rule set.  Use of Emerging Threats is recommended with Suricata.  If you use the Snort VRT or Snort GPLv2 Community Rules with Suricata, don't be surprised if some of those rules fail to load.

    New Features In This Release

    • Barnyard2 now offers Bro-IDS output capability for sending alert data to a remote Bro-IDS instance.

    • The Update tab has been significantly overhauled and the appearance brought more in line with the other package tabs. A new "force update" button is now available to force a new rules package download. The MD5 hashes and files dates for the currently downloaded and installed rules packages are now shown in the dialog.  Note– although it did not make it into this latest BETA, the next update will also show the results of the last auto-update attempt (succeeded or failed and the date/time).

    • A built-in Dashboard Widget is now installed along with the Suricata package. The widget will show up in the list of available widgets in the Dashboard window.  It will not be displayed upon the initial installation, but will show in the list of available widgets when clicking the "add widget" icon on the Dashboard.  When removing the Suricata package, the state of the Dashboard Widget will be remembered and restored when the package is subsequently reinstalled (if "save settings on uninstall" is enabled on the GLOBAL tab).

    Bill



  • Suricata Basic Setup Instructions

    Install the package under System…Packages from the Available Packages tab.

    When the installation is complete, go to Services…Suricata from the pfSense menu to launch the Suricata GUI.

    Click on the Global Settings tab and make your initial configuration selections (such as choosing the rule sets) and save them.

    Click on the Update Rules tab and then click the Check button to download the rules you enabled on the Global Settings tab.

    When the download completes, click on the Suricata Interfaces tab and then click the plus (+) sign on the right to add an interface.

    Check the Enable box and choose the interface you wish to run Suricata on and give it a friendly name.  Either WAN or LAN is a good choice depending on which traffic you want to inspect.  Look through the other settings and configure them as you desire.  Click Save when done.

    You will be returned to the Suricata Interfaces tab.  Double-click on the newly added interface (or click the e icon on the right) to edit the settings again.

    This time click the Categories tab and select the rule categories you wish to use.  Click Save when done.

    You can alter settings on the other tabs as desired, but if you are just starting out with Suricata the defaults should be fine.

    Click the Suricata Interfaces tab again and click on the red X icon to toggle Suricata to the running state.  It will take a few seconds to start.  Be patient.

    When the icon changes to a green arrow, click the Logs Browser tab and then select the suricata.log file in the "Log file to view" drop-down.  Read through the log and see if Suricata started successfully.  Here is an example of a successful startup:

    4/3/2014 -- 10:52:58 - <info>-- all 4 packet processing threads, 1 management threads initialized, engine started.
    4/3/2014 -- 10:53:03 - <info>-- 2 rule files processed. 14175 rules successfully loaded, 0 rules failed
    4/3/2014 -- 10:53:12 - <info>-- 14183 signatures processed. 614 are IP-only rules, 4037 are inspecting packet payload, 11013 inspect application layer, 76 are decoder event only
    4/3/2014 -- 10:53:12 - <info>-- building signature grouping structure, stage 1: adding signatures to signature source addresses... complete
    4/3/2014 -- 10:53:12 - <info>-- building signature grouping structure, stage 2: building source address list... complete
    4/3/2014 -- 10:53:16 - <info>-- building signature grouping structure, stage 3: building destination address lists... complete
    4/3/2014 -- 10:53:18 - <info>-- Signature(s) loaded, Detect thread(s) activated.
    4/3/2014 -- 10:54:09 - <info>-- No packets with invalid checksum, assuming checksum offloading is NOT used</info></info></info></info></info></info></info></info> 
    

    The last line shown above takes several seconds to appear as Suricata automatically examines the packets it sees and from them determines if checksum offloading is used or not.  If you don't see the checksum offloading line yet, don't worry.

    The other logs available on the Logs Browser tab are populated according to the specific features you enable on the Settings tab for the particular interface.  The default setup described above does not enable all of the features.

    To enable the Dashboard Widget, return to the pfSense Dashboard and click the plus icon (+) to add a new widget.  Select Suricata Alerts from the list.  Click Save to save the new Dashboard configuration.

    Note – pretty much every clickable icon and button in the Suricata GUI contains a tool tip to show what it does.  Just hover your mouse over an icon or button and wait for the tool tip to appear.

    Bill


  • Banned

    Can this be ported to the 2.0.3 release??

    Reason I am asking is that a lot of people have driver issues and panics in 2.1 rel.



  • @Supermule:

    Can this be ported to the 2.0.3 release??

    Reason I am asking is that a lot of people have driver issues and panics in 2.1 rel.

    Well, not without a fair amount of rework.  It makes use of system calls and features that are only available in the 2.1 code base.  The goal originally was to target 2.1 and higher for Suricata.

    EDIT – let me rephrase "a fair amount of rework" to "some amount of rework".  I have not tested it personally yet, but does the new 2.1.1 pre-release work better for folks?

    Bill


  • Banned

    Seeing a lot of questions regarding drivers not finding the hardware and Squid not working….

    2.0.3 seems to be the most stable yet for production when its loaded with traffic. I had to revert on one FW that slowed down to a grinding halt efter a few hours.



  • @Supermule:

    Seeing a lot of questions regarding drivers not finding the hardware and Squid not working….

    2.0.3 seems to be the most stable yet for production when its loaded with traffic. I had to revert on one FW that slowed down to a grinding halt efter a few hours.

    How about this?  I will trade you your Internet connection for a 2.0.3 version of the Suricata package… ;D

    I am envious of your speeds posted in your Forum signature.  Where I am located in the Southeastern U.S. we are just being offered 24 Mbps cable modem service for residential use.

    Bill


  • Banned

    You can have access to that when you use my cloud services ;)


  • Moderator

    Hi Bill,

    I updated my Suricata and noticed the following:

    • I noticed a 10% memory increase. Without any changes to my settings.

    • When I view an alert, and either Suppress or Remove a Rule, it doesn't bring the page refresh back to the same location. (Snort package does it correctly.)

    • In the Suricata Dashboard, i see alerts that are not in the Main Alerts Tab? (See the attachment). After I added the Dashboard, Saved, and edited the number of alerts to display       
        (20), it gave an error. (Didn't record the error). I did a page refresh and it seemed to work after that. 
        –Maybe its using the wrong descriptor for the Detail Column?--




  • @BBcan17:

    Hi Bill,

    I updated my Suricata and noticed the following:

    • I noticed a 10% memory increase. Without any changes to my settings.

    Not 100% sure, but this could be caused by fixing a bug in the first release where the default decoder-events, file, http-events, smtp-events and stream-events rules files were not being included in the default enabled rules.  That's fixed in this update.  Those extra files mean some extra rules being loaded.  That could be the difference.

    • When I view an alert, and either Suppress or Remove a Rule, it doesn't bring the page refresh back to the same location. (Snort package does it correctly.)

    Do you mean upon return to the ALERTS tab, if you have a long list of alerts, it doesn't scroll down to where you were?  If so, Snort is doing that more by accident than design.  Suricata (and the new upcoming Snort package) uses a more secure method of passing parameters, but this takes away the automatic ability for the browser to scroll back to previous history.  I may be able to emulate that a bit with some tricks.  I will investigate for a future release.

    • In the Suricata Dashboard, i see alerts that are not in the Main Alerts Tab? (See the attachment). After I added the Dashboard, Saved, and edited the number of alerts to display (20), it gave an error. (Didn't record the error). I did a page refresh and it seemed to work after that. 
        –Maybe its using the wrong descriptor for the Detail Column?--

    If you can capture the specific error message for me, that would be helpful.  That doggone Dashboard Widget has been a royal pain to get working even mostly correctly.  Not surprising to hear there may still be problems.  I fiddled with it for about 3 days learning by trial and error… :P.  If there is someone who is a better Ajax and jQuery programmer than me, I could use some help with that widget.  The two files are here on the box:

    /usr/local/www/widgets/javascript/suricata_alerts.js
    /usr/local/www/widgets/widgets/suricata_alerts.widget.php

    Bill



  • 2 things I've noticed. First is that unless I'm messing something up bad, and that's certainly known to happen, I don't think the suppress list is working. I added an item to the default suppress list and did a full stop/restart of Suricata and still got alerts.

    The other is I don't think there is any port scanning detection. I noticed a lack of any port scan hits in the alerts so I ran GRC's Shields Up! and did a full service port scan and nothing new was in my Suricata logs.



  • @DigitalDeviant:

    2 things I've noticed….

    I don't think the suppress list is working. I added an item to the default suppress list and did a full stop/restart of Suricata and still got alerts.

    I just noticed this in my testing as well.  I will look into it.  The curious part is if you look in the suricata.log file, you will see where it parsed the list (they call it threshold) and read the entries from it.  However, it still does not seem to actually suppress the alerts.

    The other is I don't think there is any port scanning detection. I noticed a lack of any port scan hits in the alerts so I ran GRC's Shields Up! and did a full service port scan and nothing new was in my Suricata logs.

    I get alerts on scans. I have a virtual machine test environment where a Kali Linux VM scans my various firewall VMs.  When I do a nmap scan against the Suricata VM I get alerts.  However, you do have to enable the Emerging Threats scan rules (or at least that's what I did).  From some Mailing List posts, it appears Suricata does not contain a native port scan processor like Snort does.  The omission is apparently by design according to this thread:

    https://lists.openinfosecfoundation.org/pipermail/oisf-users/2011-April/000598.html

    There is some validity to the "false positive" comment in that thread.  I see port scan FPs frequently in my Snort setups.

    Bill



  • @bmeeks:

    @DigitalDeviant:

    2 things I've noticed….

    I don't think the suppress list is working. I added an item to the default suppress list and did a full stop/restart of Suricata and still got alerts.

    I just noticed this in my testing as well.  I will look into it.  The curious part is if you look in the suricata.log file, you will see where it parsed the list (they call it threshold) and read the entries from it.  However, it still does not seem to actually suppress the alerts.

    The other is I don't think there is any port scanning detection. I noticed a lack of any port scan hits in the alerts so I ran GRC's Shields Up! and did a full service port scan and nothing new was in my Suricata logs.

    I get alerts on scans. I have a virtual machine test environment where a Kali Linux VM scans my various firewall VMs.  When I do a nmap scan against the Suricata VM I get alerts.  However, you do have to enable the Emerging Threats scan rules (or at least that's what I did).  From some Mailing List posts, it appears Suricata does not contain a native port scan processor like Snort does.  The omission is apparently by design according to this thread:

    https://lists.openinfosecfoundation.org/pipermail/oisf-users/2011-April/000598.html

    There is some validity to the "false positive" comment in that thread.  I see port scan FPs frequently in my Snort setups.

    Bill

    I just ran an nmap remotely and show nothing on my alerts and I'm running all ET open rules except the p2p rules.

    It does seem to pick up nmap scans after I figured out a few PEBKAC issues. However, the scanning tool here triggers a TCP portscan in Snort and nothing in Suricata. I guess there isn't much you can do about it since it's by design so it will be somewhat of a tradeoff with Snort.



  • were any fixes put in to correct the multiple instances because of filter reload when interfaces go up and down? You were able to correct it for snort (barnyard2 in snort still fires up multiples tho)

    
    root   65751  6.0  1.8 73192 55536  ??  SNs   3:45PM   0:01.19 /usr/pbi/suricata-i386/bin/suricata -i em3 -D -c /usr/pbi/suricata-i386/etc/suricata/suricata_34793_em3/suricata.yaml --pidfile /var/r
    root   65005  0.6  0.0  3644  1440  ??  IN    3:45PM   0:00.01 /bin/sh /usr/local/etc/rc.d/suricata.sh start
    root   65187  0.3  0.9 31008 27220  ??  SN    3:45PM   0:00.23 /usr/pbi/suricata-i386/bin/suricata -i em3 -D -c /usr/pbi/suricata-i386/etc/suricata/suricata_34793_em3/suricata.yaml --pidfile /var/r
    
    


  • @Cino:

    were any fixes put in to correct the multiple instances because of filter reload when interfaces go up and down? You were able to correct it for snort (barnyard2 in snort still fires up multiples tho)

    
    root   65751  6.0  1.8 73192 55536  ??  SNs   3:45PM   0:01.19 /usr/pbi/suricata-i386/bin/suricata -i em3 -D -c /usr/pbi/suricata-i386/etc/suricata/suricata_34793_em3/suricata.yaml --pidfile /var/r
    root   65005  0.6  0.0  3644  1440  ??  IN    3:45PM   0:00.01 /bin/sh /usr/local/etc/rc.d/suricata.sh start
    root   65187  0.3  0.9 31008 27220  ??  SN    3:45PM   0:00.23 /usr/pbi/suricata-i386/bin/suricata -i em3 -D -c /usr/pbi/suricata-i386/etc/suricata/suricata_34793_em3/suricata.yaml --pidfile /var/r
    
    

    No, I'm sorry that one slipped my mind in the rush to get Suricata 0.2 out.  I will write it down on my list of TODO fixes for the next version.

    So you are saying you get multiple instances of Barnyard2 on Snort with the exact same command-line parameters?  I did not know that was still happening for anyone.

    Bill



  • @bmeeks:

    No, I'm sorry that one slipped my mind in the rush to get Suricata 0.2 out.  I will write it down on my list of TODO fixes for the next version.

    So you are saying you get multiple instances of Barnyard2 on Snort with the exact same command-line parameters?  I did not know that was still happening for anyone.

    Bill

    Np, it happens. Hoping its an easy fix… Barnyard2, yes. I will see multiple instances of it running. I can reproduce on the fly by resetting my cable modem. I think I may have reported it but to be honest, I can't remember...



  • @Cino:

    @bmeeks:

    No, I'm sorry that one slipped my mind in the rush to get Suricata 0.2 out.  I will write it down on my list of TODO fixes for the next version.

    So you are saying you get multiple instances of Barnyard2 on Snort with the exact same command-line parameters?  I did not know that was still happening for anyone.

    Bill

    Np, it happens. Hoping its an easy fix… Barnyard2, yes. I will see multiple instances of it running. I can reproduce on the fly by resetting my cable modem. I think I may have reported it but to be honest, I can't remember...

    Since you reported this, I checked last night and I am seeing that Barnyard2 problem on my Snort setup as well.  When my cable modem reboots and bounces my firewall's WAN interface, Barnyard2 is not reliably restarting.  I'm working on it.  Hopefully it's just a dumb typo someplace since Snort works OK.  They are both supposed to be using essentially the same shell script commands.  I will put my glasses on and examine the code carefully to find the mistake.  There has to be one someplace… :-[

    Bill



  • Very minor thing, but passing it along.  When the widget gets an IPv6 alert, it causes the right side border to extend past the normal alignment.  The Snort widget wraps the address.

    Here it is with only IPv4 alerts and with an IPv6 alert changing the alignment.






  • ** Problem - Cannot Disable Interface **

    Problem:  Cannot disable Suricata on an interface, it faults to "The following input errors were detected: The value for Maximum-Pending-Packets must be between 1 and 65,000!"

    Steps to Reproduce:

    1. Have Suricata enable and running on an interface.  Max Pending Packets is at the default 1024.

    2. Uncheck "Enable" and hit "Save".

    3. The error box "The following input errors were detected: The value for Maximum-Pending-Packets must be between 1 and 65,000!"  pops up.

    4. Go back to interfaces and the disable action did not take.



  • Moderator

    @priller:

    ** Problem - Cannot Disable Interface **

    FYI - This also occurred in the previous version. Hope that helps diagnose.



  • @priller:

    ** Problem - Cannot Disable Interface **

    Problem:  Cannot disable Suricata on an interface, it faults to "The following input errors were detected: The value for Maximum-Pending-Packets must be between 1 and 65,000!"

    Steps to Reproduce:

    1. Have Suricata enable and running on an interface.  Max Pending Packets is at the default 1024.

    2. Uncheck "Enable" and hit "Save".

    3. The error box "The following input errors were detected: The value for Maximum-Pending-Packets must be between 1 and 65,000!"  pops up.

    4. Go back to interfaces and the disable action did not take.

    I will fix it.  I screwed up the order of input validation and also forgot to skip it all when just disabling the interface.  My bad… :-[

    I will post the Pull Request today, and hopefully one of the Core Team devs will have a chance to review and approve.

    Bill



  • @priller:

    Very minor thing, but passing it along.  When the widget gets an IPv6 alert, it causes the right side border to extend past the normal alignment.  The Snort widget wraps the address.

    Here it is with only IPv4 alerts and with an IPv6 alert changing the alignment.

    I will try to get this fixed in the next update as well.  The only way I've found around this is to insert zero-length spaces next to every colon in an IPv6 address.  These don't display, but they offer the browser a "line break" opportunity.  This makes the prettiest line break (breaking on a colon, that is).  The other option is a forced wrap, but that can happen in odd places and makes readability more difficult.

    Related to this, what is the preference among users for how to delimit ports when displaying IPv6 addresses?  The IPv4 standard is a colon at the end of the address, but since IPv6 already has colons, things are more confusing.

    Bill



  • @bmeeks:

    Related to this, what is the preference among users for how to delimit ports when displaying IPv6 addresses?  The IPv4 standard is a colon at the end of the address, but since IPv6 already has colons, things are more confusing.

    I believe square brackets around the address portion of the address is the standard.



  • @AhnHEL:

    @bmeeks:

    Related to this, what is the preference among users for how to delimit ports when displaying IPv6 addresses?  The IPv4 standard is a colon at the end of the address, but since IPv6 already has colons, things are more confusing.

    I believe square brackets around the address portion of the address is the standard.

    Thanks!  I will make the adjustment in the widget display.

    Bill



  • Bug Fix Update

    Just FYI.  A new Pull Request was posted today containing fixes for the bugs reported thus far with the Suricata package.  The version number will remain the same for now, but I will post an update when the pull request is merged and then interested parties can do a quick reinstall of the Suricata package GUI components to pick up the fixes.

    Here is a link to the Pull Request with the details:  https://github.com/pfsense/pfsense-packages/pull/622

    Bill



  • What are the possibilities of adding in some log file rotation routines?  alerts.log and http.log have grown to the point that it's not practical to view them in the Logs Browser.

    1041187808 Mar 13 21:52 alerts.log    ( a very unhappy checksum rule filled this up rather quickly )
    47180176    Mar 14 07:31 http.log

    Even just a daily rotation with date in the file name (ex: alerts_20140314.log) would be nice.



  • @priller:

    What are the possibilities of adding in some log file rotation routines?  alerts.log and http.log have grown to the point that it's not practical to view them in the Logs Browser.

    1041187808 Mar 13 21:52 alerts.log    ( a very unhappy checksum rule filled this up rather quickly )
    47180176    Mar 14 07:31 http.log

    Even just a daily rotation with date in the file name (ex: alerts_20140314.log) would be nice.

    I can do that.  I also noticed that Suricata can be quite chatty.  I will make the rotation a configurable cron job so the user can select from several rotation options.

    Bill


  • Moderator

    ET has finally killed the RBN rulesets.

    http://www.emergingthreats.net/2014/03/14/daily-ruleset-update-summary-03142014-%CF%80-edition/?utm_source=rss&utm_medium=rss&utm_campaign=daily-ruleset-update-summary-03142014-%25cf%2580-edition

    "Emerging Threats would like to remind and/or inform everyone that this ruleset does not contain the Russian Business Network (RBN) rules. These rules are obsolete and will not be distributed in future releases."

    Another feature for Snort/Suricata that would help is to have two Alert screens.

    One for the noisy alerts like Scans/CINS/DROP/MYSQL/SQL etc.
    One for all other alerts which would make it easier to see from the Alert screen without all of the other alerts on the same log.



  • Time to update the blueprint with the removed rules then. Open to suggestions for lists to replace those.


  • Moderator

    For ET changes, these three seem to still be online -

    pfBlocker ET Blocker
    http://rules.emergingthreats.net/fwrules/emerging-Block-IPs.txt
    http://rules.emergingthreats.net/blockrules/compromised-ips.txt
    http://doc.emergingthreats.net/pub/Main/RussianBusinessNetwork/RussianBusinessNetworkIPs.txt

    For Snort/Suricta, I would always recommend that people start with as many rules as their box can handle (Memory and CPU) and start in non-blocking mode, remove all the false positives over several weeks of review. And then putting it into Blocking mode. With Bills new tweeks removing Rules from the Alert Page makes it easier. If we had the endablesid.conf and disablesid.conf files we could populate those files with our settings and it would be even easier to manage.

    –-----------------------------------------

    Here is a list for pfBlocker.

    I like to keep the lists separate so I can see what is triggering a block. This helps to weed out False Positives.

    pfblockerlists

    pfBlocker iBlockList
    http://list.iblocklist.com/?list=bt_hijacked&fileformat=p2p
    http://list.iblocklist.com/?list=ficutxiwawokxlcyoeye&fileformat=p2p
    http://list.iblocklist.com/?list=ghlzqtqxnzctvvajwwag&fileformat=p2p
    http://list.iblocklist.com/?list=tbnuqfclfkemqivekikv&fileformat=p2p
    http://list.iblocklist.com/?list=bt_spyware&fileformat=p2p
    http://list.iblocklist.com/?list=bt_templist&fileformat=p2p

    pfBlocker ET Blocker
    http://rules.emergingthreats.net/fwrules/emerging-Block-IPs.txt
    http://rules.emergingthreats.net/blockrules/compromised-ips.txt
    http://doc.emergingthreats.net/pub/Main/RussianBusinessNetwork/RussianBusinessNetworkIPs.txt

    Spamhaus
    http://www.spamhaus.org/drop/drop.txt
    http://www.spamhaus.org/drop/edrop.txt

    pfBlocker Other
    http://www.ciarmy.com/list/ci-badguys.txt
    http://danger.rulez.sk/projects/bruteforceblocker/blist.php
    http://www.us.openbl.org/lists/base_30days.txt
    http://malc0de.com/bl/IP_Blacklist.txt

    pfBlocker Zeus/SpyEye/Palevo
    https://zeustracker.abuse.ch/blocklist.php?download=ipblocklist
    https://spyeyetracker.abuse.ch/blocklist.php?download=ipblocklist
    https://palevotracker.abuse.ch/blocklists.php?download=ipblocklist

    pfBlocker dShield
    http://feeds.dshield.org/top10-2.txt

    pfBlocker Arbor Networks - Atlas
    https://atlas.arbor.net/summary/attacks.csv
    https://atlas.arbor.net/summary/botnets.csv
    https://atlas.arbor.net/summary/fastflux.csv
    https://atlas.arbor.net/summary/phishing.csv
    https://atlas.arbor.net/summary/scans.csv
    http://atlas-public.ec2.arbor.net/public/ssh_attackers

    pfBlocker Malware Domain List
    http://www.malwaredomainlist.com/hostslist/ip.txt

    pfBlocker No Think!
    http://www.nothink.org/blacklist/blacklist_malware_http.txt
    http://www.nothink.org/blacklist/blacklist_ssh_week.txt
    http://www.nothink.org/blacklist/blacklist_malware_dns.txt

    pfBlocker SRI
    http://cgi.mtc.sri.com/download/attackers/01-17-2014/Get_Top-51_30-Day_Filterset.html
    http://cgi.mtc.sri.com/download/cc_servers/01-17-2014/Get_Top-1_30-Day_Filterset.html

    pfBlocker Infiltrated
    http://www.infiltrated.net/blacklisted

    pfBlocker AlienVault
    https://reputation.alienvault.com/reputation.snort

    DRG
    http://www.dragonresearchgroup.org/insight/sshpwauth.txt
    http://www.dragonresearchgroup.org/insight/vncprobe.txt
    http://www.dragonresearchgroup.org/insight/http-report.txt

    pfBlocker Feodo
    https://feodotracker.abuse.ch/blocklist/?download=ipblocklist
    https://feodotracker.abuse.ch/blocklist/?download=badips

    pfBlocker Blocklist.de
    http://lists.blocklist.de/lists/all.txt
    http://www.senderbase.org/static/spam/#tab=2

    pfBlocker StopForumSpam
    Local List (.CSV script to convert)

    pfBlocker Autoshun
    Local List (.CSV script to convert)



  • http://doc.emergingthreats.net/pub/Main/RussianBusinessNetwork/RussianBusinessNetworkIPs.txt

    I think that's the one that was causing problems for a number of people, so I switched from that to the "new" RBN list (now obsolete).

    A couple of interesting lists there, will test them out. If you are ok with it, I'll add them in due time to the blueprint and credit you.


  • Moderator

    I had that link with the other ET links and never noticed that it wasn't updating properly.

    If you use the pffetch script that I wrote previously, you can add that to the script and add a link in pfBlocker to the local file.

    fetch http://doc.emergingthreats.net/pub/Main/RussianBusinessNetwork/RussianBusinessNetworkIPs.txt
    It will download as "RussianBusinessNetworkIPs.txt"

    The more effort we all make the better off we all are. Open Source all the way!

    ** SORRY Bill for taking over this Thread… ***


  • Moderator

    I took another look at the RBN text document in VI, and noticed that each line has a "^M" carriage return. This is probably what was causing issues with pfBlocker not reading the file properly. The RBN list is out of date, but there are still alot of hits on my Router from Russia!!

    You can filter the ^M with -

    fetch http://doc.emergingthreats.net/pub/Main/RussianBusinessNetwork/RussianBusinessNetworkIPs.txt
    returncode=$?
    echo $returncode

    if [ "$returncode" -eq "0" ]; then
            cat RussianBusinessNetworkIPs.txt | tr -d '\r' > RBN.txt
    fi

    and use the RBN.txt in pfBlocker local file.



  • The funny thing is that I personally never had a problem with that list. It downloaded and added the IPs in the table  (checked it myself, and the IPs were there), as well as updated for over a year with no issues at all. Some other people though always had problems with it.

    That list belongs to the ET guys, so I'm assuming that it too will be made obsolete. I know that you should never assume but…

    yea, sorry Bill for taking over the thread  :P



  • Not sure if this is the place to post, but I figure it's a good starting point if nothing else, is there an easy way to get Suricata to throw the logs to Kibana like Suricata shows on their site?
    http://idsips.files.wordpress.com/2014/03/kibana300.png

    Per this walkthrough:
    https://redmine.openinfosecfoundation.org/projects/suricata/wiki/_Logstash_Kibana_and_Suricata_JSON_output

    It wants Suricata to have libjansson support enabled… the only thing they're missing is how to get Suricata and the install of Kibana, etc to talk to each other, but this all may be way too much to ask this early on in the game, not sure if anyone has any tips on it. Thank you for helping if possible!



  • @felesaerius:

    Not sure if this is the place to post, but I figure it's a good starting point if nothing else, is there an easy way to get Suricata to throw the logs to Kibana like Suricata shows on their site?
    http://idsips.files.wordpress.com/2014/03/kibana300.png

    Per this walkthrough:
    https://redmine.openinfosecfoundation.org/projects/suricata/wiki/_Logstash_Kibana_and_Suricata_JSON_output

    It wants Suricata to have libjansson support enabled… the only thing they're missing is how to get Suricata and the install of Kibana, etc to talk to each other, but this all may be way too much to ask this early on in the game, not sure if anyone has any tips on it. Thank you for helping if possible!

    I am not familiar with Kibana but will check it out.  So long as an external log stash package can accept data over a network connection then pushing Suricata logs should be possible.  It gets much more dicey to try and add another package to pfSense itself.  Besides, it's not a good idea to run a bunch of applications on your firewall because that increases the security vulnerability exposure substantially.

    Bill



  • any options to have suricata 2.0 and have options to block ip?

    can i have only to drop packet, not to block ip (snort or. suricata)?



  • @simby:

    any options to have suricata 2.0 and have options to block ip?

    can i have only to drop packet, not to block ip (snort or. suricata)?

    Suricata 2.0 was not in the FreeBSD ports repository last time I checked (about a week ago).  So we will need to wait for FreeBSD ports to update Suricata to 2.0 before it can come to pfSense.

    I am working on the blocking code for Suricata now.

    Bill


  • Banned

    Thanks a billion Bill!! Youre SO much the man of this project right now!



  • @Supermule:

    Thanks a billion Bill!! Youre SO much the man of this project right now!

    Thank you.  One caveat for Suricata blocking.  Initially it will have to operate the same way as Snort does using libpcap.  Thus it won't be true inline-mode IPS.  Ermal has to make some changes in the ipfw code within pfSense in order to accommodate true inline IPS mode.  However, due to the problem of context switching between kernel mode and user-land, IPS mode when it comes won't be nearly as fast as the pseudo-IPS mode Snort uses (and that Suricata will use initially).  So true inline IPS is probably not going to be very useful for heavily loaded firewalls.  That's just the nature of the beast unless you go to highly customized code, and if you do that then you can't easily follow the upstream updates.

    The kernel changes to support true IPS may or may not make it into 2.2.  That is not up to me.  It is up to the pfSense team.  However, I can include the pseudo-IPS mode without those kernel changes.  That means pseudo-IPS can work with 2.1.x releases.  The pseudo-IPS mode is what I am working on now.

    Bill


Log in to reply