Snort update coming soon – please read about an important change!
-
@BBcan17:
Ok - Ill start from the top and work my way down :)
xrdp is ok if you are local, but we support alot of customers all over the country and alot don't have the fastest inet connections, so tunneling x win over their links isn't feasible. :)
As far as database credentials, SO uses root with no password for local only logins… you can't login a remote sensor with root, nor would you want to, so we create a user per sensor - i.e. location-fw-interface in mysql and only give them privs on the snorby DB.
I found this link http://community.spiceworks.com/topic/466735-security-onion-and-pfsense
Does this look correct?
===========================
Open the terminal run the following commands
This will allow your pfsense to connect through Security Onions firewall to mysql
sudo ufw allow proto tcp from xx.xx.xx.xx/32 to any port 3306
mysql
This should open the prompt for mysql, run the following commands hereshow databases;
Make sure the databasesnorby
exists.This part will setup a user for Barnyard2 to use
create user 'sensorname'@'xx.xx.xx.xx' IDENTIFIED BY 'SENSORPASSWORD'
grant all privileges on snorby. to 'sensorname'@'xx.xx.xx.xx' with grant option;*
flush privileges;
exit
You should be out of the mysql prompt now, run these commands in terminalThis will allow any device to connect to mysql as MySQL will be listening for connections on any IP instead of just the loopback address (assuming its allowed through Security Onions firewall)
sudo vi /etc/mysql/my.cnf
You should have the mysql config open, use the down arrows to move down until you see the part that says
bind-address
Move the cursor (The one in the terminal window that indicates where you're typing) to where it says "= 127.0.0.1" and type 'i' (Without the ' )
It should say "– INSERT --" at the the bottom of the terminal window
Delete 127.0.0.1 and replace it with 0.0.0.0
Press 'esc' so that "– INSERT --" is gone, then type ':wq!' and hit return (Without the ' but with the Colon )
This should save and exit viNow run the command
sudo mysql restart
Wait for the MySQL Service to restartSetup Barnyard2 on pfSense in Snort
In web interface for pfSense goto Services > Snort, Under Snort Interfaces Click the
Edit
button and open the Barnyard2 tabChange and input the following settings in the
Log to MySQL Database
fieldoutput database: alert, mysql, dbname=snorby user=sensorname host=xx.xx.xx.yy password=SENSORPASSWORD
Save
goto Services > Snort > Snort Interfaces
Click the start Icon next under the Barnyard 2 Column, wait for it to go green.===========================
We run our main sensors at the perimeter via our build of PFsense as we use the block function of snort on the WAN and LAN via a cutomized ET Pro ruleset and we use pfblocker with several customized ip block lists.. but since snort runs in userspace, we get reports on the blocked ip's via snort. It's important to us to catch, log an block any attacks at the perimeter, which Pfsense is amazingly efficient at.
As SO is using "SALT", it would be good to try to incorporate that into the mix to mange the rules update process. But I guess as pfSense Snort is not using Pulled Pork that might not work? Salt is on the Freebsd ports.
Snort now has the IPrep processor which could replace pfBlocker functionality. The current pfsense integration doesn't reload the blocklists automatically when they are updated so will need to wait for the next release to see if that is fixed.
We don't run any sensors on SO at the moment..though I would not mind testing Bro in conjunction with snort as it doesn't use signatures from what we have read.
The biggest thing for us is to be able to monitor and alerts exception (high severity) alerts as we monitor alot of networks (hundreds) along with our SIEM to keep any breach attempts to a minimum.
Will changing mysql to 0.0.0.0 affect SO at all?
Getting the pfSense snort into Sguil would make this a whole lot better. The BRO integration would be much better.
I need to see if I can get Snorby to only listen to the pfSense boxes and not the local sensor installations.
Does pfSense push the full pcaps of the alert to SO Snorby? or just the alert?
Thanks for your help.
Everything looks good but I will make a few comments:
You really don't need "GRANT" options with the sensor user from what I can see.. no need to add extra vulnerabilities if not needed, no matter how remote.
Changing the mysql bind address to 0.0.0.0 just tell's mysql to listen on all interfaces… if you have more than 1 you can force it to listed on a specific interface by putting that interface's IP address as the bind address instead.SQUIL is good for looking at all alerts in detail, but we look more for exceptions and then if we see an issue dig into the SIEM for details, unless they are obvious from the snort alerts.
If you install SO as a "Server" instead of a "Sensor" or "Both" it will run as a collector versus a sensor (no local instances of snort or suricata).
PFsense just pushes the alerts, no pcap data from what we use.
-
SQUIL is good for looking at all alerts in detail, but we look more for exceptions and then if we see an issue dig into the SIEM for details, unless they are obvious from the snort alerts.
If you install SO as a "Server" instead of a "Sensor" or "Both" it will run as a collector versus a sensor (no local instances of snort or suricata).
PFsense just pushes the alerts, no pcap data from what we use.
Once you F8 (categorize) events in SGUIL or Snorby you need ELSA or another SIEM to do the rest of the detective work. ELSA is really nice can pivot on Snort/Suricata/BRO/OSSEC/Sancp/pads and Syslogs. I push all of my pfSense syslogs (Firewall and Syslogs) to ELSA.
pfSense needs a one-line patch to get it to work properly with ELSA.
http://files.pfsense.org/jimp/patches/pf-log-oneline-option-2.1.1.diffIf you arent getting the full pcap you are missing a lot of info. This is why I like Security Onions Distributed Architecture. The Sensors can be deployed anywhere. All of the local devices push their data to the remote SO Sensor and the sensor records all of the info locally. Than from the SO Server, it can pull as required from any of the sensors. So SGUIL or Snorby will be able to read the full packet capture from any of the remote sites without having a large footprint.
Also SO uses SALT which can manage all of the RULES for all of the Sensors and it makes life a lot easier.
If we can get the full Pcap from pfSense into SO, that would be a big help to manage False Positives. Managing several devices under one umbrella makes life a little easier so we can spend our time analyzing versus configuring and updated all over the place.
Hopefully we can convince (nudge nudge) Mr. Meeks in helping get pfSense to work well with SO?? ;) ;)
-
This is a barnyard error. Are you using i386 or x64?
x64. This error appears limited to only Barnyard, as Snort will startup without it but I'm unsure given that the issue initially began with signature errors which was a past problem pfSense version issue with Snort's rules.
This version of the Snort package made some big changes to Barnyard2 like adding more output plugins. One side effect of that was it needed to reconfigure how the MySQL DB login info was stored. I wrote some code in the installation routines that attempts to migrate the old MySQL settings to the new format. It is possible some old settings could trip it up. Your error is a MySQL login error, and when the login fails Barnyard2 will stop. Go to the Barnyard tab in Snort and manually re-enter the login ID and password for the MySQL DB, then click Save. Try to start Barnyard2 after that on the Snort Interfaces tab.
Bill
Yeah, I'd believe that it was a caching thing or a transfer at the start but then I did this:
In desperation following a remove / install of Snort still facing the same issues, I cleanwiped the drive and reinstalled pfSense from scratch - that is to say, clean from a CD with no imports from a previous config file, everything reentered by hand - and the error remains the same. pfSense is determined to use root only root and nothing but root, and I'm not gonna let it; especially when everything says it should be using the proper user that for some reason it is ignoring.
Gonna go out on a limb and say that it isn't a transfer issue. It simply isn't reading the configuration I gave it - the files I checked in Barynard's conf and the conf.xml for pfSense, as well as the GUI, are all pointing to the correct login but pfSense is still attempting to login with root and getting refused. Reentering the information by hand has no effect.
I think I found the cause of Barnyard2 trying to login to the DB as "root". It happened when a sensor name value was provided (that is, when the field was not blank). The code was including a comma after the sensor name in the barnyard2.conf file and there is not supposed to be one there. That was apparently confusing the parser for Barnyard2. The latest update for the Snort package that is being reviewed by the Core Team has this fixed. When they approve and merge the change, a new Snort update will appear under System…Packages.
Bill
-
I went ahead and grab the changes you made before the merge. Would it be possible to change the order of the database output? If I select the disable_signature_reference_table option (BTW you beat me too it, I started to test this option last night and was able to turn up 2 interfaces with no dup error), the 'root' error happens again when it tries to connect to mysql.
Right now the order is:
output database: log, mysql, sensor_name=pfsense_lan disable_signature_reference_table user=snort password=ABC123 dbname=snort host=somebox
After reading barnyard2's README.database doc, it seems to work in this order:
output database: log, mysql, user=snort password=ABC123 dbname=snort host=somebox sensor_name=pfsense_lan disable_signature_reference_table
-
I went ahead and grab the changes you made before the merge. Would it be possible to change the order of the database output? If I select the disable_signature_reference_table option (BTW you beat me too it, I started to test this option last night and was able to turn up 2 interfaces with no dup error), the 'root' error happens again when it tries to connect to mysql.
Right now the order is:
output database: log, mysql, sensor_name=pfsense_lan disable_signature_reference_table user=snort password=ABC123 dbname=snort host=somebox
After reading barnyard2's README.database doc, it seems to work in this order:
output database: log, mysql, user=snort password=ABC123 dbname=snort host=somebox sensor_name=pfsense_lan disable_signature_reference_table
Did you disable the reference table on both interfaces? Do you know what effect (if any) this has on the logging of snort alerts in mysql?
Thanks!
Greg
-
I went ahead and grab the changes you made before the merge. Would it be possible to change the order of the database output? If I select the disable_signature_reference_table option (BTW you beat me too it, I started to test this option last night and was able to turn up 2 interfaces with no dup error), the 'root' error happens again when it tries to connect to mysql.
Right now the order is:
output database: log, mysql, sensor_name=pfsense_lan disable_signature_reference_table user=snort password=ABC123 dbname=snort host=somebox
After reading barnyard2's README.database doc, it seems to work in this order:
output database: log, mysql, user=snort password=ABC123 dbname=snort host=somebox sensor_name=pfsense_lan disable_signature_reference_table
Sure. I will swap the order since the Pull Request is still open and I can update it. Thanks for testing and reporting back.
UPDATE: the fix to swap the order has been added to the open Pull Request.
Bill
-
I went ahead and grab the changes you made before the merge. Would it be possible to change the order of the database output? If I select the disable_signature_reference_table option (BTW you beat me too it, I started to test this option last night and was able to turn up 2 interfaces with no dup error), the 'root' error happens again when it tries to connect to mysql.
Right now the order is:
output database: log, mysql, sensor_name=pfsense_lan disable_signature_reference_table user=snort password=ABC123 dbname=snort host=somebox
After reading barnyard2's README.database doc, it seems to work in this order:
output database: log, mysql, user=snort password=ABC123 dbname=snort host=somebox sensor_name=pfsense_lan disable_signature_reference_table
Did you disable the reference table on both interfaces? Do you know what effect (if any) this has on the logging of snort alerts in mysql?
Thanks!
Greg
Greg:
My understanding is this just prevents the insertion of the references information included with most signatures. It does not alter the insertion of the other parts of the alert data.
Bill
-
Thanks Bill!!! There shouldn't be any effect in how the alerts are sent to mySql. I only disabled it on my LAN interface, and left WAN alone so it would update the DB.
-
Question. Is there a technical reason why you can't setup snort more than once on a single interface?
I was running some performance tests on my C2758 for cmb this morning and it occurred to me that since snort is single-threaded per interface, that it would be a lot more efficient on a multi-core box (C2758 has 8 cores) to offload rules that don't have anything to do with each other to a separate process. For example, run VRT Balanced on one process and the ET rulesets relating to rbn, botcc, and currentevents on another.
I've had a couple occasions where snort wouldn't shutdown correctly and I end up with two processes on a single interface which both grab traffic (based on CPU usage). Why not do it intentionally?
Thoughts?
-
Question. Is there a technical reason why you can't setup snort more than once on a single interface?
I was running some performance tests on my C2758 for cmb this morning and it occurred to me that since snort is single-threaded per interface, that it would be a lot more efficient on a multi-core box (C2758 has 8 cores) to offload rules that don't have anything to do with each other to a separate process. For example, run VRT Balanced on one process and the ET rulesets relating to rbn, botcc, and currentevents on another.
I've had a couple occasions where snort wouldn't shutdown correctly and I end up with two processes on a single interface which both grab traffic (based on CPU usage). Why not do it intentionally?
Thoughts?
Within the current GUI code that can't happen because it tests for an interface already being "Snort-configured" before letting you save a newly added interface. So from that narrow point of view the answer is "no, it's not technically possible". However, from a broader point of view, I suppose it could be done but it would take a lot of additional complexity within the GUI code to handle that situation. Some additional UUIDs would have to be assigned to each interface to identify each configured "thread" on an interface. This is because each Snort process needs it own separate configuration file and logging directories. Currently those are identified and named using the physical interface name and a UUID as part of the path name.
According to the Snort people (when comparing themselves to Suricata), the claim is no huge benefit to "threading" with Snort. I am not expert enough in all the subtleties to know, though.
I am quite close to having the Suricata blocking plugin ready, and Suricata is multi-threaded out of the gate. For the initial release, Suricata will use the same libpcap and pf table block mechanism as Snort does. This is because true inline IPS mode needs some critical changes in the ipfw code in pfSense.
Bill
-
They claim there isn't a huge benefit to threading. They didn't say there wasn't a benefit to multiple processes. I believe that's the whole point behind PF_RING.
The difference here is that instead of distributing packets on a round-robin basis to identically-configured snort processes, I was proposing using multiple snort instances, each getting all the packets, with different rule-sets.
I agree that suricata should come first. If you plan to continue to maintain snort after suricata's "completion" though, I'd like to push this as a feature request.
-
They claim there isn't a huge benefit to threading. They didn't say there wasn't a benefit to multiple processes. I believe that's the whole point behind PF_RING.
The difference here is that instead of distributing packets on a round-robin basis to identically-configured snort processes, I was proposing using multiple snort instances, each getting all the packets, with different rule-sets.
I agree that suricata should come first. If you plan to continue to maintain snort after suricata's "completion" though, I'd like to push this as a feature request.
OK. After I get Suricata out with blocking, then I will see what it would take to implement this and then discuss with the Core Team devs. Sometimes they have alternative suggestions or desires.
Bill
-
Multiple processes doesn't bring any use if it isn't combined with using pf_ring or other similarly capable packet capture interface. The currently used libcap doesn't have support for stuff like this. This would mean that you run the same ruleset with multiple Snort instanced for the same physical interface. Traffic is routed to each Snort interface by pf_ring.
Using multiple Snort interface with different rulesets sounds like a really stupid idea to me.
Then again, I don't actually know anything at all about networking and IDS efficiency.
-
If you want run multiple snort interfaces using different rule-sets (alerting/blocking), comment out line 114 to line 122 in /usr/local/www/snort/snort_interfaces_edit.php.
Use at your own risk
look for
/* See if assigned interface is already in use */ if (isset($_POST['interface'])) { foreach ($a_rule as $k => $v) { if (($v['interface'] == $_POST['interface']) && ($id <> $k)) { $input_errors[] = gettext("The '{$_POST['interface']}' interface is already assigned to another Snort instance."); break; } } }
change to
/* See if assigned interface is already in use if (isset($_POST['interface'])) { foreach ($a_rule as $k => $v) { if (($v['interface'] == $_POST['interface']) && ($id <> $k)) { $input_errors[] = gettext("The '{$_POST['interface']}' interface is already assigned to another Snort instance."); break; } } } */
-
bmeeks, can you add a way to filter out the:" [Classification: Attempted Information Leak] [Priority: 2]" (classification and priority) from the logs? Logging those to a syslog server is just adding extra unneeded info and makes managing the logs a bit of a headache.
-
@jflsakfja:
bmeeks, can you add a way to filter out the:" [Classification: Attempted Information Leak] [Priority: 2]" (classification and priority) from the logs? Logging those to a syslog server is just adding extra unneeded info and makes managing the logs a bit of a headache.
Not sure. I will have to see if Snort offers any native log output filtering. I don't recall off the top of my head. Suricata does. There are some log regex filters that can be used (but I don't have those enabled yet in the Suricata package for pfSense).
As I've said a few times, I am almost done with the initial blocking plugin for Suricata. I actually have it working as of last night on my VMs. Testing and refining the next few days, and then I will update the GUI code to accommodate the feature. As part of that update I can examine implementing the regex filtering.
Bill
-
Would really appreciate it if something like that is implemented in suricata, since i'm also waiting for the blocking feature to be available to move off snort.
Thank you for all the time you've put so far into the IDS/IPS packages.
-
I had this exact issue (along with continuing "Duplicate Entry" issues), but to fix it, I removed the "Sensor Name" in the Barnyard2 portion of the web gui. This corrected barnyard2 logging in as root. I was able to repeat the issue and correct it by adding a name to the "Sensor Name" - broke, and then removing it.. fixed.
Yes, confirmed fix for me as well. Removing sensor names stops it from using root to try to access MySQL. Unfortunately, ever since the update I'm unable to get all the interfaces working with Snort; if I turn on more than one the signature clashes start occurring after a few minutes of otherwise stable operation. None of the old fixes is correcting it, and rebuilding the DB still only grants a few mins of operation before signature errors.
For whoever it was talking about Snorby dashboard issues, my fix to this was just a time check issue. I ran the pfSense using local time instead of UTC ( as was defined on the linux host for Snorby - pfSense was on AzT, with the DB server and the Snorby server on UTC ) which caused my dashboard to never update due to the differences in timezones for the database and the data coming in without a truncate tables command run on the db. Which would allow the dashboard to update once and then die.
Internet searches will provide a lot of posts with regards to Snorby's dashboard being a pain in the ass though, so barring something simple like a time mismatch it's a veritable field day of answers to try.
-
I had this exact issue (along with continuing "Duplicate Entry" issues), but to fix it, I removed the "Sensor Name" in the Barnyard2 portion of the web gui. This corrected barnyard2 logging in as root. I was able to repeat the issue and correct it by adding a name to the "Sensor Name" - broke, and then removing it.. fixed.
Yes, confirmed fix for me as well. Removing sensor names stops it from using root to try to access MySQL. Unfortunately, ever since the update I'm unable to get all the interfaces working with Snort; if I turn on more than one the signature clashes start occurring after a few minutes of otherwise stable operation. None of the old fixes is correcting it, and rebuilding the DB still only grants a few mins of operation before signature errors.
For whoever it was talking about Snorby dashboard issues, my fix to this was just a time check issue. I ran the pfSense using local time instead of UTC ( as was defined on the linux host for Snorby - pfSense was on AzT, with the DB server and the Snorby server on UTC ) which caused my dashboard to never update due to the differences in timezones for the database and the data coming in without a truncate tables command run on the db. Which would allow the dashboard to update once and then die.
Internet searches will provide a lot of posts with regards to Snorby's dashboard being a pain in the ass though, so barring something simple like a time mismatch it's a veritable field day of answers to try.
armouredking:
A new Pull Request is posted on Github containing a fix for both of these issues (well, a fix for the "root login thing" and a workaround for the duplicate key errors when running multiple Snort/Barnyard2 processes against a single DB). The Core Team has not yet approved and merged the fix, though. Hopefully they can get to it this week. In the meantime, if you know how to grab PHP files off of Github you can do that. Some other users have done that already. All of the fixes are within the PHP files. Here is a link: https://github.com/pfsense/pfsense-packages/pull/648
Bill
-
How would one go about requesting a feature in the snort package?
Right now, the package provides IP info links for DNS Stuff and WHOIS. I'd love to see a link to Virus Total. Should be pretty straight forward, for instance:
https://www.virustotal.com/en/ip-address/23.62.236.97/information/
Virus Total provides a lot of history information on an IP, including reverse DNS history and seeding potential for infected files.
Thanks,
Ben