Snort 3
-
I doubt I'll get an answer, but worth a shot. Anyone have any guess on when we might see Snort 3 available on pfSense? I think that is going to be a BOOM when it comes available. It seems like it's been available directly on the Snort website for a while so I'm surprised it's not an optional package for pfSense.
Even if a beta (or 'devel') version was available for pfSense, I would be willing to give it a shot.
Rumors? Anything?
Thanks.
-
Since I maintain the Snort 2.9.x package on pfSense, I guess it's logical that I will be the one to create the Snort3 package...
I tried to compile Snort3 as a test package for pfSense back a few months ago. The compilation failed. I have not tried since then, but I do have plans to see about introducing a Snort3 package in the future. Mostly waiting for it to go to RELEASE from the current BETA mode.
I will also have to completely rewrite the custom blocking plugin used on pfSense. The new Snort3 architecture is quite different in terms of the internal plugin plumbing as compared to Snort 2.9.x. Because of that, it is likely the first version of Snort3 might offer IDS mode only with no blocking available. Depends on how hard it is to rewrite the blocking plugin and integrate it with Snort3.
So to shorten that response down, I would not expect to see Snort3 on pfSense in the near term (where near term means in the next 3 to 6 months). Maybe later this year, though.
-
Thanks for the quick reply. It sounds like it's not just as simple as just dropping it in.
If there is any way for me to help, just let me know.
You probably don't have a way to do this, or allowed, but I would be willing to donate $100 to a fund to help expedite this process, so you could make it a priority**. Also, by what you said, this may not even be worth the effort until it leaves BETA.
**(I say this with the assumption that others like me have the same motivation, where a legitimate pool could be created and funds released upon completion (in a reasonable amount of time).
Just trying to find a positive way to make everyone happy :) If this is even possible, maybe someone else knows a way to set this up. (or the reply may be NO, don't even try)
A year sounds SOOO far away. Sigh...
-
@talaverde said in Snort 3:
Thanks for the quick reply. It sounds like it's not just as simple as just dropping it in.
If there is any way for me to help, just let me know.
You probably don't have a way to do this, or allowed, but I would be willing to donate $100 to a fund to help expedite this process, so you could make it a priority**. Also, by what you said, this may not even be worth the effort until it leaves BETA.
**(I say this with the assumption that others like me have the same motivation, where a legitimate pool could be created and funds released upon completion (in a reasonable amount of time).
Just trying to find a positive way to make everyone happy :) If this is even possible, maybe someone else knows a way to set this up. (or the reply may be NO, don't even try)
A year sounds SOOO far away. Sigh...
pfSense has a mechanism called "Bounties". Here is a link to that sub-forum: https://forum.netgate.com/category/30/bounties.
I will try compiling Snort3 again in the near future (this month) and see what happens. I am also working on improving Netmap integration in DAQ, the data acquisition library used by Snort. I have been having an email conversation with the Snort principals (Cisco folks, now) about this work. Still a long way from being ready because Netmap documentation is sparse and programming examples are even rarer. The Netmap work should carry over to both Snort 2.9.x and Snort3.
I dropped the Snort3 work primarily because of the compile failure and because Snort 2.9.12 came out along with Suricata 4.1.x. Both of those packages took the higher priority.
-
Great Answer. I'll check out the bounties.
If it wasn't for the the lack multi-threading, I wouldn't really care. I have a gigabit fiber connection and Snort takes a big toll, even on my 4+ GHz i7. I opted against $uricata because their paid options aren't affordable for the little guy. I'm willing to pay for a subscription, but not $500 or even thousands per year for home use.
As you know, the multi-threading is just one benefit. I can't wait to put 3.0 into action. Thanks for the dedication and hard work.
-
@talaverde said in Snort 3:
Great Answer. I'll check out the bounties.
If it wasn't for the the lack multi-threading, I wouldn't really care. I have a gigabit fiber connection and Snort takes a big toll, even on my 4+ GHz i7. I opted against $uricata because their paid options aren't affordable for the little guy. I'm willing to pay for a subscription, but not $500 or even thousands per year for home use.
As you know, the multi-threading is just one benefit. I can't wait to put 3.0 into action. Thanks for the dedication and hard work.
Be aware that multithreading is not necessarily all that it's marketed as. Suricata has multithreading and can still have bottlenecks in some situations. There are several whitepapers scattered around the web that document testing scenarios with multithreading on Suricata. Also, the Snort folks for years pooh-poohed the advantages of multithreading; especially after Suricata came out and had multithreading while Snort 2.9.x did not ... . Funny how multithreading wasn't that big of a deal until Snort3 got it.
Not saying multithreading can't be a good thing, but the performance gains are all wrapped up in the details of its implementation and exactly how packets pass from module to module (decoding, protocol detection, signature matching and alerting).
-
Beware that multi-threading is not necessarily all that it's marketed as...
Being that I currently have Snort installed, instead of Suricata, is a way of agreeing with that comment (to a point). I'm not expecting the holy grail, but i am 'excited' to see what improvements there are.
I took a 50% hit with Snort (though who really needs 80MB/s+ for daily driving). If Snort 3.0 doesn't give back some of my bandwidth, I'll probably drop my fiber down to 30MB/s, even it's on $20/mo difference.
Not to have a topic change, but I originally had Snort monitoring both my WAN and LAN. I lost too much bandwidth. So, I cut it to just my LAN. I'm hoping I can either go back to monitoring both WAN and LAN, or sticking with LAN, but getting my speeds back... some? a little? hopefully it's measurable.
We'll see. Thanks.
-
BTW, just in case there may have been some confusion, my comment about multithreading not being a big deal until Snort3 had it was a small jab/joke aimed at the Snort developers and not you personally . Just wanted to clarify that to be sure no offense was taken.
Personally, for home use, I see no point in running the IDS/IPS on the WAN. The default WAN rule set will drop all unsolicited traffic anyway, so putting the IDS/IPS there is just going to get a lot of stuff logged that the firewall is dropping anyway. Snort sits in front of the firewall on the WAN side, so it sees and logs everything before the firewall rules processing occurs.
I suggest home users, and even many business users, put Snort (or Suricata) on the LAN. That does two things. First, it removes a bunch of "noise" from the logs that the firewall dropped. No sense in my view of logging and alerting on a bunch of stuff the firewall rules are going to drop anyway. Second, it means all the local host addresses in alerts will be the actual LAN address of the host and not the WAN's public IP (like happens when you run Snort on the WAN). This is because on the WAN, Snort sees traffic before the NAT rules are "undone". Of course if you are not using NAT, then this second point is moot.
A 50% performance hit is unusual unless your NIC and/or CPU is a little low on horsepower. Of course the number of enabled rules plays a significant role as well. The packet size can also be a big factor in performance. A bunch of small packets like ICMP replies consume a lot of CPU as opposed to fewer packets but with bigger payload sizes. So this means your MB/sec will increase with larger packets while CPU utilization may even decrease a bit.
-
re: multi-threading, completely understood your jab. I just meant I don't expect a miracle. I think it's funny how they've been careful to use the proper wording for that new 'feature'.
re: WAN. Just like pfBlockerNG, the WAN blocks by default anyway, so why filter? I was torn on that one with Snort. I thought maybe it's good to have another check in case an internal (bad) app requested data (i.e. virus). Your vote helped and your logic is sound. LAN only -- got it.
I'm using the 'Security' IPS Policy, all the ET Open Rules, and all the OPENAPPI rules. So, a lot of rules! (I'm still tweaking)
(AT&T Gigabit Fiber, Intel NICs w teaming, 10GB RAM, i7-4790, SSD)
I've had to disable many APPI rules for apps I'm running (or false alerts), and still working on that.
The 50% hit was when I had both WAN and LAN enabled. When I have time, I'll post a comparison of speeds w/ Snort on and off. May do separate thread. I wish I had a better way to identify false alerts while also test effectiveness (see Snort work in action). I have lots to learn.
(Side question - would using RAM disks help, as I have plenty of RAM, or is Snort all in RAM anyway? Could it hurt?)
-
@talaverde said in Snort 3:
(Side question - would using RAM disks help, as I have plenty of RAM, or is Snort all in RAM anyway? Could it hurt?)
Snort is totally a RAM resident beast other than of course the log files. So a RAM disk is of zero benefit really, and can actually lead to issues if it's not large enough as Snort needs 256 MB or so of space in /tmp to download, extract and install rule updates.
-
A 50% performance hit is unusual unless your NIC and/or CPU is a little low on horsepower. Of course the number of enabled rules plays a significant role as well. The packet size can also be a big factor in performance. A bunch of small packets like ICMP replies consume a lot of CPU as opposed to fewer packets but with bigger payload sizes. So this means your MB/sec will increase with larger packets while CPU utilization may even decrease a bit.
In a very rough test, I'm getting 700 Mb/s down / 600 Mb/s up on my 1Gb/s fiber w Snort OFF. With Snort on, it drops to 496 Mb/s down / 525 Mb/s up.
I turned off all the OPENAPPI rules as I'm not really sure how they are helping me with just one or two people in my home. With those off, my speeds went back up to 680 Mb/s down / 600 Mb/s up.
(Note - all these speeds are through a VPN. Big Brother AT&T has BLOCKED all speed test sites. The only way I can run an legitimate speed tests are through a VPN. The main goal here is to measure the affect of Snort anyway)
Would you agree that the OPENAPPI rules aren't worth the speed drop they produce?
-
@talaverde said in Snort 3:
A 50% performance hit is unusual unless your NIC and/or CPU is a little low on horsepower. Of course the number of enabled rules plays a significant role as well. The packet size can also be a big factor in performance. A bunch of small packets like ICMP replies consume a lot of CPU as opposed to fewer packets but with bigger payload sizes. So this means your MB/sec will increase with larger packets while CPU utilization may even decrease a bit.
In a very rough test, I'm getting 700 Mb/s down / 600 Mb/s up on my 1Gb/s fiber w Snort OFF. With Snort on, it drops to 496 Mb/s down / 525 Mb/s up.
I turned off all the OPENAPPI rules as I'm not really sure how they are helping me with just one or two people in my home. With those off, my speeds went back up to 680 Mb/s down / 600 Mb/s up.
(Note - all these speeds are through a VPN. Big Brother AT&T has BLOCKED all speed test sites. The only way I can run an legitimate speed tests are through a VPN. The main goal here is to measure the affect of Snort anyway)
Would you agree that the OPENAPPI rules aren't worth the speed drop they produce?
Yes, I would not use the OpenAppID rules in a home environment. Those are primarily aimed at the corporate world to help IT Security admins enforce employee policies (such as no Twitter, Facebook or Instagram on company time, etc.). That is of limited benefit at home unless you are just curious and want to experiment (or you want to torture any teenage kids in the house ).
-
Yes, I would not use the OpenAppID rules in a home environment. Those are primarily aimed at the corporate world to help IT Security admins enforce employee policies (such as no Twitter, Facebook or Instagram on company time, etc.). That is of limited benefit at home unless you are just curious and want to experiment (or you want to torture any teenage kids in the house ).
Thanks for the confirmation.
I've managed to manipulate my AT&T modem to allow speed tests (DSLReports / Ookla). With no VPN, I'm getting roughly 850 Mb/s down / 825 Mb/s up. That's through pfSense (HyperV VM), 24-port L2 Switch, Snort ON, and pfBlockerNG ON. (Oh, yea, I turned off OpenAppID in Snort.) Losing 150 Mb/s to all that isn't bad IMO.
As a VPN is a MUST these days, those test above aren't reality. Doing the same test as above but with a VPN has been a nightmare. I can't get any consistent results. Not only are VPN services notoriously inconsistent, Snort is throwing all sorts of alerts with DSLReprorts speed test. Even w/o the alerts, I can't get any constant speed test results worth reporting.
Some of the Snort alerts, as an example:
Sensitive Data was Transmitted Across the Network / (spp_sdf) SDF Combination Alert
Sensitive Data was Transmitted Across the Network / SENSITIVE-DATA Email Addresses
... multiple times
Other alerts too, I think.Both DSLReports and Ookla are giving inconstant results.
In short, best I can tell, I'm getting roughly 600-700 Mb/s down / 300-500 Mb/s up with VPN. Sometimes upload drops to <100Mb/s for some reason (my guess, Snort is blocking it)
I've tried both PIA and NortdVPN (integrated into pfSense)
Once I can provide more meaningful data, I'll report back. In short, it seems like Snort isn't playing well with either VPN or the speed tests, or the combination of the two.
-
In attempt to close this out:
After a reboot, my speeds were 800 Mbps / 655 Mbps through VPN, Snort and pfBlockerNG. I'm quite happy with these results. (Only concern is that this required a reboot. What happens after time?? Only time will tell). Moral of this story is that Snort doesn't have a 'debilitating' affect on speeds with reasonable number of rules. Nor does pfB. (VPN is the most inconsistent)
The biggest problem is combining inconsistent VPN speeds, along with Snort analyzing various speed test scripts. Too many variables to give good results. Best bet is to measure each of these individually.
(For the average users) The best set of Snort rules seems to be:
- LAN (Only)
- 'Security' IPS Policy (Subscribe to paid Oinkmaster code)
- ET Open rules (Non-paid - too expensive for personal use)
- No OpenAppi Rules - Unless you have a family to torture ;)
I haven't seen any significant performance issues with this setup. My main conclusion from all this is that pfSense needs a separate section to discuss VPN speeds. Most articles on the web are paid for and don't take pfSense into consideration. (To those listening - PLEASE set up separate board for VPN discussions. thanks!)
-
@talaverde
Why do you think a VPN is a necessity for web browsing? For most home users, the only real need for a VPN is to allow secure remote access back into a home network from outside (think Remote Desktop or SSH, for example). I am not an advocate of VPN use for privacy reasons. But to each his own, I guess.As for your Snort alerts on Sensitive Data, you do not need those rules enabled in my opinion. They are, much like the OpenAppID rules, aimed primarily at corporations and others who have either HIPPA or financial data on their networks that needs protection from ex-filtration. Those rules will false positive on a lot of normal home network traffic (such as simply signing up on a web site with your email address). Who wants an alert and block about that?
Dont' take this in a negative way, 'cause that's not how I mean it, but you need to do some research on IT security and tuning an IDS/IPS. You have way too many rules enabled for a home network. You really should not be running the IPS-Security policy either at home. IPS-Connectivity is more than adequate for home users and will generate next to no false positives. For the truly paranoid home user, maybe bump up to IPS-Balanced. IPS-Security is more for Ft. Knox or the pentagon ... . You will get false positive with IPS-Security enabled.
-
(Nothing taken negative. I welcome contrarian views. - so now that's out of the way..)
Multiple topics here:
I'm actually an IT consultant, installing, configuring and training users on financial software for firms larger than I'd like to mention here. Most connections are through... wait for it... VPN connections. With that said, I still want to harden my local environment the best I can. Nightmare scenario would be a virus I get through casual browsing which somehow gets passed on to their corporate environment. (1st thing you'd say is separate the hardware I use to connect to these clients. Reality isn't always so cut and dry)
Regardless of my clients and my career, I know too much about these vulnerabilities to not worry about my own data. HOWEVER, I agree with you to a point. I'm trying to find a balance. I may not need all these rules, but some may be helpful. I sure wish I had some good documentation to figure out which is which. I do this for a living with other software. In those cases, I have resources available. (Manuals, Knowledge Bases)
Now, the VPN topic could go own for ages. We have two types of VPNs. A 'tunnel' to connect a remote machine to a hardened corporate network. Then, you have personal VPNs so your ISP, Google, Microsoft, AT&T, Apple, and everyone else don't have a never-ending history of your every move for all time, which can be used against you in any legal matter for the rest of your life. Am I exaggerating? Maybe. Do I want to take the chance? No. I tell you what - take a vote. See how many others prefer to use a VPN like me vs the option of 'why bother'.
Avoiding that endless conversation, one thing you said that caught my interest:
"Snort alerts on sensitive data". How did you identify those? How can * I * identify those?
I feel like there is some Snort rule/alert catalog that I've missed somewhere.
Thanks for the feedback.
-
Extension on my last post:
Do you feel 'Emerging Threat (ET) Rules are over-kill for non-corporate systems? Or, are you just saying I should use a 'less strict' IPS Policy rule set?
"you need to do some research on IT security and tuning an IDS/IPS." - OK, so where do you think is the best place to start? The Snort manual seems to be nothing more than a code book. Are there better documents available for people that don't want to make a profession out of interpreting these rules? (or is that the idea? ;) ;).
I simply want to be able to:
- Get an alert
- Identify what triggered it (app, protocol, port, what ever)
- Have some idea if it's a threat
- Have some idea how much of a risk it is to disable the rule
This all doesn't have to be black and white. Just something better than programming code. I get enough of that during the day.
-
@talaverde
It's true the documentation for all the Snort and Emerging Threats rules is very sparse. Rarely can you find a complete description of what a rule is looking for and how it is looking for it. If you become a rule syntax guru, you can discern generally what payload content a particular rule is looking for within a packet, but that may not be much help in determining "why" it is looking for that content. It gets even murkier with Snort's shared object rules (those rules files with "so" in the extension). Those are actually precompiled (source code written in C) binary modules that Snort loads to assist with content inspection. The vast majority of those rules are treated as proprietary info by the authors, so the content matching information is never provided. Truly good IDS/IPS admins are rare and usually well paid. It takes knowledge of the rule syntax and then of course very specific knowledge of the various threats to decipher alerts and make the call of "real alert" or "false positive". Or stated another way, you just about need to be one of those black-hat guys who actually writes the trojans to also be a good IDS/IPS admin who can parse the alerts and pick the real ones from the false positives. You need to understand how the threats work and how they spread in order to write detection rules for them. That same knowledge is needed when poring through alerts to decide if they represent a real threat or not. SIEM tools can help here by correlating lots of data and organizing it by timeline to help important things stand out. A popular big-dollar tool for this is ArcSight, but there are some open-source alternatives.I fully understand using a VPN for remote corporate work. The company I worked for before retirement used a Citrix remote desktop system via a VPN with two-factor authentication. The only thing that worked across the connection was keystrokes and the display. All clipboard activity (copy and paste) was blocked, as was any file access on my local machine. My PC became essentially a dumb TTY terminal when connected to the corporate network.
So I support VPNs for point-to-point networking transport across the Internet and for secure remote access. I'm just not a fan of using one for general browsing. One reason I say that is I see a fair number of issues folks post about here on the forums when using (or trying to use) VPNs. VPNs for secure access is one thing, but using VPNs for personal privacy is treated by many Internet content providers as quasi-nefarious. Netflix is a case in point: they ban all the VPN provider IP ranges because folks were using VPNs to circumvent Netflix's viewing restrictions. I don't mean everyone using a VPN for privacy is a potential nefarious character, but there is some amount of "guilt-by-association" in the eyes of many content providers the same as there is with TOR. VPNs also inevitably add CPU loading to the firewall and increase overall latency as your connection bounces around to get from you to the VPN host and then from the VPN host on to the actual final destination. But as I said in my post, to each his own.
With regards to the Sensitive Data Rules in Snort, you can find a little bit about them in the Snort manual at Snort.org. Basically they are just looking for patterns of numbers or letters (in the case of email addresses). So think of them as a fancy use of regular expression searches for data patterns typical of things like credit card numbers, phone numbers, Social Security numbers and email addresses. That's mostly what the rules are looking for. The main purpose of using them would be in a situation where you are concerned about the possible exfiltration of such data from your network. However, today everything pretty much goes over HTTPS/SSL in one form or the other, so these rules are becoming less effective unless you have MITM proxy. I mean only a truly dumb cyber criminal would steal your credit card info database and then send it out of your corporate network as cleartext using HTTP or FTP ... .
Finally, here is how I use the Emerging Threats rules. First off, there are two versions: the very expensive paid ET-Pro version which has quite comprehensive coverage of current threats; and the free ET-Open version which has much less coverage of current threats. I use the ET-Open rules, but only six categories total. I use emerging-botcc, emerging-malware, emerging-mobile_malware and emerging-trojan rules on my LAN along with Snort's IPS Policy-Balanced (I ran the IPS Policy-Connectivity for several years, though). On my WAN, I use two of the ET-Open "bad IP actors" rules, but only so I see regular alerts from Snort. I like to have those to assist with my package coding tasks. I don't use those WAN rules for any security reason. They are solely to generate alerts I can use when working on the PHP source code for the Snort package. Those two WAN rules are emerging-ciarmy and emerging-dshield. I get several alerts per minute from those rules, but they are alerting on stuff my firewall drops anyway. I get maybe one or two alerts per month on my LAN, and those are usually false positives. But that is expected. Just me and my wife here browsing.
-
Great stuff. All very interesting. (Though I'm surprised we haven't had any other users join the thread. This feels like we're having an email conversation ;)
I was really hoping there was some resource out there (e-book, website) that would help me (slowly) begin to interpret these alerts and decide which were false or not. My training is currently focused on T-SQL, PowerShell, C# and dotNet, I don't think I have the bandwidth to learn IP translation (in regards to Snort). Maybe next year.
It seems that I'm completely blind when it comes to dividing false alerts to true threats, and there is no easy way to change that. So, I think I might put more thought into getting Barnyard2 up and running. Maybe if I start logging enough data over an extended period of time, I could begin to analyze that data. If understanding the language isn't an option, maybe focusing on the patterns will tell me more. It may tell me nothing in respects to this conversation, but at least I'll learn a new skill (using Barnyard2).
Thanks for all the great information.
(My anticipation for Snort3 coming to pfSense is even GREATER with this new information. I can't wait !! :)
-
@talaverde
The best place to start learning how to interpret IDS alerts is to do a little studying on the art of the "darkside". By that I mean learning about how things like buffer overflows work and how they are utilized to escalate privilege on computer systems. If you have a programming background, that really helps in understanding the underlying mechanisms of such compromise techniques.Next you want to have a decent handle on how TCP/IP networking really works down at the packet level. There are lots of tutorials about this on the web. The SANS Institute web site has a Reading Room that is an excellent source of this information as well as information on how popular exploits work.
Armed with this information, it becomes a little easier to sort through the alerts. But to really dive in and truly dissect a Snort or Suricata alert, you will need the captured raw packets and a tool to analyze them. This lets you look into the packet payloads to see if the byte patterns present match up with known exploits. But I would say 6 or 7 times out of 10 you will see just false positives. A lot of false positives come from tacky programming by web folks (not strictly following the RFCs with regards to session setup and teardown, for example). Many rules, especially the Snort HTTP_INSPECT rules, are examining web traffic against the RFC standards. Any little deviation from the standards can generate an alert, but usually the deviation is not harmful and thus the alert is considered a false positive.
The fact we have poor documentation of the rules is our own fault as computer folk. You say you program in a few languages, so you will likely identify with this rhetorical question. What's more fun: (1) documenting code you've already written; or (2) writing new code? For me, it's certainly writing new code! I think the same applies to the rule authors. They know what their rule is or was looking for when they wrote it, but they don't have time to document it for someone else to reference in the future because they are off right now to write another rule for that next big exploit... .
-
-
@talaverde @bmeeks I would be happy to join in on a bounty for accelerated package development with Snort 3. Snort has been a great tool for me, and Im all-in on pfSense, and I would be happy to support the package and the developer if I could.
-
@talaverde @bmeeks I would be happy to join in on a bounty for accelerated package development with Snort 3. Snort has been a great tool for me, and Im all-in on pfSense, and I would be happy to support the package and the developer if I could.
Sounds good. Busy week for me, but I'll try to organize something next week. If someone wants to take the lead before then, feel free. Hopefully we'll get some action on this! yay!
-
b
After some time, I have even more respect for your post.
-
With all the other coding projects I have on my plate (PowerShell, T-SQL, C#, dotNet), not to mention my corporate applications, making Snort packet interpretation a priority is WAY down the list.
-
I haven't given up on the 'Security' IPS Policy, but likely it's just a matter of time that I learn my lesson. Can you give me more detail why you went with a 'lower' level? To many false alerts? (Do the 'higher' levels cause more latency? (That's a burning question of mine)
-
For the Emerging Threat rules, I started with the four your mentioned, then tried to branch out. I've had alerts pop up, but I don't think they've had anything to do with my experimentation of ET rules. NEVERTHELESS. Any chance you want to expand on why you picked these four rules? If you were to add some more ET rules, what would you start with? (I realize that's asking a lot - very open ended question)
In short, I'm not as worried about false alarms. I can sort those out... I think (says the green little piglet). What has been MOST concerning to me recently is the added latency to my system as I've been adding more and more security measures. pfBlockerNG, Snort, FireFox and all it's fixins.. and so on. All these added up, I'm finding it taking longer and longer to connect to things like GoToMeeting. My RDP connections are dropping more frequently. I can't help but thing it has to do with all these filters, checks and balances.
Ah, balance. The key. I think that's what you might have been trying to get to. I may be willing to push that more than you, but I'd live to get more insight to your decisions.
So, again, why did you pick those ET rules? What made you drop your IPS Policy? To much noise? Latency?
BTW, you mentioned Selective Data Rules. At risk of admitting my ignorance, I don't know which ones those are? I ask because I've been getting credit card alerts. I'm almost positive they are false, but still makes me nervous. Why did you bring them up again?
I think I've hit my limit on questions. Hopefully you find yourself in a talkative mood in the near future.
Thanks again for your previous feedback.
-
-
Several long-winded answers follow:
@talaverde said in Snort 3:
I haven't given up on the 'Security' IPS Policy, but likely it's just a matter of time that I learn my lesson. Can you give me more detail why you went with a 'lower' level? To many false alerts?
The Snort team will tell you that their IPS Policies work like this:
-
IPS-Connectivity = good protection with minimal to zero false positives in most networks.
-
IPS-Balanced = better protection, but a higher risk of some false positives in most networks due to several more "hair-trigger" rules being enabled as compared to the IPS-Connectivity policy.
-
IPS-Security = best protection, but a very high risk (actually almost a certainty) of false positives in most networks due to the inclusion of many more rules that trigger on anomalies. (that "almost a certainty" quote is mine and not from the Snort team).
I downgraded from IPS-Security to IPS-Balanced because of false positives in my network. And I still have a few of the rules within that policy suppressed if I recall correctly. Again, that would be due to false positives in my environment.
@talaverde said in Snort 3:
Do the 'higher' levels cause more latency? (That's a burning question of mine). What has been MOST concerning to me recently is the added latency to my system as I've been adding more and more security measures. pfBlockerNG, Snort, FireFox and all it's fixins.. and so on. All these added up, I'm finding it taking longer and longer to connect to things like GoToMeeting. My RDP connections are dropping more frequently. I can't help but thing it has to do with all these filters, checks and balances.
The latency issue is a function of the number of rules being processed. The more rules a packet has to traverse, the longer it will take. So it's the number of enabled rules that has the biggest impact. And by the way, on pfSense, the latency you may notice with Snort would be due to CPU load and not because the packets have to get through Snort before going any place. Snort on pfSense works in a PCAP mode where Snort gets copies of every packet traversing the physical interface to analyze. The original packet went straight to the
pf
firewall and then on to the host stack. Snort analyzes its copy of the packet (or copies in the case of multiple packets in a stream) and then decides whether to block or not. It informs thepf
firewall when it wants to block the remaining packets from a stream. So if the CPU is busy doing Snort rule comparisons, it takes more time to answer interrupts from the NIC card to grab packets and send them on to the host stack. That will increase latency a little bit. Same with encrypting and decrypting every single packet across an interface for VPNs. That will increase latency.Layer in a lot of this CPU busy-work and you can definitely bog your firewall down. With Snort enabled with thousands of rules, and then pfBlocker-NG enabled with thousands of IP-address rules for the CPU to check every single packet against, it is self-evident something has to give somewhere. That's what makes IDS/IPS a balancing act. And getting it working well requires knowing your network hosts' exposure and the threats they are most vulnerable to, and mitigating just those. No sense having your CPU sort through a ton of DNS and mail server rules if you are not running exposed DNS or mail servers on your network. Those are just two examples, but there are many more.
@talaverde said in Snort 3:
- For the Emerging Threat rules, I started with the four your mentioned, then tried to branch out. I've had alerts pop up, but I don't think they've had anything to do with my experimentation of ET rules. NEVERTHELESS. Any chance you want to expand on why you picked these four rules? If you were to add some more ET rules, what would you start with? (I realize that's asking a lot - very open ended question)
Rules are designed for specific network environments. Maybe "targeted" is a better word than designed. Let's take an example - the ET-DNS rules. Those rules are designed primarily to detect attacks against a DNS server. They are not really focused on protecting DNS clients. So if I don't have DNS servers in my network, especially none exposed to the Internet, then those ET-DNS rules are useless in my setup. So why waste CPU resources examining traffic against those rules? Several other rule categories I eliminate using the same logic. I don't have any exposed attack surfaces that need those rules looking for threats. I chose the malware and bot-cc rules because that is probably the biggest risk area for any network, home or corporate. I use those ET-Open rule categories as an "extra set of eyes" looking at my network along with the Snort Subscriber Rules. There are Snort Subscriber Rules covering malware and bot-cc IP combos, but since I have the ET-Open rules and have the CPU resources to spare, I run those along with the Snort Subscriber Rules just in case one catches something the other misses. But I only chose to do that in what I think of as the higher risk categories for my network. Just a personal opinion and decision, though. There are no hard and fast rules for choosing rules and policies in the IDS/IPS world.
As for which other ET-Open categories I might choose, I would base the decision on the exposures in my network. If I ran a mail server, I would choose the emerging-smtp, emerging-pop3 and emerging-imap rules. If I had teenagers in the house or even pre-adolescents, I might enable the emerging-inappropriate rules to start catching that kind of stuff. It's simply a matter of trying to balance CPU and RAM usage against the threats your network is vulnerable to (or stuff you want to limit access to from within your network). And, in the end, it's simply a judgement call. I may judge right, or something may "get me" and I find out I judged wrong. If that happens, I learn from the mistake, take corrective action, and move on. The Equifax IT security guys recently graduated from that same school of "bad experience" ... .
@talaverde said in Snort 3:
What made you drop your IPS Policy? To much noise? Latency?
Mostly too many false positive alerts. For example, I would get alerts on "EXE download attempt" when Windows was simply trying to perform some security updates. Then there were always JavaScript alerts from the browser rules, but 99% of those are false positives triggering on obfuscated JavaScript. Once, long ago, that was "the thing" for hiding malicious code, but these days it is more likely to simply be the advertising networks trying to thwart ad blockers. Granted there still is malicious stuff out there, but every single piece of obfuscated JavaScript is not a malware payload; and you grow weary of getting blocks on half of the web sites you visit.
So I dialed down the Snort IPS Policy a bit and just make sure my machines stay updated with the latest security patches. I have all my computers on a UPS and they stay powered on and connected 24x7. Faithfully installing automatic updates from Microsoft and Linux land is the best security practice there is. That's better than 50,000 Snort rules ... . Even with all that, a zero-day can still get you. So backups are critical!
@talaverde said in Snort 3:
BTW, you mentioned Selective Data Rules. At risk of admitting my ignorance, I don't know which ones those are? I ask because I've been getting credit card alerts. I'm almost positive they are false, but still makes me nervous. Why did you bring them up again?
It's "Sensitive Data" instead of "Selective Data". They are part of the three built-in Snort rules bundles: (1) decoder rules, (2) preprocessor rules and (3) sensitive data rules. The Sensitive Data rules are not enabled by default. You have to turn on the Sensitive Data preprocessor on the PREPROCESSORS tab to enable these rules.
The Sensitive Data rules were designed to detect, through regular expression pattern matching, attempts to send credit card numbers, phone numbers, email addresses and social security numbers over a network connection. They simply look for patterns. For instance, a pattern of 3 digits, a dash, 3 more digits, another dash, and then 4 digits in a packet's payload is how phone numbers generally are written (area code-local exchange prefix-individual phone number). Way back when web traffic was plaintext HTTP, these rules could help a corporate network admin detect the outbound transmission of this potentially sensitive data. Now days, with web traffic being majority HTTPS with SSL encryption, the rules don't help much because they can no longer see the plaintext payload. Plus these rules can "false positive" easily because they are simply looking for ASCII code patterns. So if some random piece of encrypted payload happened to decode like this as ASCII data: "123-456-7890", the Sensitive Date rule for phone numbers might trigger. But this string sequence might really just be the random result of encrypting some entirely different set of data bytes. I don't think very many folks enable the Sensitive Data rules these days.
-
-
Since I maintain the Snort 2.9.x package on pfSense, I guess it's logical that I will be the one to create the Snort3 package...
What are we likely to get as a replacement for barnyard2?
I manage a growing number of various models of Netgate devices, all with Snort installed. To attempt to monitor them all from one screen i'm trying to use an external SIEM setup. This has proved problematical to say the least.
-
What are we likely to get as a replacement for barnyard2?
I manage a growing number of various models of Netgate devices, all with Snort installed. To attempt to monitor them all from one screen i'm trying to use an external SIEM setup. This has proved problematical to say the least.
Probably something like this blog post talks about. Except instead of Logstash something like Filebeats might work better on a FreeBSD platform like pfSense. I have not investigated any of that in much detail, though. Here is a tutorial on converting from Logstash Forwarder to Filebeats (or Beats in FreeBSD): https://www.elastic.co/guide/en/beats/filebeat/current/migrating-from-logstash-forwarder.html.
-
Do you think it will be posible to create IDS policies and apply them to firewall rules like in the commercial firewalls?
Basically you can create a policy with a personalized configuration and rules and apply this policy to a fw rule, so the traffic of that firewall rule is the only affected by that IDS policy.
THis can be to a firewall rule or to a port, or host. -
@l0rdraiden said in Snort 3:
Do you think it will be posible to create IDS policies and apply them to firewall rules like in the commercial firewalls?
Basically you can create a policy with a personalized configuration and rules and apply this policy to a fw rule, so the traffic of that firewall rule is the only affected by that IDS policy.
THis can be to a firewall rule or to a port, or host.No, that is not something that I predict is on the horizon. The packet filter firewall used by pfSense is totally unaware of the presence of any installed IDS/IPS package and any policies defined in the IDS/IPS. Today the IDS/IPS component sits completely outside of the firewall. Changing that would require substantially reworking the internal network plumbing of the FreeBSD kernel used beneath pfSense.
-
I was just curious if there was any update to this. I am very interested in using Snort 3 with Pfsense. Thanks!
-
I was just curious if there was any update to this. I am very interested in using Snort 3 with Pfsense. Thanks!
Snort3 will likely be a long time in coming -- if ever. I started working on a package for it, but the effort got to be very frustrating because so much is different from Snort 2.9.x. Migrating an existing pfSense Snort 2.9.x configuration over to Snort3 proved to be a tough challenge. That's one of the reasons I put the package development back into mothballs. I never did get a working system going with Snort3 on pfSense. The binary part is not really the issue. The difficulties are in the PHP GUI code and all the gymnastics required to create the LUA configuration file for the binary to use.
Anybody is free to take up the challenge and work on a Snort3 package if they desire, but my enthusiasm for it has evaporated for now.
-
@bmeeks Thank you for the update.
-
@Paych3ck I am not a developer nor have any vested interest in snort. But like you was curious and I came across this thread. Kinda bummed out that at this time no further development was going to be done and to be fair it is a large task at hand. But I wanted to offer others some context who are like us curious as about snort 3.
Checking the official snort blog:
https://blog.snort.org/
-https://blog.snort.org/2018/08/snort-3-beta-available-now.html -8/2018 beta releasedOther points from the snort download page:
-Up to now its been receiving updates (still beta stage)
-2.9.16 is still listed as stable but not 3.0So I dunno maybe another reason is that the dust hasn't settled.
-
According to twitter snort3 is on it's final beta with release later this year.
-
As the OP of this thread, I sorta felt bad because I lost interest. This is because I ended up installing Suricata, even if only just to try it out. Surprisingly, I was able to significantly drop the RAM used by my pfSense (VMs) and even noticed a slight improvement in speeds. I may have just had things mis-configured with Snort, but I'm happy at the moment. While I'll almost definitely try out Snort 3 when it's available, I'm not anxiously waiting, like I was before.
I have noticed many more alerts with Suricata, than with Snort. I don't know that that means more protection or more false alarms. It may be a little of both.
-
any updates on snort 3.0? Single Threading is killing my use of it but their rule sets are far and away cheaper than suricata. Single threading kill throughput to the point it's pointless to even use the package on higher end network speeds.
-
@beachbum2021 said in Snort 3:
any updates on snort 3.0? Single Threading is killing my use of it but their rule sets are far and away cheaper than suricata. Single threading kill throughput to the point it's pointless to even use the package on higher end network speeds.
No more progress, and I have no plans at present to resume work on a Snort3 package. If someone else wishes to tackle that project, they are welcome to do so.
-
@bmeeks thanks for the update