IDS/IPS and antivirus on 2.1 release
-
However, checked seclists and sourcefire claimed that it shall remain open sourced
Look at what happened to MySQL.
Since snort and clamav are GPLed, I guess they Cisco is required to release further changes, don't they?
Nothing in GPL requires you to make new versions publicly available.
-
@amirkabir:
From Pfsense site:
"This project started in 2004 as a fork of the m0n0wall project, but focused towards full PC installations rather than the embedded hardware focus of m0n0wall.I do NOT want IDS with zillions of false positives and daily hours of babysitting, nor do I want useless antivirus such as ClamAV - embedded or not. Kindly install those yourself if you find a need for them.
Just because some functionality is well integrated, tested and part of the distribution doesn't mean you have to use it and enable it.
There's plenty of functionality in pfSense that I don't use, and a bunch of packages that sort of bite each other in unpredictable way, because there are no "standard" packages and the various interactions are not part of the testing.There may be other valid reasons not to include package <x>as part of the distribution, but the fact that n% of users won't use it or don't need it, isn't one of them.</x>
-
The quality of a function is not dependent on whether it's a package or in the base system.
Quite true, but the integration needs to be better. e.g. naming conventions: the base system names the menus usually BY FUNCTION, while packages tend to name the menues BY PROJECT NAME, which frankly is annoying.
Similarly, the packages as we have them now, do not signal in any way what is mutually exclusive, e.g. Dansguardian does not exclude HAVP and/or SquidGuard from being installed, which is something that could be solved with meta packages that are sorted by function, such that one can install only one IDS/IPS system, only one content filter, etc.
It's probably the majority of users that have one or more packages installed, but unfortunately, due to a lack of conventions and/or a system of enforcing these conventions, the fit and finish of the entire installation quickly takes a nose dive once a few packages are installed, which makes it feel inferior to other products, just because things get inconsistent, not because things really are inferior in substance.
-
Plus Cisco just bought Sourcefire. Which means snort and clamav are now both Cisco products. Looks like we need to find replacements ASAP. :-)
Suricata would seem to be a good choice, seems to also scale better and have other advantages, while still being able to work with Snort rules, if required.
ClamAV can be forked, if need be, but of course updating the rules may become an issue after a while. -
Would you trust anything coming out of Navy/HLS?? It means a lot of backdoors for the US gov to use.
Could we fork of SNORT instead and establish a community ourselves?
-
Would you trust anything coming out of Navy/HLS?? It means a lot of backdoors for the US gov to use.
Could we fork of SNORT instead and establish a community ourselves?
If it's open source, yes. Otherwise one couldn't use Tor, or for that matter the entire internet, which is just a renamed milnet/[d]arpanet.
If it were some close-sourced thing, that would be a different issue.
-
I think the Snort code is open-source but updates is something else.
But if they implemented SNort community rules anyway, then a merger would be good and the updates along the way.
-
Just because some functionality is well integrated, tested and part of the distribution doesn't mean you have to use it and enable it.
There's plenty of functionality in pfSense that I don't use, and a bunch of packages that sort of bite each other in unpredictable way, because there are no "standard" packages and the various interactions are not part of the testing.There may be other valid reasons not to include package <x>as part of the distribution, but the fact that n% of users won't use it or don't need it, isn't one of them.</x>
Actually, it is a perfectly valid reason. We recently removed OLSRD and RIP (routed) from base for that very reason. They were better as packages, they were no longer used by enough people to justify keeping them in the base system.
Quite true, but the integration needs to be better. e.g. naming conventions: the base system names the menus usually BY FUNCTION, while packages tend to name the menues BY PROJECT NAME, which frankly is annoying.
That's no reason to put them in base. Just fix/change the menu names. But that's a Bikeshed discussion for another thread. Naming by function breaks the moment you have two packages that have the same function but different actual names, and you can't tell them apart.
And again, if a package doesn't work like you want, submit patches or convince the maintainer to fix/make changes. That does not have any relationship to it being in base.
Similarly, the packages as we have them now, do not signal in any way what is mutually exclusive, e.g. Dansguardian does not exclude HAVP and/or SquidGuard from being installed, which is something that could be solved with meta packages that are sorted by function, such that one can install only one IDS/IPS system, only one content filter, etc.
It could use some "conflicts" type checking but the examples you gave aren't always really conflicts. I'm sure someone has a valid reason for combining those in some weird way, and if we stopped them from being installed, something else would break.
It's probably the majority of users that have one or more packages installed, but unfortunately, due to a lack of conventions and/or a system of enforcing these conventions, the fit and finish of the entire installation quickly takes a nose dive once a few packages are installed, which makes it feel inferior to other products, just because things get inconsistent, not because things really are inferior in substance.
If you have some examples of this with non-beta/non-development packages, feel free to cite actual examples and not generalities that aren't really true. Lots of us run many packages with no problems. Certain specific packages may have issues, but again, that's a problem with the individual package's code that isn't going to magically get fixed by bringing it into the base system.
If you want better packages, then help the maintainers make them better, submit patches to make them better, or fund the maintainers to help them get better.
-
Actually, it is a perfectly valid reason. We recently removed OLSRD and RIP (routed) from base for that very reason. They were better as packages, they were no longer used by enough people to justify keeping them in the base system.
The biggest downside to me using a package as opposed to something built in by default is what happens when you upgrade a system. There is extra downtime while the package gets re-installed. With CARP enabled for HA and you upgrade your primary server it will take over after it reboots but before packages are done upgrading/reinstalled. This is one of the reasons I prefer not to use haproxy. Snort, openvpn client export, etc and non-critical packages like that are acceptable to come later but important routing and loadbalancing packages are critical functions.
A way to help with that scenario though would be if there was an easy way to manually disable CARP on the primary that would survive reboots. Better yet would be for the firewall to know when it is done with upgrades and just delay CARP failover back to primary automatically. I would actually prefer the manual method though to make sure that all the packages get upgraded properly before putting load on it. The automatic method though would just be a nice thing for people who don't want to worry about it but where hosted webserver,etc uptime is still very critical to them.
-
Actually, it is a perfectly valid reason. We recently removed OLSRD and RIP (routed) from base for that very reason. They were better as packages, they were no longer used by enough people to justify keeping them in the base system.
The biggest downside to me using a package as opposed to something built in by default is what happens when you upgrade a system. There is extra downtime while the package gets re-installed. With CARP enabled for HA and you upgrade your primary server it will take over after it reboots but before packages are done upgrading/reinstalled. This is one of the reasons I prefer not to use haproxy. Snort, openvpn client export, etc and non-critical packages like that are acceptable to come later but important routing and loadbalancing packages are critical functions.
A way to help with that scenario though would be if there was an easy way to manually disable CARP on the primary that would survive reboots. Better yet would be for the firewall to know when it is done with upgrades and just delay CARP failover back to primary automatically. I would actually prefer the manual method though to make sure that all the packages get upgraded properly before putting load on it. The automatic method though would just be a nice thing for people who don't want to worry about it but where hosted webserver,etc uptime is still very critical to them.
Which is still, technically, not a package system issue, but a general CARP issue.
Firmware upgrades, for most users, happen once or twice a year, or less often. A few minutes of downtime during a scheduled maintenance window for a firmware upgrade is not a huge issue for most people.
Yes, if you're tracking snapshots it's more often and more disruptive, but the fact of the matter is that is not how most people work.
Still, that's an issue to take up with the CARP procedures, it's not specific to things being a package really.
-
I agree with your reasoning. Upgrades don't happen often on production sites. When they happen for me I fail over to a backup site so it isn't such a big deal. I was somehow overlooking that seemingly obvious point.
I also failed to mention that main reason I prefer the built-in load balancer is because it doesn't drop connections during firewall fail-over where haproxy does. That again shouldn't happen often.
-
I also failed to mention that main reason I prefer the built-in load balancer is because it doesn't drop connections during firewall fail-over where haproxy does. That again shouldn't happen often.
The built-in balancer works because relayd works like NAT rules, so it fails over with the normal state table entries.
Until HAproxy grows a way to sync its internal connection states in a cluster-like fashion, there isn't anything we can do for that. There's a separate thread for that somewhere, I seem to recall posting about it in the last couple weeks. Usually web sessions are fleeting enough that it doesn't matter a ton that the states are gone.
-
Well, there's still the killer argument: unneccessary complexity.
More complex systems are more likely to break. I guess we don't need to discuss this.
Is IDS neccessary? For most of us, no.
Of course, everyone has his favouriet package "which should be in the base system". Yup, if it's in the base system it gives the user the illusion of real solid support. However, when thinking again about illusions and additonal complexity, I'd rather prefer the modular approach, where I can simply remove addons which begin to act up.
-
"Well, there's still the killer argument: unneccessary complexity." ;D
-
How about a system like the WP Plugin Repository?
Bring in some sort of order to the chaos of broken, semi-functional packages that gives Pfsense a bad name?
-
Well, there's still the killer argument: unneccessary complexity.
More complex systems are more likely to break. I guess we don't need to discuss this.
Is IDS neccessary? For most of us, no.
It all depends how you use pfSense. e.g. if I tried to program the firewall, it would be a fairly insane effort for a little home network, because there's no starting point, and the number and kinds of software people run these days isn't anything like the old days where you'd open up the mail, DNS and SSH ports, and shut everything else.
We have things like dropbox, drobo, back-to-my-mac, iCloud, email, DNS, VoIP, Skype, P2P, Tor, RDP, VNC, Apple Remote Desktop, remote server admin tools, push services, photo streams, proprietary protocols of ever more license demons, web apps, etc.
I'd end up with a mess of rules, and probably still a bunch of things that don't work quite well.Or I can use SNORT's blocking features, which simply selectively block hosts for a few hours if they do things that look suspicious, and features like the ssh lock-out feature that block a host after a certain number of failed login attempts, and pfBlocker with some list subscription of known bad hosts in combination with something like Apple's application firewall, which isn't ports/protocol based, but decides which apps can accept incoming network traffic.
As things get more and more internet driven, old school firewall programming gets more and more difficult, and smart, behavior driven thread detection/blocking more important.
As such I think IDS/IPS along with crowd-sourced blocking lists are the future of firewalls, and while rule based firewalls work well for systems that are essentially shut out of the internet except for a very few carefully poked holes, that model doesn't work too well for the way most people use the net these days and in the future.
-
Fine, if you want SNORT, install it as a package.
As such I think IDS/IPS along with crowd-sourced blocking lists are the future of firewalls
Nope, definitely not.
In case you haven't noticed yet: blocking lists are regularily manipulated to put companies out of business. You are probably not a prime target, but others definitely are.
However, if you're not a target, who bother? And if you are, why open up a new attack vector to get DoSed?
-
I think having pfsense start up with nothing, exactly as it does now, is the best way to go. It ensures maximum compatibility across a broad range of platforms. I think if you want a package, add a package is the correct mentality. SNORT especially, is something that some just MUST have and others should avoid like the plague.
-
Let's take an objective look at this:
Pros to having things as a package:
- Able to update more frequently independent of a firmware release
- Allows the package development to proceed with more freedom
- Faster response to upstream changes for things like snort versions, rule formats, AV databases, CVEs, etc.
- Keeps the main system simpler, smaller, etc. (less complexity)
- Less bloat in the main system for things many users do not need (snort users are in the minority)
- Most packages works fine as-is for most users
- Smaller download sizes, less bandwidth used for hosting pfSense downloads and mirrors
- Lots of other good things discussed here and elsewhere about the package system.
Cons for having things as a package
- Making those 4 clicks to install the package can be tiresome and tough work the one time you have to do it. I'm surprised people haven't broken their fingers.
- Some people want it pre-installed
- Somehow magically better code and support because it's in the base system.
Things that have little or no bearing on whether or not it's a package:
- The menu name (snort vs IDS/IPS vs some other confusing name)
- How the package behaves during a firmware upgrade that happens for most people 1-2 times per year, if that.
- How a package interacts with other features such as CARP
So, in other words:
- Some valid complaints are not about the package itself, but about how packages in general are handled. This is a base system issue, and you can't "fix" that by moving a package into the base system. You're treating the symptom not curing the disease.
- Some valid complaints are about the code of the packages themselves. This will not be solved by moving it to the base system. Someone still has to fix it. Bringing it into base would only happen if a package was already in good shape, or if someone funding getting it into good shape and other factors deemed it worthy. The same fixes would happen if it were kept a package and someone fixed the package. Hence, broken code is broken no matter where it is, and good code is good no matter where it is.
In conclusion:
- Help the package maintainers fix packages by contributing code, funds, etc.
- Help identify and fix issues with package handling in the base system (patches are always welcome, doubly so if they actually fix things in an elegant and useful way)
- Keep using packages and make them even more awesome than they already are.
If anything arguments could be made for taking even more things out of the base system, though what we have now strikes a good default balance.
If you want a real debate about whether a package should be a package, or in the base system, take a closer look at a small, simple package such as sudo or blinkled. It's a much better debate than this one for large/complex add-ons. ;D
-
I think modularizing things into packages is per se a good thing, but having started with pfSense as a total noob not all that long time ago, I remember the confusions the packages create.
And that boils largely down to competing/conflicting packages that are not recognizable as such, and the vastly differing quality and level of support for various packages.
If there were some sort of curation process, e.g. official packages (things that are maintained by the pfSense team, and are just taken out of the base system for a more modular setup), qualified packages (very commonly used things that for license or other reasons are not part of the base system or official packages, but are tested and have a seal of approval that they work and are consistent with the UI and naming conventions), and user contributed packages (for which no assertion is made in regards to anything), that would be a step forward, particularly if one could restrict the list of packages shown in the "available packages" section to a certain quality level, e.g. only official packages, or qualified and official packages, or everything.
As long as there's a better way of handling package conflicts, and as long as package isn't synonym with less support, less quality, less consistency, etc. I'm all for modularizing things. However right now, I consistently get the feel that anything that's in a package is "second class citizen", and that's pretty much the main reason that I'd love to see everything that I consider important part of the base system.
