How secure are the packages

michmoor

@bmeeks
Lots of things to digest here. Dont want to get into the weeds of where an OSS product would be used or could be used.
I do agree that when dealing with an org with a budget the reality is that pfsense is not sitting in-line anywhere. Exceptions to the rule and there is probably a case to be made where it makes sense to migrate from a Fortigate or Checkpoint to a pfsense. Networks are like snowflakes. Each is different with different motives.

To digress for a moment, I'm working in a business where they want to virtualize all the things..NFV for anything that passes a packet. Lots of technical debt for a bunch of engineers who don't have experience in operating and securing a virtualized world yet here I am deploying vmware and vsphere and virtualizing a firewall. Throwing my lot in with big established vendors doesn't always guarantee a successful outcome. Just because something is done doesn't mean it always makes sense to do so. PFsense has its place at certain businesses but sometimes money dictates design.

Overall, to bring it back to the packages it sounds like they are as secure as they can be, and depending on the amount of technical debt I want to ingest would determine if I want to split up all the functions that pfsense provides out of the box?

edit: I worked at a fintech company that for the most part was pretty enterprise-y. Ran the Ciscos and Palos. But when it came to the OS running their business applications it was Centos all day. Apparently, RedHat was deemed to expensive and there was a considerable amount of staff that were experienced enough in Linux.

Side note. The support of community packages has always worried me with pfsense. I generally don't install a package unless I'm willing to maintain it long-term and I'm not a coder....haha..

bmeeks

@michmoor said in How secure are the packages:

Side note. The support of community packages has always worried me with pfsense.

This is the most important potential issue that should be thoroughly analyzed before committing to use packages as part of a business installation with pfSense. The packages are maintained by volunteers who are not affiliated with Netgate. It's pretty much all free effort and labor from these maintainers. And a maintainer can disappear at any instant. That leaves packages to wither and die. The Squid package family is the most recent example of such an event. There have been a few others in the past as well. For example, if I left the ecosystem tonight, then both Snort and Suricata would likely wither and die quite quickly. There is nobody waiting in the wings to pick up support and maintenance of either of these packages. Ditto for a package such as pfBlockerNG.

michmoor

@bmeeks
Im glad we are on the same page there.
I always thought that popular packages such as pfBlcoker or Suricata are operated under a consultant type of relationship due to the inherent value-add and popularity these packages offer. So you have billable hours towards the package and get paid in return. In my mind that makes sense and ensures some level of support. Im not a businessman so perhaps it cant work like this.
That said, pfsense should probably operate under default conditions then to ensure a base level of security without technical debt. In other words, don't install packages. Bringing it back to my initial post, it probably makes sense to run the reverse proxy on a separate system then.

Bill, you should know we appreciate you and we know you do this for the love as you stated numerous times. So thank you for always taking the time out to answer questions and provide insight.

bmeeks

@michmoor said in How secure are the packages:

you have billable hours towards the package and get paid in return.

Not how it works -- at least not for any package maintainer I'm aware of. I've never been paid a dime for my work on Snort or Suricata. I did get gifted an SG-3100 once in the past to debug a problem with Snort on 32-bit ARM hardware. Later I was also given an SG-5100 as a gift. That's as close to being "compensated" as I've gotten for my work on the IDS/IPS packages.

But to be fair, I've never asked to be compensated and really do not desire it. If I were compensated, then I would feel a strong obligation to continue the work. As it is, I've been doing the work for free because I enjoy it and don't mind contributing my efforts to the greater pfSense community. But I am also free to move on to something else if I desire without having the strings of compensation and its expectations attached.

I think many users lose sight of the fact pfSense CE is totally free as are the add-on packages. Thus neither generates any revenue for Netgate. I believe that's why the move to pfSense Plus happened, and I expect that to pick up speed. It is totally unrealisitc to expect Netgate to support all of the third-party add-on packages available for pfSense for free. That would keep a small army of programmers busy full time with absolutely no incoming revenue from their efforts. Netgate is focused on the core pfSense product (and primarily on pfSense Plus). Any work they do on packages is usually limited to just the packages they use as part of the core functionality (think OpenVPN and Wireguard, for example).

michmoor

@bmeeks said in How secure are the packages:

I think many users lose sight of the fact pfSense CE is totally free as are the add-on packages. Thus neither generates any revenue for Netgate.

Couldnt agree with you anymore here. To be fair to users I think most don't understand the package support structure so they assume that if it's part of the ecosystem then its supported. To believe otherwise hasn't been made clear in any way. So to most, if there's a problem in a community package "Oh netgate will help let me submit a redmine" but that's not how it works.
I do want to say that a strong delineation of the CE and Plus versions would help. The sister project, TrueNAS does something similar (Core vs Scale) but its fair to say they make it a point to tell you whats community supported and whats meant for enterprises.

SteveITS

@michmoor lurking on by, there’s a list at https://www.netgate.com/supported-pfsense-plus-packages

michmoor

@SteveITS
Yep. +1 on this.
I try to make it a point to stick with the 'Maintained by Netgate' packages as best i can.

Patch

@michmoor said in How secure are the packages:

The support of community packages has always worried me with pfsense. I generally don't install a package unless I'm willing to maintain it long-term and I'm not a coder

@michmoor said in How secure are the packages:

it probably makes sense to run the reverse proxy on a separate system then

My interpretation is a bit different.
If pfsense with current packages is a good solution to a businesses requirement then use the current offering provided if support for a potentially non core package is lost then an alternative solution is available which maybe

get someone else to step up and maintain the package, or
add an external device or VM to provide that function.

So in summary, it there is an external solution to a pfsense package, then potential loss of future support for a package may not be a real barrier. In addition if the only sensible solution to a functional requirement is firewall functionality / or a package then support for the package becomes more critical.

JonathanLee

It’s all open source software nothing to hide here… so yeah they are secure, plus if you find a bug everyone helps fix it too. Super secure

bmeeks

@JonathanLee said in How secure are the packages:

It’s all open source software nothing to hide here… so yeah they are secure, plus if you find a bug everyone helps fix it too. Super secure

Sounds like you might need to read up on the xz-utils backdoor . That one got into some BETA/RC pipelines and was just about to get pushed out everywhere when it was discovered almost by accident.

Thinking open-source software is secure "because everyone is looking at the source code" is not a valid supposition. As I mentioned earlier, the code is now so complex, and the development environments and associated tools are so varied, that hardly anybody is actually looking at the code and really understanding what it does. Do you know what is in the Snort package code you are using? Have you analyzed it to see if there are any backdoors in the binary piece or the PHP piece? I'm guessing not. You are depending on others to do that for you. But those "others" may themselves be assuming that "others" are looking so they don't have to, and on it goes with the final result being pretty much nobody is really looking. A ton of the open source stuff is really dependent on just one or two core maintainers on GitHub.

Don't misunderstand my comments to mean I am against open source software. On the contrary, I think it is a great thing. But to think open source is inherently more "secure" than closed source is not really valid these days.

JonathanLee

@bmeeks that XZ bug was bad, glad the community caught it.

michmoor

@JonathanLee A Microsoft developer caught it

JonathanLee

@michmoor did you fix the RE update issue yet? That update required a full repartition of the RE with a format to expand it. That was a nightmare I think that a big bug was hiding in the RE (recovery partition) the one I always go on a tangent for the invasive containers, the cache stopped seeing random stuff after I wiped out the RE and rebuilt it. I wouldn’t worry about it unless you can’t install the updates. Windows 10 stuff. They wanted me to just reinstall to get rid of it I refused because I wanted it fixed permanently on all systems. Now it’s working perfect, Microsoft fixed it !!!

michmoor

@JonathanLee
Huh?

dennypage

@michmoor said in How secure are the packages:

I try to make it a point to stick with the 'Maintained by Netgate' packages as best i can.

I think 'Maintained by Netgate' is a bit of a misnomer.

Most of the packages have two categories of components. The first category is the pfSense package itself, named pfSense-pkg-XXXXX. This package generally contains php, xml, and html code. There are a few of these that Netgate directly maintains, but most are maintained by volunteers.

The second category is usually the underlying XXXXX package and its dependencies. These are generally external projects, and not maintained by Netgate. Netgate may make contributions to these projects, but they do not own them. They have their own maintainers.

The point is that all the packages, even the simple ones, have substantial supply chains behind them that is outside of Netgate's control. That Netgate maintains the php, xml, and html code really doesn't make them any safer.

michmoor

@dennypage
Thanks for the background Denny. Pfsense package has always remained an interesting area for me as it pertains to the overall platform security.
You and Bill gave really good info on this.

So where does that leave us overall at least to my original question? Do we continue to use packages on the firewall in my case HA Proxy and just be mindful that it’s still an external thing with its own set of challenges or are there at least ways that pfsense can silo these packages so if there are issues the blast radius isn’t the entire system? That’s the part I’m not clear on.

Dobby_

@michmoor said in How secure are the packages:

So where does that leave us overall at least to my original question?

If you install the pure pfSense either 23/24.xx or 2.7.xx version and you
may not install any package, you perhaps tend to setup squid and
other software external and if this is also OpenSource software you
may standing then also again in front of the same "door" trust or not.

So it is more or less the question to trust opensource software or not and
when you do, wich software exactly.

Do we continue to use packages on the firewall in my case HA Proxy
and just be mindful that it’s still an external thing with its own set of
challenges

Trusting Squid & SquidGuard & ClamAV internal or external where should
be the difference, I mean if you trust it, you can install it on the pfSense
or external like you want, must or need it, but the "thing" with the trust
you must answer even internal or external used.

or are there at least ways that pfsense can silo these packages so if there are issues the blast radius isn’t the entire system? That’s the part I’m not clear on.

The given ability to set up pfSense as a pure firewall or "pimp it up" to
a fully UTM device make it in my eyes in real interesting! And it is only
based on the taken hardware that you will be able to serve for home
only till bigger companies.

JonathanLee

I am gonna say they are secure, I like that it is open source so nothing is hidden, you can look at anything.

bmeeks

@Dobby_ makes an excellent point. You either trust the open-source world and the packages on pfSense -- or you don't. Whether it is installed on the firewall or run on separate hardware is not hugely significant in the grand scheme of things if the flexibility of UTM is what you seek.

True that anything "extra" installed on the firewall constitutes an additional attack surface, and a weakness exploited from that could lead to a bad actor having some level of control over the firewall itself. If the "extra" software is installed on a separate machine, then only that host machine is likely to be compromised by an exploit.

So, there will always be trade offs. Is flexibility and maximizing the use of a single hardware platform more important to you, or is absolute security what you strive for? The answer to those questions determines your course.

In my mind, the biggest consideration when using a third-party package on pfSense is how severe is the impact to your security posture if the chosen package were to suddenly disappear (or otherwise become unsupported). @dennypage did a good job explaining that most packages on pfSense actually consist of two pieces: a GUI part written in PHP and sometimes with XML and JavaScript woven in, and then a binary portion that comes from a totally separate source. I'll use Suricata as a example of how this shakes out.

When you install the Suricata package on pfSense, what you actually specify is the package pfSense-pkg-suricata-x.xx_x. But this is only the GUI component that you interact with in the pfSense GUI. It is written entirely in PHP with a little sprinking in of JavaScript for client-side functionality. It has absolutely nothing to do with analyzing traffic or creating the alert logs. That part is handled by the suricata binary that comes from the FreeBSD ports tree. The pkg utility reads the manifest associated with the GUI portion of the package and determines what additional packages it needs to download and install to make the chosen package functional. One of those is the suricata binary package.

The FreeBSD ports tree has its own maintainer for the Suricata binary port (actually it happens to be the guy who programs for OPNsense). He imports and then modifies as necessary the suricata binary package from OISF (the folks who created and maintain the Suricata binary). Then there is another wrinkle in pfSense. Because I implemented a custom blocking mode for the binary (Legacy Mode Blocking), I wrote my own output plugin that is compiled into the Suricata binary only for use on pfSense. There is a custom patch that is applied to only the pfSense version of Suricata that tacks on the custom blocking plugin for Legacy Mode blocking. See how complicated this has gotten? Lots of folks involved to make this all work. And Snort works pretty much the same way just with a different upstream FreeBSD ports maintainer.

This complexity means any one of three things could result in Suricata disappearing. First, OISF could drop the product. That would mean no binary and thus no functionality. Second, the FreeBSD port maintainer could disappear and that could result in Suricata becoming orphaned in the ports tree with nobody maintaining it. That's more or less what happened to Squid (and also to the Barnyard port formerly used in Snort). Because pfSense pulls its package binaries from the FreeBSD ports tree, if a port becomes orphaned there the impact is immediately felt in pfSense. And finally, on the pfSense side, if I were to disappear then Suricata would also likely slowly die as it would cease to be updated. Nearly all the pfSense packages operate in the manner thus described (although probably very few have custom patches like the IDS/IPS packages).

Later Edit:
To finish and wrap this up so it meshes with the overall discussion of package security, consider that with a package like Suricata there are at least three distinct "supply chain" exploit insert points.

First, some malware could get slipped into the OISF source tree. As this is the root upstream source for FreeBSD ports, that would mean anything using the code from FreeBSD ports would be compromised as soon as the tainted Suricata update was merged into FreeBSD ports.
Second, the FreeBSD ports maintainer could slip in a piece of malware. This would mean users of Suricata on FreeBSD or its derivatives (think pfSense) would potentially be compromised.
And third, I as the custom plugin maintainer for Suricata on pfSense could slip in some piece of malware and thus compromise any pfSense box using the package.

So, three possible supply chain exploit insertion points for just a single package. Multiply this by the number of dependent libraries most packages use and you can begin to see the enormity of the potential issue. The Suricata binary by itself has maybe a dozen or so shared library dependencies it incorporates into an installed instance.

bmeeks

@JonathanLee said in How secure are the packages:

you can look at anything.

But my point here was that if you don't know what you are looking for, does it really matter that you can look?

In the xz-utils case, the actual malicious code got inserted during the binary build process by the M4 macro used by the development environment tools. It never did actually exist as cleartext in the C source code. You would need to be quite conversant and experienced in the M4 and autotools compilation and linker process to have discovered this. And the Microsoft guy who discovered it did not find it by just chance looking at the M4 macro. Instead, he noticed some peculiarity in the behavior of SSHD on an already infected system and went looking for what changed.