Feedback on challenges with the pfSense package development process

[michmoor]

@bmeeks I would just say that the package support list aids in the confusion. It should only list the ones maintained by Netgate. Including others is , in my opinion, bad marketing as they are not supported. I’m not directing this to you Bill, just “talking out loud”. That page needs clarification/ cleanup more so because it’s mentioned byTAC within Reddit forums when they communicate supported packages which just isn’t true for most of them.

[andrew_cb]

I don't recall seeing https://www.netgate.com/supported-pfsense-plus-packages. Is this the only place with the package support status? I don't see it linked anywhere on the website. Regardless, there are numerous references to non-Netgate maintained packages throughout the website, which give the appearance of at least being endorsed by Netgate, and no mention of their "risky", unofficial status.

Here's a few just from browsing around:

https://docs.netgate.com/pfsense/en/latest/packages/list.html < The first two paragraphs at the top of the page are sort of a disclaimer, but still doesn't make it clear about the support status of packages. There's no link to the official page.
"Zabbix-proxy
Zabbix Agent proxy. Collects performance and availability data on behalf of the Zabbix server, lowering the burden on the server. Available in multiple versions."

https://www.netgate.com/pfsense-plus-applications/firewall
"Observe key operating metrics like network utilization, CPU load and disk space usage with built-in Zabbix monitoring"

https://www.netgate.com/blog/alternatives-to-ntopng
"pfBlockerNG: DNS-based Blocking and Filtering
pfBlockerNG is a pfSense Plus software package that allows you to block access to malicious domains, ads, and other unwanted content at the DNS level. It uses various lists and feeds to stay up-to-date with the latest threats and can be customized to block specific categories or domains based on your organization's policies. pfBlockerNG integrates with pfSense Plus software's firewall and routing capabilities, providing an additional layer of security."

AND

"Network Monitoring Tools
When it comes to network monitoring, open-source tools like Nagios, Zabbix, and Prometheus are widely used in various organizations.
...
There is a pfSense Plus package called NRPE which allows a Nagios server to monitor pfSense Plus software, and
The node_exporter package in pfSense will export the Prometheus metrics to that external server."

https://www.netgate.com/pfsense-plus-technology
"Telegraf
An agent written in Go for collecting, processing, aggregating, and writing metrics."

"Zabbix-agent
Zabbix Monitoring agent. The agent gathers operational information locally and reports data to Zabbix server for further processing. The agent can also generate alerts in case of failures. Available in multiple versions."

The Packages menu in pfSense similarly doesn't include any disclaimers or link to the official package status page.

According to the package lists, NO monitoring packages are supported (Prometheus, Nagios, Zabbix, Telegraf). Is SNMP really the only official option?

You can see how this creates a very confusing experience for an end-user when trying to use packages.

[andrew_cb]

@bmeeks Thanks for your insight. The complexities with creating/maintaining a GUI for packages makes total sense.

Again with Zabbix, based on my use of it with other systems, no changes to the GUI configuration parameters are necessary as the basic parameters are the same. Currently, pfSense has Zabbix 5.0, 6.0, and 6.4 packages and associated configuration values/pages. If it can accommodate 3 versions, adding a 4th that requires no/minimal modifications should be relatively simple.

If there is a complex issue that is delaying the update or release of a package, then at least some communication about that would go a long way.

If Netgate is not the maintainer of a package, and building packages for pfSense requires a very particular FreeBSD build environment, then are those essentially orphaned from a pfSense perspective? What is the trigger and motivator for them to be updated?

Netgate staff have do not respond to these posts saying "Package xyz is not maintained by pfSense, please ask the maintainer to submit an updated version to be reviewed for inclusion in the package repository", rather they respond to some of the posts and set a target version in redmine. If Netgate is truly not maintaining the FreeBSD/pfSense port builds of these packages, then this behavior further adds to the confusion.

Another point I want to mention - Netgate staff caution against installing packages from the FreeBSD repository as this can break things, and instead recommend to only install packages from the pfSense repository. Again, this further reinforces the impression that Netgate is maintaining and curating the packages available in Packages.

[michmoor]

@andrew_cb
Just want to reiterate what was stated in these forums awhile ago.
Do not install any package you cannot directly support yourself. This is for Suricata/Snort,pfblocker,zabbix.
Once you make these tools part of your production system, you run the risk that at any point in time these packages stop receiving support. Then what?
As I stated above, the ONLY thing that Netgate supports is 'pf'. Nothing more nothing less.

[bmeeks]

@andrew_cb said in Feedback on challenges with the pfSense package development process:

@bmeeks Thanks for your insight. The complexities with creating/maintaining a GUI for packages makes total sense.

Again with Zabbix, based on my use of it with other systems, no changes to the GUI configuration parameters are necessary as the basic parameters are the same. Currently, pfSense has Zabbix 5.0, 6.0, and 6.4 packages and associated configuration values/pages. If it can accommodate 3 versions, adding a 4th that requires no/minimal modifications should be relatively simple.

If there is a complex issue that is delaying the update or release of a package, then at least some communication about that would go a long way.

If Netgate is not the maintainer of a package, and building packages for pfSense requires a very particular FreeBSD build environment, then are those essentially orphaned from a pfSense perspective? What is the trigger and motivator for them to be updated?

Netgate staff have do not respond to these posts saying "Package xyz is not maintained by pfSense, please ask the maintainer to submit an updated version to be reviewed for inclusion in the package repository", rather they respond to some of the posts and set a target version in redmine. If Netgate is truly not maintaining the FreeBSD/pfSense port builds of these packages, then this behavior further adds to the confusion.

Another point I want to mention - Netgate staff caution against installing packages from the FreeBSD repository as this can break things, and instead recommend to only install packages from the pfSense repository. Again, this further reinforces the impression that Netgate is maintaining and curating the packages available in Packages.

There are complexities invovled in building packages for FreeBSD (and by extension, pfSense) that are not always appreciated by those that do not have extensive experience in the area. Let me try to provide a high level overview.

The binary portions of packages must be built using the exact same FreeBSD kernel that the package will ultimately be installed on. That means, since pfSense uses a customized/modified FreeBSD kernel, any packages installed on pfSense need to be built in a builder that uses the same customized FreeBSD kernel and any shared library versions. That's where the warning about installling packages from the standard FreeBSD repo comes from. Those packages will be built for stock FreeBSD kernels and are also likely configured for and expecting to find particular versions of shared libraries. Those exact conditions may not exist on a given pfSense version. The kernel code is different from stock FreeBSD and the versions of shared libaries can well be different. When packages are installed, they automatically pull in the dependent library versions they were built with. That may overwrite something that pfSense needs and thus can break pfSense itself.

Netgate has their own set of package builder servers. Those servers build (meaning compile) the binary and PHP portions of packages using an enviroment created from a particular pfSense version (CE 2.7.2, or CE 2.8.0 DEVEL, or 24.03 Plus). That means the packages built by pfSense servers and served up in the pfSense GUI will have shared dependencies that match what is already installed by pfSense. Thus no conflict and no possibility of breaking pfSense.

When pfSense itself is updated, the packages are all rebuilt using the new kernel and shared library tree.

The binary portions of a package are tied to the underlying kernel version and thus the pfSense version. The PHP portion of a package is usually NOT tied to the kernel version althought it might have some dependency on the exact version of PHP used in the pfSense GUI.

The pfSense package repo source code is hosted on GitHub here: https://github.com/pfsense/FreeBSD-ports. Anyone is free to submit Pull Requests that contain updates for existing packages or that add entirely new packages to the pfSense list. That has happened many times over the years. Over the course of history, someone has had need of some particular function or package and created it themselves. They then may decide to "share it" with the larger community by creating a Pull Request and posting their package code to the GitHub link I provided. If the Netgate developers think the Pull Request has merit, they merge it into the tree. That's how most of the packages found their way into pfSense in the first place. Only a very few were created by Netgate themselves.

If you look at some of the recently closed Pull Requests on GitHub you will see updates I've provided for Suricata and updates that the pfBlockerNG developer has provided for that package: https://github.com/pfsense/FreeBSD-ports/pulls?q=is%3Apr+is%3Aclosed. Note that neither he nor I are Netgate employees. We are just pfSense users like you that decided to produce a package (or update an existing one) and shared our work with the larger community. For some of the really popular packages, the Netgate team may step up and fix small bugs to help out the larger community and to aid the volunteer package maintainer. That has happened occasionally for the Snort and Suricata packages I maintain. It has also happened from time to time with the pfBlockerNG package. But generally speaking Netgate will not take it upon themselves to update the binary portion of a package that they do not directly maintain. They wait on the volunteer maintainer who has more detailed knowledge of the inner workings of a package to submit any update via a Pull Request at the GitHub link posted above.

[michmoor]

@bmeeks
Curious, do you think the current system of 3rd party packages and support of a volunteer community is scalable?
If you look at other vendor firewalls, the entirety of the system is under control of the company. Unifi may run Suricata in the background but if something breaks on their system they take ownership of it and not wait for a fix to hopefully come upstream.

[SteveITS]

@michmoor On one hand, arguably no, but OTOH there’s no free version of UniFi AFAIK.

Open source is a weird middle ground of lots of people wanting updates, but few have the skill and time and will to do the work for free.

[bmeeks]

@SteveITS said in Feedback on challenges with the pfSense package development process:

Open source is a weird middle ground of lots of people wanting updates, but few have the skill and time and will to do the work for free.

This is most certainly true and very well describes the overall situation with regards to packages on pfSense and any other similar open source platform.

[bmeeks]

@michmoor said in Feedback on challenges with the pfSense package development process:

@bmeeks
Curious, do you think the current system of 3rd party packages and support of a volunteer community is scalable?
If you look at other vendor firewalls, the entirety of the system is under control of the company. Unifi may run Suricata in the background but if something breaks on their system they take ownership of it and not wait for a fix to hopefully come upstream.

@michmoor, not trying to pick on you with this reply, but your questions and remarks make it appear you seem to expect the same level of support from pfSense and Netgate as you get from the other firewall vendors I see in your signature (Palo Alto and Juniper).

At the price you are paying for pfSense versus what I suspect you are paying for Palo Alto and Juniper, that is just never going to happen . Especially so for the free open source edition of pfSense (and remember the packages tree is essentially the same for both the free and paid editions of pfSense, with the one exception I recall being an AWS package for Plus that is not available in CE).

Netgate could certainly follow the lead of a Palo Alto or Juniper or Cisco and bring all the packages in house and maintain them. But the price of Plus would have to increase substantially and I would not expect to see that support spread to the open source side for free.

[marcosm]

@michmoor said in Feedback on challenges with the pfSense package development process:

@marcosm Marcos, are packages dependent on point releases of pfsense plus?
What constitutes a new package?
Whats a realistic timeframe new packages like Zabbix can be added to the pfsense repo?

I doubt this answer is satisfactory but it's what I've got...

Sometimes. Generally anything with its own page at https://www.freshports.org/. Depends on the situation.

Off-topic: I personally hesitate to spend much effort browsing/responding in social platforms because it often tends to result in additional stress that I prefer to avoid in my life. That's become more and more true as the years pass. Anyway, I say this to offer some insight into why I may not respond to things. I imagine I'm not alone in that.

[michmoor]

@marcosm No worries Marcos. I appreciate you even responding.
Your answer is satisfactory. I will leave it at that :)

[michmoor]

@bmeeks Always picking on me..why is that ( kidding).

Im not comparing any level of support between vendors in my profile for example and Netgate. There is a gulf between them but that's not the issue (or my issue) in particular.
For some background, I have been using PfSense for about five years now. When I started using the platform, I assumed that everything running on it or installed on it was supported. Much like the OP here, I figured it was the total package, and if you needed white glove support, you could get a TAC support contract.

I later found out that there are nuances in what is supported and by whom. The website's marketing and documentation do lead anyone to believe that there is support for third-party packages, but I found out years ago that is not true. Hence why i would advocate for cleaning up the package support list page and only list the packages that are supported by Netgate. Documentation can state otherwise.

[tinfoilmatt]

@michmoor said in Feedback on challenges with the pfSense package development process:

There is a gulf between them but that's not the issue (or my issue) in particular.

Speaking for myself and rather confidently on behalf of others, many of us feel like that "gulf" exists in the opposite direction from what you suggest and are nothing but grateful that Netgate continues to maintain a CE and a free Plus which can utilize third-party packages at all.

@michmoor said in Feedback on challenges with the pfSense package development process:

The website's marketing and documentation do lead anyone to believe that there is support for third-party packages

That's just not true unless you're claiming to speak on behalf of literally everyone.

[Mauriced]

But if I understand this correctly, the community has to provide the necessary files to successfully compile an application/package in PFSense in order to get it in the web-gui and pkg list, right? (which is done using the pfsense/FreeBSD-ports)

Then the netgate team needs to (compile, ) test and put this in the upcoming PFSense release so that every PFSense admin has access to the package, right?

So if that is correct, then why is it taking so long for a new package to be added that is already 5 months in the main branch?(https://github.com/pfsense/FreeBSD-ports/tree/main/net-mgmt/zabbix7-server)

Again, please correct me if I am wrong, I am trying to understand the complete process, and if I can help in any way to speed up this process I am willing to try what I can for the Zabbix package.

[michmoor]

@tinfoilmatt I think there is a reading comprehension issue here.....I have no idea what you are talking about or really what you are trying to convey.

[Gertjan]

@Mauriced said in Feedback on challenges with the pfSense package development process:

I am trying to understand the complete process

The very moment Netgate decides to place a pfSense package on their "package servers", the one pfSense uses to get updates, upgrades and new pfSense packages, even if the package wasn't created by Netgate internally, they will take "some" responsibility for it.

A pfSense package, as for example Notes or Cron or Shellcmd, are simple packages, with not much more as a collection of php files for the GUI interaction.
Some more data (scripts) like 'where to place it on the menu' and 'what do do when placing the files on the file system' and 'what to do when removing the packages', and that's it.
Then the 'pkg' tool is used to build a real 'FreeBSD' package and voila, a new pfSense-package-xxxx.pkg is born. The file is place on the Netgate pfSense package server, and everybody can now install it.
Easy to understand : every externally written pfSense package has to be authored by Netgate.
A bit like Apple authors every phone app before it enters the app store.
I guess these question must be asked and answered by the Netgate team :
Does the new functionality belong on a firewall ?
Is there an audience (market, usage, etc ?) for it ?
And also :
Who will write the manuals ?
How much config storage will be needed (our xml config file has become rather big lately)
The pfSense documentation web site has to be maintained.
And the most important one : is there any security aspect ? Will there be one ?
These are the questions I could find after a thorough 5 minutes reflection.
Things become much harder when binaries are involved. True, most binaries used are often maintained by others, and if big brother (the FreeBSD repository itself), hosts them, they are probably ok, and tested. Still, using 'whatever' on a desk top, or even a server environment is one thing, using it on a firewall is another.
If one pfSense package provokes a major security issue, the big world out there will not understand that its just a failing "add-on", but it was pfSense (Netgate) who has an issue. That's very bad for the business.

Also, keep in mind : what will happen when a package becomes extremely popular ?
And then then author stops maintaining it for whatever reason. Netgate won't have a choice, they have to 'incorporate' the package into their maintenance list.
Only this reason alone will make them think twice before they 'accept' a package from 'some one'.

As always : these just my - a pfSense users as we all are - thoughts - and keep in mind, I 'm sure I just scratched the surface of the question.

[andrew_cb]

There are lots of good discussions and viewpoints in this thread!

So far, the only way I've found https://www.netgate.com/supported-pfsense-plus-packages is by searching. I would suggest that https://docs.netgate.com/pfsense/en/latest/packages/index.html and https://www.netgate.com/supported-pfsense-plus-packages be linked together.
On https://www.netgate.com/supported-pfsense-plus-packages, I think the list should be updated to include ALL available packages so that the supported and non-supported status is clearer.
Having a link to the supported packages page from within the Packages page of pfSense would also help improve awareness. An official/unofficial label or filter on the Packages page would be great, too.

Maybe it's due to the ignorance of people like me comparing pfSense against Big Name Firewalls.
Netgate could improve the messaging about what it means to have an "open source" firewall and how support for third-party packages works.

With Zabbix, one could think, "The Zabbix project is still maintained and is releasing new versions," and not be aware that

• a) Zabbix is not releasing FreeBSD versions,
• b) Even if there is a plain FreeBSD version, it's likely not compatible, and
• c) There's no active maintainer of Zabbix for pfSense.

These issues are not immediately apparent when starting out with pfSense.

The numerous Redmin and forum posts go unanswered, and Netgate staff put a target version on Redmine's request, which can reasonably be interpreted as an agreement and commitment by Netgate. I've never seen Netgate staff pushing back on Zabbix and explaining that Netgate does not maintain the package and that if someone else builds a pfSense-compatible version, they can submit it for inclusion.

Regarding paying support to help drive pfSense development, I think that issue is irrelevant with the current paid support options as both CE and Plus have almost the same packages. There is no express or implied statement that purchasing support will affect package development whatsoever. I could purchase $10,000 of support and still not get Zabbix 7.0 anytime soon. There's a Bounty forum, but I haven't seen it widely promoted as a means to get the community to maintain packages.

I'm not trying to trash Netgate or anyone who works on pfSense. I genuinely appreciate their hard work and love using pfSense at home and work.
I'm trying to help move forward with ways to improve it from a business/commercial perspective. All the effort to deliver MiM is great, and better package support would make pfSense even better! Perhaps Plus could get more regular package updates and support compared to CE, either on its own or by requiring TAC Lite or a "Packages Only" subscription. This would be similar to how Plus gets the VPN export package, but CE doesn't.

[andrew_cb]

I spent some time yesterday learning how to build from ports, and I was able to create Zabbix 7.0.6 proxy and agent binaries using a fully updated FreeBSD 14.2 VM. I didn't use Poudriere jails or anything, just ran

cd /usr/ports/net-mgmt/zabbix7-server/ && make

I copied the binaries to a pfSense 24.11 box for testing and so far they work perfectly.
This is not a proper build and testing process, but it's a good indication that Zabbix 7.0 should work fine on pfSense 24.11.

I'm not a developer and don't have a Github repo, so I guess this doesn't help anyone unless I setup an account and submit a PR for my binaries to be included?

Also, is there a guide of what the statuses mean on Redmine? The Zabbix 7.0 package request is "Confirmed", "In Progress", and target is 25.01. Does this mean that Zabbix 7.0 will NEVER be released for 24.03 or 24.11? Is Netgate staff working on the packages to include with 25.01, or is it waiting on someone in the community to submit it?

If 24.11 is the currently supported version of pfSense, and assuming the Zabbix package doesn't require modifications to pfSense, then it should be possible for an official package to be released for 24.11, right? As far as I can tell, it can 7.0 behaves exactly the same as 6.4.

I think my test port uses different default paths for the config, log, and pid files than pfSense does, but the log and pid paths can be set in the config file, and the the config file path is given as a parameter to the binary. Or someone with more knowledge than me could just set those correctly in the build.