Feedback on challenges with the pfSense package development process
-
@marcosm
The questions asked @michmoor are a good summary of the basic information needed for pfSense packages. I'm not sure of the best way to track that - a single table in the pfSense documentation would be the most convenient for reference. Another option would be to put the current package version and next scheduled update on the package's documentation page, but not all packages have pages. Redmine is another option, but it's less convenient to navigate. Maybe something in the packages list in Pfsense itself?If the default policy is that packages are normally only updated with each new pfSense release, then I think that needs to be more clearly stated.
Going back to the Zabbix example, when there was an update to 6.4.10, it was only made available in 24.03 and not in 23.09 or 23.05.
There have been no further mainstream updates to the Zabbix packages since April 2024. The issue is still open and "confirmed".
https://redmine.pfsense.org/issues/1531324.11 RC contains Zabbix 6.4.19 from Sept 2024, so it is relatively up to date.
This is becoming urgent because Zabbix's End of Full Support for 6.4 was 30 June 2024, and End of Limited Support is 31 December, 2024.
https://www.zabbix.com/life_cycle_and_release_policyIt's frustrating that multiple comments and posts, both on the forums and on Redmine, have asked about updates to Zabbix 6.4 and a package for 7.0, and Netgate has not responded other than changing the target version on the 7.0 Redmine.
I have no idea how easy or difficult it is to release a package. Can you comment on roughly how much time/effort is required for a knowledgeable person (i.e Netgate staff) to release a Zabbix package for:
- a) an existing version (i.e. 6.0, 6.4), and
- b) a new version (i.e. 7.0)
p.s. I appreciate that you are responding to the discussion and I do not mean for this post to come across as hostile.
-
As a volunteer package developer/maintainer, let me chime in here with a point.
One thing I believe many users do not grasp is that most packages used with pfSense consist of two very distinct parts. There is a binary component that you see posted in FreeBSD-ports and this piece is installed by generic FreeBSD users. For these packages, users do their own configuration and set up via the command line interface (meaning no GUI).
But on pfSense, most packages also have a customized GUI component written in PHP. That GUI piece is created and then maintained by a developer familiar with the GUI interface of pfSense. The user then "interacts with" and "configures" the said package using the customized GUI on pfSense. The configuration parameters of the package are stored in the firewall's
config.xmlfile and are only written to any file system based configuration file used by the package's binary piece when the user "saves" the setting in pfSense. This "save" operation writes a new conf file for the package (and also generally will overwrite anything the user attempted to do via the CLI outside of the pfSense GUI). This type of operation also means that the PHP GUI code must be intimately familiar with all of the exact parameters and their associated values required by the package binary portion. So, if a new version of the binary package requires new or updated conf parameters in order to start, the PHP GUI code that creates the conf file may need modification.This can many times mean that when a new version of the package binary comes out (let's use Zabbix 7 as an example), updating to the new version means potentially rewriting or at least modifying the customized PHP code that constitutes the GUI interface. And therein lies the potential issue. The PHP code used to make the package integrate into pfSense is not something the upstream maintainers of the binary (Zabbix in this example) have knowledge of nor do they care to learn. Thus someone on the pfSense side of the equation must edit the PHP GUI code to make sure everything still works with the updated binary piece from upstream.
The packages subsystem in pfSense began as volunteer contributions. Most packages are still supported by non-paid volunteers not associated with Netgate in any manner. Over the years, as these volunteer maintainers have come and gone, either their package dies or in a few very rare cases Netgate takes over its maintenance. The disconnect I see is that many users seem to think Netgate develops and maintains all the third-party packages available in the pfSense Package Manager menu. That is not the case at all.
-
@bmeeks
To be fair the disconnect is introduced by the advertisement on the product websitehttps://www.netgate.com/supported-pfsense-plus-packages
Support is not clearly defined. It’s understandable to me why most see a package in a pfsense repo is supported by pfsense dev team.
The reality is that only pf in pfsense is supported. Nothing else. -
@michmoor said in Feedback on challenges with the pfSense package development process:
@bmeeks
To be fair the disconnect is introduced by the advertisement on the product websitehttps://www.netgate.com/supported-pfsense-plus-packages
Support is not clearly defined. It’s understandable to me why most see a package in a pfsense repo is supported by pfsense dev team.
The reality is that only pf in pfsense is supported. Nothing else.Yeah, I did not get into "why" many users think Netgate developers support all the packages. Just stating that the perception is not always reality
.To be fair to Netgate, it is clearly spelled out on that page which packages they directly support. That would be all those marked "Maintained by Netgate".
Since Zabbix is mentioned in this thread, note that it is not listed at all on the linked page.
-
@bmeeks I would just say that the package support list aids in the confusion. It should only list the ones maintained by Netgate. Including others is , in my opinion, bad marketing as they are not supported. I’m not directing this to you Bill, just “talking out loud”. That page needs clarification/ cleanup more so because it’s mentioned byTAC within Reddit forums when they communicate supported packages which just isn’t true for most of them.
-
I don't recall seeing https://www.netgate.com/supported-pfsense-plus-packages. Is this the only place with the package support status? I don't see it linked anywhere on the website. Regardless, there are numerous references to non-Netgate maintained packages throughout the website, which give the appearance of at least being endorsed by Netgate, and no mention of their "risky", unofficial status.
Here's a few just from browsing around:
https://docs.netgate.com/pfsense/en/latest/packages/list.html < The first two paragraphs at the top of the page are sort of a disclaimer, but still doesn't make it clear about the support status of packages. There's no link to the official page.
"Zabbix-proxy
Zabbix Agent proxy. Collects performance and availability data on behalf of the Zabbix server, lowering the burden on the server. Available in multiple versions."https://www.netgate.com/pfsense-plus-applications/firewall
"Observe key operating metrics like network utilization, CPU load and disk space usage with built-in Zabbix monitoring"https://www.netgate.com/blog/alternatives-to-ntopng
"pfBlockerNG: DNS-based Blocking and Filtering
pfBlockerNG is a pfSense Plus software package that allows you to block access to malicious domains, ads, and other unwanted content at the DNS level. It uses various lists and feeds to stay up-to-date with the latest threats and can be customized to block specific categories or domains based on your organization's policies. pfBlockerNG integrates with pfSense Plus software's firewall and routing capabilities, providing an additional layer of security."AND
"Network Monitoring Tools
When it comes to network monitoring, open-source tools like Nagios, Zabbix, and Prometheus are widely used in various organizations.
...
There is a pfSense Plus package called NRPE which allows a Nagios server to monitor pfSense Plus software, and
The node_exporter package in pfSense will export the Prometheus metrics to that external server."https://www.netgate.com/pfsense-plus-technology
"Telegraf
An agent written in Go for collecting, processing, aggregating, and writing metrics.""Zabbix-agent
Zabbix Monitoring agent. The agent gathers operational information locally and reports data to Zabbix server for further processing. The agent can also generate alerts in case of failures. Available in multiple versions."The Packages menu in pfSense similarly doesn't include any disclaimers or link to the official package status page.
According to the package lists, NO monitoring packages are supported (Prometheus, Nagios, Zabbix, Telegraf). Is SNMP really the only official option?
You can see how this creates a very confusing experience for an end-user when trying to use packages.
-
@bmeeks Thanks for your insight. The complexities with creating/maintaining a GUI for packages makes total sense.
Again with Zabbix, based on my use of it with other systems, no changes to the GUI configuration parameters are necessary as the basic parameters are the same. Currently, pfSense has Zabbix 5.0, 6.0, and 6.4 packages and associated configuration values/pages. If it can accommodate 3 versions, adding a 4th that requires no/minimal modifications should be relatively simple.
If there is a complex issue that is delaying the update or release of a package, then at least some communication about that would go a long way.
If Netgate is not the maintainer of a package, and building packages for pfSense requires a very particular FreeBSD build environment, then are those essentially orphaned from a pfSense perspective? What is the trigger and motivator for them to be updated?
Netgate staff have do not respond to these posts saying "Package xyz is not maintained by pfSense, please ask the maintainer to submit an updated version to be reviewed for inclusion in the package repository", rather they respond to some of the posts and set a target version in redmine. If Netgate is truly not maintaining the FreeBSD/pfSense port builds of these packages, then this behavior further adds to the confusion.
Another point I want to mention - Netgate staff caution against installing packages from the FreeBSD repository as this can break things, and instead recommend to only install packages from the pfSense repository. Again, this further reinforces the impression that Netgate is maintaining and curating the packages available in Packages.
-
@andrew_cb
Just want to reiterate what was stated in these forums awhile ago.
Do not install any package you cannot directly support yourself. This is for Suricata/Snort,pfblocker,zabbix.
Once you make these tools part of your production system, you run the risk that at any point in time these packages stop receiving support. Then what?
As I stated above, the ONLY thing that Netgate supports is 'pf'. Nothing more nothing less. -
@andrew_cb said in Feedback on challenges with the pfSense package development process:
@bmeeks Thanks for your insight. The complexities with creating/maintaining a GUI for packages makes total sense.
Again with Zabbix, based on my use of it with other systems, no changes to the GUI configuration parameters are necessary as the basic parameters are the same. Currently, pfSense has Zabbix 5.0, 6.0, and 6.4 packages and associated configuration values/pages. If it can accommodate 3 versions, adding a 4th that requires no/minimal modifications should be relatively simple.
If there is a complex issue that is delaying the update or release of a package, then at least some communication about that would go a long way.
If Netgate is not the maintainer of a package, and building packages for pfSense requires a very particular FreeBSD build environment, then are those essentially orphaned from a pfSense perspective? What is the trigger and motivator for them to be updated?
Netgate staff have do not respond to these posts saying "Package xyz is not maintained by pfSense, please ask the maintainer to submit an updated version to be reviewed for inclusion in the package repository", rather they respond to some of the posts and set a target version in redmine. If Netgate is truly not maintaining the FreeBSD/pfSense port builds of these packages, then this behavior further adds to the confusion.
Another point I want to mention - Netgate staff caution against installing packages from the FreeBSD repository as this can break things, and instead recommend to only install packages from the pfSense repository. Again, this further reinforces the impression that Netgate is maintaining and curating the packages available in Packages.
There are complexities invovled in building packages for FreeBSD (and by extension, pfSense) that are not always appreciated by those that do not have extensive experience in the area. Let me try to provide a high level overview.
The binary portions of packages must be built using the exact same FreeBSD kernel that the package will ultimately be installed on. That means, since pfSense uses a customized/modified FreeBSD kernel, any packages installed on pfSense need to be built in a builder that uses the same customized FreeBSD kernel and any shared library versions. That's where the warning about installling packages from the standard FreeBSD repo comes from. Those packages will be built for stock FreeBSD kernels and are also likely configured for and expecting to find particular versions of shared libraries. Those exact conditions may not exist on a given pfSense version. The kernel code is different from stock FreeBSD and the versions of shared libaries can well be different. When packages are installed, they automatically pull in the dependent library versions they were built with. That may overwrite something that pfSense needs and thus can break pfSense itself.
Netgate has their own set of package builder servers. Those servers build (meaning compile) the binary and PHP portions of packages using an enviroment created from a particular pfSense version (CE 2.7.2, or CE 2.8.0 DEVEL, or 24.03 Plus). That means the packages built by pfSense servers and served up in the pfSense GUI will have shared dependencies that match what is already installed by pfSense. Thus no conflict and no possibility of breaking pfSense.
When pfSense itself is updated, the packages are all rebuilt using the new kernel and shared library tree.
The binary portions of a package are tied to the underlying kernel version and thus the pfSense version. The PHP portion of a package is usually NOT tied to the kernel version althought it might have some dependency on the exact version of PHP used in the pfSense GUI.
The pfSense package repo source code is hosted on GitHub here: https://github.com/pfsense/FreeBSD-ports. Anyone is free to submit Pull Requests that contain updates for existing packages or that add entirely new packages to the pfSense list. That has happened many times over the years. Over the course of history, someone has had need of some particular function or package and created it themselves. They then may decide to "share it" with the larger community by creating a Pull Request and posting their package code to the GitHub link I provided. If the Netgate developers think the Pull Request has merit, they merge it into the tree. That's how most of the packages found their way into pfSense in the first place. Only a very few were created by Netgate themselves.
If you look at some of the recently closed Pull Requests on GitHub you will see updates I've provided for Suricata and updates that the pfBlockerNG developer has provided for that package: https://github.com/pfsense/FreeBSD-ports/pulls?q=is%3Apr+is%3Aclosed. Note that neither he nor I are Netgate employees. We are just pfSense users like you that decided to produce a package (or update an existing one) and shared our work with the larger community. For some of the really popular packages, the Netgate team may step up and fix small bugs to help out the larger community and to aid the volunteer package maintainer. That has happened occasionally for the Snort and Suricata packages I maintain. It has also happened from time to time with the pfBlockerNG package. But generally speaking Netgate will not take it upon themselves to update the binary portion of a package that they do not directly maintain. They wait on the volunteer maintainer who has more detailed knowledge of the inner workings of a package to submit any update via a Pull Request at the GitHub link posted above.
-
@bmeeks
Curious, do you think the current system of 3rd party packages and support of a volunteer community is scalable?
If you look at other vendor firewalls, the entirety of the system is under control of the company. Unifi may run Suricata in the background but if something breaks on their system they take ownership of it and not wait for a fix to hopefully come upstream.Firewall: NetGate,Palo Alto-VM,Juniper SRX
Routing: Juniper, Arista, Cisco
Switching: Juniper, Arista, Cisco
Wireless: Unifi, Aruba IAP
JNCIP,CCNP Enterprise -
@michmoor On one hand, arguably no, but OTOH there’s no free version of UniFi AFAIK.
Open source is a weird middle ground of lots of people wanting updates, but few have the skill and time and will to do the work for free.
-
@SteveITS said in Feedback on challenges with the pfSense package development process:
Open source is a weird middle ground of lots of people wanting updates, but few have the skill and time and will to do the work for free.
This is most certainly true and very well describes the overall situation with regards to packages on pfSense and any other similar open source platform.
-
@michmoor said in Feedback on challenges with the pfSense package development process:
@bmeeks
Curious, do you think the current system of 3rd party packages and support of a volunteer community is scalable?
If you look at other vendor firewalls, the entirety of the system is under control of the company. Unifi may run Suricata in the background but if something breaks on their system they take ownership of it and not wait for a fix to hopefully come upstream.@michmoor, not trying to pick on you with this reply, but your questions and remarks make it appear you seem to expect the same level of support from pfSense and Netgate as you get from the other firewall vendors I see in your signature (Palo Alto and Juniper).
At the price you are paying for pfSense versus what I suspect you are paying for Palo Alto and Juniper, that is just never going to happen
. Especially so for the free open source edition of pfSense (and remember the packages tree is essentially the same for both the free and paid editions of pfSense, with the one exception I recall being an AWS package for Plus that is not available in CE).Netgate could certainly follow the lead of a Palo Alto or Juniper or Cisco and bring all the packages in house and maintain them. But the price of Plus would have to increase substantially and I would not expect to see that support spread to the open source side for free.
-
@michmoor said in Feedback on challenges with the pfSense package development process:
@marcosm Marcos, are packages dependent on point releases of pfsense plus?
What constitutes a new package?
Whats a realistic timeframe new packages like Zabbix can be added to the pfsense repo?I doubt this answer is satisfactory but it's what I've got...
Sometimes. Generally anything with its own page at https://www.freshports.org/. Depends on the situation.
Off-topic: I personally hesitate to spend much effort browsing/responding in social platforms because it often tends to result in additional stress that I prefer to avoid in my life. That's become more and more true as the years pass. Anyway, I say this to offer some insight into why I may not respond to things. I imagine I'm not alone in that.
-
@marcosm No worries Marcos. I appreciate you even responding.
Your answer is satisfactory. I will leave it at that :) -
@bmeeks Always picking on me..why is that ( kidding).
Im not comparing any level of support between vendors in my profile for example and Netgate. There is a gulf between them but that's not the issue (or my issue) in particular.
For some background, I have been using PfSense for about five years now. When I started using the platform, I assumed that everything running on it or installed on it was supported. Much like the OP here, I figured it was the total package, and if you needed white glove support, you could get a TAC support contract.I later found out that there are nuances in what is supported and by whom. The website's marketing and documentation do lead anyone to believe that there is support for third-party packages, but I found out years ago that is not true. Hence why i would advocate for cleaning up the package support list page and only list the packages that are supported by Netgate. Documentation can state otherwise.
-
@michmoor said in Feedback on challenges with the pfSense package development process:
There is a gulf between them but that's not the issue (or my issue) in particular.
Speaking for myself and rather confidently on behalf of others, many of us feel like that "gulf" exists in the opposite direction from what you suggest and are nothing but grateful that Netgate continues to maintain a CE and a free Plus which can utilize third-party packages at all.
@michmoor said in Feedback on challenges with the pfSense package development process:
The website's marketing and documentation do lead anyone to believe that there is support for third-party packages
That's just not true unless you're claiming to speak on behalf of literally everyone.
-
But if I understand this correctly, the community has to provide the necessary files to successfully compile an application/package in PFSense in order to get it in the web-gui and pkg list, right? (which is done using the pfsense/FreeBSD-ports)
Then the netgate team needs to (compile, ) test and put this in the upcoming PFSense release so that every PFSense admin has access to the package, right?
So if that is correct, then why is it taking so long for a new package to be added that is already 5 months in the main branch?(https://github.com/pfsense/FreeBSD-ports/tree/main/net-mgmt/zabbix7-server)
Again, please correct me if I am wrong, I am trying to understand the complete process, and if I can help in any way to speed up this process I am willing to try what I can for the Zabbix package.
-
@tinfoilmatt I think there is a reading comprehension issue here.....I have no idea what you are talking about or really what you are trying to convey.
-
@Mauriced said in Feedback on challenges with the pfSense package development process:
I am trying to understand the complete process
The very moment Netgate decides to place a pfSense package on their "package servers", the one pfSense uses to get updates, upgrades and new pfSense packages, even if the package wasn't created by Netgate internally, they will take "some" responsibility for it.
A pfSense package, as for example Notes or Cron or Shellcmd, are simple packages, with not much more as a collection of php files for the GUI interaction.
Some more data (scripts) like 'where to place it on the menu' and 'what do do when placing the files on the file system' and 'what to do when removing the packages', and that's it.
Then the 'pkg' tool is used to build a real 'FreeBSD' package and voila, a new pfSense-package-xxxx.pkg is born. The file is place on the Netgate pfSense package server, and everybody can now install it.
Easy to understand : every externally written pfSense package has to be authored by Netgate.
A bit like Apple authors every phone app before it enters the app store.
I guess these question must be asked and answered by the Netgate team :
Does the new functionality belong on a firewall ?
Is there an audience (market, usage, etc ?) for it ?
And also :
Who will write the manuals ?
How much config storage will be needed (our xml config file has become rather big lately)
The pfSense documentation web site has to be maintained.
And the most important one : is there any security aspect ? Will there be one ?
These are the questions I could find after a thorough 5 minutes reflection.
Things become much harder when binaries are involved. True, most binaries used are often maintained by others, and if big brother (the FreeBSD repository itself), hosts them, they are probably ok, and tested. Still, using 'whatever' on a desk top, or even a server environment is one thing, using it on a firewall is another.
If one pfSense package provokes a major security issue, the big world out there will not understand that its just a failing "add-on", but it was pfSense (Netgate) who has an issue. That's very bad for the business.Also, keep in mind : what will happen when a package becomes extremely popular ?
And then then author stops maintaining it for whatever reason. Netgate won't have a choice, they have to 'incorporate' the package into their maintenance list.
Only this reason alone will make them think twice before they 'accept' a package from 'some one'.As always : these just my - a pfSense users as we all are - thoughts - and keep in mind, I 'm sure I just scratched the surface of the question.
