HEADS UP: Package Maintainers - ACTION REQUIRED - Packages Cleanup / PBI Prep
-
For those of you who maintain packages, we need some things cleaned up and updated. This applies to all packages for 2.0.x and 2.1.
DISCLAIMER: This is not a rant or aimed at anyone specifically, so relax! It's all for making things better/smoother with packages, and to remind package maintainers of the policies about the package system. :)
Most of these items only apply to packages that have binary dependencies, but pure-PHP packages could use a review also (but none of the build items apply there).
Make sure if you have binaries, they are on files.pfsense.org - they should not be pulled in from any other source (with the possible exception of FreeBSD's official package repository, but that doesn't have PBIs so it doesn't matter). This is for security/control reasons, we have to make sure they are on servers that our under our control.
This also goes for any files/libraries pulled in from install scripts - this really shouldn't be necessary if your ports are built right but isn't forbidden, they just need to be on our servers.
All packages with binary dependencies should have build_port_path tags defined to point to the path in /usr/ports where the FreeBSD port can be found. See examples throughout pkg_config.8.xml* files.
If there is no FreeBSD port, you can create one and we can import it into pfPorts in the tools repository. If you need help with this, ask on the development mailing list or the development board here.
If your package requires custom patches to a port, we need those in pfPorts also
If your package requires custom options set for a port, we need those in build_options format (examples are in various ports in pkg_config.8.xml)
We can build packages as needed for you if the binaries do not exist on our servers. We must be able to build these automatically at all times. Copying over files from an outside source should not be needed so long as the build options and port paths are set right in the package. If your port needs special handling in some way, see above about getting the changes into a pfPort. If the name is the same as another port, there are ways to accommodate that, but it still needs to be in a unique pfPort.
If you need to test these things on your private servers, please setup your own custom package repository and point your pfSense install there (See /pkg_mgr_settings.php), you can checkout your own forked copy of the package repository to that location, and test as you please without altering the "live" packages repository. Once you know your package works, let one of us know and we can make sure it gets built on our side. This should eliminate any need to point the "live" package install paths at a non-pfSense server. If you need changes to the package repo, they can be committed or submitted as a pull request after testing.
For 2.1 we are building PBI format packages so they will not conflict with each other (yay), in the past we have disabled automated builds for new packages but that should be OK now, so things like Squid3 can join in the fun.
Make sure a binary exists on files.pfsense.org before adding it as a depends_on_package or depends_on_package_pbi. If it's not there, things will break. Add/update build_port_path and build_options as needed, then wait overnight (automated builds happen ~1am EDT) or ping a developer to trigger a build for you. Once the binaries are present, only then should you add/edit a depends_on_package or depends_on_package_pbi tag referring to the new binary name.
Also, make sure you are using autocrlf on git to avoid committing badly-formed newlines. These will on occasion cause problems for others using git, forcing them to commit files they made no changes to simply because git doesn't like the newlines to be mismatched.
Remember - Absolutely no binaries should be committed to the packages repository!
If, even after all of the automated build processes, we determine that we need a file from you directly, contact us and we will copy the file to files.pfsense.org from your site, again, they should never be hosted on a server that we do not control.
We will be cleaning up the links to files/packages in the near future so be sure to act on this as soon as possible to make sure your packages do not break.
If you saw a package disappear from 2.1 today, it's because it had no PBI and wasn't in the process of being built yet. Once a PBI is built, we'll bring it back. We are actively developing/improving the PBI build process so we need things to be correct so that all PBIs can be built properly. If all packages have proper build tags this will happen automagically and we can fix the tags once the PBIs appear.
If you notice an issue with a package that you do not maintain, let us know and we'll look it over. Or submit a pull request on github with a proposed fix if you like, so it may be reviewed. As a general rule, however, it's best to only alter your own packages that you maintain.
Thanks, and keep all the great contributions coming, it's appreciated! If you need help, don't be afraid to ask, especially on the development list or board here.
I'll edit this post if anything else need attention.
