Suggested change to upgrade release announcement language
Notice the upgrade announcement language includes:
"As always, you can upgrade from any previous version straight to 2.2.2"
Given past experience (for example total email forwarding postfix "won't start" breakage with no user way to fix when upgrading to the previous release)
I suggest changing the release language to:
"As always, you can upgrade from any previous version straight to 2.2.2. Note users of any add-on package may experience loss of function without notice."
I support this change.
I am getting used to recrafting from ISO since 2.1.5 and I have changed my upgrade process accordingly. Following today's failed upgrade I am now investigating stripping back my pfSense configuration by offloading Snort and Squid3 onto separate hardware. I will only need NUT as a package on pfSense. I am also seriously considering pulling DHCP and DNS and putting them back onto dedicated hardware.
I think packages should earn their 'supported' status for each release version/upgrade. They should be marked 'untested' in the packages list until ten or so positive test results from different users have been submitted.
This arena of 'package testing' is, for example, why newcomer Linux Mint has come from nowhere to lead all the others in new installs. They review every package upgrade, and in their package manger rate them in order of 'disruption danger' from 1 to 5. It won't even list upgrades from packages level 4 and 5, other than security related.
These are the sort of touches that make the difference between a lab project and a long term mission critical one. It would be one thing if PF architecture left every package alone other than its core so the user could decide when to upgrade known working packages or not. But PF architecture doesn't work that way. It's all or nothing.
The only way forward I can see is for the PF upgrade notice system to say 'upgrade available' with no qualification only when the particular packages each install is using have a flag somewhere qualifying they don't have material regressions (only new features buggy / broken). Who sets does the testing and how that flag is set I don't know.
I suppose the 'upgrade available, some packages untested' should pop up when there is an upgrade of the usual sort–- for those who want to take a ride on the wild side, or who think of PF as a learning tool or lab project.
I'd bet there is a paid subscription business out there for someone to offer an add-on that requires a subscription fee, but will delay the upgrade process until regression testing they do has completed.