Suggestion to make a 2.3.3 release



  • IMHO the rate of discovery of "dumb bugs" in RELENG_2_3 seems to have dropped in the last few weeks. And there are quite a few things that have been back-ported into RELENG_2_3 since 2.3.2 was released.

    With the imminent release of FreeBSD 11.0 and thus a natural concentration on testing and tweaking all things pfSense 2.4/FreeBSD 11.0, I reckon it would be worth getting the current (good) state of RELENG_2_3 released as 2.3.3.

    That will get recent user code changes out into wide use, so they won't be mixed (and possibly confused) with changes that happen due to FreeBSD 11.0 etc, and provide a nice base for those stuck on 32-bit for the short term.



  • At least some good news. 2.3.2_1 shows 97% complete all of a sudden.

    https://redmine.pfsense.org/projects/pfsense/roadmap



  • It would be great if correct iface names would display in new traffic graph widgets. I hope it makes it to this release…


  • Rebel Alliance Developer Netgate

    @Harvy66:

    At least some good news. 2.3.2_1 shows 97% complete all of a sudden.

    https://redmine.pfsense.org/projects/pfsense/roadmap

    Most of the bugs were fixed but needed closed, and a few needed moved. We'll be doing 2.3.2_1 this week, assuming everything goes well.



  • @jimp:

    @Harvy66:

    At least some good news. 2.3.2_1 shows 97% complete all of a sudden.

    https://redmine.pfsense.org/projects/pfsense/roadmap

    Most of the bugs were fixed but needed closed, and a few needed moved. We'll be doing 2.3.2_1 this week, assuming everything goes well.

    I think there are quite a few "new" things that were backported to RELENG_2_3 but not RELENG_2_3_2. (I will check that…) It would be nice to get the code in RELENG_2_3 released.



  • @phil.davis:

    I think there are quite a few "new" things that were backported to RELENG_2_3 but not RELENG_2_3_2. (I will check that…) It would be nice to get the code in RELENG_2_3 released.

    Ditto.


  • Rebel Alliance Developer Netgate

    Depending on what they are, they'll probably have to wait at this point. We'd like to get it out ASAP and can't bring in a bunch of things that would need extra testing.



  • @jimp:

    Depending on what they are, they'll probably have to wait at this point. We'd like to get it out ASAP and can't bring in a bunch of things that would need extra testing.

    When did this testing religion take hold?  Thought it was all guinea pig.  ;)



  • The recent copyright text changes are in RELENG_2_3 but not in RELENG_2_3_2.
    And things like "Save widget settings per user", "Backport Add missing recommended key lengths/digest to Cert system"…

    Anyone using the 2.3.3-DEVELOPMENT snapshots has been using the code from RELENG_2_3. It seems a shame not to release it.

    I attached the output of:

    git diff RELENG_2_3_2..RELENG_2_3
    

    But there is a lot of noise in that because of the 100's of files that have the copyright diff.

    diff.txt


  • Rebel Alliance Developer Netgate

    The keys, perhaps, could be for security but not the widget settings, that's a feature. The _x releases are intended to be strictly security and significant bugs. Those changes make sense in 2.3.3 but not 2.3.2_x.



  • @jimp:

    The keys, perhaps, could be for security but not the widget settings, that's a feature. The _x releases are intended to be strictly security and significant bugs. Those changes make sense in 2.3.3 but not 2.3.2_x.

    Yes, I agree. I am just suggesting to release 2.3.3 (rather than bothering with 2.3.2_1)


  • Rebel Alliance Developer Netgate

    That would require a significantly larger amount of testing which would delay a release further. Better to get a quick security release out without the extra hoopla of a full release with new installers and whatnot.



  • @phil.davis:

    @jimp:

    The keys, perhaps, could be for security but not the widget settings, that's a feature. The _x releases are intended to be strictly security and significant bugs. Those changes make sense in 2.3.3 but not 2.3.2_x.

    Yes, I agree. I am just suggesting to release 2.3.3 (rather than bothering with 2.3.2_1)

    For what it's worth, irrespective of whatever it gets called, it would be great if this release would include the "DHCP6c before RA" fix, assuming it's been back-ported. I and others have been using it for quite a while and it's been solid. (I was using the "2.3.3" snapshot since early August, but switched to the 2.4 shapshot a couple of weeks ago.)



  • @bimmerdriver:

    @phil.davis:

    @jimp:

    The keys, perhaps, could be for security but not the widget settings, that's a feature. The _x releases are intended to be strictly security and significant bugs. Those changes make sense in 2.3.3 but not 2.3.2_x.

    Yes, I agree. I am just suggesting to release 2.3.3 (rather than bothering with 2.3.2_1)

    For what it's worth, irrespective of whatever it gets called, it would be great if this release would include the "DHCP6c before RA" fix, assuming it's been back-ported. I and others have been using it for quite a while and it's been solid. (I was using the "2.3.3" snapshot since early August, but switched to the 2.4 shapshot a couple of weeks ago.)

    Indeed, that is in RELENG_2_3 but not in RELENG_2_3_2



  • @jimp:

    The keys, perhaps, could be for security but not the widget settings, that's a feature. The _x releases are intended to be strictly security and significant bugs. Those changes make sense in 2.3.3 but not 2.3.2_x.

    So much for _x releases being "intended to be strictly security and significant bugs."

    Show system platform and serial / UUID
    https://github.com/pfsense/pfsense/commit/27663052c2b5e199a96f4f5ea766abafa8f1ec08#diff-7f8270168d6aa91bbc90278269ffba9b

    Make serial/UUID bold
    https://github.com/pfsense/pfsense/commit/27663052c2b5e199a96f4f5ea766abafa8f1ec08#diff-7f8270168d6aa91bbc90278269ffba9b

    Aren't double standards great.  Apply/enforce them when they suit, ignore them when they don't.


  • Rebel Alliance Developer Netgate

    That was a bug in our internal tracker. That serial number readout was in the factory images but not the CE images, it was intended to be in both, and the code is well-tested.



  • That is so shallow.  Those same kinds of justifications can be applied to other "bug" commits too.

    Please note, the criteria is supposed to be STRICTLY SECURITY and SIGNIFICANT bug fixes.  That is neither a security nor significant bug.



  • more what you'd call "guidelines" than actual rules



  • @Harvy66:

    more what you'd call "guidelines" than actual rules

    All the more reason many of these other fixes are just as valid to be included.  Some of them even more so because they actually have functionality fixes/corrections.  Not just WebGUI window dressing.


  • Rebel Alliance Developer Netgate

    We have to draw a line somewhere, or 2.3.2_1 would practically be 2.3.3, which would be a lot more work. We need to get it built, tested, and out for security reasons which means being conservative in what we pull in. The ones that made the cut, made it. The rest can wait for 2.3.3 or maybe 2.3.2_2. The _1 release was built today, and is already being tested.

    Could we have included more? Sure, but the time to advocate for that was much sooner than the day of/day before a release is to be built. If they were significant enough they should have been sent as PRs against RELENG_2_3_2 a while ago.



  • We did advocate for them much sooner.  That's why we submitted the PR's.  Why where they not include in 2_3_2 when they where merged?  They should have been there was no significant reason for them not to be.


  • Rebel Alliance Developer Netgate

    We don't default to cherry picking them all the time unless it's requested. If a separate PR is submitted against RELENG_2_3_2 it's more likely to be included. It isn't always immediately clear if a bug applies to all branches. I usually check on the things I fix myself, but for PRs it's a bit of an extra burden to expect us to check all of the branches to see if it applies.



  • For change tracking later on it is much neater if a change in master is then cherry-picked back to whatever old branches are appropriate. Making separate pull requests for the same code makes it somewhat harder to verify that a change has happened across multiple branches.

    Because this is a company-managed project (rather than community-managed), there is nobody in the community with commit access. So it is company people who have to do the cherry-picking. Also, the community is not made aware of the impending cutting of a release, so we do not know to scan through looking for things that could go into a branch. Even with a security release like this, it would be nice to put an announcement somewhere to say "we are building a security/bug fix release in 48 hours for testing and then release at the end of the week, please submit any proposals for bug fixes by x time."

    I had noticed that all bugs were being routinely back-ported to RELENG_2_3 but only some bugs were also put into RELENG_2_3_2. Since there seemed to be no consistent rule, I assumed that it was just that some company devs remembered RELENG_2_3_2 and others had forgotten about it. Also 2.3.3-DEVELOPMENT snapshot builder was routinely running. I concluded that there would likely be a 2.3.3 release anyway, and the content of RELENG_2_3_2 would not matter. So I didn't take time to ask for more stuff to go back to RELENG_2_3_2.

    I agree with NOYB comments - it seems that code written by a company dev will be back-ported pretty routinely to all branches, but code contributed by the community is not treated equally. That is a shame.

    Conclusion: giving the community information (about long-term release schedules, old version support plans, "emergency" releases…) and taking community feedback/suggestions on the forward plan and the methodology for handling what goes in what branch would be appreciated.



  • Ditto that for sure.



  • @jimp:

    We don't default to cherry picking them all the time unless it's requested. If a separate PR is submitted against RELENG_2_3_2 it's more likely to be included. It isn't always immediately clear if a bug applies to all branches. I usually check on the things I fix myself, but for PRs it's a bit of an extra burden to expect us to check all of the branches to see if it applies.

    Here is a case where no checking if applies was required.  It was spelled out.

    Even when there is a referenced bug report with both the target and affected versions set it still doesn't happen.  Seem like that is a reasonable form of request for it to be back ported.
    Delete NAT rule with associated firewall rule does not update firewall separators position
    https://github.com/pfsense/pfsense/pull/3089
    https://redmine.pfsense.org/issues/6676

    Plus additional reference if it were needed.
    https://forum.pfsense.org/index.php?topic=116099.0

    If it not clear.  Simple ask or at least let the submitter know.  There simply isn't enough communication to the active community.  We are and have been for quite some time been left in a black hole.


  • Rebel Alliance Developer Netgate

    Fair point, but it does go both ways. A gentle reminder that it needs cherry picking would likely lead to it being pulled in a majority of the time.



  • So jimp, can you confirm that there will be a 2.3.3 and it will include all those the fixes and "enhancements" back-ported to RELENG_2_3 that did not make it in 2.3.2_1 then?


  • Rebel Alliance Developer Netgate

    I don't think we've officially decided either way. There are snapshots, so it's likely, but it depends on how quickly 2.4 progresses.



  • @jimp:

    … it's likely, but it depends on how quickly 2.4 progresses.

    Please remember that a bunch of users would be cut off the things discussed above with 2.4 being 64 bit only.
    Having that on the last 32 bit release will help some, don't you think?


  • Rebel Alliance Developer Netgate

    @jahonix:

    @jimp:

    … it's likely, but it depends on how quickly 2.4 progresses.

    Please remember that a bunch of users would be cut off the things discussed above with 2.4 being 64 bit only.
    Having that on the last 32 bit release will help some, don't you think?

    We'll keep doing errata releases as needed for security updates and such (and to pull in more of the referenced bug fixes from this thread and beyond), it's just not clear if we'll need another full release. Depends on what comes up.



  • @jimp:

    We'll keep doing errata releases as needed for security updates and such (and to pull in more of the referenced bug fixes from this thread and beyond), it's just not clear if we'll need another full release. Depends on what comes up.

    What will trigger a 2.3.3 release?  If we knew maybe we could help it along.  ;)


  • Rebel Alliance Developer Netgate

    @NOYB:

    What will trigger a 2.3.3 release?  If we knew maybe we could help it along.  ;)

    Now that we have the ability to do errata style releases it's a much tougher question. Probably something that would require a rebuild of the installer images in the extreme case, or when there have been enough changes to warrant a new installer. Maybe a significant change that would need more long-term testing in snapshots. Or a significant enough security issue that we wouldn't want new installations to be vulnerable without an update.

    With the errata releases and pkg we can update binaries, kernels, etc, in between without much fuss. It's a lot better system than we had in the past.



  • @jimp:

    With the errata releases and pkg we can update binaries, kernels, etc, in between without much fuss. It's a lot better system than we had in the past.

    I'd still vote for having one steady release (or patch-level rebuild) every 2 months with ISO/memstick made available for download and installing on clean systems.

    Updating clean installs on non-patch-level build takes considerable time when updated packages is high. Having a regular flow of releases would be helpful.

    So far I am relying on the snapshots, while researching on what's incorrect when things don't happen as expected and submitting bug reports/fixes afterwards on those.



  • @jimp:

    Fair point, but it does go both ways. A gentle reminder that it needs cherry picking would likely lead to it being pulled in a majority of the time.

    It may be a two way street, but the traffic is going in only one direction.