Package Version Control, can anyone explain? (I am lost)

louis2

Hello,

I am struggling with the actual PIMD-package, which is based on PIMD 2.3.2.

So that seems to be OK, does not it ……. Euh ….

PIMD was released on 10 Mar 2016 …… and then I did read on
on https://troglobit.com/projects/pimd/
@troglobit troglobit released this on 10 Mar 2016 · 380 commits to master since this release

But that is not all June this year a FreeBSD developer …… did make some corrections and ..I can not find his name in the commit list

And the version is still 2.3.2 ....

Today I tried to install PIMD from the netgate repository:

Fetching pfSense-pkg-pimd-0.0.3.txz: .. done
Installing pfSense-pkg-pimd-0.0.3...
Newer FreeBSD version for package pfSense-pkg-pimd:
To ignore this error set IGNORE_OSVERSION=yes
package: 1201519
running kernel: 1200086
Ignore the mismatch and continue? [Y/n]:
Failed to install the following 1 package(s): https://beta.pfsense.org/packages/pfSense__amd64-pfSense_devel/All/pfSense-pkg-pimd-0.0.3.txz

So ...

How many 2.3.2 version are there !!!????
And how do I know that the patches made in June made it into the pimd-master??
And how do I know if the actual pfSense PIMD-package is using the latest version?? (including that patch)

And yep pimd -v still displaying version 2.3.2 ….. does that not sound nice ?

To summarize “I am lost”

Louis
Other sites e.g.:
https://github.com/troglobit/pimd/releases/tag/2.3.2
https://www.freshports.org/net/pimd

bmeeks

@louis2:
Many (if not most) packages on pfSense have two different components, and each of those components can have a different version. Packages need a GUI piece (based on PHP code usually) to "create" the GUI interface you manipulate in your browser. However, that package code only creates the GUI. It will also be responsible for translating the configurable parameters you enter within the GUI to statements in a configuration file for the actual binary portion of the package pair that does the work. So in the case of PIMD there will be two packages. The GUI part is pfSense-pkg-pimd-0.0.3.txz, while the binary portion is pimd-2.3.2.txz. It is up to the GUI package creator to create his own versioning system. Different maintainers do it different ways, but seldom does the GUI package version number match the binary portion precisely.

When you install the PIMD package using the pfSense menu under SYSTEM > PACKAGE MANAGER it will install both packages by downloading each from the pfSense repository that corresponds to your pfSense version (2.4.5 or 2.5).

These lines from your post are unusual and make me wonder if you have added an additional pkg repo to your setup.

Newer FreeBSD version for package pfSense-pkg-pimd:
To ignore this error set IGNORE_OSVERSION=yes
package: 1201519
running kernel: 1200086
Ignore the mismatch and continue? [Y/n]:

louis2

@bmeeks,

I am very pleased with the first part of your description. I knew that, but I think not every one.

However, my concern is not the versioning of the "GUI-piece", but is the "binairy portion".

And to say it in an unfriendly way that seems to be a mess in the given example. If a package still has the samen version number after years (2016 => 2020) and many hunderd patches (380) ........ something is IMHO not OK!

But lets for the moment assume that the PIMD-maintainer at trolobit.com did manage to keep in sync with the different OS related packages. .....

How for the hell (sorry) can I check that

the package FreeBSD has created is based on the latest "trolobit" patch level!?
And how can I see which "last moment" FreeBSD modifications are in the actual FreeBSD package!!??
And how can I see that pfSense is using that Package!!??

What I would expect is that if I enter a command like "PIMD -v" the answer is not 2.3.2. but: 2.3.2.xyz_abc where

xyz is the revision-number from the main source (trolobit) and
abc is the additional FreeBSD patch level

That would make things trackable.

As said, at the moment I can only hope that the recent FreeBSD-patch is included in the actual pfSense package.

@jimp can you perhaps clearify how this works / how to check if you (also read NetGate here) is using the correct version?

Related to the latest lines, yep I had a special build installed to test something. That build did not include PIMD so I did try to add that package. And I was surprised to see that that install was saying "hey I am not going to do that, there is a newer version"

Louis

bmeeks

@louis2 said in Package Version Control, can anyone explain? (I am lost):

@bmeeks,

I am very pleased with the first part of your description. I knew that, but I think not every one.

However, my concern is not the versioning of the "GUI-piece", but is the "binairy portion".

And to say it in an unfriendly way that seems to be a mess in the given example. If a package still has the samen version number after years (2016 => 2020) and many hunderd patches (380) ........ something is IMHO not OK!

But lets for the moment assume that the PIMD-maintainer at trolobit.com did manage to keep in sync with the different OS related packages. .....

How for the hell (sorry) can I check that

the package FreeBSD has created is based on the latest "trolobit" patch level!?

And how can I see which "last moment" FreeBSD modifications are in the actual FreeBSD package!!??

And how can I see that pfSense is using that Package!!??

What I would expect is that if I enter a command like "PIMD -v" the answer is not 2.3.2. but: 2.3.2.xyz_abc where

xyz is the revision-number from the main source (trolobit) and

abc is the additional FreeBSD patch level

That would make things trackable.

As said, at the moment I can only hope that the recent FreeBSD-patch is included in the actual pfSense package.

@jimp can you perhaps clearify how this works / how to check if you (also read NetGate here) is using the correct version?

Related to the latest lines, yep I had a special build installed to test something. That build did not include PIMD so I did try to add that package. And I was surprised to see that that install was saying "hey I am not going to do that, there is a newer version"

Louis

Here is the link to the pimd binary package on FreeBSD ports. This is the official ports repository for FreeBSD.

https://svnweb.freebsd.org/ports/head/net/pimd/?pathrev=503015. (note it may take many seconds for this link to open as the FreeBSD server is not the fastest).

Every FreeBSD port has some (usually volunteer) maintainer who is responsible for updates. According to the official link below, the pimd package has not been materially updated in 4 years. The best place to see the history of updates is to check out the Change Log for the package Makefile. Here is that link:

https://svnweb.freebsd.org/ports/head/net/pimd/Makefile?view=log&pathrev=503015.

Many times the FreeBSD folks themselves make changes to a bunch of ports for reasons not related to actually changing the code. These are usually changes related to new compiler tools, directives and/or changes in dependent libraries. The last actual code version update I see in the Change Log was done March 11, 2016.

pfSense pulls in its binary packages directly from the FreeBSD ports tree. So that is why version 2.3.2 is in pfSense-2.5. It is coming directly from FreeBSD ports. The GUI portion is obviously a pfSense-specific thing and thus you won't find the pfSense-pkg-pimd package listed in the link above.

So to get back to what seems to be your central beef: the pfSense package for pimd is the most current one in FreeBSD ports. Therefore the author/creator of pimd itself has not ported any of his new code over to FreeBSD. That may be his fault, or it may be that the FreeBSD maintainer for pimd abandoned the project and thus the port has fallen behind. That happens sometimes.

louis2

@bmeeks

If that is true ........ we are using a completely wrong pimd-version.
I mean not including the 380 pathes from trolobit. and not those they (FreeBSD) made themselfs.

I can hardly believe its that bad ......

Louis

bmeeks

@louis2 said in Package Version Control, can anyone explain? (I am lost):

@bmeeks

If that is true ........ we are using a completely wrong pimd-version.
I mean not including the 380 pathes from trolobit. and not those they (FreeBSD) made themselfs.

I can hardly believe its that bad ......

Louis

The links I provided are to the current packages (current for FreeBSD). I don't know who "trolobit" is. Is he someone who has recently assumed pimd responsibility? Also, what FreeBSD changes are you referring to? Do you have a link to them?

I will say that it is not an uncommon occurrence for a binary package that is maintained primarily for say Linux to have its FreeBSD version sometimes lag behind. Developers usually have a "favorite" OS they target. Or perhaps in the past a FreeBSD enthusiast ported over the Linux version of pimd to FreeBSD, but then later lost interest and abandoned the FreeBSD project. Not saying this happened here, because I have no idea, but there are similar examples with other packages in the past. The fact the current FreeBSD official package seems to have last been updated in March of 2016 indicates probably some level of "abandonment" by the original FreeBSD package creator.

bmeeks

This is how FreeBSD packages work when building them manaually.

The Makefile included in a package has instructions for compiling and installing the package. One of the lines in that file specifies the URL to be used for downloading the source code tarball. There is also a VERSION, and optionally, a PORTREVISION line in the Makefile. That tells the package building process which version of the source code tarball to download. Other settings in the file determine which build options to use, what libraries are needed and so forth.

FreeBSD ports repos don't keep the actual C source files, for example. They instead consist of the Makefile, a pkg-plist file telling what files to install where after the package is built, and then an optional files directory containing any necessary patches. These are usually changes needed in order for the port to build on a FreeBSD system. But other patches may be included as well. So in the case of pimd, you could examine the files directory in the link I provided and see if any pertinent changes are included.

louis2

Here some IMHO relevant links .....

https://troglobit.com/projects/pimd/
https://github.com/troglobit/pimd/releases/tag/2.3.2
https://github.com/troglobit/pimd/commits?author=troglobit
https://github.com/troglobit/pimd
https://www.openhub.net/p/pimd/
https://www.freshports.org/net/pimd

freebsd pimd source ???

Something like that .....

Louis

louis2

Here the link to the recently fixed FreeBSD bug
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=246629

I think the actual patch is in
https://svnweb.freebsd.org/base?view=revision&revision=362472
https://svnweb.freebsd.org/base?view=revision&revision=362494

Directorystable/12/sys/netinet/ip_mroute.c

bmeeks

@louis2 said in Package Version Control, can anyone explain? (I am lost):

Here the link to the recently fixed FreeBSD bug
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=246629

I think the actual patch is in
https://svnweb.freebsd.org/base?view=revision&revision=362472
https://svnweb.freebsd.org/base?view=revision&revision=362494

Directorystable/12/sys/netinet/ip_mroute.c

Those are patches within the FreeBSD source code itself and not patches to pimd. Those kinds of patches are part of the FreeBSD versioning and won't be reflected in any pimd version info. pfSense-2.5 DEVEL is based on FereeBSD-12.1/STABLE, and is periodically "rebased" to pick up the latest changes to the STABLE code.

bmeeks

@louis2 said in Package Version Control, can anyone explain? (I am lost):

Here some IMHO relevant links .....

https://troglobit.com/projects/pimd/
https://github.com/troglobit/pimd/releases/tag/2.3.2
https://github.com/troglobit/pimd/commits?author=troglobit
https://github.com/troglobit/pimd
https://www.openhub.net/p/pimd/
https://www.freshports.org/net/pimd

freebsd pimd source ???

Something like that .....

Louis

These links reinforce (in my mind, anyway) what I expressed earlier. There is a generic pimd code base meant for use on multiple operating systems (Linux and FreeBSD, for example). So anyone running their own FreeBSD setup could download that pimd source code and compile it on their box. But that would only work for someone who has a FreeBSD setup with all the software development tools on it (compiler, linker, etc.). The pfSense firewall distro has none of that because installing all of that adds a large new attack surface via potential vulnerabilities in the various tools.

So in the past someone took that pimd code and ported it into a "package" in the FreeBSD ports tree. So there exists in the FreeBSD ports tree a precompiled package ready to be installed. Unfortunately, whoever created that original FreeBSD package has not maintained it. And it is now way behind. As I mentioned in an earlier post, many times package maintainers/creators are volunteers. They can sometimes disappear and the package they contributed languishes when nobody else steps up and picks it up.

So that is what appears to have happened with pimd. The package itself on FreeBSD is quite old and likely out-of-date (or at least not at the current patch level). That won't change unless someone steps up and takes over maintenance of the package in FreeBSD ports. If the package is maintained in FreeBSD ports, then the updated code will always get pulled into pfSense as well since it uses the binary ports straight from FreeBSD.

Finally, looking at the Github links you provided, it seems 2.3.2 is the current RELEASE, but that person (troglobit) has made 380 commits (code changes) to a branch he is calling Master since the release of the 2.3.2 version back in 2016. So the whole thing appears to be in somewhat of a mess as changes have been made (at least 380 of them via commits) since the initial release in 2016, but during that time the version has not been incremented. So the only way someone else could keep up is if they had a Github repo clone tracking his Master branch and they periodically did a "git pull upstream master" and then compiled a new binary daemon from the resulting updated source code.

And once more, those FreeBSD updates you linked are actually changes to the FreeBSD kernel source code. Those are going into pfSense-2.5 on some periodic basis now that 2.5-DEVEL is tracking FreeBSD-12.1/STABLE. But those are not the same as the 380 commits that have been pushed to the pimd Github repository.

Quarkz

https://github.com/pfsense/FreeBSD-src/commit/df4360fdf61dded94249adcffc5e2cc6b7b2702b

Pretty sure that FreeBSD fix was already committed to 2.5.0

Quarkz

On a side note, I’m seeing packages try to start multiple times as well, avahi is another example. Pretty much anything that ends up with a .sh file in /usr/local/etc/rc.d. PIMD is also one of those. I have tracked it back to code in /etc/rc.start_packages but haven’t had cycles to determine why packages are getting started twice yet.

louis2

That is another issue, but I completely agree. I allready raised that a couple of times. Packages starting and stopping many times, where they schould only be started ones and on the correct moment please! (after the OS is started and after pfSense basic functions are started). I did create a bug report related to that see bugzilla (I think there even multiple related bug reports).

Louis

louis2

Someone has to compile the PIMD-master as it is now, including all those bug fixes (380! +) , in favor of FreeBSD .....

That should preferable be someone who has already the required development setup ...... and experience ....

Then we have to do some testing to assure that it is at least running correctly in the majority of situations.

Can't hardly be worse than it is now

Louis

skogs

@louis2 said in Package Version Control, can anyone explain? (I am lost):

Someone has to compile the PIMD-master as it is now, including all those bug fixes (380! +) , in favor of FreeBSD .....

That should preferable be someone who has already the required development setup ...... and experience ....

Then we have to do some testing to assure that it is at least running correctly in the majority of situations.

Can't hardly be worse than it is now

Louis

...Agreed...the author of PIMD...and he should probably do a version bump sometime. Somebody else doing it may unnecessarily fork it, and perhaps the author hasn't done a version bump because he doesn't feel good about where it is at.