Working with Packages on 2.3 and Converting to Bootstrap
-
I have collected and combined a lot of information and added some more to come up with what I hope is a good set of info to help others make the jump into working with packages on 2.3 and converting things to bootstrap as needed.
Basic info for those who have not yet done any development work:
https://doc.pfsense.org/index.php/Getting_Started_with_pfSense_Development
https://doc.pfsense.org/index.php/Contributor_License_Agreement_for_DevelopersOverview of the new pfSense Package port directory structure:
https://doc.pfsense.org/index.php/Package_Port_Directory_StructureA list of current packages on 2.3 and where they are located inside the ports repository:
https://doc.pfsense.org/index.php/Package_Port_ListUpdated doc about Developing Packages with 2.3-specific info:
https://doc.pfsense.org/index.php/Developing_PackagesNotes about converting packages to bootstrap:
https://doc.pfsense.org/index.php/Converting_Packages_to_Bootstrap2.3 Package Conversion and Operation Status
Here is a Google Docs sheet with the current list of 2.3 packages and their status. Many use XML forms and do not need any help (which is good!) but still need testing to confirm they function as expected on 2.3.The master ticket in Redmine for the conversion of packages is here: https://redmine.pfsense.org/issues/5568
Let me know if anything seems to be missing or if there are any specific topics missed that need covered.
Thanks to all for their effort!
-
bmeeks, I was thinking of taking a shot at converting Suricata so I had somewhere to dive in if you need a hand.
-
I added a link above to a Google docs sheet with the current status of the packages we have moved to 2.3 already. Many do not need conversion but still could use testing.
Feel free to report success or failure here in this thread but if you do find a problem please start a new thread to investigate the problem to avoid crosstalk this thread.
-
bmeeks, I was thinking of taking a shot at converting Suricata so I had somewhere to dive in if you need a hand.
Thank you! I will be glad for the help and to help you. Suricata has not been started. I had been working on Snort until I took a break to work on the netmap stuff. Send me a PM and we can swap e-mail addresses (if you don't have mine already). Will be easier probably to correspond that way or by phone even.
Bill
-
bmeeks, I was thinking of taking a shot at converting Suricata so I had somewhere to dive in if you need a hand.
Thank you! I will be glad to help. Suricata has not been started. I had been working on Snort until I took a break to work on the netmap stuff. Send me a PM and we can swap e-mail addresses (if you don't have mine already). Will be easier probably to correspond that way or by phone even.
Bill
Will do. I figured it would be best to attempt a smaller package, like cron, so I can get the process nailed down before moving onto something larger like Suricata. I'll be in touch once I get that done and we can come up with a plan of attack.
-
Guys… those packages that are adding their own user/group accounts - is that now safe to remove that code?
-
Such as?
If it's for a binary package, at least in theory, the pkg code should be handling that properly. I know PBIs had some issues there. When in doubt, pkg add just the underlying program (not the pfSense pkg) and check for the user after.
-
Such as?
If it's for a binary package, at least in theory, the pkg code should be handling that properly. I know PBIs had some issues there. When in doubt, pkg add just the underlying program (not the pfSense pkg) and check for the user after.
Well, the first one I hit was Avahi. Then, there's the infamous Squid thing with the ClamAV stuff where PBI was adding something with wrong UID/GID and the proxy user/group confusion/mess. Also, I recall there vaguely there was some issue related to the UID/GIDs handling with certain range being reserved for pfSense user manager and the accounts removed on reboot???
-
Another note - I don't think this package makes any sense? Never had any GUI.
https://github.com/pfsense/FreeBSD-ports/tree/devel/net-mgmt/pfSense-pkg-iftop
-
Another note - I don't think this package makes any sense? Never had any GUI.
https://github.com/pfsense/FreeBSD-ports/tree/devel/net-mgmt/pfSense-pkg-iftop
It's just there so it gets built and available in the repo. It's console only and very handy to have available.
-
Such as?
If it's for a binary package, at least in theory, the pkg code should be handling that properly. I know PBIs had some issues there. When in doubt, pkg add just the underlying program (not the pfSense pkg) and check for the user after.
Well, the first one I hit was Avahi. Then, there's the infamous Squid thing with the ClamAV stuff where PBI was adding something with wrong UID/GID and the proxy user/group confusion/mess. Also, I recall there vaguely there was some issue related to the UID/GIDs handling with certain range being reserved for pfSense user manager and the accounts removed on reboot???
None of the "official" users would collide with pfSense:
https://github.com/pfsense/FreeBSD-ports/blob/devel/UIDs
https://github.com/pfsense/FreeBSD-ports/blob/devel/GIDsPackages in FreeBSD are supposed to specify their users in the Makefile, and the specifics of UID/GID/etc are pulled from the above files. For example:
https://github.com/pfsense/FreeBSD-ports/blob/devel/www/squid/Makefile#L36I don't think we should have any code in the .inc files or elsewhere to manually make users any longer, if one isn't in the above list we should add it in and also tweak the pfSense package Makefile appropriately.
-
I don't think we should have any code in the .inc files or elsewhere to manually make users any longer, if one isn't in the above list we should add it in and also tweak the pfSense package Makefile appropriately.
OK, sounds good. Junk nuked. :)
-
I had a bit of a look at this task and didn't notice any mention of anything relating to REST..
I noticed the pfSense blog post about converting to REST in 3.0 (https://blog.pfsense.org/?p=1588) - its quite old though.. Is this still the plan?
While in there changing the PHP pages it could be good if people were taking this into account as much as possible..
Something like:
-
Split functional code out into a [file]_rest.php that returns JSON
-
Change HTML page to make REST HTTP requests to execute their functions
Doing this now could get the process of converting to REST early (unless 3 is the next release directly after 2.3?).
I was taking a look at the sipproxd and it seems relatively small and thinking - depending on the time frame that packages need to be converted in - I might be able to take a look at converting that one in this manner? Could give a basic template to start moving to the REST structure..
I'm primary a web developer and work a lot with REST APIs (built in Django) and web front ends (the Ember JavaScript framework)
-
-
REST will be for 3.0 only and some time off (we haven't even began drafting the API) – Seems like overkill to start all that now when we haven't decided on what we'll be using for any of that.
We'll be on 2.3 for a while, most likely there will even be a 2.4, that blog post is long-term goals.
-
I had a bit of a look at this task and didn't notice any mention of anything relating to REST..
I noticed the pfSense blog post about converting to REST in 3.0 (https://blog.pfsense.org/?p=1588) - its quite old though.. Is this still the plan?
While in there changing the PHP pages it could be good if people were taking this into account as much as possible..
Something like:
-
Split functional code out into a [file]_rest.php that returns JSON
-
Change HTML page to make REST HTTP requests to execute their functions
Doing this now could get the process of converting to REST early (unless 3 is the next release directly after 2.3?).
I was taking a look at the sipproxd and it seems relatively small and thinking - depending on the time frame that packages need to be converted in - I might be able to take a look at converting that one in this manner? Could give a basic template to start moving to the REST structure..
I'm primary a web developer and work a lot with REST APIs (built in Django) and web front ends (the Ember JavaScript framework)
I agree with Jimp, we should focus on getting 2.3 out to prevent what could be massive scope creep. Not saying it isn't important though. I was actually looking at make graph.php utilize an API over the weekend so that I could convert the traffic graphs to d3.j3 using JSON for the datasource. If you want to take a stab at converting various small data points to JSON APIs and need someone to bounce ideas off of let me know. I could probably squeeze a little bit of time in the week to hack on stuff like that.
-
-
NUT?
Let me know if anything seems to be missing or if there are any specific topics missed that need covered.
-
Not addressing missing packages yet. The ones we have listed must all be complete first before we can consider expanding that list.
-
Okay, thanks Jim.
Not addressing missing packages yet. The ones we have listed must all be complete first before we can consider expanding that list.
-
Overview of the new pfSense Package port directory structure:
https://doc.pfsense.org/index.php/Package_Port_Directory_StructureLooks great!!!
I've send the first pull request to postfix package. Structure looks extremely easier to port packages to pfsense. Excelent work!!!!!!
-
I've send the first pull request to postfix package.
Still not showing up under Available Packages
