Making changes to haproxy package; how do I make them available to everyone?

Briantist

Okay, I had to take a short hiatus from this, but I think I'm ready to have these changes made public. Here's a list of what I've changed:

HAProxy Version and Platform Folders
I needed version 1.4.x of HAProxy and it wasn't available, so I compiled it myself. I went with the latest version at the time, which is 1.4.18. Since I am using a 64 bit pfSense, I created a FreeBSD 8.1 am64 environment and compiled it that way. The package as written didn't have a way to deal with this, so under the existing binaries8 folder I added the i386 and amd64 folders. I put the binary in the appropriate folder and modified the package code to detect the proper platform and pull from the appropriate place.

The only problem here is that I don't have an environment for 8.1 i386 or for 7.x (for 1.2.3).
<aftersaveredirect></aftersaveredirect>
Although I don't think this package uses it, there was a typo in this URL which I fixed.
New Server Status Options
I've added the backup and disabled options to the status dropdown. Disabled is an actual HAProxy option. I've kept inactive, and it does what it always did (leaves it in the package but does not write it out to the HAProxy config). Check HAProxy docs if you're unfamiliar with those options. They worked just fine previously by simply adding them to the advanced options but it's nice to use them in the UI.
Monitor URI is now optional
Monitor URI was a required field on the form, but putting something in there caused TCP type frontends to always be in failure unless they were actually load balancing an HTTP server. This option should be optional because it's not required even on HTTP.
XMLRPC Sync
The XMLRPC sync was broken in two ways. First, the code was attempting to remove the 3 members from the config it sent to others (so that they didn't try to send their configurations also) but the code was missing an array element so it never removed them. There was also a typo that prevented you from entering a second or third member (it would disappear on save).
New Stats Options
I added an option to show a node name in the stats. By default, if enabled, it will use the system's name (useful in a cluster where the config will be propagated) or you can override it with whatever you want.
You can also show a node description.
You can specify an auto-refresh, in any units that HAProxy supports (5s, 2d, 40000us).
New load balancing options
I added two new load balancing options to the frontends: static-rr and leastconn. There are other options that HAProxy supports but I didn't add them because they are more complex and would need additional fields to use them properly.
Ports text box
The box accepts a comma separated list of ports but the limit was set to 10 characters. It's easy to run into this limit with high ports. I changed the box size to 30 characters with a maxsize of 500 (displays 30, can fit up to 500).
Global Advanced Options
The advanced options in the global tab were not being saved correctly because of the placement of the base64 encoding (if you saved options it would write it to the config as plain text and then base64 decode it when it went to to display the data or write it out to haproxy config).
Changed Default Tab
Originally the frontends tab was the default, and saving on any tab brought you back there. I changed it to the global tab, because I felt it was more useful; I guess it's a personal preference.

So as I wrote in the original post, I have a local package repo where I'm making these changes.

I need someone to compile the freebsd 8.1 i386 package and the 7.x i386 binary for 1.2.3, and I need detailed instructions on how I get this included into the master repo. I've never done anything like this before; I don't know if there's a review process, where exactly I have to put the files, etc. I signed up an account on redmine though I'm not sure if that helps.

Thanks!

Edit: Added the default tab modification to the list.

stephenw10

Do you really need to support 1.2.3 anymore?

Steve

Briantist

@stephenw10:

Do you really need to support 1.2.3 anymore?

Steve

Well, I don't care about it too much. I still support some systems running it that can't be upgraded yet, but I don't plan on running HAProxy on them.

That being said, HAProxy currently does run on 1.2.3, and nothing I've changed prevents that. We only need an updated binary. I think I might just create the VMs for the other versions of FreeBSD and compile them myself if no one else wants to step up. Apparently pfSense 2.0.1 is coming out soon and will use FreeBSD 9? Maybe I'll do that one too.

I would really love some help on how to get all this back to the community though.

marcelloc

I can help. I will configure these vms and compile it.

If all you done was just fixes, i can publish it for you.
But you can create an account at Github and Pull your update request.

2.0.1 uses the same freebsd version 8.1

Briantist

@marcelloc:

I can help. I will configure these vms and compile it.

If all you done was just fixes, i can publish it for you.
But you can create an account at Github and Pull your update request.

2.0.1 uses the same freebsd version 8.1

Thanks marcelloc, you've been very helpful to me throughout this whole process. I wrote out everything that was changed. As you can see there are fixes as well as new features, but I believe that someone who upgrades from the current package to this new one will not see any issue. I can also post the actual code changes if you want to go over them in detail before publishing.

I have created a free account at github but I don't know what to do from there. I don't know what it means to pull my update request. Sorry for my ignorance on that! :)

Briantist

Okay I've asked my manager about it as he's more familiar with git and github so I have an idea of what I need to do with that. I'll work on it and I'll post again once I'm done or when I have more questions. Thanks!

marcelloc

the repo for packages is https://github.com/bsdperimeter/pfsense-packages

marcelloc

the ftp-archive has 1.4.x haproxy packager version. Test if any of these versions works on your pfsense

amd64 (tested with no issues)
(8.1 packages)
pkg_add -r http://ftp-archive.freebsd.org/pub/FreeBSD-Archive/ports/amd64/packages-8.1-release/All/haproxy-1.4.8.tbz
(8.2 packages)
pkg_add -r http://ftp-archive.freebsd.org/pub/FreeBSD-Archive/ports/amd64/packages-8.2-release/All/haproxy-1.4.10.tbz
(8-stable packages)
pkg_add -r ftp://ftp.freebsd.org/pub/FreeBSD/ports/amd64/packages-8-stable/All/haproxy-1.4.16.tbz

I386
(8.1 packages)
pkg_add -r http://ftp-archive.freebsd.org/pub/FreeBSD-Archive/ports/i386/packages-8.1-release/All/haproxy-1.4.8.tbz
(8.2 packages)
pkg_add -r http://ftp-archive.freebsd.org/pub/FreeBSD-Archive/ports/i386/packages-8.2-release/All/haproxy-1.4.10.tbz
(8-stable packages)
pkg_add -r ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-8-stable/All/haproxy-1.4.16.tbz

for 1.2.3 version(need test)
http://ftp-archive.freebsd.org/pub/FreeBSD-Archive/ports/i386/packages-7.4-release/All/haproxy-1.4.10.tbz

Briantist

We did try 1.4.16 but there was something about it that didn't work correctly with RPC/MAPI (we're load balancing Exchange 2010). Whatever it was, it worked when we used 1.4.18 without any changes to the config.

marcelloc

ok, I will build it from ports.

Briantist

Okay, I've created the github repo:
https://github.com/briantist/pfSense-Packages

Please let me know if I've done something wrong with it or whatever. What should I do next? I see a pull request option in my repo, but when I click it I don't really understand what it's asking.

marcelloc

Using the gui, browse https://github.com/bsdperimeter/pfsense-packages and find file you want to change, click edit this file.

After this, github will clone pfsense-package and allow you edit the file and pull file change request.

Briantist

@marcelloc:

Using the gui, browse https://github.com/bsdperimeter/pfsense-packages and find file you want to change, click edit this file.

After this, github will clone pfsense-package and allow you edit the file and pull file change request.

When you say "the gui" do you just mean to browse that URL in a browser? When I go to one of the file that way, there is a button that says "fork and edit". Is that the one to click? Do I have to do this to every file individually?

Again, sorry for being so green. I really appreciate your patience.

marcelloc

yes, press fork and edit.
this way you get a clone from pfsense-packages.

Then you choose between edit each file individually or apply via git on your clone and then pull all file changes in a single request.

Briantist

Okay, I think I'm almost there:
https://github.com/briantist/pfsense-packages-1

I used to git to put all the changes up to this repo at once. I started to do the pull request but I got scared when it said that I was asking bsdperimeter to pull in 333 commits, most of which looked like they were from other people, so I thought maybe I was doing something wrong.

marcelloc

try to edit each file and see what happens.

Briantist

Should I be editing it on my fork, or in the main repo?

marcelloc

choose file on main repo. then fork and edit

Briantist

Okay I did it all one file at a time. There should be 6 new pull requests. I can't add the i386 folder under the binaries7 folder, nor the i386 and amd64 folders underneath binaries8 but they will need to be there.

If there's anything else I need to do please let me know. Thanks so much!

marcelloc

test compiled version

i386
http://e-sac.siteseguro.ws/pfsense/8/All/haproxy-1.4.16.tbz

amd64
http://e-sac.siteseguro.ws/pfsense/8/amd64/All/haproxy-1.4.16.tbz