Making changes to haproxy package; how do I make them available to everyone?

marcelloc

Github.com is the place to Pull file changes request.

You can go to redmine and Open a ticket with your code as a feature

Briantist

I'm new to this stuff. There are changes in several of the text files, but also I am using a new binary that I compiled myself. I only created a FreeBSD 8.1 64 bit VM for compiling so I don't have the newer versions of the binary for 8.1 32 bit, or for 7.x, but if I really had to I guess I could create new VMs for that. Goes GitHub also provide for a way to include the new binaries? Thanks!

marcelloc

Take a look on files.pfsense.com and see if your version isn't there.

If don't, include ports compilation path to package XML and Open a redmine requesting compilation.

marcelloc

What kind of bugs did you found?

I saw some forum users having trouble after some hours/days.

Briantist

I plan to make a full writeup, but some of the bugs I found are:

Global advanced options don't work (misplaced base64 encode)
Cluster sync won't let you save the 2nd or 3rd member (forget which) due to a typo in the check
Cluster sync will throw loads of errors because the code that unsets the sync members before sending the config over was missing an array element
Monitor URI is incorrectly applied to TCP mode, which makes it impossible to balance a non-HTTP protocol, and it's a required field

There is a lot more that was changed that wouldn't exactly be called a bug. I don't have it all in front of me right now.

These are all bugs in the pfSense package though, not HAProxy itself. Could you link me to the threads where people were having issues? Don't know that I'll be able to do anything but I'll take a look while I'm involved with making changes anyway.

marcelloc

Take a look on postfix or varnish sync page. I've changed sync options to a list instead of fixed limit.

Briantist

@marcelloc:

Take a look on postfix or varnish sync page. I've changed sync options to a list instead of fixed limit.

I will definitely look at that! That's something I wanted to do, but I didn't know if I would have the time. I'll look at what you've done with those and try to get that implemented too.

Briantist

@Briantist:

@marcelloc:

Take a look on postfix or varnish sync page. I've changed sync options to a list instead of fixed limit.

I will definitely look at that! That's something I wanted to do, but I didn't know if I would have the time. I'll look at what you've done with those and try to get that implemented too.

I don't think I can use that code. You seem to be using edit_package.php (or whatever it is) and an XML file that lays out the pages. Unfortunately, HAProxy is not written like that and it's direct PHP, so if I change the UI to use a dynamic range of boxes, and I also have to change the way it's stored in the configuration and how that gets interpreted and processed. It's not impossible but I have to see how much time I have for that change.

Briantist

Okay, I had to take a short hiatus from this, but I think I'm ready to have these changes made public. Here's a list of what I've changed:

HAProxy Version and Platform Folders
I needed version 1.4.x of HAProxy and it wasn't available, so I compiled it myself. I went with the latest version at the time, which is 1.4.18. Since I am using a 64 bit pfSense, I created a FreeBSD 8.1 am64 environment and compiled it that way. The package as written didn't have a way to deal with this, so under the existing binaries8 folder I added the i386 and amd64 folders. I put the binary in the appropriate folder and modified the package code to detect the proper platform and pull from the appropriate place.

The only problem here is that I don't have an environment for 8.1 i386 or for 7.x (for 1.2.3).
<aftersaveredirect></aftersaveredirect>
Although I don't think this package uses it, there was a typo in this URL which I fixed.
New Server Status Options
I've added the backup and disabled options to the status dropdown. Disabled is an actual HAProxy option. I've kept inactive, and it does what it always did (leaves it in the package but does not write it out to the HAProxy config). Check HAProxy docs if you're unfamiliar with those options. They worked just fine previously by simply adding them to the advanced options but it's nice to use them in the UI.
Monitor URI is now optional
Monitor URI was a required field on the form, but putting something in there caused TCP type frontends to always be in failure unless they were actually load balancing an HTTP server. This option should be optional because it's not required even on HTTP.
XMLRPC Sync
The XMLRPC sync was broken in two ways. First, the code was attempting to remove the 3 members from the config it sent to others (so that they didn't try to send their configurations also) but the code was missing an array element so it never removed them. There was also a typo that prevented you from entering a second or third member (it would disappear on save).
New Stats Options
I added an option to show a node name in the stats. By default, if enabled, it will use the system's name (useful in a cluster where the config will be propagated) or you can override it with whatever you want.
You can also show a node description.
You can specify an auto-refresh, in any units that HAProxy supports (5s, 2d, 40000us).
New load balancing options
I added two new load balancing options to the frontends: static-rr and leastconn. There are other options that HAProxy supports but I didn't add them because they are more complex and would need additional fields to use them properly.
Ports text box
The box accepts a comma separated list of ports but the limit was set to 10 characters. It's easy to run into this limit with high ports. I changed the box size to 30 characters with a maxsize of 500 (displays 30, can fit up to 500).
Global Advanced Options
The advanced options in the global tab were not being saved correctly because of the placement of the base64 encoding (if you saved options it would write it to the config as plain text and then base64 decode it when it went to to display the data or write it out to haproxy config).
Changed Default Tab
Originally the frontends tab was the default, and saving on any tab brought you back there. I changed it to the global tab, because I felt it was more useful; I guess it's a personal preference.

So as I wrote in the original post, I have a local package repo where I'm making these changes.

I need someone to compile the freebsd 8.1 i386 package and the 7.x i386 binary for 1.2.3, and I need detailed instructions on how I get this included into the master repo. I've never done anything like this before; I don't know if there's a review process, where exactly I have to put the files, etc. I signed up an account on redmine though I'm not sure if that helps.

Thanks!

Edit: Added the default tab modification to the list.

stephenw10

Do you really need to support 1.2.3 anymore?

Steve

Briantist

@stephenw10:

Do you really need to support 1.2.3 anymore?

Steve

Well, I don't care about it too much. I still support some systems running it that can't be upgraded yet, but I don't plan on running HAProxy on them.

That being said, HAProxy currently does run on 1.2.3, and nothing I've changed prevents that. We only need an updated binary. I think I might just create the VMs for the other versions of FreeBSD and compile them myself if no one else wants to step up. Apparently pfSense 2.0.1 is coming out soon and will use FreeBSD 9? Maybe I'll do that one too.

I would really love some help on how to get all this back to the community though.

marcelloc

I can help. I will configure these vms and compile it.

If all you done was just fixes, i can publish it for you.
But you can create an account at Github and Pull your update request.

2.0.1 uses the same freebsd version 8.1

Briantist

@marcelloc:

I can help. I will configure these vms and compile it.

If all you done was just fixes, i can publish it for you.
But you can create an account at Github and Pull your update request.

2.0.1 uses the same freebsd version 8.1

Thanks marcelloc, you've been very helpful to me throughout this whole process. I wrote out everything that was changed. As you can see there are fixes as well as new features, but I believe that someone who upgrades from the current package to this new one will not see any issue. I can also post the actual code changes if you want to go over them in detail before publishing.

I have created a free account at github but I don't know what to do from there. I don't know what it means to pull my update request. Sorry for my ignorance on that! :)

Briantist

Okay I've asked my manager about it as he's more familiar with git and github so I have an idea of what I need to do with that. I'll work on it and I'll post again once I'm done or when I have more questions. Thanks!

marcelloc

the repo for packages is https://github.com/bsdperimeter/pfsense-packages

marcelloc

the ftp-archive has 1.4.x haproxy packager version. Test if any of these versions works on your pfsense

amd64 (tested with no issues)
(8.1 packages)
pkg_add -r http://ftp-archive.freebsd.org/pub/FreeBSD-Archive/ports/amd64/packages-8.1-release/All/haproxy-1.4.8.tbz
(8.2 packages)
pkg_add -r http://ftp-archive.freebsd.org/pub/FreeBSD-Archive/ports/amd64/packages-8.2-release/All/haproxy-1.4.10.tbz
(8-stable packages)
pkg_add -r ftp://ftp.freebsd.org/pub/FreeBSD/ports/amd64/packages-8-stable/All/haproxy-1.4.16.tbz

I386
(8.1 packages)
pkg_add -r http://ftp-archive.freebsd.org/pub/FreeBSD-Archive/ports/i386/packages-8.1-release/All/haproxy-1.4.8.tbz
(8.2 packages)
pkg_add -r http://ftp-archive.freebsd.org/pub/FreeBSD-Archive/ports/i386/packages-8.2-release/All/haproxy-1.4.10.tbz
(8-stable packages)
pkg_add -r ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-8-stable/All/haproxy-1.4.16.tbz

for 1.2.3 version(need test)
http://ftp-archive.freebsd.org/pub/FreeBSD-Archive/ports/i386/packages-7.4-release/All/haproxy-1.4.10.tbz

Briantist

We did try 1.4.16 but there was something about it that didn't work correctly with RPC/MAPI (we're load balancing Exchange 2010). Whatever it was, it worked when we used 1.4.18 without any changes to the config.

marcelloc

ok, I will build it from ports.

Briantist

Okay, I've created the github repo:
https://github.com/briantist/pfSense-Packages

Please let me know if I've done something wrong with it or whatever. What should I do next? I see a pull request option in my repo, but when I click it I don't really understand what it's asking.

marcelloc

Using the gui, browse https://github.com/bsdperimeter/pfsense-packages and find file you want to change, click edit this file.

After this, github will clone pfsense-package and allow you edit the file and pull file change request.