Making changes to haproxy package; how do I make them available to everyone?



  • I've been working on changes to the HAProxy package. I've updated the version, made several bug fixes, and enhancements. I've created a local repo for packages and have pointed all four of my pfSense boxes at it, so I've been making changes on my repo and then pulling the package down from there. I'm not completely done yet but I'm getting there, and I want to know what the procedure is for getting these changes put into the "global" repo so that everyone has access to them. I imagine there's some process where I submit the code and it gets reviewed by some of the main developers of the project? I still have some stuff to do but as I get close I want to make sure I'm doing things the right way.



  • Github.com is the place to Pull file changes request.

    You can go to  redmine and Open a ticket with your code as a feature



  • I'm new to this stuff. There are changes in several of the text files, but also I am using a new binary that I compiled myself. I only created a FreeBSD 8.1 64 bit VM for compiling so I don't have the newer versions of the binary for 8.1 32 bit, or for 7.x, but if I really had to I guess I could create new VMs for that. Goes GitHub also provide for a way to include the new binaries? Thanks!



  • Take a look on files.pfsense.com and see if your version isn't there.

    If don't, include ports compilation path to package XML and Open a redmine requesting compilation.



  • What kind of bugs did you found?

    I saw some forum users having trouble after some hours/days.



  • I plan to make a full writeup, but some of the bugs I found are:

    • Global advanced options don't work (misplaced base64 encode)

    • Cluster sync won't let you save the 2nd or 3rd member (forget which) due to a typo in the check

    • Cluster sync will throw loads of errors because the code that unsets the sync members before sending the config over was missing an array element

    • Monitor URI is incorrectly applied to TCP mode, which makes it impossible to balance a non-HTTP protocol, and it's a required field

    There is a lot more that was changed that wouldn't exactly be called a bug. I don't have it all in front of me right now.

    These are all bugs in the pfSense package though, not HAProxy itself. Could you link me to the threads where people were having issues? Don't know that I'll be able to do anything but I'll take a look while I'm involved with making changes anyway.



  • Take a look on postfix or varnish sync page. I've changed sync options to a list instead of fixed limit.



  • @marcelloc:

    Take a look on postfix or varnish sync page. I've changed sync options to a list instead of fixed limit.

    I will definitely look at that! That's something I wanted to do, but I didn't know if I would have the time. I'll look at what you've done with those and try to get that implemented too.



  • @Briantist:

    @marcelloc:

    Take a look on postfix or varnish sync page. I've changed sync options to a list instead of fixed limit.

    I will definitely look at that! That's something I wanted to do, but I didn't know if I would have the time. I'll look at what you've done with those and try to get that implemented too.

    I don't think I can use that code. You seem to be using edit_package.php (or whatever it is) and an XML file that lays out the pages. Unfortunately, HAProxy is not written like that and it's direct PHP, so if I change the UI to use a dynamic range of boxes, and I also have to change the way it's stored in the configuration and how that gets interpreted and processed. It's not impossible but I have to see how much time I have for that change.



  • Okay, I had to take a short hiatus from this, but I think I'm ready to have these changes made public. Here's a list of what I've changed:

    • HAProxy Version and Platform Folders
      I needed version 1.4.x of HAProxy and it wasn't available, so I compiled it myself. I went with the latest version at the time, which is 1.4.18. Since I am using a 64 bit pfSense, I created a FreeBSD 8.1 am64 environment and compiled it that way. The package as written didn't have a way to deal with this, so under the existing binaries8 folder I added the i386 and amd64 folders. I put the binary in the appropriate folder and modified the package code to detect the proper platform and pull from the appropriate place.

      The only problem here is that I don't have an environment for 8.1 i386 or for 7.x (for 1.2.3).

    • <aftersaveredirect></aftersaveredirect>
      Although I don't think this package uses it, there was a typo in this URL which I fixed.

    • New Server Status Options
      I've added the backup and disabled options to the status dropdown. Disabled is an actual HAProxy option. I've kept inactive, and it does what it always did (leaves it in the package but does not write it out to the HAProxy config). Check HAProxy docs if you're unfamiliar with those options. They worked just fine previously by simply adding them to the advanced options but it's nice to use them in the UI.

    • Monitor URI is now optional
      Monitor URI was a required field on the form, but putting something in there caused TCP type frontends to always be in failure unless they were actually load balancing an HTTP server. This option should be optional because it's not required even on HTTP.

    • XMLRPC Sync
      The XMLRPC sync was broken in two ways. First, the code was attempting to remove the 3 members from the config it sent to others (so that they didn't try to send their configurations also) but the code was missing an array element so it never removed them. There was also a typo that prevented you from entering a second or third member (it would disappear on save).

    • New Stats Options
      I added an option to show a node name in the stats. By default, if enabled, it will use the system's name (useful in a cluster where the config will be propagated) or you can override it with whatever you want.
      You can also show a node description.
      You can specify an auto-refresh, in any units that HAProxy supports (5s, 2d, 40000us).

    • New load balancing options
      I added two new load balancing options to the frontends: static-rr and leastconn. There are other options that HAProxy supports but I didn't add them because they are more complex and would need additional fields to use them properly.

    • Ports text box
      The box accepts a comma separated list of ports but the limit was set to 10 characters. It's easy to run into this limit with high ports. I changed the box size to 30 characters with a maxsize of 500 (displays 30, can fit up to 500).

    • Global Advanced Options
      The advanced options in the global tab were not being saved correctly because of the placement of the base64 encoding (if you saved options it would write it to the config as plain text and then base64 decode it when it went to to display the data or write it out to haproxy config).

    • Changed Default Tab
      Originally the frontends tab was the default, and saving on any tab brought you back there. I changed it to the global tab, because I felt it was more useful; I guess it's a personal preference.

    So as I wrote in the original post, I have a local package repo where I'm making these changes.

    I need someone to compile the freebsd 8.1 i386 package and the 7.x i386 binary for 1.2.3, and I need detailed instructions on how I get this included into the master repo. I've never done anything like this before; I don't know if there's a review process, where exactly I have to put the files, etc. I signed up an account on redmine though I'm not sure if that helps.

    Thanks!

    Edit: Added the default tab modification to the list.


  • Netgate Administrator

    Do you really need to support 1.2.3 anymore?

    Steve



  • @stephenw10:

    Do you really need to support 1.2.3 anymore?

    Steve

    Well, I don't care about it too much. I still support some systems running it that can't be upgraded yet, but I don't plan on running HAProxy on them.

    That being said, HAProxy currently does run on 1.2.3, and nothing I've changed prevents that. We only need an updated binary. I think I might just create the VMs for the other versions of FreeBSD and compile them myself if no one else wants to step up. Apparently pfSense 2.0.1 is coming out soon and will use FreeBSD 9? Maybe I'll do that one too.

    I would really love some help on how to get all this back to the community though.



  • I can help. I will configure these vms and compile it.

    If all you done was just fixes, i can publish it for you.
    But you can create an account at Github and Pull your update request.

    2.0.1 uses the same freebsd version 8.1



  • @marcelloc:

    I can help. I will configure these vms and compile it.

    If all you done was just fixes, i can publish it for you.
    But you can create an account at Github and Pull your update request.

    2.0.1 uses the same freebsd version 8.1

    Thanks marcelloc, you've been very helpful to me throughout this whole process. I wrote out everything that was changed. As you can see there are fixes as well as new features, but I believe that someone who upgrades from the current package to this new one will not see any issue. I can also post the actual code changes if you want to go over them in detail before publishing.

    I have created a free account at github but I don't know what to do from there. I don't know what it means to pull my update request. Sorry for my ignorance on that!  :)



  • Okay I've asked my manager about it as he's more familiar with git and github so I have an idea of what I need to do with that. I'll work on it and I'll post again once I'm done or when I have more questions. Thanks!







  • We did try 1.4.16 but there was something about it that didn't work correctly with RPC/MAPI (we're load balancing Exchange 2010). Whatever it was, it worked when we used 1.4.18 without any changes to the config.



  • ok, I will build it from ports.



  • Okay, I've created the github repo:
    https://github.com/briantist/pfSense-Packages

    Please let me know if I've done something wrong with it or whatever. What should I do next? I see a pull request option in my repo, but when I click it I don't really understand what it's asking.



  • Using the gui, browse https://github.com/bsdperimeter/pfsense-packages and find file you want to change, click edit this file.

    After this, github will clone pfsense-package and allow you edit the file and pull file change request.



  • @marcelloc:

    Using the gui, browse https://github.com/bsdperimeter/pfsense-packages and find file you want to change, click edit this file.

    After this, github will clone pfsense-package and allow you edit the file and pull file change request.

    When you say "the gui" do you just mean to browse that URL in a browser? When I go to one of the file that way, there is a button that says "fork and edit". Is that the one to click? Do I have to do this to every file individually?

    Again, sorry for being so green. I really appreciate your patience.



  • yes, press fork and edit.
    this way you get a clone from pfsense-packages.

    Then you choose between edit each file individually or apply via git on your clone and then pull all file changes in a single request.



  • Okay, I think I'm almost there:
    https://github.com/briantist/pfsense-packages-1

    I used to git to put all the changes up to this repo at once. I started to do the pull request but I got scared when it said that I was asking bsdperimeter to pull in 333 commits, most of which looked like they were from other people, so I thought maybe I was doing something wrong.



  • try to edit each file and see what happens.



  • Should I be editing it on my fork, or in the main repo?



  • choose file on main repo. then fork and edit



  • Okay I did it all one file at a time. There should be 6 new pull requests. I can't add the i386 folder under the binaries7 folder, nor the i386 and amd64 folders underneath binaries8 but they will need to be there.

    If there's anything else I need to do please let me know. Thanks so much!





  • Version should be 1.4.18.. any reason why it can't be? As I said we tried 1.4.16 before, even tried it on a linux VM to see it was something platform specific.



  • This is the freebsd ports version. not so easy to compile and build a package without ports. :-\

    did you tried package modifications with haproxy devel version 1.5?





  • I was going to say, I had no problem compiling 1.4.18, but you've already completed it. That's great. Is there anything else I need to do?



  • Install package 0.3 on a pfsense other then your production server and test it.

    I'll check here too.

    When all tests are done, I'll change version to 1.0 release.



  • @Briantist:

    We did try 1.4.16 but there was something about it that didn't work correctly with RPC/MAPI (we're load balancing Exchange 2010). Whatever it was, it worked when we used 1.4.18 without any changes to the config.

    The RPC/MAPI you use with 1.4.18 is for owa or all exchange services? Can I replace Micro$oft NLB with haproxy?

    It could be very usefull to me. Exchange NLB freaks out my network everytime I enable it.



  • All exchange services. We are using this for that on RPC/MAPI, OWA/EWS (both the HTTPS access and the HTTP listener which redirects to HTTPS), IMAP, POP3, SMTP (both internal and external).

    I and a few co-workers have been eating our own dog food by running our own Outlook clients through our pfSense HAProxy setup for the better part of a month now and it's working great.

    About 50% of our desktops are Mac and are running Outlook 2011, which uses EWS for all of its mail access, and in my limited testing so far it seems to work well that way too.

    NLB is pretty crappy, so yeah we're definitely looking forward to replacing it. Once this package goes live, we're going to be doing some strict penetration testing since our pfSense cluster straddles our internal and DMZ (so that it can deal with external SMTP), and then we're going to get the rest of our group and department on it before rolling it out for the whole organization.

    Once I get it all set I'll write up a post about it.

    I'm doing some limited testing here at home of 0.3 version. Tomorrow when I get into work I will load it on there and try it out (it's not production yet, just the few people including myself who are using it live for ourselves) and I'll report back.



  • All exchange services. We are using this for that on RPC/MAPI, OWA/EWS

    Great I`ll test too, any specific balance option to do this?

    Once this package goes live, we're going to be doing some strict penetration testing since our pfSense cluster straddles our internal and DMZ (so that it can deal with external SMTP)

    Try postfix forwarder package, it works really nice together with exchange. it keeps out more then 80% misconfigured/fake spam servers and protects your exchange servers from internet.



  • The balance option depends on the service. For OWA, use source because you need to keep the same client on the same CAS server and since it's HTTPS you can't insert a tracking cookie. For SMTP/POP3/IMAP use round robin. RPC is a bit complex because by default Exchange uses three different services over RPC (the endpoint mapper, the address book, and MAPI) and a giant range of ports for RPC. You have to make changes so that the address book and MAPI use a single port. Then you need to create a separate frontend for each of those and add advanced options to keep the client connections on the same CAS servers (technically you don't need to create separate frontends to do this in HAProxy, but with the way it's implemented in pfSense you do). For the MAPI stuff I'm using the newly added leastconn balance option, but I was using round robin before that. I think that the advanced options end up overriding it anyway so I'm not sure it makes a difference.

    It'll be clearer once I have time to put together a real write-up.

    As for spam protection, we have that covered for now with a single appliance. Once this is in place, we plan making use of multiple spam gateways which we will also load balance with this setup.



  • Also one quick question before I go to sleep, how big is your exchange environment?



  • There are 14 exchange servers distributed in some locations, but my problem is with 02 exchange servers in the main site.

    Total mailboxes are 60k.


Locked