HAProxy package overwrite



  • Back in October I started work on the HAProxy package. In November, I posted a thread about it, asking how to make the changes public:
    http://forum.pfsense.org/index.php/topic,43188.0.html

    That thread kept on through January, where I discussed my changes and worked with Marcello on making those changes public. I documented all of the changes on the documentation wiki:
    http://doc.pfsense.org/index.php/Haproxy_package

    My package maintained backward compatibility with the previous version, so that existing configurations worked with with the new changes. It was made live, everything seemed fine, no complaints.

    Around the end of January Marcello PMed me to tell me that Andrew Thompson published a new version of HAproxy that overwrote mine and did not include my changes:
    https://github.com/bsdperimeter/pfsense-packages/commit/c0d7d735d6da067633da3fe4b0dd2db9d19b36db

    There was no thread posted, and these changes are not documented anywhere. We're nearly a month out from this release and there is still no documentation on it or mention of it on the forums (I may have missed it if there was a post but I was looking out). This release breaks compatibility with existing configurations.

    I began talking with Andrew in the comments on the github page linked above, but I have not heard back in weeks and nothing has changed.

    I understand that maybe Andrew is a core developer and did not need anyone's permission to overwrite these files, but I have to wonder if there is anything in place to prevent things like this from happening, and how this can be resolved.

    Personally, I think that his version should be forked since it does not keep compatibility and doesn't include the latest fixes. My changes are documented if he wants to add my changes to his, but I have nothing to go on with his changes other than to reverse engineer them. The wiki contains the high level overview of my changes but I already have details of every line of code that was changed documented in a confluence wiki and I can provide those pages as PDFs.

    If need be, I'll put my version as a separate package instead of his, but I think it should be the other way around.

    I've been a fan of this project for a long time, and have just started actually contributing anything to it. I'm just really surprised that the package management has been so sloppy, and I'd like to be a part of the process of making it better. I'm hoping to hear from Andrew and anyone else on the core team about this. Thanks.



  • How do we install your package currently?



  • +1, I built my HA cluster with Briantist's v1.0. It was working great.

    I recently bumped the package up to v1.2 on my backup node which destroyed the config.

    Can we please get Briantist's version back?



  • I'll do it today.

    To do not conflict with current version, I'll release it as haproxy-legacy.



  • I've just published haproxy legacy version with Briantist updates.




  • I would prefer a name other than legacy, since the package is quite up-to-date, and I plan on releasing new fixes and features that will make it very full-featured. Maybe we can take a nod from the beverage industry and call is HAProxy Classic  ;)

    Thanks Marcello, couldn't have done any of this without you. Thanks joem for your support as well.



  • I also want to say I'm dismayed that there has been no response from the admins or people who run the pfSense project. I really am amazed that this situation was possible in the first place.



  • As it was published on ports

    I've just pushed haproxy 1.4.19 with latest bug fixes.

    Until we find a way to merge both packages, I've updated haproxy and haproxy-legacy.



  • Hi marcelloc, thank you very much for your assistance and quick reply. Thank you as well Briantist for your hard work and updates to this package.

    I removed v1.2 and attempted to install haproxy-legacy this evening, but I receive the following download failed message:

    Beginning package installation for haproxy-legacy…
    Downloading package configuration file... done.
    Saving updated package information... done.
    Downloading haproxy-legacy and its dependencies...
    Checking for package installation...
    Downloading http://e-sac.siteseguro.ws/pfsense/8/All/haproxy-1.4.19.tbz ...  could not download from there or http://ftp2.FreeBSD.org/pub/FreeBSD/ports/i386/packages-8.1-release/All/haproxy-1.4.19.tbz.
    of haproxy-1.4.19 failed!

    I am running 2.0.1-RELEASE (i386). Any advice you could share would be appreciated. Thank you again.



  • fixed. Try to install again.



  • Any chance of a name change that doesn't involve the word legacy? It's confusing because it implies that the package is not current, or is deprecated..



  • I think That I should rename it to Classic instead of legacy as you suggested.



  • The installation for amd64 still fails, it seems the previous fix was only for i386.

    Beginning package installation for haproxy…
    Downloading package configuration file... done.
    Saving updated package information... done.
    Downloading haproxy and its dependencies...
    Checking for package installation...
    Downloading http://e-sac.siteseguro.ws/packages/8/amd64/All/haproxy-1.4.19.tbz ...  could not download from there or http://ftp2.FreeBSD.org/pub/FreeBSD/ports/amd64/packages-8.1-release/All/haproxy-1.4.19.tbz.
    of haproxy-1.4.19 failed!

    Can this be fixed also for AMD64?
    Thanks



  • there was a typo on url path, I've just fixed it.

    Try again in 15 minutes.



  • @marcelloc:

    I think That I should rename it to Classic instead of legacy as you suggested.

    I still see the legacy name. I'm trying to finalize my production installations and I want to move to the final name. Any timeframe on the classic name? Thanks.



  • I'm seeing now That I just changed i386 version.



  • Hi everyone,

    Sort of off topic but involves HAProxy….

    Any chance of a version of the plugin for 1.4.19 that just uses a user specified config file?  We'd simply like to create a haproxy.cfg and copy that over to the PFsense box and that be used.  No fancy gui setups - could be one box to paste the entire config into.

    Reason for this is ease of migration off working HAProxy boxes and compatibility with our staging environment.  Right now the HAproxy config gui also doesn't expose all the options that 1.4.19 supports.

    Any thoughts?



  • It will work.

    There are more features on 1.0

    Using the gui you can export generated config and paste it on other system.

    You can configure multiple pfsense boxes as well and use it's package config and XML sync.

    Briantist is the guy behind haproxy improvements, paste here option you do not see on this package. Maybe It could be on next release.



  • @marcelloc:

    It will work.

    There are more features on 1.0

    Using the gui you can export generated config and paste it on other system.

    You can configure multiple pfsense boxes as well and use it's package config and XML sync.

    Briantist is the guy behind haproxy improvements, paste here option you do not see on this package. Maybe It could be on next release.

    Hi, I see how it will generate the full config output for cutting but I don't see where you can paste a complete config in?

    We are running a config with 1 front end but back end switching based on acl's and such, and it would be far easier to just paste the complete config in.  I guess I could always do that into the haproxy.cfg on the pfsense but then is someone goes into the gui they risk wiping that out.



  • Now I undestood.

    A way to do it is installing filter package to be able to create and edit config files on pfsense.
    With this package you have also the option to execute a command after file saving.

    The problem is That package gui will always try to override config file.
    To workaround this, disable package gui and only use filer package to edit config.

    Can you share with us the config you use on haproxy and options That are not on gui?



  • @marcelloc:

    Now I undestood.

    A way to do it is installing filter package to be able to create and edit config files on pfsense.
    With this package you have also the option to execute a command after file saving.

    The problem is That package gui will always try to override config file.
    To workaround this, disable package gui and only use filer package to edit config.

    Can you share with us the config you use on haproxy and options That are not on gui?

    How do I disable the package gui?

    I will put up an example config once I sanitize it.



  • The package name is filer, not filter  :)



  • I agree with marcello, and would have recommended filer. I am not sure how you would disable the gui of haproxy though. If you want to start editing the package files yourself you could just comment out the lines that write over the config file based on the xml config. It's in haproxy.inc, line 99 I believe:

    
    function haproxy_configure() {
    	global $config, $g;
    
    	$a_global = &$config['installedpackages']['haproxy'];
    	$a_backends = &$config['installedpackages']['haproxy']['ha_backends']['item'];
    	$a_frontends = &$config['installedpackages']['haproxy']['ha_frontends']['item'];
    	$a_servers = &$config['installedpackages']['haproxy']['ha_servers']['item'];
    
    	$fd = fopen("{$g['varetc_path']}/haproxy.cfg", "w");
    
    	if(is_array($a_global)) {
    		fwrite ($fd, "global\n");
    		if($a_global['advanced'])
    			fwrite ($fd, "\t" . base64_decode($a_global['advanced']) . "\n");
    		fwrite ($fd, "\tmaxconn\t\t\t".$a_global['maxconn']."\n");
    		if($a_global['remotesyslog'])
    			fwrite ($fd, "\tlog\t\t\t{$a_global['remotesyslog']}\tlocal0\n");
    		fwrite ($fd, "\tuid\t\t\t80\n");
    		fwrite ($fd, "\tgid\t\t\t80\n");
    		// Set numprocs if defined or use system default (#cores)
    		if($a_global['nbproc'])
    			$numprocs = $a_global['nbproc'];
    		else 
    			$numprocs = trim(`/sbin/sysctl kern.smp.cpus | cut -d" " -f2`);			
    		fwrite ($fd, "\tnbproc\t\t\t$numprocs\n");
    		fwrite ($fd, "\tchroot\t\t\t/var/empty\n");
    		fwrite ($fd, "\tdaemon\n");
    		fwrite ($fd, "\n");
    	}
    
    

    I think that at the beginning of that function, if you just stick a "return;" in there it would probably do what you want. Then you just have to be sure that the package doesn't get overwritten. That shouldn't happen automatically except on pfSense upgrades or configuration restores.



  • Actually, you could use filer to overwrite haproxy.inc. Just copy and paste the whole file and make your modifications, and have filer save it back over the original. There is no guarantee that on a package reinstall, the file would get overwritten again before the "bad" code is run (in fact, I'm almost sure that it won't), but the nice thing about using filer is that if your files do get overwritten on the filesystem you need only save the filer config again to have it rewrite your versions, since your versions are stored in the XML config too.



  • @marcelloc:

    I'm seeing now That I just changed i386 version.

    I'm still seeing an old name on 64 bit. The 32 bit .xml on github is using the name haproxy-full it seems, but while still storing all the files in a directory called haproxy-legacy. Just a bit confused by the name, and I'd like the clear the whole thing up all at once.

    Also it still seems to be downloading from your site (e-sac.siteseguro.ws). Previously, I thought this would change to point back to the pfsense servers.

    Let me know if I can make any of these changes myself without putting the burden on you?



  • @Briantist:

    I'm still seeing an old name on 64 bit. The 32 bit .xml on github is using the name haproxy-full it seems, but while still storing all the files in a directory called haproxy-legacy. Just a bit confused by the name, and I'd like the clear the whole thing up all at once.

    I'll do on next update.

    @Briantist:

    Also it still seems to be downloading from your site (e-sac.siteseguro.ws). Previously, I thought this would change to point back to the pfsense servers.

    Again files.pfsense has not the latest version(1.4.19) of haproxy.

    jwelter99,

    to disable haproxy gui package, go on services -> haproxy and uncheck Enable HAProxy



  • @marcelloc:

    @Briantist:

    I'm still seeing an old name on 64 bit. The 32 bit .xml on github is using the name haproxy-full it seems, but while still storing all the files in a directory called haproxy-legacy. Just a bit confused by the name, and I'd like the clear the whole thing up all at once.

    I'll do on next update.

    @Briantist:

    Also it still seems to be downloading from your site (e-sac.siteseguro.ws). Previously, I thought this would change to point back to the pfsense servers.

    Again files.pfsense has not the latest version(1.4.19) of haproxy.

    jwelter99,

    to disable haproxy gui package, go on services -> haproxy and uncheck Enable HAProxy

    Thanks, what I did was remove the haproxy package from pfsense, and then just did a pkg add to add it back to freebsd and used filter to do the rest.  Seems to be a reasonable approach as the config doesnt change often and pfsense updates don't happen often either(thats a good thing!)


Log in to reply