HAProxy vs Builtin LoadBalancer

[boujid]

Hi
i used to work with the builtin loadbalancer in PfSense, howerver this load balancer not give choices about loadbalancing algorithm.
as i am using it to loadbalance smtp and https i was forced to activate sticky connections cayse its a mandatory thing for https.

i checked HAProxy, it appear as a more powerful one, there is a lot of loadbalancing algorithms and we can choose an algorithm of each load balancer we create

however in HAProxy i can see only ports, there is no send/expect ability to test the selected port/protocol.

in my case we have many servers that hang so the port is still active/open but its not responding

is there a way to let HAProxy check protocols rather than just checkin ports ?

what do you advise me  ?

can someone enlighten us about strengths and weaknesses of each scenarios ?

Thanks

[PiBa]

cant really help you with a objective compare..

HAProxy can perform several basic health checks for a variety of protocols. http, http-over-ssl, smtp, mysql, other.. also its possible to use an agent-check with which its possible to implement other checks if needed..

if your willing to give it a try, the haproxy-devel package has most options, and is currently pretty stable i think.. (i made most of the additions to the current haproxy-devel package).. also though the HAProxy software it uses is from a devel tree it is based on handpicked "tested" builds.. if you have any issues in either the package, or the HAProxy software itself i'm willing to help fix issues with the package. As in the haproxy mailing list its very active, and critical issues are fixed fast when the need is there.

[Briantist]

I did the latest changes on the pfSense Haproxy-full package.

You want to look at the documentation here:
http://haproxy.1wt.eu/download/1.4/doc/configuration.txt

Do a find on page for "smtpchk". You would have to put this in the advanced options box manually (in the pfSense package) for the server, not the frontend.

When you choose HTTPS instead of TCP for the frontend in pfSense, it will do an HTTPS health check, but be aware that this is basically an SSL check and doesn't mean that the underlying HTTP request was successful. That might become possible in HAProxy 1.5 since it will have SSL offloading built-in.

[PiBa]

For truly 'custom' protocol checks you could use the HAProxy 1.5 agent check: http://cbonte.github.io/haproxy-dconv/configuration-1.5.html#option lb-agent-chk.

As for a simple http (or SMTP or mySQL) requests there are a few possibilities also for 1.4: http://cbonte.github.io/haproxy-dconv/configuration-1.4.html#option%20httpchk
option httpchk OPTIONS /myHealthCheck.aspx HTTP/1.1\r\nHost:\ www

As for running checks over SSL and offloading SSL those are not necessarily related, but both are supported in the current 1.5 based HAproxy-devel package.

[Briantist]

@PiBa:

For truly 'custom' protocol checks you could use the HAProxy 1.5 agent check: http://cbonte.github.io/haproxy-dconv/configuration-1.5.html#option lb-agent-chk.

As for a simple http (or SMTP or mySQL) requests there are a few possibilities also for 1.4: http://cbonte.github.io/haproxy-dconv/configuration-1.4.html#option%20httpchk
option httpchk OPTIONS /myHealthCheck.aspx HTTP/1.1\r\nHost:\ www

As for running checks over SSL and offloading SSL those are not necessarily related, but both are supported in the current 1.5 based HAproxy-devel package.

When you refer to HAProxy-devel package, do you mean a pfSense package? I don't see one listed in packages. I have a lot of changes planned for the pfSense HAProxy-full package, so if you're working on something separately we should keep in touch!  :)

[PiBa]

Hey Briantist,
Yes haproxy-devel is a pfSense package, though HAProxy1.5 is only compiled for pfSense2.1 so that also became the minimum version for the package.. And as it is a development version that is 'pretty stable', and people daring enough to use HAProxy1.5 should also be able to use pfSense2.1.. I discussed with JimP that it wouldn't make sense to put effort in getting it into 2.0, though i would have liked that, as it would make the packege available to a few more people.. Though that technically is (or was.?. a few versions ago) possible with a little workaround: https://raw.github.com/PiBa-NL/pfsense-packages/3c1278fbdecdc07108124b17de943c55589075a6/config/haproxy-devel/haproxy-devel_install_on_pfs_2_0.php

[Briantist]

@PiBa:

Hey Briantist,
Yes haproxy-devel is a pfSense package, though HAProxy1.5 is only compiled for pfSense2.1 so that also became the minimum version for the package.. And as it is a development version that is 'pretty stable', and people daring enough to use HAProxy1.5 should also be able to use pfSense2.1.. I discussed with JimP that it wouldn't make sense to put effort in getting it into 2.0, though i would have liked that, as it would make the packege available to a few more people.. Though that technically is (or was.?. a few versions ago) possible with a little workaround: https://raw.github.com/PiBa-NL/pfsense-packages/3c1278fbdecdc07108124b17de943c55589075a6/config/haproxy-devel/haproxy-devel_install_on_pfs_2_0.php

Thanks, I won't mess with trying to put it in 2.0; I'd rather just install an early 2.1 (or wait for 2.1). I have additional questions but I'll PM you so we don't derail the thread.

[boujid]

thanks a lot for your answers

well, i will try to explain what i want to do :
we have commercials proxies, there is a global policy but there is also a policy for secure sites
so the proxy is working with 3 ports : 8021 (ftp) 8080 (http) and 8443 for (https)

the problem is there is freaking genius people in the company that use standalone browsers and they configure the port 8080 for the https protocol rather than 8443
so their SSL sites are not inspected and they are bypassing the proxies policy.

we have 2 proxies, so i am trying to loadbalance with round robin on 8021 & 8080 and balance with source for 8443

so basically this outlaw people will have headaches cause round-robin is not suitable for https

what i was trying to do from the beginning is :

health check for protocols and not just seeing if a port open or not :

1)health check for proxy port 8080, proxy port 8080, proxy port 8443
2)balance round-robin for proxy port 8080
3)balance least conn for proxy port 8021
4)balance source for proxy port 8443

unfortunately i cannot find health check for all this ports !!!
the least conn is no more available in the package "Release 1.4.24 pkg v 1.2.1 ", there is just 2 balance method : round-robin and source !!!!

and when i uninstalled haproxy and tried to reinstall it, the installtion stuck in
"Executing custom_php_install_command()…"

so i am too disappointed cause the builtin loadbalancer is more stable, unfortunately there is no choice of balancing method and once the sticky connection is enabled, its enabled for the whole pools and not just some of them !!!

i believe that HAProxy is a powerful tool but i cannot make it work !

HELP !!!

Thanks

[Briantist]

Leastconn is one of the connection methods I added in haproxy-full . Use that version if you need that method.

[PiBa]

Im not sure if there is a actual difference with the proxy ports for http and https.., But it seams at least with squid i can successfully use the following option in my haproxy.cfg

option httpchk HEAD http://127.0.0.1/
server SquidProxyServer 127.0.0.1:3128  check inter 1000  weight 1

So to get that fill in the following in the backend config page:
Health check method: HTML
Http check Method: HEAD
Http check URI: http://127.0.0.1/

Also "Least Connections" is available in haproxy-devel package..

Where the first 127.0.0.1 points to the webinterface of pfSense (needs to be a running/available website) (i do have pfSense WbGUI running on https.. but left the default redirect enabled)
The second 127.0.0.1:3128 points to a locally running squid installation.

Then when looking at the 'stats' page of HAproxy it tells the backend is UP and LastChk: L7OK/301 in 1ms.
Did require squid to also allow also listen and allow connections on/from the 127.0.0.1 network.

For the 8443 you could try the same.. Or maybe add a advanced option "check-ssl", not sure it there is technically any difference in those proxy ports.. ???

As for FTP, im not sure if its possible to perform health checks on it.. I guess you might need the agent check.. Or maybe give smtpchk a try, ive read/seen its 'sort off' compatible.. at least it will pass the check if the welcome message is a single line…  :-\