Haproxy setup help

rajbps

Hi team,

Can anyone assist in setting up haproxy with details please? I have a 3 websites different domains running on port 80 and 3 other sites running on 443 one of them is for exchange 2013 web access.

i am willing to buy a drink to someone who can assist with details.

Raj

PiBa

Hi Raj,

Where did you get stuck? What did you try?

https://github.com/PiBa-NL/pfsense-haproxy-package-doc/wiki/Single-frontend-serving-multiple-different-domains-using-http
https://github.com/PiBa-NL/pfsense-haproxy-package-doc/wiki/pfsense_2_3_haproxy_sni_plus_offloading_backends

Or try the Templates/create config button in the package to setup a basic 'example'.?.

Then post your haproxy.conf and describe what doesnt work yet.

Regards,
PiBa-NL

rajbps

Hiya,

Sorry for the late reply. This is the link that I used:

https://blog.briantruscott.com/how-to-serve-multiple-domains-from-a-single-public-ip-using-haproxy-on-pfsense/

I will try to follow your links and update shortly.

Cheers,

raj

rajbps

Ok I think I got it working

Created a backend for each server and 2 main front ends one for http for 80 and one for https.

Then created a few sub frontends that link to either http or https. Got owa and phones to sync for exchange and websites to work. Just have one last thing. Have a support site on https and I can load it good. When i try to open it on http it does not get redirected to https. Can anyone assist on how to get this done pls?

Rajbps

PiBa

You could make a 'action' in a frontend that listens on :80 and then performs a "http-request redirect scheme https" if the acl for the support domain matches.

rajbps

Could you provide a screenshot pls. Cant seem to get my head round it.

PiBa

Something like this.

![2016-10-26 21_16_17-pfSe.localdomain - HAProxy_ Frontend_ Edit.png](/public/imported_attachments/1/2016-10-26 21_16_17-pfSe.localdomain - HAProxy_ Frontend_ Edit.png)
![2016-10-26 21_16_17-pfSe.localdomain - HAProxy_ Frontend_ Edit.png_thumb](/public/imported_attachments/1/2016-10-26 21_16_17-pfSe.localdomain - HAProxy_ Frontend_ Edit.png_thumb)

rajbps

Hiya, the "scheme https" is that the main front end for https or the shared front end which uses https front end as primary please?

PiBa

It just redirects the client from http://support to https://support, so after that initial http request the 'usual route' for that website will end up on one of your frontends that listen on :443.

rajbps

is there a way for me to get the config out of pfsense and send it to you in private pls?

PiBa

The generated haproxy configuration can be seen at the bottom of the settings tab in the package, there is a button to show it.

As for getting it to me you could send me a PM through the forum if thats ok for you?. Anyhow replace passwords/public ip's/domainnames by similar but dummy values where needed. (do make sure to replace a name or ip you want to replace by the same value each time so it still makes sense..)

rajbps

pm sent

PiBa

Config recieved. Looks like you didnt configure the 'action' part from my screenshot above?

Is the redirect the only current issue.? Or are there others i should look at?

rajbps

thats the bit and if you can see anything to harden security thats always welcome

PiBa

Ok so add the redirect 'action' as in my screenshot ? I think that should fix the redirect part.

As a minor observation i'm wondering about for example 'email' and 'support' are using different acl's matches v.s. contains is that intentional? should othersubdomain.support.mydomain.com also be send to the same backend as support.mydomain.com ?

You passing https straight to the backend, and using transparent-client-ip, should work.. and i assume you have read the warning, and understand it can cause some issues.. (it might bite you later when you want to connect another way and cant seem to get it working, the 'transparent-client-ip' might be the culprit..) Though there arn't really much other options for some backends.. Just saying ;)

rajbps

when I add the second bit and try to save and apply, I get the following error :

Errors found while starting haproxy
[ALERT] 299/234604 (87967) : parsing [/var/etc/haproxy_test/haproxy.cfg:39] : error detected in frontend 'Loki-merged' while parsing 'http-request redirect' rule : expects 'code', 'prefix', 'location', 'scheme', 'set-cookie', 'clear-cookie', 'drop-query' or 'append-slash' (was 'Support').
[ALERT] 299/234604 (87967) : Error(s) found in configuration file : /var/etc/haproxy_test/haproxy.cfg

rajbps

http-request redirect See below httpsredirect

rule: Support

rajbps

"As a minor observation i'm wondering about for example 'email' and 'support' are using different acl's matches v.s. contains is that intentional? should othersubdomain.support.mydomain.com also be send to the same backend as support.mydomain.com ?"

In regards to the email and support, if i got it correctly, i have a different backend for each url as they go to different servers. Then I have one https frontend and sub frontend that share the primary https front end. If there is a better more efficient way to do it please advise as I am very new to haproxy and this is a learning curve for me .

Cheers for all the help though.

Rajbps

PiBa

@rajbps:

http-request redirect See below httpsredirect

rule: Support

For a simple https redirect the 'rule' should be:

scheme https

no more no less :)

rajbps

yep error gone :-)

PiBa

The usage of multiple backends and acl's is good. And it will probably work OK.

The question i got was for example about these acl's:

   acl         OWA   req.ssl_sni -i email.mydomain.com
   acl         Support   req.ssl_sni -m sub -i support.mydomain.com

Where email is using an 'exact match', the other for support is checking if the it is 'part of' the requested domain. I would expect a little that you would use "Server Name Indication TLS extension matches:" for all those acl expression's, and not have a few with "Server Name Indication TLS extension contains:".

If it would look like this it would probably be a little better, though probably it wouldnt make much of a difference :)..

   acl         OWA   req.ssl_sni -i email.mydomain.com
   acl         Support   req.ssl_sni -i support.mydomain.com

rajbps

Is there a way to get a log of ips of live connections to the webservers please?

PiBa

The dashboard widget has the ability to show currently connected clients..
But best is probably to run a syslog server on the network and let haproxy send its syslog messages there. On the settings tab og haproxy package you can fill in the syslog server to send the udp traffic to.

rajbps

Hi PiBA,

I have exchange running behind haproxy fine and just added a new server with a new domain and vlan and that works well for webinterface but not for connecting the phones. Email box is Iredmail. If I redirect all https traffic direct to that Iredmail box then the phone will connect from external but as soon as i put haproxy in the mix I get stuck. Could you advise please?

Rajbps

PiBa

Whats the config, what requests are send, why does your application not work? As i'm not using Iredmail and you haven't included much info about requests/configs/errors/logs there ain't much i can advice..

rajbps

Hi Piba,

I am new to it myself so I do not know much. The maintenainer of the project for Iredmail said just to open port 443. If i point port 443 direct to the the Iredmail server, the phone works but when I put HAproxy back in the mix then it does not work. There is not much in the logs for sogo also :-(

rajbps

ok might have made some progress. installed a syslog and changed HAproxxy to log everything. When I try to connect the phone from external this is what syslog shows me as error:

Nov 24 22:40:44 192.168.3.251 ffww syslog notice haproxy[96712] Proxy Iredmail_https_ipvANY started.
Nov 24 22:40:52 192.168.3.251 ffww syslog info haproxy[96902] X.X.X.X:59323 [24/Nov/2016:22:40:52.070] HTTPS-merged Iredmail_https_ipvANY/mailer 1/0/111 1866 – 1/1/0/0/0 0/0
Nov 24 22:40:57 192.168.3.251 ffww syslog err haproxy[96902] X.X.X.X:23380 [24/Nov/2016:22:40:52.089] HTTPS-merged HTTPS-merged/ <nosrv>-1/-1/5002 0 SC 0/0/0/0/0 0/0
Nov 24 22:41:07 192.168.3.251 ffww syslog err haproxy[96902] X.X.X.X:40317 [24/Nov/2016:22:41:07.378] HTTPS-merged HTTPS-merged/ <nosrv>-1/-1/0 0 SC 0/0/0/0/0 0/0
Nov 24 22:41:25 192.168.3.251 ffww syslog info haproxy[96902] X.X.X.X:10799 [24/Nov/2016:22:41:25.175] HTTPS-merged Iredmail_https_ipvANY/mailer 1/0/132 1866 – 1/1/0/0/0 0/0

Hope this helps

Rajbps</nosrv></nosrv>

PiBa

The "<nosrv>" looks like it might be a problem. It usually means the request does not match the acl's used for the use_backend action. Can you share the haproxy.cfg file from bottom of settings tab?</nosrv>

rajbps

will send you in a pm in a few min

PiBa

Could you try configuring the Iredmail backend as the default backend?

Also afaik a SNIvalue will never contain a 'path' like /owa it might contain a portnumber though.. So add a acl for both mailer.example.com and mailer.example.com:443 also try capturingnthe phone requests in a wireshark capture, the ssl connection packetshould show if/what sni value is send.

rajbps

Cheers for that by changing the default to of the Iredmail frontend, the backend bit from default to Iredmail did the trick. Thanks so much for all the help. I am going to change the owa part now and see what happens

rajbps

Just send you a new pm with the new change. Would you suggest doing the same with the other front ends?

Cheers,

Rajbps

PiBa

Ok so with the 'default backend' set it works, that confirms the problem was with the requests not matching the acl checks and because of that ending up at <nosrv>.

A frontend as configured in haproxy.cfg can effectively only have 1 "default_backend" so no you should not set the default on each 'shared frontend', as available in the webgui. So best would be to figure out why the acl was not matched..

If the phones dont send SNI information another workaround could be to 'decrypt/offload' ssl traffic on haproxy preferably with a wildcard certificate to allow haproxy to read the host header in the http request to choose a backend.. Or ofcourse keep it like it is now.. with only the the frontend thats for the phones having a default.</nosrv>

rajbps

Hi PiBa,

Sorry to post on this ols post. You assisted me before and hopefully you can assist again. I have an iphone and its been working fine. I added an android today and it does not register with exchange. I done a bit of digging and believe that the issue is haproxy. I can see the following in syslog. Message: x.x.x.x:20655 [13/Jun/2017:20:49:43.498] HTTPS-merged majesty_https_ipvANY/ <nosrv>-1/-1/0 0 SC 1/0/0/0/0 0/0
I will send you the haproxy.cfg file in a pm if that`s ok. Could you assist please.

Cheers,
Rajbps</nosrv>

PiBa

The iphone is also using this majesty backend? The backend got picked, but the <nosrv>seems to tell no server could be connected to in that backend. That also seems to match the description for your SC status code (see below). The backend server shows as 'up' on the stats page? Can you tcpdump/wireshark the traffic while the android attempts to connect?

SC The server or an equipment between it and haproxy explicitly refused
the TCP connection (the proxy received a TCP RST or an ICMP message
in return). Under some circumstances, it can also be the network
stack telling the proxy that the server is unreachable (eg: no route,
or no ARP response on local network).</nosrv>

rajbps

The iphone has worked with no issues and still work. Its connecting the the right backend as owa. Just the android seems to try to connect to majesty with is not the exchange server :-(

PiBa

Ok 'default_backend majesty' is probably the reason it ends up there.. could be that none of the acl's matched..

The current acl might not always match.:
acl OWA req.ssl_sni -i mail.mydomian.com
Could you add also a:
acl OWA req.ssl_sni -i mail.mydomian.com:443

So including the port? that might solve something..

p.s. it seems you have to many 'default_backend' configured anyhow. but if the acl's pick up the traffic you shouldnt end up on majesty. (when requesting mail.mydomian.com)