HAproxy HTTPS + Openvpn
-
Hi,
Im trying to set up haproxy in a way so I can use openvpn and https on 1 public ip on port 443. Im a bit stuck at the point where im at now.
I am using a tcp mode main frontend with a default backend which goes to the openvpn server. This works without issues, but my website on https is no longer working.. I found a few articles on google using "req.ssl_sni -i ..." and with this im able to get my website working again over https ... to get it working I needed to turn on SSL offloading on the main frontend which I dont want because then open vpn wont work anymore. So im stuck at a working website, no vpn or a working vpn and no website.
I was thinking of setting it up next using a tcp mode frontend -> to a backend if a sni header matches -> from the backend back to a http frontend listening on 127.0.0.1:<randomport> -> from the http frontend to the website. This didnt work either.
Below is my current configuration (all my backend websites are running on port 443 with a valid cert, maybe I need it to be non-ssl port 80? )
Im hoping someone can point me in the right direction
# Automaticaly generated, dont edit manually. # Generated on: 2019-10-11 18:26 global maxconn 1000 stats socket /tmp/haproxy.socket level admin gid 80 nbproc 1 hard-stop-after 15m chroot /tmp/haproxy_chroot daemon tune.ssl.default-dh-param 4096 log-send-hostname haproxy-node server-state-file /tmp/haproxy_server_state listen HAProxyLocalStats bind 127.0.0.1:2200 name localstats mode http stats enable stats admin if TRUE stats show-legends stats uri /haproxy/haproxy_stats.php?haproxystats=1 timeout client 5000 timeout connect 5000 timeout server 5000 ## trying to set the mode to HTTP frontend Frontend-loopback bind 127.0.0.1:5555 name 127.0.0.1:5555 mode http log global option http-keep-alive option forwardfor acl https ssl_fc http-request set-header X-Forwarded-Proto http if !https http-request set-header X-Forwarded-Proto https if https timeout client 30000 http-request del-header X-Forwarded-Proto http-request set-header X-Forwarded-Proto "\ https if { ssl_fc }" default_backend bw-test_ipvANY ## main incoming frontend for openvpn and HTTPS frontend Main bind 192.168.2.252:443 name 192.168.2.252:443 mode tcp log global timeout client 30000 tcp-request content accept if HTTP tcp-request content accept if { req.ssl_hello_type 1 } acl Acl1 req.ssl_sni -i .domain.com use_backend Backend-to-front-loopback_ipvANY if Acl1 default_backend v.domain.com_ipvANY ## the website im trying to go to from the "Frontend-loopback" backend bw-test_ipvANY mode http id 110 log global timeout connect 30000 timeout server 30000 retries 3 server bw 172.16.252.9:443 id 101 ssl verify none crt /var/etc/haproxy/server_clientcert_5d9e36a600d17.pem ## backend to go back to HTTP frontend backend Backend-to-front-loopback_ipvANY mode tcp id 111 log global timeout connect 30000 timeout server 30000 retries 3 server Bk-to-front 127.0.0.1:5555 id 112 ## open vpn server on pfsense backend v.domain.com_ipvANY mode tcp id 108 log global timeout connect 30000 timeout server 30000 retries 2 server v.domain.com 192.168.2.252:6785 id 109Thanks!
-
@Actionhenk said in HAproxy HTTPS + Openvpn:
frontend Frontend-loopback
bind 127.0.0.1:5555 name 127.0.0.1:5555Can you enable ssl-offloading on this loopback frontend?
-
Thanks for your reply. I had already tried that but unfortunately it wasnt working. I changed a few things and ended up with a working solution.
I made a change in the main front end, I was using req.ssl_sni in the front end to filter domain where I should have been using req.ssl_hello_type 1
I then set the action to go to a backend and put all my sites in a single backend using req.ssl_sni on the backend to filter for sites. Then forwarding it back to individual frontends for each site. Not sure if it is best practice but for now its OK :))
# Automaticaly generated, dont edit manually. # Generated on: 2019-10-11 22:33 global maxconn 1000 stats socket /tmp/haproxy.socket level admin gid 80 nbproc 1 hard-stop-after 15m chroot /tmp/haproxy_chroot daemon tune.ssl.default-dh-param 4096 log-send-hostname haproxy-node server-state-file /tmp/haproxy_server_state ssl-default-bind-options no-sslv3 no-tls-tickets no-tlsv10 no-tlsv11 no-tlsv12 ssl-default-bind-ciphers ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256 listen HAProxyLocalStats bind 127.0.0.1:2200 name localstats mode http stats enable stats admin if TRUE stats show-legends stats uri /haproxy/haproxy_stats.php?haproxystats=1 timeout client 5000 timeout connect 5000 timeout server 5000 frontend frontend-ssl bind 192.168.2.252:443 name 192.168.2.252:443 mode tcp log global timeout client 30000 tcp-request inspect-delay 5s tcp-request content accept if HTTP acl acl1 req.ssl_hello_type 1 use_backend main-ssl_ipvANY if acl1 default_backend openvpn_ipvANY frontend main-https-zim-mail bind 127.0.0.1:4646 name 127.0.0.1:4646 ssl crt-list /var/etc/haproxy/main-https-zim-mail.crt_list mode http log global option http-keep-alive timeout client 30000 acl zim-mail var(txn.txnhost) -m str -i zim-mail.domain.com acl aclcrt_main-https-zim-mail var(txn.txnhost) -m reg -i ^zim-mail\.domain\.com(:([0-9]){1,5})?$ http-request set-var(txn.txnhost) hdr(host) use_backend backend-zim-mail.domain.com_ipvANY if zim-mail aclcrt_main-https-zim-mail frontend main-https-eas bind 127.0.0.1:5050 name 127.0.0.1:5050 ssl crt-list /var/etc/haproxy/main-https-eas.crt_list mode http log global option http-keep-alive timeout client 30000 acl eas var(txn.txnhost) -m str -i eas.domain.com acl aclcrt_main-https-eas var(txn.txnhost) -m reg -i ^eas\.domain\.com(:([0-9]){1,5})?$ http-request set-var(txn.txnhost) hdr(host) use_backend backend-eas.domain.com_ipvANY if eas aclcrt_main-https-eas frontend main-https-bw bind 127.0.0.1:4747 name 127.0.0.1:4747 ssl crt-list /var/etc/haproxy/main-https-bw.crt_list mode http log global option http-keep-alive option forwardfor acl https ssl_fc http-request set-header X-Forwarded-Proto http if !https http-request set-header X-Forwarded-Proto https if https timeout client 30000 acl bw var(txn.txnhost) -m str -i bw.domain.com acl aclcrt_main-https-bw var(txn.txnhost) -m reg -i ^bw\.domain\.com(:([0-9]){1,5})?$ http-request set-var(txn.txnhost) hdr(host) use_backend backend-bw.domain.com_ipvANY if bw aclcrt_main-https-bw frontend main-https-domain bind 127.0.0.1:4848 name 127.0.0.1:4848 ssl crt-list /var/etc/haproxy/main-https-domain.crt_list mode http log global option http-keep-alive option forwardfor acl https ssl_fc http-request set-header X-Forwarded-Proto http if !https http-request set-header X-Forwarded-Proto https if https timeout client 30000 acl domain var(txn.txnhost) -m str -i domain.com acl aclcrt_main-https-domain var(txn.txnhost) -m reg -i ^domain\.com(:([0-9]){1,5})?$ http-request set-var(txn.txnhost) hdr(host) use_backend backend-domain.com_ipvANY if domain aclcrt_main-https-domain frontend main-https-www bind 127.0.0.1:4949 name 127.0.0.1:4949 ssl crt-list /var/etc/haproxy/main-https-www.crt_list mode http log global option http-keep-alive option forwardfor acl https ssl_fc http-request set-header X-Forwarded-Proto http if !https http-request set-header X-Forwarded-Proto https if https timeout client 30000 acl eas var(txn.txnhost) -m str -i eas.domain.com acl aclcrt_main-https-www var(txn.txnhost) -m reg -i ^www\.domain\.com(:([0-9]){1,5})?$ http-request set-var(txn.txnhost) hdr(host) use_backend backend-eas.domain.com_ipvANY if eas aclcrt_main-https-www backend main-ssl_ipvANY mode tcp id 110 log global timeout connect 30000 timeout server 30000 retries 3 acl zim-mail req.ssl_sni -i zim-mail.domain.com acl bw req.ssl_sni -i bw.domain.com acl domain req.ssl_sni -i domain.com acl www req.ssl_sni -i www.domain.com acl eas req.ssl_sni -i eas.domain.com use-server zim-mail if zim-mail use-server bw if bw use-server domain if domain use-server www if www use-server eas if eas server zim-mail 127.0.0.1:4646 id 123 server bw 127.0.0.1:4747 id 124 server domain 127.0.0.1:4848 id 125 server www 127.0.0.1:4949 id 126 server eas 127.0.0.1:5050 id 111 backend openvpn_ipvANY mode tcp id 112 log global timeout connect 30000 timeout server 30000 retries 3 server openvpn 192.168.2.252:6785 id 113 backend backend-zim-mail.domain.com_ipvANY mode http id 116 log global timeout connect 30000 timeout server 30000 retries 3 server backend-zim-mail.domain.com 172.16.252.7:443 id 117 ssl verify none backend backend-eas.domain.com_ipvANY mode http id 114 log global timeout connect 30000 timeout server 30000 retries 3 server backend-eas.domain.com 172.16.252.8:443 id 115 ssl verify none backend backend-bw.domain.com_ipvANY mode http id 118 log global timeout connect 30000 timeout server 30000 retries 3 server backend-bw.domain.com 172.16.252.9:443 id 119 ssl verify none backend backend-domain.com_ipvANY mode http id 122 log global timeout connect 30000 timeout server 30000 retries 3 server backend-domain.com 172.16.252.12:443 id 121 -
@Actionhenk
That looks overly complicated.. I do see that the 'loopback' kinda frontends now are using offloading as suggested. Also i see that the bw server is no longer using a client-certificate.. Perhaps that is the part that made the original configuration fail while you where testing/changing it.?.Also you are using SNI to switch between the 5 webservers, in general its better to do that on the Host header. Are there 5 different certificates used for the 5 websites? Otherwise switching to a different domain could fail if the browser decides it can keep the connection alive with the same certificate..
-
@Actionhenk said in HAproxy HTTPS + Openvpn:
req.ssl_sni -i zim-mail.domain.com
Thanks for the suggestions (didnt notice bw was missing a certificate, checked in the gui and it is there), I have tried to change it to match host headers but haproxy shows me an error saving, " error detected while parsing switching rule : no such ACL : 'acl-eas'" The error somehow doesnt show if I have a "req.ssl_sni -i www.domain.com" in the acl.
Maybe something different, do you know if it is possible to deny connections from an external source ip in tcp mode ? There are options using "Source IP matches IP or Alias" but its not matching the external ip's, send-proxy is only picking up internal ips. Is it possible to send to, or get originating IP through to the backends from a tcp frontend ?
Thanks!
-
Hard to tell what go's wrong without the haproxy.cfg content that you might have. When using proxy-protocol between backend and next frontend the 'src' should still match the external client ip's afaik.. as for acl's some automatically disaprear depending on the mode chosen.. But when traffic is offloaded the 'ssl_fc_sni' returns the sni value. Not the 'ssl_sni' which is only used with tcp-passthrough.. Such acl's are also automatically hidden in the webgui on a frontend which knows the mode its going to run with better..