HAproxy HTTPS + Openvpn



  • Hi,

    Im trying to set up haproxy in a way so I can use openvpn and https on 1 public ip on port 443. Im a bit stuck at the point where im at now.

    I am using a tcp mode main frontend with a default backend which goes to the openvpn server. This works without issues, but my website on https is no longer working.. I found a few articles on google using "req.ssl_sni -i ..." and with this im able to get my website working again over https ... to get it working I needed to turn on SSL offloading on the main frontend which I dont want because then open vpn wont work anymore. So im stuck at a working website, no vpn or a working vpn and no website.

    I was thinking of setting it up next using a tcp mode frontend -> to a backend if a sni header matches -> from the backend back to a http frontend listening on 127.0.0.1:<randomport> -> from the http frontend to the website. This didnt work either.

    Below is my current configuration (all my backend websites are running on port 443 with a valid cert, maybe I need it to be non-ssl port 80? )

    Im hoping someone can point me in the right direction

    # Automaticaly generated, dont edit manually.
    # Generated on: 2019-10-11 18:26
    global
    	maxconn			1000
    	stats socket /tmp/haproxy.socket level admin 
    	gid			80
    	nbproc			1
    	hard-stop-after		15m
    	chroot				/tmp/haproxy_chroot
    	daemon
    	tune.ssl.default-dh-param	4096
    	log-send-hostname		haproxy-node
    	server-state-file /tmp/haproxy_server_state
    
    listen HAProxyLocalStats
    	bind 127.0.0.1:2200 name localstats
    	mode http
    	stats enable
    	stats admin if TRUE
    	stats show-legends
    	stats uri /haproxy/haproxy_stats.php?haproxystats=1
    	timeout client 5000
    	timeout connect 5000
    	timeout server 5000
    
    ## trying to set the mode to HTTP
    frontend Frontend-loopback
    	bind			127.0.0.1:5555 name 127.0.0.1:5555   
    	mode			http
    	log			global
    	option			http-keep-alive
    	option			forwardfor
    	acl https ssl_fc
    	http-request set-header		X-Forwarded-Proto http if !https
    	http-request set-header		X-Forwarded-Proto https if https
    	timeout client		30000
    	http-request del-header X-Forwarded-Proto 
    	http-request set-header X-Forwarded-Proto "\ https if { ssl_fc }" 
    	default_backend bw-test_ipvANY
    
    ## main incoming frontend for openvpn and HTTPS
    frontend Main
    	bind			192.168.2.252:443 name 192.168.2.252:443   
    	mode			tcp
    	log			global
    	timeout client		30000
    	tcp-request content accept if HTTP
    	tcp-request content accept if { req.ssl_hello_type 1 }
    	acl			Acl1	req.ssl_sni -i .domain.com
    	use_backend Backend-to-front-loopback_ipvANY  if  Acl1 
    	default_backend v.domain.com_ipvANY
    
    ## the website im trying to go to from the "Frontend-loopback"
    backend bw-test_ipvANY
    	mode			http
    	id			110
    	log			global
    	timeout connect		30000
    	timeout server		30000
    	retries			3
    	server			bw 172.16.252.9:443 id 101 ssl  verify none crt /var/etc/haproxy/server_clientcert_5d9e36a600d17.pem 
    
    ## backend to go back to HTTP frontend
    backend Backend-to-front-loopback_ipvANY
    	mode			tcp
    	id			111
    	log			global
    	timeout connect		30000
    	timeout server		30000
    	retries			3
    	server			Bk-to-front 127.0.0.1:5555 id 112  
    
    ## open vpn server on pfsense
    backend v.domain.com_ipvANY
    	mode			tcp
    	id			108
    	log			global
    	timeout connect		30000
    	timeout server		30000
    	retries			2
    	server			v.domain.com 192.168.2.252:6785 id 109
    

    Thanks!



  • @Actionhenk said in HAproxy HTTPS + Openvpn:

    frontend Frontend-loopback
    bind 127.0.0.1:5555 name 127.0.0.1:5555

    Can you enable ssl-offloading on this loopback frontend?



  • Thanks for your reply. I had already tried that but unfortunately it wasnt working. I changed a few things and ended up with a working solution.

    I made a change in the main front end, I was using req.ssl_sni in the front end to filter domain where I should have been using req.ssl_hello_type 1

    I then set the action to go to a backend and put all my sites in a single backend using req.ssl_sni on the backend to filter for sites. Then forwarding it back to individual frontends for each site. Not sure if it is best practice but for now its OK :))

    # Automaticaly generated, dont edit manually.
    # Generated on: 2019-10-11 22:33
    global
    	maxconn			1000
    	stats socket /tmp/haproxy.socket level admin 
    	gid			80
    	nbproc			1
    	hard-stop-after		15m
    	chroot				/tmp/haproxy_chroot
    	daemon
    	tune.ssl.default-dh-param	4096
    	log-send-hostname		haproxy-node
    	server-state-file /tmp/haproxy_server_state
    	ssl-default-bind-options no-sslv3 no-tls-tickets no-tlsv10 no-tlsv11 no-tlsv12
    	ssl-default-bind-ciphers ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256
    	
    
    listen HAProxyLocalStats
    	bind 127.0.0.1:2200 name localstats
    	mode http
    	stats enable
    	stats admin if TRUE
    	stats show-legends
    	stats uri /haproxy/haproxy_stats.php?haproxystats=1
    	timeout client 5000
    	timeout connect 5000
    	timeout server 5000
    
    frontend frontend-ssl
    	bind			192.168.2.252:443 name 192.168.2.252:443   
    	mode			tcp
    	log			global
    	timeout client		30000
    	tcp-request inspect-delay 5s
    	tcp-request content accept if HTTP
    	acl			acl1	req.ssl_hello_type 1
    	use_backend main-ssl_ipvANY  if  acl1 
    	default_backend openvpn_ipvANY
    
    frontend main-https-zim-mail
    	bind			127.0.0.1:4646 name 127.0.0.1:4646   ssl crt-list /var/etc/haproxy/main-https-zim-mail.crt_list  
    	mode			http
    	log			global
    	option			http-keep-alive
    	timeout client		30000
    	acl			zim-mail	var(txn.txnhost) -m str -i zim-mail.domain.com
    	acl			aclcrt_main-https-zim-mail	var(txn.txnhost) -m reg -i ^zim-mail\.domain\.com(:([0-9]){1,5})?$
    	http-request set-var(txn.txnhost) hdr(host)
    	use_backend backend-zim-mail.domain.com_ipvANY  if  zim-mail aclcrt_main-https-zim-mail
    
    frontend main-https-eas
    	bind			127.0.0.1:5050 name 127.0.0.1:5050   ssl crt-list /var/etc/haproxy/main-https-eas.crt_list  
    	mode			http
    	log			global
    	option			http-keep-alive
    	timeout client		30000
    	acl			eas	var(txn.txnhost) -m str -i eas.domain.com
    	acl			aclcrt_main-https-eas	var(txn.txnhost) -m reg -i ^eas\.domain\.com(:([0-9]){1,5})?$
    	http-request set-var(txn.txnhost) hdr(host)
    	use_backend backend-eas.domain.com_ipvANY  if  eas aclcrt_main-https-eas
    
    frontend main-https-bw
    	bind			127.0.0.1:4747 name 127.0.0.1:4747   ssl crt-list /var/etc/haproxy/main-https-bw.crt_list  
    	mode			http
    	log			global
    	option			http-keep-alive
    	option			forwardfor
    	acl https ssl_fc
    	http-request set-header		X-Forwarded-Proto http if !https
    	http-request set-header		X-Forwarded-Proto https if https
    	timeout client		30000
    	acl			bw	var(txn.txnhost) -m str -i bw.domain.com
    	acl			aclcrt_main-https-bw	var(txn.txnhost) -m reg -i ^bw\.domain\.com(:([0-9]){1,5})?$
    	http-request set-var(txn.txnhost) hdr(host)
    	use_backend backend-bw.domain.com_ipvANY  if  bw aclcrt_main-https-bw
    
    frontend main-https-domain
    	bind			127.0.0.1:4848 name 127.0.0.1:4848   ssl crt-list /var/etc/haproxy/main-https-domain.crt_list  
    	mode			http
    	log			global
    	option			http-keep-alive
    	option			forwardfor
    	acl https ssl_fc
    	http-request set-header		X-Forwarded-Proto http if !https
    	http-request set-header		X-Forwarded-Proto https if https
    	timeout client		30000
    	acl			domain	var(txn.txnhost) -m str -i domain.com
    	acl			aclcrt_main-https-domain	var(txn.txnhost) -m reg -i ^domain\.com(:([0-9]){1,5})?$
    	http-request set-var(txn.txnhost) hdr(host)
    	use_backend backend-domain.com_ipvANY  if  domain aclcrt_main-https-domain
    
    frontend main-https-www
    	bind			127.0.0.1:4949 name 127.0.0.1:4949   ssl crt-list /var/etc/haproxy/main-https-www.crt_list  
    	mode			http
    	log			global
    	option			http-keep-alive
    	option			forwardfor
    	acl https ssl_fc
    	http-request set-header		X-Forwarded-Proto http if !https
    	http-request set-header		X-Forwarded-Proto https if https
    	timeout client		30000
    	acl			eas	var(txn.txnhost) -m str -i eas.domain.com
    	acl			aclcrt_main-https-www	var(txn.txnhost) -m reg -i ^www\.domain\.com(:([0-9]){1,5})?$
    	http-request set-var(txn.txnhost) hdr(host)
    	use_backend backend-eas.domain.com_ipvANY  if  eas aclcrt_main-https-www
    
    backend main-ssl_ipvANY
    	mode			tcp
    	id			110
    	log			global
    	timeout connect		30000
    	timeout server		30000
    	retries			3
    	acl			zim-mail	req.ssl_sni -i zim-mail.domain.com
    	acl			bw	req.ssl_sni -i bw.domain.com
    	acl			domain	req.ssl_sni -i domain.com
    	acl			www	req.ssl_sni -i www.domain.com
    	acl			eas	req.ssl_sni -i eas.domain.com
    	use-server zim-mail  if  zim-mail 
    	use-server bw  if  bw 
    	use-server domain  if  domain 
    	use-server www  if  www 
    	use-server eas  if  eas 
    	server			zim-mail 127.0.0.1:4646 id 123  
    	server			bw 127.0.0.1:4747 id 124  
    	server			domain 127.0.0.1:4848 id 125  
    	server			www 127.0.0.1:4949 id 126  
    	server			eas 127.0.0.1:5050 id 111  
    
    backend openvpn_ipvANY
    	mode			tcp
    	id			112
    	log			global
    	timeout connect		30000
    	timeout server		30000
    	retries			3
    	server			openvpn 192.168.2.252:6785 id 113  
    
    backend backend-zim-mail.domain.com_ipvANY
    	mode			http
    	id			116
    	log			global
    	timeout connect		30000
    	timeout server		30000
    	retries			3
    	server			backend-zim-mail.domain.com 172.16.252.7:443 id 117 ssl  verify none 
    
    backend backend-eas.domain.com_ipvANY
    	mode			http
    	id			114
    	log			global
    	timeout connect		30000
    	timeout server		30000
    	retries			3
    	server			backend-eas.domain.com 172.16.252.8:443 id 115 ssl  verify none 
    
    backend backend-bw.domain.com_ipvANY
    	mode			http
    	id			118
    	log			global
    	timeout connect		30000
    	timeout server		30000
    	retries			3
    	server			backend-bw.domain.com 172.16.252.9:443 id 119 ssl  verify none 
    
    backend backend-domain.com_ipvANY
    	mode			http
    	id			122
    	log			global
    	timeout connect		30000
    	timeout server		30000
    	retries			3
    	server			backend-domain.com 172.16.252.12:443 id 121
    


  • @Actionhenk
    That looks overly complicated.. I do see that the 'loopback' kinda frontends now are using offloading as suggested. Also i see that the bw server is no longer using a client-certificate.. Perhaps that is the part that made the original configuration fail while you where testing/changing it.?.

    Also you are using SNI to switch between the 5 webservers, in general its better to do that on the Host header. Are there 5 different certificates used for the 5 websites? Otherwise switching to a different domain could fail if the browser decides it can keep the connection alive with the same certificate..



  • @Actionhenk said in HAproxy HTTPS + Openvpn:

    req.ssl_sni -i zim-mail.domain.com

    Thanks for the suggestions (didnt notice bw was missing a certificate, checked in the gui and it is there), I have tried to change it to match host headers but haproxy shows me an error saving, " error detected while parsing switching rule : no such ACL : 'acl-eas'" The error somehow doesnt show if I have a "req.ssl_sni -i www.domain.com" in the acl.

    Maybe something different, do you know if it is possible to deny connections from an external source ip in tcp mode ? There are options using "Source IP matches IP or Alias" but its not matching the external ip's, send-proxy is only picking up internal ips. Is it possible to send to, or get originating IP through to the backends from a tcp frontend ?

    Thanks!



  • Hard to tell what go's wrong without the haproxy.cfg content that you might have. When using proxy-protocol between backend and next frontend the 'src' should still match the external client ip's afaik.. as for acl's some automatically disaprear depending on the mode chosen.. But when traffic is offloaded the 'ssl_fc_sni' returns the sni value. Not the 'ssl_sni' which is only used with tcp-passthrough.. Such acl's are also automatically hidden in the webgui on a frontend which knows the mode its going to run with better..


Log in to reply