Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    HAProxy + ACME [FIXED]

    Scheduled Pinned Locked Moved ACME
    20 Posts 2 Posters 5.1k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • U
      uwscia
      last edited by

      Ok, That was going to be my next question… how to stop the loopback issue.

      Transparent ClientIP not checked...

      So if I have my servers on 192.168.1.1 and my devices on 192.168.2.1 ect... I'll be able to access the domains locally?
      Or, Is there a way to redirect my domain internally to a local domain... sub1.domain.com to sub1.domain.local?
      Which would be best practice?

      1 Reply Last reply Reply Quote 0
      • P
        PiBa
        last edited by

        Internally you could go for a split-dns solution to reply the webserver 192.168.1.100 ip when a request for sub1.domain.com is made.. But i prefer to keep it simple and have all clients both externally and internally follow (more or less) the same route to the destination.

        Sending a redirect could be done also, but well the users would be using 2 different domains for the same website / or a specific page on it, making a favorite to a page wont work when going outside anymore.. Or sending a link to a colleague thats outside..

        There shouldn't be much of a 'loopback issue' when using haproxy (not like you would have with portforwards..). as the client resolves the pfSense wan-ip, haproxy accepts the connection, and makes another connection to the webserver.. That should 'just work'.. or is the wan-ip not actually on pfSense but on a upstream isp provided router.?.

        So to check, the clients do resolve the wan-ip of pfSense when internally requesting the domain? You have made simple 'pass' rules? (no portforwards should be needed..) Does any error appear? Does stats page 'count' a new connection on frontend when you try to connect?

        1 Reply Last reply Reply Quote 0
        • U
          uwscia
          last edited by

          Ok so I started working on getting HTTPS to work…

          I've set HTTP redirect to HTTPS, but can't seem to disable it to just test basic http...  not a big deal right now... but a pain for troubleshooting...

          I'm having trouble finding "good practice" guides for setting up the backend SSL to talk with cert manager...

          Here is my HAproxy settings ATM..

          
          # Automaticaly generated, dont edit manually.
          # Generated on: 2017-12-07 22:10
          global
          	maxconn			10
          	stats socket /tmp/haproxy.socket level admin
          	uid			80
          	gid			80
          	nbproc			1
          	chroot			/tmp/haproxy_chroot
          	daemon
          	tune.ssl.default-dh-param	2048
          	log-send-hostname		HAproxy
          	server-state-file /tmp/haproxy_server_state
          
          listen HAProxyLocalStats
          	bind 127.0.0.1:2200 name localstats
          	mode http
          	stats enable
          	stats admin if TRUE
          	stats uri /haproxy/haproxy_stats.php?haproxystats=1
          	timeout client 5000
          	timeout connect 5000
          	timeout server 5000
          
          frontend WAN_HTTP
          	bind			wan_ip:80 name wan_ip:80   
          	mode			http
          	log			global
          	option			http-keep-alive
          	timeout client		30000
          	default_backend ssl-redirect_http_ipvANY
          
          frontend WAN_HTTPS
          	bind			wan_ip:443 name wan_ip:443   
          	mode			http
          	log			global
          	option			http-keep-alive
          	timeout client		30000
          	acl			www-acl	hdr(host) -i www.domain.ca
          	acl			cloud-acl	hdr(host) -i cloud.domain.ca
          	acl			aclcrt_WAN_HTTPS	hdr_reg(host) -i ^www\.domain\.ca(:([0-9]){1,5})?$
          	acl			aclcrt_WAN_HTTPS	hdr_reg(host) -i ^cloud\.domain\.ca(:([0-9]){1,5})?$
          	acl			aclcrt_WAN_HTTPS	hdr_reg(host) -i ^domain\.ca(:([0-9]){1,5})?$
          	use_backend www_http_ipvANY  if  www-acl aclcrt_WAN_HTTPS 
          	use_backend cloud_http_ipvANY  if  cloud-acl aclcrt_WAN_HTTPS 
          
          backend ssl-redirect_http_ipvANY
          	mode			http
          	log			global
          	timeout connect		30000
          	timeout server		30000
          	retries			3
          	server			redirect 127.0.0.1  
          
          backend www_http_ipvANY
          	mode			http
          	log			global
          	timeout connect		30000
          	timeout server		30000
          	retries			3
          	server			www-acl 192.168.1.100:8090 ssl  verify none crt /var/etc/haproxy/server_clientcert_5a22d36348213.pem 
          
          backend cloud_http_ipvANY
          	mode			http
          	log			global
          	timeout connect		30000
          	timeout server		30000
          	retries			3
          	server			cloud 192.168.1.100:8082
          
          
          1 Reply Last reply Reply Quote 0
          • P
            PiBa
            last edited by

            For https does your webserver 'require' sending a client certificate? If not make the ssl related selection boxes there on the backend server empty..
            As for the frontend youve got it configured with http/https(offloading) but have not configured the server certificate on the frontend. That is required to be able to read the host-header for the "hdr_reg(host)" acl's youve used.

            As for the 'redirect' backend it seems your pointing it to the pfSense localhost webgui that is then sending a redirect.. Better configure the redirect in haproxy itself as a 'action' if desired, or point it to the actual webserver:80 for testing.?

            Also it might be that the overall webgui redirect in pfSense advanced settings is confusing you.. Or a cached HSTS header that the webgui sends.. To remove that HSTS redirect from browser cache special steps are needed in the browser used..

            1 Reply Last reply Reply Quote 0
            • U
              uwscia
              last edited by

              So the certs get set in the frontend or backend?

              1 Reply Last reply Reply Quote 0
              • U
                uwscia
                last edited by

                Okay, I'm getting somewhere, I finally got working frontend and backends… I'm just getting an error when I try to use acme cert vs the default cert.

                
                # Automaticaly generated, dont edit manually.
                # Generated on: 2017-12-08 18:36
                global
                	maxconn			10
                	stats socket /tmp/haproxy.socket level admin
                	uid			80
                	gid			80
                	nbproc			1
                	chroot			/tmp/haproxy_chroot
                	daemon
                	tune.ssl.default-dh-param	2048
                	log-send-hostname		HAproxy
                	server-state-file /tmp/haproxy_server_state
                	ssl-default-bind-ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!3DES:!MD5:!PSK
                
                listen HAProxyLocalStats
                	bind 127.0.0.1:2200 name localstats
                	mode http
                	stats enable
                	stats admin if TRUE
                	stats uri /haproxy/haproxy_stats.php?haproxystats=1
                	timeout client 5000
                	timeout connect 5000
                	timeout server 5000
                
                frontend ssl-redirect
                	bind			wan_ip:80 name wan_ip:80   
                	mode			http
                	log			global
                	option			http-keep-alive
                	timeout client		30000
                	default_backend ssl-redirect_http_ipvANY
                
                frontend HTTPS_Frontend-merged
                	bind			wan_ip:443 name wan_ip:443 ssl no-sslv3 crt /var/etc/haproxy/HTTPS_Frontend.pem  
                	mode			http
                	log			global
                	option			http-keep-alive
                	option			forwardfor
                	acl https ssl_fc
                	http-request set-header		X-Forwarded-Proto http if !https
                	http-request set-header		X-Forwarded-Proto https if https
                	timeout client		7200000
                	# Remove headers that expose security-sensitive information.
                	rspidel ^Server:.*$
                	rspidel ^X-Powered-By:.*$
                	rspidel ^X-AspNet-Version:.*$
                	acl			aclcrt_HTTPS_Frontend	hdr_reg(host) -i ^pfSense-5a22d36348213(:([0-9]){1,5})?$
                	acl			sub1	hdr(host) -i sub1.domain.ca
                	use_backend dummy_backend_http_ipvANY  if   aclcrt_HTTPS_Frontend
                	default_backend dummy_backend_http_ipvANY
                	default_backend sub1_http_ipvANY
                
                backend ssl-redirect_http_ipvANY
                	mode			http
                	log			global
                	timeout connect		30000
                	timeout server		30000
                	retries			3
                	redirect scheme https code 301
                
                backend dummy_backend_http_ipvANY
                	mode			http
                	log			global
                	timeout connect		30000
                	timeout server		30000
                	retries			3
                	server			dummy_backend 127.0.0.1:8888 disabled 
                
                backend sub1_http_ipvANY
                	mode			http
                	log			global
                	stats			enable
                	stats			uri /euc_haproxy?stats
                	stats			realm .
                	timeout connect		30000
                	timeout server		30000
                	retries			3
                	option			httpchk GET /services/Heartbeat 
                	server			sub1-svr 192.168.1.100:8090 check inter 1000
                
                
                1 Reply Last reply Reply Quote 0
                • P
                  PiBa
                  last edited by

                  Getting 'an error' is fixed by performing 'an solution'.
                  Does haproxy report an error while starting? Does the browser report an error while loading the page? Does the machine catch fire ::) Please specify 'what' error specifically you get and where/when it happens that really helps both you and me..

                  Anyhow..

                  	acl			aclcrt_HTTPS_Frontend	hdr_reg(host) -i ^pfSense-5a22d36348213(:([0-9]){1,5})?$
                  	acl			sub1	hdr(host) -i sub1.domain.ca
                  	use_backend dummy_backend_http_ipvANY  if   aclcrt_HTTPS_Frontend
                  	default_backend dummy_backend_http_ipvANY
                  	default_backend sub1_http_ipvANY
                  
                  

                  Having 2 default's and a unused sub1 acl probably creates a few issues.
                  Use the use-backend action to select a backend when the desired sub1 acl matches instead of specifying a default backend for each frontend without further certificate derived 'automatic' acl's.

                  As for certificates, those that are presented to a client/browser are indeed specified on the frontend. But seeing your config that seems good now.

                  If the webserver also uses https, and requires a client certificate or you just want to verify its a valid certificate, then haproxy can do that with the certificate options on the server.. But this is usually not needed to get 'things working', but could improve security if the network between haproxy and webserver is not 'trusted'.. For now lets leave this alone and focus on getting the basics working.

                  Locally http://192.168.1.100:8090/ works without https right.?

                  1 Reply Last reply Reply Quote 0
                  • U
                    uwscia
                    last edited by

                    Ok, the error I get if I move away from the webConfigurator default cert is…
                    [ALERT] 342/082804 (46521) : parsing [/var/etc/haproxy_test/haproxy.cfg:36] : 'bind WAN_IP:443' : unable to load SSL certificate from PEM file

                    Also when I try to add another shared front end with separate backend it will throw a 503 for all https requests

                    So I don't need seperate backend for each server?

                    I need plex.domain to go to 1 server
                    and www.domain to go to a pool of 4 servers
                    plus a few more domains…

                    and then for the cert(s) I was thinking I could use 1 for all domains but then I'll need a seperate 1 "vpn.domain" for openVPN

                    Yes http://192.168.1.100:8090/ works

                    1 Reply Last reply Reply Quote 0
                    • U
                      uwscia
                      last edited by

                      
                      # Automaticaly generated, dont edit manually.
                      # Generated on: 2017-12-10 08:09
                      global
                      	maxconn			10
                      	stats socket /tmp/haproxy.socket level admin
                      	uid			80
                      	gid			80
                      	nbproc			1
                      	chroot			/tmp/haproxy_chroot
                      	daemon
                      	tune.ssl.default-dh-param	2048
                      	log-send-hostname		HAproxy
                      	server-state-file /tmp/haproxy_server_state
                      	ssl-default-bind-ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!3DES:!MD5:!PSK
                      
                      listen HAProxyLocalStats
                      	bind 127.0.0.1:2200 name localstats
                      	mode http
                      	stats enable
                      	stats admin if TRUE
                      	stats uri /haproxy/haproxy_stats.php?haproxystats=1
                      	timeout client 5000
                      	timeout connect 5000
                      	timeout server 5000
                      
                      frontend Frontend1-http
                      	bind			WAN_IP:80 name WAN_IP:80   
                      	mode			http
                      	log			global
                      	option			http-keep-alive
                      	timeout client		60000
                      	acl			httpRedirectAcl	hdr(host) -i portal.domain.ca
                      	acl			httpRedirectAcl	hdr_beg(host) -i support.
                      	acl			httpRedirectAcl	hdr(host) -i piwik.domain.ca
                      	acl			httpRedirectAcl	hdr(host) -i clan.domain.ca
                      	acl			httpRedirectAcl	hdr(host) -i plex.domain.ca
                      	acl			httpRedirectAcl	hdr(host) -i mine.domain.ca
                      	acl			httpRedirectAcl	hdr(host) -i cloud.domain.ca
                      	http-request redirect scheme https  if  httpRedirectAcl 
                      	default_backend backend-www_http_ipvANY
                      
                      frontend Frontend2-SNI
                      	bind			WAN_IP:443 name WAN_IP:443   
                      	mode			tcp
                      	log			global
                      	timeout client		60000
                      	tcp-request inspect-delay	5s
                      	acl			aclSupport	req.ssl_sni -i support.domain.ca
                      	acl			aclForum	req.ssl_sni -m beg -i forum.
                      	tcp-request content accept if { req.ssl_hello_type 1 }
                      
                      	tcp-request content accept if { req.ssl_hello_type 1 }
                      
                      	use_backend backend-support_https_ipvANY  if  aclSupport 
                      	use_backend backend-forum_https_ipvANY  if  aclForum 
                      	default_backend fronend3-offloading_https_ipvANY
                      
                      frontend Frontend3-offloading
                      	bind			127.0.0.1:1443 name 127.0.0.1:1443 ssl no-sslv3 crt /var/etc/haproxy/Frontend3-offloading.pem  
                      	bind /tmp/haproxy_chroot/Frontend3-offloading.socket name unixsocket uid 80 accept-proxy ssl no-sslv3 crt /var/etc/haproxy/Frontend3-offloading.pem 
                      	mode			http
                      	log			global
                      	option			http-keep-alive
                      	option			forwardfor
                      	acl https ssl_fc
                      	http-request set-header		X-Forwarded-Proto http if !https
                      	http-request set-header		X-Forwarded-Proto https if https
                      	timeout client		60000
                      	acl			aclPortal	hdr(host) -i portal.domain.ca
                      	acl			aclClan	hdr(host) -i clan.domain.ca
                      	acl			aclCloud	hdr(host) -i cloud.domain.ca
                      	acl			aclPiwik	hdr(host) -i piwik.domain.ca
                      	acl			aclPlex	hdr(host) -i plex.domain.ca
                      	acl			aclmineOS	hdr(host) -i mine.domain.ca
                      	use_backend backend-portal_http_ipvANY  if  aclPortal 
                      	use_backend backend-clan_http_ipvANY  if  aclClan 
                      	use_backend backend-cloud_http_ipvANY  if  aclCloud 
                      	use_backend backend-piwik_http_ipvANY  if  aclPiwik 
                      	use_backend backend-plex_http_ipvANY  if  aclPlex 
                      	use_backend backend-mineOS_http_ipvANY  if  aclmineOS 
                      	default_backend backend-www_http_ipvANY
                      
                      backend backend-www_http_ipvANY
                      	mode			http
                      	log			global
                      	timeout connect		30000
                      	timeout server		30000
                      	retries			3
                      	option			httpchk OPTIONS / 
                      	server			Default 192.168.1.100:19999 check inter 1000  
                      
                      backend backend-support_https_ipvANY
                      	mode			tcp
                      	log			global
                      	timeout connect		30000
                      	timeout server		30000
                      	retries			3
                      	option			httpchk OPTIONS / 
                      
                      backend backend-forum_https_ipvANY
                      	mode			tcp
                      	log			global
                      	timeout connect		30000
                      	timeout server		30000
                      	retries			3
                      	option			httpchk OPTIONS / 
                      
                      backend fronend3-offloading_https_ipvANY
                      	mode			tcp
                      	log			global
                      	timeout connect		30000
                      	timeout server		30000
                      	retries			3
                      	server			frontend3-srv /Frontend3-offloading.socket send-proxy-v2-ssl-cn check inter 5000  
                      
                      backend backend-portal_http_ipvANY
                      	mode			http
                      	log			global
                      	timeout connect		30000
                      	timeout server		30000
                      	retries			3
                      	option			httpchk OPTIONS / 
                      
                      backend backend-clan_http_ipvANY
                      	mode			http
                      	log			global
                      	timeout connect		30000
                      	timeout server		30000
                      	retries			3
                      	option			httpchk OPTIONS / 
                      	server			Clan 192.168.1.100:8090 check inter 1000  
                      
                      backend backend-cloud_http_ipvANY
                      	mode			http
                      	log			global
                      	rspadd Strict-Transport-Security:\ max-age=15552000;
                      	timeout connect		30000
                      	timeout server		30000
                      	retries			3
                      	server			Cloud 192.168.1.100:8082 check inter 1000  
                      
                      backend backend-piwik_http_ipvANY
                      	mode			http
                      	log			global
                      	timeout connect		30000
                      	timeout server		30000
                      	retries			3
                      	option			httpchk OPTIONS / 
                      	server			Piwik 192.168.1.100:8190 check inter 1000  
                      
                      backend backend-plex_http_ipvANY
                      	mode			http
                      	log			global
                      	timeout connect		30000
                      	timeout server		30000
                      	retries			3
                      	server			Plex 192.168.1.100:32400 check inter 1000  
                      
                      backend backend-mineOS_http_ipvANY
                      	mode			http
                      	log			global
                      	timeout connect		30000
                      	timeout server		30000
                      	retries			3
                      	option			httpchk OPTIONS /
                      
                      
                      1 Reply Last reply Reply Quote 0
                      • U
                        uwscia
                        last edited by

                        COMPLETED… TY PiBa

                        1 Reply Last reply Reply Quote 0
                        • First post
                          Last post
                        Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.