Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Haproxy to DMZ not working

    Scheduled Pinned Locked Moved Cache/Proxy
    6 Posts 3 Posters 1.6k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • H
      hhbarnes
      last edited by

      Everyone,

      I'm trying to setup Haproxy to load balance between 4 servers hosting an NGNIX server listening on port 3001 that I have set up in my DMZ. I've tested each of the servers from the Lan and they are working properly. I've also checked Haproxy and it is also reporting that the servers are alive as well. However, when I try to access the site from the internet I get a 503 error and I can't seem to find anything in the logs to correspond with the error.

      To see if I did something wrong in the DMZ, I moved the servers on to my LAN. I then reconfigured the Haproxy backends to point to the servers in the LAN. In this configuration, I was able to successfully reach the servers, so I've determined its probably something I haven't configured correctly in the DMZ. Currently, my DMZ is locked down and I'm only allowing egress traffic over a few ports to allow the web applications to work. Are there any other rules I need to add to allow Haproxy to route the requests properly?

      I'm new to Haproxy & pfSense so I apologize if I'm missing something obvious.

      Thanks in advance,
      Harry

      P 1 Reply Last reply Reply Quote 0
      • P
        PiBa @hhbarnes
        last edited by

        @hhbarnes
        What does your haproxy.cfg file look like? (can be seen on the bottom of settings tab of haproxy)

        1 Reply Last reply Reply Quote 0
        • H
          hhbarnes
          last edited by

          Sure thing, I've attached two configurations, the first is when the server is on my LAN, the second is when it is on the DMZ. I've also attached the firewall logs for the two configurations as well. I noticed that the DMZ configuration is blocking communication from the firewall to the client, but there are no issues when it is connected to the LAN. Another observation is that on the Haproxy dashboard widget I can see the session number increase when I request the page from my browser so it looks like Haproxy is doing its job correctly.

          ===========================================
Configuration when Server is on LAN (Works)
===========================================
Content
# Automaticaly generated, dont edit manually.
# Generated on: 2019-02-02 18:30
global
	maxconn			1000
	stats socket /tmp/haproxy.socket level admin  expose-fd listeners
	gid			80
	nbproc			1
	nbthread			1
	hard-stop-after		15m
	chroot				/tmp/haproxy_chroot
	daemon
	tune.ssl.default-dh-param	2048
	server-state-file /tmp/haproxy_server_state

listen HAProxyLocalStats
	bind 127.0.0.1:2200 name localstats
	mode http
	stats enable
	stats admin if TRUE
	stats show-legends
	stats uri /haproxy/haproxy_stats.php?haproxystats=1
	timeout client 5000
	timeout connect 5000
	timeout server 5000

frontend shared-frontend-merged
	bind			71.XXX.XXX.XXX:443 name 71.XXX.XXX.XXX:443   ssl crt-list /var/etc/haproxy/shared-frontend.crt_list  
	mode			http
	log			global
	option			http-keep-alive
	option			forwardfor
	acl https ssl_fc
	http-request set-header		X-Forwarded-Proto http if !https
	http-request set-header		X-Forwarded-Proto https if https
	timeout client		30000
	acl			aclcrt_shared-frontend	var(txn.txnhost) -m reg -i ^AAAAAA\.in(:([0-9]){1,5})?$
	acl			aclcrt_shared-frontend	var(txn.txnhost) -m reg -i ^([^\.]*)\.AAAAAA\.in(:([0-9]){1,5})?$
	acl			aclcrt_shared-frontend	var(txn.txnhost) -m reg -i ^([^\.]*)\.BBBBBB\.com(:([0-9]){1,5})?$
	acl			aclcrt_shared-frontend	var(txn.txnhost) -m reg -i ^([^\.]*)\.CCCCCC\.com(:([0-9]){1,5})?$
	acl			aclcrt_shared-frontend	var(txn.txnhost) -m reg -i ^([^\.]*)\.DDDDDD\.com(:([0-9]){1,5})?$
	acl			aclcrt_shared-frontend	var(txn.txnhost) -m reg -i ^BBBBBB\.com(:([0-9]){1,5})?$
	acl			aclcrt_shared-frontend	var(txn.txnhost) -m reg -i ^CCCCCC\.com(:([0-9]){1,5})?$
	acl			aclcrt_shared-frontend	var(txn.txnhost) -m reg -i ^DDDDDD\.com(:([0-9]){1,5})?$
	acl			BBBBBB.com	var(txn.txnhost) -m end -i BBBBBB.com
	http-request set-var(txn.txnhost) hdr(host)
	use_backend BBBBBB.LAN_ipv4  if  BBBBBB.com 

frontend http-to-https
	bind			71.XXX.XXX.XXX:80 name 71.XXX.XXX.XXX:80   
	mode			http
	log			global
	option			http-keep-alive
	timeout client		30000
	http-request redirect scheme https 

backend BBBBBB.LAN_ipv4
	mode			http
	id			10107
	log			global
	option			log-health-checks
	timeout connect		30000
	timeout server		30000
	retries			3
	source ipv4@ usesrc clientip
	option			httpchk GET / 
	server			pine-04 192.168.1.44:3001 id 10101 check inter 1000
	
====================================================
Firewall Logs for Page when server is on LAN (Works)
====================================================
Feb 2 18:37:59	WAN	 Allow HTTPS Inbound (1546899421)	  107.182.230.243:33164	  71.XXX.XXX.XXX:443	TCP:S
Feb 2 18:37:59	WAN	 Allow HTTPS Inbound (1546899421)	  107.182.230.243:33165	  71.XXX.XXX.XXX:443	TCP:S
Feb 2 18:37:59	WAN	 Allow HTTPS Inbound (1546899421)	  107.182.230.243:33163	  71.XXX.XXX.XXX:443	TCP:S
Feb 2 18:37:59	WAN	 Allow HTTPS Inbound (1546899421)	  107.182.230.243:33162	  71.XXX.XXX.XXX:443	TCP:S
Feb 2 18:37:59	WAN	 Allow HTTPS Inbound (1546899421)	  107.182.230.243:33160	  71.XXX.XXX.XXX:443	TCP:S
Feb 2 18:37:59	WAN	 Allow HTTPS Inbound (1546899421)	  107.182.230.243:33141	  71.XXX.XXX.XXX:443	TCP:S



==================================================
Configuration when Server is on DMZ (Doesn't work)
==================================================
# Automaticaly generated, dont edit manually.
# Generated on: 2019-02-02 18:47
global
	maxconn			1000
	stats socket /tmp/haproxy.socket level admin  expose-fd listeners
	gid			80
	nbproc			1
	nbthread			1
	hard-stop-after		15m
	chroot				/tmp/haproxy_chroot
	daemon
	tune.ssl.default-dh-param	2048
	server-state-file /tmp/haproxy_server_state

listen HAProxyLocalStats
	bind 127.0.0.1:2200 name localstats
	mode http
	stats enable
	stats admin if TRUE
	stats show-legends
	stats uri /haproxy/haproxy_stats.php?haproxystats=1
	timeout client 5000
	timeout connect 5000
	timeout server 5000

frontend shared-frontend-merged
	bind			71.XXX.XXX.XXX:443 name 71.XXX.XXX.XXX:443   ssl crt-list /var/etc/haproxy/shared-frontend.crt_list  
	mode			http
	log			global
	option			http-keep-alive
	option			forwardfor
	acl https ssl_fc
	http-request set-header		X-Forwarded-Proto http if !https
	http-request set-header		X-Forwarded-Proto https if https
	timeout client		30000
	acl			aclcrt_shared-frontend	var(txn.txnhost) -m reg -i ^AAAAAA\.in(:([0-9]){1,5})?$
	acl			aclcrt_shared-frontend	var(txn.txnhost) -m reg -i ^([^\.]*)\.AAAAAA\.in(:([0-9]){1,5})?$
	acl			aclcrt_shared-frontend	var(txn.txnhost) -m reg -i ^([^\.]*)\.BBBBBBBBBBBBB\.com(:([0-9]){1,5})?$
	acl			aclcrt_shared-frontend	var(txn.txnhost) -m reg -i ^([^\.]*)\.CCCCCCC\.com(:([0-9]){1,5})?$
	acl			aclcrt_shared-frontend	var(txn.txnhost) -m reg -i ^([^\.]*)\.DDDDDDD\.com(:([0-9]){1,5})?$
	acl			aclcrt_shared-frontend	var(txn.txnhost) -m reg -i ^BBBBBBBBBBBBB\.com(:([0-9]){1,5})?$
	acl			aclcrt_shared-frontend	var(txn.txnhost) -m reg -i ^CCCCCCC\.com(:([0-9]){1,5})?$
	acl			aclcrt_shared-frontend	var(txn.txnhost) -m reg -i ^DDDDDDD\.com(:([0-9]){1,5})?$
	acl			BBBBBBBBBBBBB.com	var(txn.txnhost) -m end -i BBBBBBBBBBBBB.com
	http-request set-var(txn.txnhost) hdr(host)
	use_backend BBBBBBBBBBBBB.com_ipv4  if  BBBBBBBBBBBBB.com 

frontend http-to-https
	bind			71.XXX.XXX.XXX:80 name 71.XXX.XXX.XXX:80   
	mode			http
	log			global
	option			http-keep-alive
	timeout client		30000
	http-request redirect scheme https 

backend BBBBBBBBBBBBB.com_ipv4
	mode			http
	id			10102
	log			global
	option			log-health-checks
	timeout connect		30000
	timeout server		30000
	retries			3
	source ipv4@ usesrc clientip
	option			httpchk GET / 
	server			pine-01 10.0.1.41:3001 id 10101 check inter 1000
	
	
===========================================================
Firewall Logs for Page when server is on DMZ (Doesn't Work)
===========================================================
Feb 2 19:01:21	► WAN	Default deny rule IPv4 (1000000104)	  71.XXX.XXX.XXX:45513	  107.182.230.242:53148	TCP:SA
Feb 2 19:01:19	► WAN	Default deny rule IPv4 (1000000104)	  71.XXX.XXX.XXX:12325	  107.182.230.242:18353	TCP:SA
Feb 2 19:01:18	► WAN	Default deny rule IPv4 (1000000104)	  71.XXX.XXX.XXX:10087	  107.182.230.242:53148	TCP:SA
Feb 2 19:01:17	► WAN	Default deny rule IPv4 (1000000104)	  71.XXX.XXX.XXX:7195	  107.182.230.242:53148	TCP:SA
Feb 2 19:01:15	► WAN	Default deny rule IPv4 (1000000104)	  71.XXX.XXX.XXX:12836	  107.182.230.242:53148	TCP:SA
Feb 2 19:01:13	► WAN	Default deny rule IPv4 (1000000104)	  71.XXX.XXX.XXX:13321	  107.182.230.242:53148	TCP:SA
Feb 2 19:01:12	► WAN	Default deny rule IPv4 (1000000104)	  71.XXX.XXX.XXX:53919	  107.182.230.242:53148	TCP:SA
Feb 2 19:01:12	► WAN	Default deny rule IPv4 (1000000104)	  71.XXX.XXX.XXX:15333	  107.182.230.242:18362	TCP:SA
Feb 2 19:01:05	► WAN	Default deny rule IPv4 (1000000104)	  71.XXX.XXX.XXX:39372	  107.182.230.242:18353	TCP:SA
Feb 2 19:01:04	► WAN	Default deny rule IPv4 (1000000104)	  71.XXX.XXX.XXX:56889	  107.182.230.242:18362	TCP:SA
Feb 2 19:01:02	► WAN	Default deny rule IPv4 (1000000104)	  71.XXX.XXX.XXX:6576	  107.182.230.242:18362	TCP:SA
Feb 2 19:00:58	► WAN	Default deny rule IPv4 (1000000104)	  71.XXX.XXX.XXX:29797	  107.182.230.242:18362	TCP:SA
Feb 2 19:00:57	► WAN	Default deny rule IPv4 (1000000104)	  71.XXX.XXX.XXX:25852	  107.182.230.242:18332	TCP:SA
Feb 2 19:00:57	► WAN	Default deny rule IPv4 (1000000104)	  71.XXX.XXX.XXX:38859	  107.182.230.242:18353	TCP:SA
Feb 2 19:00:54	► WAN	Default deny rule IPv4 (1000000104)	  71.XXX.XXX.XXX:35318	  107.182.230.242:18362	TCP:SA
Feb 2 19:00:54	► WAN	Default deny rule IPv4 (1000000104)	  71.XXX.XXX.XXX:48667	  107.182.230.242:18353	TCP:SA
Feb 2 19:00:51	► WAN	Default deny rule IPv4 (1000000104)	  71.XXX.XXX.XXX:53569	  107.182.230.242:18362	TCP:SA
Feb 2 19:00:50	► WAN	Default deny rule IPv4 (1000000104)	  71.XXX.XXX.XXX:44300	  107.182.230.242:18353	TCP:SA
Feb 2 19:00:49	► WAN	Default deny rule IPv4 (1000000104)	  71.XXX.XXX.XXX:3502	  107.182.230.242:18324	TCP:SA
Feb 2 19:00:48	► WAN	Default deny rule IPv4 (1000000104)	  71.XXX.XXX.XXX:37541	  107.182.230.242:18362	TCP:SA
Feb 2 19:00:47	► WAN	Default deny rule IPv4 (1000000104)	  71.XXX.XXX.XXX:53128	  107.182.230.242:18353	TCP:SA
Feb 2 19:00:47	► WAN	Default deny rule IPv4 (1000000104)	  71.XXX.XXX.XXX:14512	  107.182.230.242:18362	TCP:SA
Feb 2 19:00:45	► WAN	Default deny rule IPv4 (1000000104)	  71.XXX.XXX.XXX:7715	  107.182.230.242:18362	TCP:SA
Feb 2 19:00:44	► WAN	Default deny rule IPv4 (1000000104)	  71.XXX.XXX.XXX:15218	  107.182.230.242:18353	TCP:SA
Feb 2 19:00:43	► WAN	Default deny rule IPv4 (1000000104)	  71.XXX.XXX.XXX:47133	  107.182.230.242:18362	TCP:SA
Feb 2 19:00:42	► WAN	Default deny rule IPv4 (1000000104)	  71.XXX.XXX.XXX:16575	  107.182.230.242:18362	TCP:SA
Feb 2 19:00:42	► WAN	Default deny rule IPv4 (1000000104)	  71.XXX.XXX.XXX:63709	  107.182.230.242:18332	TCP:SA
Feb 2 19:00:41	► WAN	Default deny rule IPv4 (1000000104)	  71.XXX.XXX.XXX:48244	  107.182.230.242:18353	TCP:SA
Feb 2 19:00:40	► WAN	Default deny rule IPv4 (1000000104)	  71.XXX.XXX.XXX:2504	  107.182.230.242:18353	TCP:SA
Feb 2 19:00:38	► WAN	Default deny rule IPv4 (1000000104)	  71.XXX.XXX.XXX:18308	  107.182.230.242:18353	TCP:SA
Feb 2 19:00:36	► WAN	Default deny rule IPv4 (1000000104)	  71.XXX.XXX.XXX:55553	  107.182.230.242:18353	TCP:SA
Feb 2 19:00:35	► WAN	Default deny rule IPv4 (1000000104)	  71.XXX.XXX.XXX:10284	  107.182.230.242:18353	TCP:SA
Feb 2 19:00:34	► WAN	Default deny rule IPv4 (1000000104)	  71.XXX.XXX.XXX:31715	  107.182.230.242:18324	TCP:SA
Feb 2 19:00:34	► WAN	Default deny rule IPv4 (1000000104)	  71.XXX.XXX.XXX:8533	  107.182.230.242:18332	TCP:SA
Feb 2 19:00:32	► WAN	Default deny rule IPv4 (1000000104)	  71.XXX.XXX.XXX:41424	  107.182.230.242:18332	TCP:SA
Feb 2 19:00:28	► WAN	Default deny rule IPv4 (1000000104)	  71.XXX.XXX.XXX:12983	  107.182.230.242:18332	TCP:SA
Feb 2 19:00:27	► WAN	Default deny rule IPv4 (1000000104)	  71.XXX.XXX.XXX:25773	  107.182.230.242:18324	TCP:SA
Feb 2 19:00:25	► WAN	Default deny rule IPv4 (1000000104)	  71.XXX.XXX.XXX:19495	  107.182.230.242:18332	TCP:SA
Feb 2 19:00:25	► WAN	Default deny rule IPv4 (1000000104)	  71.XXX.XXX.XXX:40138	  107.182.230.242:18324	TCP:SA
Feb 2 19:00:22	► WAN	Default deny rule IPv4 (1000000104)	  71.XXX.XXX.XXX:49320	  107.182.230.242:18332	TCP:SA
Feb 2 19:00:21	► WAN	Default deny rule IPv4 (1000000104)	  71.XXX.XXX.XXX:9931	  107.182.230.242:18324	TCP:SA
Feb 2 19:00:20	► WAN	Default deny rule IPv4 (1000000104)	  71.XXX.XXX.XXX:49203	  107.182.230.242:58725	TCP:SA
Feb 2 19:00:19	► WAN	Default deny rule IPv4 (1000000104)	  71.XXX.XXX.XXX:3118	  107.182.230.242:18332	TCP:SA
Feb 2 19:00:18	► WAN	Default deny rule IPv4 (1000000104)	  71.XXX.XXX.XXX:31800	  107.182.230.242:18324	TCP:SA
Feb 2 19:00:16	► WAN	Default deny rule IPv4 (1000000104)	  71.XXX.XXX.XXX:1879	  107.182.230.242:18332	TCP:SA
Feb 2 19:00:16	► WAN	Default deny rule IPv4 (1000000104)	  71.XXX.XXX.XXX:37333	  107.182.230.242:18332	TCP:SA
Feb 2 19:00:15	► WAN	Default deny rule IPv4 (1000000104)	  71.XXX.XXX.XXX:55368	  107.182.230.242:18324	TCP:SA
Feb 2 19:00:13	► WAN	Default deny rule IPv4 (1000000104)	  71.XXX.XXX.XXX:51370	  107.182.230.242:18332	TCP:SA
Feb 2 19:00:12	► WAN	Default deny rule IPv4 (1000000104)	  71.XXX.XXX.XXX:48781	  107.182.230.242:18332	TCP:SA
Feb 2 19:00:11	► WAN	Default deny rule IPv4 (1000000104)	  71.XXX.XXX.XXX:60948	  107.182.230.242:18324	TCP:SA
Feb 2 19:00:12	WAN	 Allow HTTPS Inbound (1546899421)	  107.182.230.242:48384	  71.XXX.XXX.XXX:443	TCP:S
          P 1 Reply Last reply Reply Quote 0
          • P
            PiBa @hhbarnes
            last edited by

            @hhbarnes
            As you are using the 'Transparent ClientIP' please make sure that when you change the IP of the server in the configuration, you also sset the correct interface to put the reply-capture-rule on in the backend advanced setting. If this is not set to the right interface, that would explain the Syn-Ack packets from the backend not getting redirected to the haproxy process but trying to leave on the wan interface.

            H 1 Reply Last reply Reply Quote 0
            • H
              hhbarnes @PiBa
              last edited by

              @piba I owe you one, that was exactly the problem. I had it configured on the LAN first and then copied the configuration and I thought I changed everything to point to the DMZ. Once I changed the interface on the transparent IP it worked.

              Thanks again!

              C 1 Reply Last reply Reply Quote 0
              • C
                carlostico @hhbarnes
                last edited by

                @hhbarnes

                Is it better to use HAPROXY with DMZ ?

                1 Reply Last reply Reply Quote 0
                • First post
                  Last post
                Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.