pfSense, Haproxy, cloudflare cname DDNS letsencrypt certs Timeout

[freak4915]

Dear All,

I am at my wits end and an sure its something simple and stupid I'm missing here, but I have been looking so long at this problem that it's like proof reading your own work and not seeing any faults in your work.

I have a small feeling it might be cloudflare as I only recently switched over to them, but ping do correctly resolve to the desired address, as I did disable proxy to try and remove things in the middle possibly causing my issue.

So I have DNSEXIT set up with an address which is updated to my WAN IP from pfSense. I have CNAMES in cloudflare for the desired public sites.

So if I ping test.{MyDomain} I hit the cname which resolves to the DNSEXIT address, which resolves to my WAN IP.

This worked for a long time with squid reverse proxy, but wish to do the same in haproxy for it's letsencrypt integration.

I have a fresh install of pfSense, where the WAN is my dynamically assigned public facing internet address.

I have two WAN Firewall rules
IPv4 TCP * Source * Port This Firewall Destination 80 (HTTP) Port * Gateway
IPv4 TCP * Source * Port This Firewall Destination 443 (HTTPS) Port * Gateway

I have switched the pfSense webgui to port 8443 (HTTPS)

I have Pure NAT set under NAT Reflection mode for port forwards and Enable NAT Reflection for 1:1 NAT and Enable automatic outbound NAT for Reflection Ticked

This allowed me to access the sites internally, but only because it bypassed haproxy completely.

DNS Resolver enabled on all ports with Register DHCP leases in the DNS Resolver and Register DHCP static mappings in the DNS Resolver enabled.

I have no NAT port forwarding rules, no 1:1 NAT rules, Outbound has no mapping and is in Automatic outbound NAT mode

My HAPROXY CONF file

Automaticaly generated, dont edit manually.

Generated on: 2020-05-13 11:01

global
maxconn 1000
stats socket /tmp/haproxy.socket level admin expose-fd listeners
uid 80
gid 80
nbproc 1
nbthread 1
hard-stop-after 15m
chroot /tmp/haproxy_chroot
daemon
tune.ssl.default-dh-param 2052
server-state-file /tmp/haproxy_server_state

listen HAProxyLocalStats
bind 127.0.0.1:2200 name localstats
mode http
stats enable
stats admin if TRUE
stats show-legends
stats uri /haproxy/haproxy_stats.php?haproxystats=1
timeout client 5000
timeout connect 5000
timeout server 5000

frontend Test
bind {WAN IP}:443 name {WAN IP}:443 ssl crt-list /var/etc/haproxy/{CERT LOCATION}.crt_list
mode http
log global
option http-keep-alive
timeout client 30000
acl ACL1 var(txn.txnhost) -m str -i test.{MyDOMAIN}
acl aclcrt_test var(txn.txnhost) -m reg -i ^([^.]*).{My}.{DOMAIN}(:([0-9]){1,5})?$
acl aclcrt_test var(txn.txnhost) -m reg -i ^{My}.{DOMAIN}(:([0-9]){1,5})?$
http-request set-var(txn.txnhost) hdr(host)
use_backend test_ipvANY if ACL1 aclcrt_test

backend test_ipvANY
mode http
id 100
log global
timeout connect 30000
timeout server 30000
retries 3
option httpchk OPTIONS /
server test {Internal IP}:443 id 101 ssl check inter 1000 verify none crt /var/etc/haproxy/server_clientcert_5eb58f9641a2d.pem

When I try and hit the address test.{MyDoamin} just get a 30sec pause before getting a connection timeout error. From what I have read usually means it's a firewall/NAT issue where it's hitting the firewall, not going anywhere and then the browser times out. but I can't see why.

I have green for the backend on haproxy stats page.
I can hit the site internally using the test.{MyDomain} so know the site is operational.

Could someone please have a look into this problem?

[PiBa]

@freak4915 said in pfSense, Haproxy, cloudflare cname DDNS letsencrypt certs Timeout:

IPv4 TCP * Source * Port This Firewall Destination 443 (HTTPS) Port * Gateway

No exactly sure how to read that, if you have a gateway filled in in the rule can you remove that? Other than that there shouldn't be any issues with the config you have.. Not regarding a client-connection timeout anyhow..

All the NAT and reflection settings should be irrelevant for the connection through haproxy..

On the haproxy stats, are the frontends counting incoming connections? I think not.. as i would then expect a different error result in the browser or perhaps a proper response..

Can you check on pfSense console that connection attempts are actually arriving on your wan interface?
Login with ssh go to the shell, then run something like

tcpdump -ni WAN-NIC port 443 and host 1.2.3.4

where WAN-NIC must be the name of the actual nic like em1 or igb2 or rt3 vtnet4 or pppoe5 so what your wan nic is actually called ;) and the 1.2.3.4 must be the public ip of the client that tries to connect. Then look for the [S] packet that tries to connect and see if a [S.] is send back for a request made from outside.. if so then haproxy (or at least something) apparently accepted the connection.. but im suspecting that modem or isp might be blocking traffic already.?.