pfSense, Haproxy, cloudflare cname DDNS letsencrypt certs Timeout



  • Dear All,

    I am at my wits end and an sure its something simple and stupid I'm missing here, but I have been looking so long at this problem that it's like proof reading your own work and not seeing any faults in your work.

    I have a small feeling it might be cloudflare as I only recently switched over to them, but ping do correctly resolve to the desired address, as I did disable proxy to try and remove things in the middle possibly causing my issue.

    So I have DNSEXIT set up with an address which is updated to my WAN IP from pfSense. I have CNAMES in cloudflare for the desired public sites.

    So if I ping test.{MyDomain} I hit the cname which resolves to the DNSEXIT address, which resolves to my WAN IP.

    This worked for a long time with squid reverse proxy, but wish to do the same in haproxy for it's letsencrypt integration.

    I have a fresh install of pfSense, where the WAN is my dynamically assigned public facing internet address.

    I have two WAN Firewall rules
    IPv4 TCP * Source * Port This Firewall Destination 80 (HTTP) Port * Gateway
    IPv4 TCP * Source * Port This Firewall Destination 443 (HTTPS) Port * Gateway

    I have switched the pfSense webgui to port 8443 (HTTPS)

    I have Pure NAT set under NAT Reflection mode for port forwards and Enable NAT Reflection for 1:1 NAT and Enable automatic outbound NAT for Reflection Ticked

    This allowed me to access the sites internally, but only because it bypassed haproxy completely.

    DNS Resolver enabled on all ports with Register DHCP leases in the DNS Resolver and Register DHCP static mappings in the DNS Resolver enabled.

    I have no NAT port forwarding rules, no 1:1 NAT rules, Outbound has no mapping and is in Automatic outbound NAT mode

    My HAPROXY CONF file

    Automaticaly generated, dont edit manually.

    Generated on: 2020-05-13 11:01

    global
    maxconn 1000
    stats socket /tmp/haproxy.socket level admin expose-fd listeners
    uid 80
    gid 80
    nbproc 1
    nbthread 1
    hard-stop-after 15m
    chroot /tmp/haproxy_chroot
    daemon
    tune.ssl.default-dh-param 2052
    server-state-file /tmp/haproxy_server_state

    listen HAProxyLocalStats
    bind 127.0.0.1:2200 name localstats
    mode http
    stats enable
    stats admin if TRUE
    stats show-legends
    stats uri /haproxy/haproxy_stats.php?haproxystats=1
    timeout client 5000
    timeout connect 5000
    timeout server 5000

    frontend Test
    bind {WAN IP}:443 name {WAN IP}:443 ssl crt-list /var/etc/haproxy/{CERT LOCATION}.crt_list
    mode http
    log global
    option http-keep-alive
    timeout client 30000
    acl ACL1 var(txn.txnhost) -m str -i test.{MyDOMAIN}
    acl aclcrt_test var(txn.txnhost) -m reg -i ^([^.]*).{My}.{DOMAIN}(:([0-9]){1,5})?$
    acl aclcrt_test var(txn.txnhost) -m reg -i ^{My}.{DOMAIN}(:([0-9]){1,5})?$
    http-request set-var(txn.txnhost) hdr(host)
    use_backend test_ipvANY if ACL1 aclcrt_test

    backend test_ipvANY
    mode http
    id 100
    log global
    timeout connect 30000
    timeout server 30000
    retries 3
    option httpchk OPTIONS /
    server test {Internal IP}:443 id 101 ssl check inter 1000 verify none crt /var/etc/haproxy/server_clientcert_5eb58f9641a2d.pem

    When I try and hit the address test.{MyDoamin} just get a 30sec pause before getting a connection timeout error. From what I have read usually means it's a firewall/NAT issue where it's hitting the firewall, not going anywhere and then the browser times out. but I can't see why.

    I have green for the backend on haproxy stats page.
    I can hit the site internally using the test.{MyDomain} so know the site is operational.

    Could someone please have a look into this problem?



  • @freak4915 said in pfSense, Haproxy, cloudflare cname DDNS letsencrypt certs Timeout:

    IPv4 TCP * Source * Port This Firewall Destination 443 (HTTPS) Port * Gateway

    No exactly sure how to read that, if you have a gateway filled in in the rule can you remove that? Other than that there shouldn't be any issues with the config you have.. Not regarding a client-connection timeout anyhow..

    All the NAT and reflection settings should be irrelevant for the connection through haproxy..

    On the haproxy stats, are the frontends counting incoming connections? I think not.. as i would then expect a different error result in the browser or perhaps a proper response..

    Can you check on pfSense console that connection attempts are actually arriving on your wan interface?
    Login with ssh go to the shell, then run something like

    tcpdump -ni WAN-NIC port 443 and host 1.2.3.4
    

    where WAN-NIC must be the name of the actual nic like em1 or igb2 or rt3 vtnet4 or pppoe5 so what your wan nic is actually called ;) and the 1.2.3.4 must be the public ip of the client that tries to connect. Then look for the [S] packet that tries to connect and see if a [S.] is send back for a request made from outside.. if so then haproxy (or at least something) apparently accepted the connection.. but im suspecting that modem or isp might be blocking traffic already.?.


Log in to reply