Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    HAProxy - SSL OffLoading Fine, Adding SSH to the Mix and Stuck

    Scheduled Pinned Locked Moved Cache/Proxy
    9 Posts 2 Posters 4.0k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • A
      ak
      last edited by

      I had HAProxy working great as a reverse proxy with SSL Offloading.

      FrontEnd:

      • Bound to 443

      • SSL Offloading setup.

      • 2 domains with SSL

      • Configured with SSL cert and 'Add ACL for certificate Subject Alternative Names

      • Mutliple ACL where if a path starts with particular value, it will go to relevant backend

      This was working great and had minimal problems.

      I then stumbled upon HAProxy's ability to inspect traffic and being able to then have SSH and SSL on the same 443 port. So I tried setting this up with

      FrontEnd [Call this Front_SSL_SSH:443] set to listen to new 443.

      • Type is tcp

      • Set with inspect delay 5s

      • Set acl 'is_ssl' if req_ssl_ver 2:3.1

      • Configured to use the new back end if is_ssl true - BackEnd_SSL_SSH (see below)

      • User ssh_server backend (essentially to a ssh port on a server in my network)

      BackEnd_SSL_SSH

      • This points to the original FrontEnd at the very top

      SSH works fine, but the web requests fail. The HAProxy logs shows a 'SSL handshake failure' when I try and access the server via a browser. I suspect that the new front end that is doing the detection has done the SSL handshake already, so when it comes the web server, this fails as the browser does not expect a second SSL? Not sure - but would like to keep the SSL Offloading from the original if possible.

      Any ideas? Or have I misunderstood the problem. I have googled but all examples I have found does not use the SSL Offloading for the web part.

      1 Reply Last reply Reply Quote 0
      • P
        PiBa
        last edited by

        Can you share the haproxy.conf from bottom of settings tab?

        1 Reply Last reply Reply Quote 0
        • A
          ak
          last edited by

          I have sanitised the config to the below and changed ip etc. Let me know if anything does not make sense.
          Note that

          • 192.168.2.1 - this is the WAN address of pfsense

          • 192.168.1.1 - this is the LAN address of pfsense

          • 192.168.1.0/24 - this is the LAN network hosting the webservices

          Automaticaly generated, dont edit manually.

          Generated on: 2017-12-10 11:24

          global
          maxconn 128
          log /var/run/log local0 info
          stats socket /tmp/haproxy.socket level admin
          uid 80
          gid 80
          nbproc 1
          chroot /tmp/haproxy_chroot
          daemon
          tune.ssl.default-dh-param 2048
          server-state-file /tmp/haproxy_server_state

          Time-to-first-Byte (TTFB) value needs to be optimized based on

          the actual public certificate chain see

          https://www.igvita.com/2013/10/24

          /optimizing-tls-record-size-and-buffering-latency/

          tune.ssl.maxrecord 1370

          Modern browser compatibility only as mentioned here:

          https://wiki.mozilla.org/Security/Server_Side_TLS

          ssl-default-bind-ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!3DES:!MD5:!PSK

          listen HAProxyLocalStats
          bind 127.0.0.1:2200 name localstats
          mode http
          stats enable
          stats admin if TRUE
          stats uri /haproxy/haproxy_stats.php?haproxystats=1
          timeout client 5000
          timeout connect 5000
          timeout server 5000

          frontend personal-domain.com
          bind 192.168.2.1:443 name 192.168.2.1:443 ssl  crt /var/etc/haproxy/personal-domain.com.pem crt /var/etc/haproxy/personal-domain.com 
          mode http
          log global
          option socket-stats
          option http-keep-alive
          maxconn 10
          timeout client 30000
          redirect scheme https code 301 if !{ ssl_fc }
          acl PersonalDomainBackend hdr_end(host) -i personal-domain.com
          acl AppOneBackend path_beg -i /secure/appone
          acl AppTwoBackend path_beg -i /secure/apptwo
          acl BusinessDomainBackend hdr_end(host) -i business-domain.com
          use_backend appone_http_ipvANY  if  PersonalDomainBackend AppOneBackend aclcrt_personal-domain.com
          use_backend apptwo_http_ipvANY  if  PersonalDomainBackend AppTwoBackend aclcrt_personal-domain.com
          use_backend businessdomain_http_ipvANY  if  BusinessDomainBackend aclcrt_personal-domain.com
          use_backend personaldomain_http_ipvANY  if  aclcrt_personal-domain.com

          frontend personal-dimain.com.http
          bind 192.168.2.1:80 name 192.168.2.1:80 
          mode http
          log global
          option http-keep-alive
          timeout client 30000
          redirect scheme https code 301 if !{ ssl_fc }

          frontend top_level_ssl_ssh_splitter
          bind 192.168.2.1:1443 name 192.168.2.1:1443 ssl  crt /var/etc/haproxy/top_level_ssl_ssh_splitter.pem crt /var/etc/haproxy/top_level_ssl_ssh_splitter 
          mode tcp
          log global
          option log-separate-errors
          option tcplog
          timeout client 30000
          tcp-request inspect-delay 5s
          tcp-request content accept if { req.ssl_hello_type 1 }
          acl is_ssh payload(1,7) -m bin 5353482d322e30
          tcp-request content accept if is_ssh
          use_backend main_ssl_web_to_frontend_tcp_ipvANY  if  !is_ssh
          use_backend ssh_server_tcp_ipvANY  if  is_ssh
          default_backend ssh_server_tcp_ipvANY

          backend appone_http_ipvANY
          mode http
          log global
          timeout connect 30000
          timeout server 30000
          retries 3
          server appone 192.168.1.9:9091 check inter 10000

          backend apptwo_http_ipvANY
          mode http
          log global
          timeout connect 30000
          timeout server 30000
          retries 3
          server apptwo 192.168.1.9:8113 check inter 10000

          backend personaldomain_http_ipvANY
          mode http
          log global
          timeout connect 30000
          timeout server 30000
          retries 3
          server personal-domain.com 192.168.1.9:10080 check inter 1000

          backend businessdomain_http_ipvANY
          mode http
          log global
          timeout connect 30000
          timeout server 30000
          retries 3
          acl businessdomain hdr_end(host) -i business-domain.com
          server business-domain.com 192.168.1.9:10080 check inter 1000

          backend main_ssl_web_to_frontend_tcp_ipvANY
          mode tcp
          log global
          timeout connect 30000
          timeout server 30000
          retries 3
          server ssl_web_server 192.168.2.1:443

          backend ssh_server_tcp_ipvANY
          mode tcp
          log global
          timeout connect 30000
          timeout server 30000
          retries 3
          server ssh 192.168.1.9:22

          1 Reply Last reply Reply Quote 0
          • P
            PiBa
            last edited by

            The https requests can only be decrypted once.. So after "top_level_ssl_ssh_splitter" decrypts the traffic with the configured certificate. The resulting traffic is plain ssh or http.

            As such the "main_ssl_web_to_frontend_tcp_ipvANY" should probably be pointing to a internal/private/localhost:80 port on haproxy that does not decrypt ssl again and further processes the desired acl's to split domains for appone/apptwo/business/personal apart tp the backends.

            1 Reply Last reply Reply Quote 0
            • A
              ak
              last edited by

              Not had an opportunity to test this out but just wanted to clarify the config

              • The entry point is port 1443 - as 443 was the original and I didn't want to change the setup to drastically. So the entry port via mapping is to port 1443, this then decides to hit the 443 or the ssh server

              • The https website is working fine. The SSH attempt shows in the haproxy.log an 'SSL handshake failure'

              • Tested via a ssh client on IOS

              1 Reply Last reply Reply Quote 0
              • P
                PiBa
                last edited by

                Seems to be the opposite of your previous problem? "SSH works fine, but the web requests fail." and yes i was assuming usage of the 1443.. So please check again which of the two is actually the problem.?.

                1 Reply Last reply Reply Quote 0
                • P
                  PiBa
                  last edited by

                  As for the ssh connection make sure to wrap it inside a ssl layer.?

                  https://blog.chmd.fr/ssh-over-ssl-episode-4-a-haproxy-based-configuration.html

                  Connecting from an SSH client

To connect to your server from linux, just drop this in your ~/.ssh/config:

Host server.com
    ProxyCommand openssl s_client -connect server.com:443 -quiet

If you are on windows and you cannot install anything client side, there is also a solution for you.
Download socat and putty (none of them requires admin rights).
Then, with socat, run:

socat TCP-LISTEN:8888 OPENSSL:server.com:443,verify=0
And with putty, direct your client to 127.0.0.1 on the port 8888.
                  1 Reply Last reply Reply Quote 0
                  • A
                    ak
                    last edited by

                    thanks - I can't remember what setting I had for 'ssh working but ssl not'. I had some different acl testing entries but as I needed the website up and running I reverted back to the current set of acl above.

                    I will try your suggestions. I think there is too much going on so will start from scratch over the holiday period and get basics working before moving up the level of complexity.

                    Just a quick question - if I have a splitter that decrypts SSL to divert between the web traffic and SSH traffice, will I be able to use SNI to run two websites on different URL's or have I lost that ability on the split as the SSL is decrypted by that time and its only HTTP sent to the web server/frontend?

                    1 Reply Last reply Reply Quote 0
                    • P
                      PiBa
                      last edited by

                      Using SNI does not need decryption of the traffic, so it should be possible to not configure any certificate on the 1443 frontend, and keep the is_ssh payload check and have it working like that.

                      Or perhaps this helps?: https://marc.info/?l=haproxy&m=132375969032305&w=2

                      1 Reply Last reply Reply Quote 0
                      • First post
                        Last post
                      Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.