Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    SSL certs handling and HAproxy

    Scheduled Pinned Locked Moved General pfSense Questions
    136 Posts 3 Posters 25.8k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • L
      lewis
      last edited by

      I'd like to do things but so far, have yet to be able to get either to work.
      I've followed several posts/articles, watched videos, nothing works.

      I'm using the haproxy to load balance web servers on the back end and pfsense forwards some public IPs to stand alone hosts as well.
      I have to share the LE certs using nfs for the web servers because they are behind pfsense.
      I'd like to put all of the certs on pfsense and have them auto updated from there.

      I'd like to start with just one non LB host to learn how this works.

      I have ACME installed, configured, able to pick up LE certs.
      On the test web server, I commented out the ssl certs and have SSLEngine on in the httpd config.
      There is no self signed cert on the server.

      Of course, it cannot respond to anything like this.

      What is not clear are the following;

      I've read that haproxy is used to forward port 80 to the web server/s.

      However, some seem to say that the web server/s should respond only to http but some say the web server also needs a self signed cert, likely for each domain if using virtualhosts.

      Some show haproxy configs that include only port 80 and others show both 80/443. Some say use redirect, some don't.
      I typically always have directives for virtualhosts redirecting port 80 to 443.

      Everything I've read shows variations of haproxy configurations so other than the above question, does anyone know of a doc/url I could follow to set up just one server to see how things work.

      I'm pretty sure the problem is on the web server side but I've tried so many combinations that I'm not sure what is what anymore and need some guidance or a proper doc that explains. I think this because no matter what I do, I keep seeing the 443 port down in the pfsense dashboard for haproxy.

      Seeing this work will also help me to better understand how I'll be able to install a varnish server that would cache for all of the servers on the LAN.

      1 Reply Last reply Reply Quote 0
      • stephenw10S
        stephenw10 Netgate Administrator
        last edited by

        Usually in this situation you would offload the SSL to HAProxy and make the communication to the backend http. But HAProxy should be able to talk https to the backend too if required. That could potentially be using the self signed cert as long as pfSense has the CA cert that was signed by.

        1 Reply Last reply Reply Quote 0
        • L
          lewis
          last edited by lewis

          Yes, this is my understanding also though it's little confusing when not having done this yet because some say optional, some say the server must have a self signed cert etc. I can set up a self signed cert but it's a multi-domain (virtualhosts) web server.

          As mentioned, I have acme set up, it's able to pick up the ssl cert so it's on pfsense now.
          I can select it, configure the haproxy etc.

          I've followed these and others but nothing ever works.

          https://www.boettrich.info/blog/beitrag/lets-encrypt-on-pfsense-haproxy/
          https://www.youtube.com/watch?v=gVOEdt-BHDY
          https://www.youtube.com/watch?v=cB6oKJjr4Ls

          f401560d-4794-410c-8b98-a2541b2933be-image.png

          I can see the host in haproxy;
          506551c1-9cf9-4214-8ff3-401e4d7d034a-image.png

          I can see the heartbeat hitting the web servers.
          In haproxy, I've tried port 80, 443 and 80/443. I can see the hits to the server but that's all.

          In the access log, I see this repeatedly from incoming connections that are trying to reach the server.
          [18/Dec/2023:07:39:51 -0700] "\x16\x03\x01" 400 226 180 - "-" "-"

          In haproxy, 'External address' the current settings in frontend are;

          Listen address is the public IP (virtual IP), port 443, SSL Offloading on.
          Type: http/https offloading

          In ACL, I have name (matching), 'Host matches', no, no and value is the domain name.

          Maybe I'm just missing a self cert on the server since the cert handshaking is not happening to begin with?

          I'm not sure I can do this in a forum, there are so many variables. I'd like to trash the whole setup and start over, one step at a time.
          That's why I wonder if someone could share a doc or url that is well known to work after following the steps.
          Once I see this working, it'll be easier to understand.

          1 Reply Last reply Reply Quote 0
          • stephenw10S
            stephenw10 Netgate Administrator
            last edited by

            Did you try using http to the backend? Do those servers respond to http?

            L 1 Reply Last reply Reply Quote 0
            • L
              lewis @stephenw10
              last edited by

              @stephenw10

              Yes, some of the articles configure for http, some for https and some for both. I've tried them all.
              I'll double check that the host can respond to http because I've make so many changes.
              I'll update shortly.

              1 Reply Last reply Reply Quote 0
              • L
                lewis
                last edited by lewis

                The host responds to http requests when using curl but not browsers. Not one browser wants to allow the http connection, even internally.

                Second problem I see is that the web site was built using all https links so even when something would connect using http, all links in the sites db are https. So that doesn't work either. This means I need to configure pfsense to use https otherwise this cannot work.

                Does it mean I mean one single self signed SSL cert on the server or one for every virtualhost on it?

                Of course, re-enabling the certs on the web server gives full access again, directly at least but not via the firewall.

                1 Reply Last reply Reply Quote 0
                • L
                  lewis
                  last edited by lewis

                  Actually, I did another test.

                  With the certs re-enabled on the web server;

                  I can connect directly to the web server just fine from any browser on the LAN.

                  I can connect from public over the Internet, going through pfsense.
                  That said, as shown above, haproxy knows only about port 80.

                  So, what's happening now?
                  Is it working because I re-enabled the SSL certs on the server itself?
                  Is the pfsense cert being used or the one on the web server itself?

                  Update:
                  When I check the SSL cert using decoder.link/sslchecker/, it shows the date for the cert on the web server, not the newer one on pfsense.
                  I guess that means that haproxy is not in the middle even though it's configured to be?

                  Confusing.

                  1 Reply Last reply Reply Quote 0
                  • stephenw10S
                    stephenw10 Netgate Administrator
                    last edited by

                    Do you have port forwards on pfSense that are bypassing HAProxy?

                    1 Reply Last reply Reply Quote 0
                    • L
                      lewis
                      last edited by

                      Yes, I did and I've disabled that now.
                      From public, nmap shows port 443 filtered so no response what so ever.

                      1 Reply Last reply Reply Quote 0
                      • L
                        lewis
                        last edited by

                        The thing is, if I don't enable the certs on the server, then it won't respond to port 443.
                        It cannot respond to port 80 because that's forwarded to port 443.

                        1 Reply Last reply Reply Quote 0
                        • stephenw10S
                          stephenw10 Netgate Administrator
                          last edited by

                          If 443 shows as filtered that implies HAProxy is not listening on it. Or there are no firewall rules to pass it?

                          Where are you forwarding port 80 to port 443? That should never happen. You can redirect it to enforce https use but that's not a forward.

                          1 Reply Last reply Reply Quote 0
                          • L
                            lewis
                            last edited by lewis

                            Wait, you just told me to remove the pfsense port forwards :).
                            The NAT rules were to forward incoming connections to that virtual IP to the web server LAN IP.

                            As mentioned, I do see HAproxy in the dashboard showing port 80 and it's live.

                            3da172af-548a-4383-9b3c-6f121bf74ffa-image.png

                            The 80 to 443 is a redirect in the httpd.conf, forwarding port 80 connections to https.

                            1 Reply Last reply Reply Quote 0
                            • stephenw10S
                              stephenw10 Netgate Administrator
                              last edited by

                              You should not have any port forwards. HAProxy listens on the interfaces directly. You especially shouldn't forward port 80 to port 443 because it always fail. http redircets are OK but you should not have them here because HAProxy is connecting with http.

                              1 Reply Last reply Reply Quote 0
                              • L
                                lewis
                                last edited by

                                I've tested with all combinations as mentioned in this post.
                                I've had the server set to http only without redirects and both http and https but nothing gets to it.

                                I'm a little confused about the port forwards now however.
                                I've had haproxy load balancing web servers for years and those have NAT rules to allow incoming traffic to get to the servers.
                                It must be working, I've seen countless times where we'd had a server or two down and the others were taking connections.

                                stephenw10S 1 Reply Last reply Reply Quote 0
                                • stephenw10S
                                  stephenw10 Netgate Administrator @lewis
                                  last edited by stephenw10

                                  Forwarding to a pool perhaps? If you have NAT rules on WAN forwarding traffic to the servers then HAProxy never sees that traffic. Maybe those rules are not actually matching the traffic. Or they are forwarding traffic to some address on the firewall where HAProxy is listening, which would be unnecessary.

                                  What are those NAT rules?

                                  1 Reply Last reply Reply Quote 0
                                  • L
                                    lewis
                                    last edited by

                                    I'll try to explain how things are set up then.

                                    I have frontend and backend set up for three web servers for example.
                                    I see haproxy showing all three servers in the dashboard.
                                    If one server goes down, it shows in the dashboard.
                                    If I reboot one of those servers, the others get the traffic.

                                    The NAT rules are for taking incoming public 80/443 connections to a virtual IP and sending that traffic to the web servers.
                                    The web servers are also defined in an alias which has the three LAN IP addresses for them.

                                    Does that explain it?

                                    1 Reply Last reply Reply Quote 0
                                    • stephenw10S
                                      stephenw10 Netgate Administrator
                                      last edited by

                                      If those NAT rule are catching incoming web traffic on both ports 80 and 443 and forwarding it to the internal server IPs directly then HAProxy never sees it.

                                      You should not have any port forward rules. HAProxy should be able to listen on port 80 and 443 on that virtual IP directly.

                                      1 Reply Last reply Reply Quote 0
                                      • L
                                        lewis
                                        last edited by

                                        How does the firewall know to send the incoming connections to the three web servers then? I've always had that rule.
                                        The NAT IP is set to the alias I mentioned and it works.

                                        Of course, I learned that setup from reading many articles so now I'm even more confused that that has been working.

                                        kiokomanK 1 Reply Last reply Reply Quote 0
                                        • L
                                          lewis
                                          last edited by

                                          @lewis said in SSL certs handling and HAproxy:

                                          How does the firewall know to send the incoming connections to the three web servers then? I've always had that rule.
                                          The NAT IP is set to the alias I mentioned and it works.

                                          What I mean is that the IP for the destination web server is the alias that has three IPs in it. I see traffic to all three web servers.

                                          1 Reply Last reply Reply Quote 0
                                          • kiokomanK
                                            kiokoman LAYER 8 @lewis
                                            last edited by kiokoman

                                            @lewis
                                            23b54e99-9e89-415a-bd13-00d37c7a637f-image.png

                                            that's what a frontend is need for

                                            443/80 -> wan -> Haproxy frontend -> haproxy backend

                                            no nat is needed

                                            if the ip you mentioned are virtual ip of pfsense you just need to cconfigure haproxy to listen to that ip

                                            6bd1db36-4e4e-439d-ad3f-0ed3872be6c0-image.png

                                            Listen adress

                                            ̿' ̿'\̵͇̿̿\з=(◕_◕)=ε/̵͇̿̿/'̿'̿ ̿
                                            Please do not use chat/PM to ask for help
                                            we must focus on silencing this @guest character. we must make up lies and alter the copyrights !
                                            Don't forget to Upvote with the 👍 button for any post you find to be helpful.

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.