• Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login
Netgate Discussion Forum
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login

SSL certs handling and HAproxy

Scheduled Pinned Locked Moved General pfSense Questions
136 Posts 3 Posters 26.0k Views
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • L
    lewis
    last edited by Dec 18, 2023, 7:46 PM

    Yes, I did and I've disabled that now.
    From public, nmap shows port 443 filtered so no response what so ever.

    1 Reply Last reply Reply Quote 0
    • L
      lewis
      last edited by Dec 18, 2023, 7:53 PM

      The thing is, if I don't enable the certs on the server, then it won't respond to port 443.
      It cannot respond to port 80 because that's forwarded to port 443.

      1 Reply Last reply Reply Quote 0
      • S
        stephenw10 Netgate Administrator
        last edited by Dec 18, 2023, 8:24 PM

        If 443 shows as filtered that implies HAProxy is not listening on it. Or there are no firewall rules to pass it?

        Where are you forwarding port 80 to port 443? That should never happen. You can redirect it to enforce https use but that's not a forward.

        1 Reply Last reply Reply Quote 0
        • L
          lewis
          last edited by lewis Dec 18, 2023, 8:37 PM Dec 18, 2023, 8:31 PM

          Wait, you just told me to remove the pfsense port forwards :).
          The NAT rules were to forward incoming connections to that virtual IP to the web server LAN IP.

          As mentioned, I do see HAproxy in the dashboard showing port 80 and it's live.

          3da172af-548a-4383-9b3c-6f121bf74ffa-image.png

          The 80 to 443 is a redirect in the httpd.conf, forwarding port 80 connections to https.

          1 Reply Last reply Reply Quote 0
          • S
            stephenw10 Netgate Administrator
            last edited by Dec 18, 2023, 8:53 PM

            You should not have any port forwards. HAProxy listens on the interfaces directly. You especially shouldn't forward port 80 to port 443 because it always fail. http redircets are OK but you should not have them here because HAProxy is connecting with http.

            1 Reply Last reply Reply Quote 0
            • L
              lewis
              last edited by Dec 18, 2023, 9:00 PM

              I've tested with all combinations as mentioned in this post.
              I've had the server set to http only without redirects and both http and https but nothing gets to it.

              I'm a little confused about the port forwards now however.
              I've had haproxy load balancing web servers for years and those have NAT rules to allow incoming traffic to get to the servers.
              It must be working, I've seen countless times where we'd had a server or two down and the others were taking connections.

              S 1 Reply Last reply Dec 18, 2023, 9:27 PM Reply Quote 0
              • S
                stephenw10 Netgate Administrator @lewis
                last edited by stephenw10 Dec 18, 2023, 9:28 PM Dec 18, 2023, 9:27 PM

                Forwarding to a pool perhaps? If you have NAT rules on WAN forwarding traffic to the servers then HAProxy never sees that traffic. Maybe those rules are not actually matching the traffic. Or they are forwarding traffic to some address on the firewall where HAProxy is listening, which would be unnecessary.

                What are those NAT rules?

                1 Reply Last reply Reply Quote 0
                • L
                  lewis
                  last edited by Dec 18, 2023, 9:41 PM

                  I'll try to explain how things are set up then.

                  I have frontend and backend set up for three web servers for example.
                  I see haproxy showing all three servers in the dashboard.
                  If one server goes down, it shows in the dashboard.
                  If I reboot one of those servers, the others get the traffic.

                  The NAT rules are for taking incoming public 80/443 connections to a virtual IP and sending that traffic to the web servers.
                  The web servers are also defined in an alias which has the three LAN IP addresses for them.

                  Does that explain it?

                  1 Reply Last reply Reply Quote 0
                  • S
                    stephenw10 Netgate Administrator
                    last edited by Dec 18, 2023, 10:20 PM

                    If those NAT rule are catching incoming web traffic on both ports 80 and 443 and forwarding it to the internal server IPs directly then HAProxy never sees it.

                    You should not have any port forward rules. HAProxy should be able to listen on port 80 and 443 on that virtual IP directly.

                    1 Reply Last reply Reply Quote 0
                    • L
                      lewis
                      last edited by Dec 18, 2023, 10:28 PM

                      How does the firewall know to send the incoming connections to the three web servers then? I've always had that rule.
                      The NAT IP is set to the alias I mentioned and it works.

                      Of course, I learned that setup from reading many articles so now I'm even more confused that that has been working.

                      K 1 Reply Last reply Dec 18, 2023, 10:38 PM Reply Quote 0
                      • L
                        lewis
                        last edited by Dec 18, 2023, 10:37 PM

                        @lewis said in SSL certs handling and HAproxy:

                        How does the firewall know to send the incoming connections to the three web servers then? I've always had that rule.
                        The NAT IP is set to the alias I mentioned and it works.

                        What I mean is that the IP for the destination web server is the alias that has three IPs in it. I see traffic to all three web servers.

                        1 Reply Last reply Reply Quote 0
                        • K
                          kiokoman LAYER 8 @lewis
                          last edited by kiokoman Dec 18, 2023, 10:42 PM Dec 18, 2023, 10:38 PM

                          @lewis
                          23b54e99-9e89-415a-bd13-00d37c7a637f-image.png

                          that's what a frontend is need for

                          443/80 -> wan -> Haproxy frontend -> haproxy backend

                          no nat is needed

                          if the ip you mentioned are virtual ip of pfsense you just need to cconfigure haproxy to listen to that ip

                          6bd1db36-4e4e-439d-ad3f-0ed3872be6c0-image.png

                          Listen adress

                          ̿' ̿'\̵͇̿̿\з=(◕_◕)=ε/̵͇̿̿/'̿'̿ ̿
                          Please do not use chat/PM to ask for help
                          we must focus on silencing this @guest character. we must make up lies and alter the copyrights !
                          Don't forget to Upvote with the 👍 button for any post you find to be helpful.

                          1 Reply Last reply Reply Quote 0
                          • L
                            lewis
                            last edited by lewis Dec 18, 2023, 10:44 PM Dec 18, 2023, 10:43 PM

                            Yes, I've understood all that's being said but why does it seem that it's been working as I've explained above?
                            Yes, HAproxy is definitely listening on the correct interface so I'm there in terms of that part.

                            Also, I cannot change that until I can solve the SSL issue since the web servers are using their SSL cert over a SAN share.
                            I need to solve the issue with being able to use pfsense, ACME and HAproxy then I can fix this new thing that's come up.

                            So, if we can focus on getting just one web server working with ACME/HAproxy, that will show me how to do the rest.

                            1 Reply Last reply Reply Quote 0
                            • S
                              stephenw10 Netgate Administrator
                              last edited by Dec 18, 2023, 10:55 PM

                              Do you actually see any traffic in HAProxy in other installs though?

                              You need firewall rules on WAN to allow traffic to HAProxy of course.

                              I imagine it's working by just bypassing HAProxy on other systems. But then you don't get any of the HAProxy features like SSL off loading.

                              I would approach this the other way. Remove the forwards and get HAProxy functioning in a basic form before adding back the SSL features. Whatever testing you have done until now appears to be invalid since nothing was actually hitting HAProxy.

                              Unless we have misunderstood your description of those NAT rules. You have a screenshot?

                              L 1 Reply Last reply Dec 19, 2023, 1:53 AM Reply Quote 0
                              • L
                                lewis @stephenw10
                                last edited by Dec 19, 2023, 1:53 AM

                                Do you actually see any traffic in HAProxy in other installs though?

                                I don't have haproxy on any other install, just this one firewall.

                                You need firewall rules on WAN to allow traffic to HAProxy of course.

                                I had NAT rules but now I'm told those aren't needed, HAproxy knows about the incoming traffic over the selected interface.

                                I imagine it's working by just bypassing HAProxy on other systems. But then you don't get any of the HAProxy features like SSL off loading.

                                This is the first time I'm trying to use ACME on the firewall. HAproxy is in use to load balance web servers. That's always worked.

                                I would approach this the other way. Remove the forwards and get HAProxy functioning in a basic form before adding back the SSL features.

                                I'll have to set it up as a test on another pfsense as I cannot disrupt services on this one I've been trying to get working.

                                Whatever testing you have done until now appears to be invalid since nothing was actually hitting HAProxy.

                                But the weird thing is, I do get fail over. And more than that, the IP for the rule was set with the alias that has three web servers and all three get traffic so, something else is goofy here.

                                Unless we have misunderstood your description of those NAT rules. You have a screenshot?

                                Where do I start? I'll get back on this tomorrow and nab a bunch of screen shots.

                                1 Reply Last reply Reply Quote 0
                                • S
                                  stephenw10 Netgate Administrator
                                  last edited by Dec 19, 2023, 2:21 AM

                                  Ah, sorry I thought you meant you had other installs running with HAProxy.

                                  Ok so disable or remove those NAT rules. Make sure there are firewall pass rules to allow the incoming traffic to access HAProxy. Check the HAProxy status to see what's happening.

                                  Grab screenshots. 😉

                                  1 Reply Last reply Reply Quote 0
                                  • L
                                    lewis
                                    last edited by lewis Dec 19, 2023, 5:01 PM Dec 19, 2023, 4:43 PM

                                    I'll work on this shortly but the post is getting a bit confused.

                                    I wanted to work on one single host, not load balanced, just using ACME/HAProxy setup to see it working.
                                    Then I could move on to what is happening with this load balancing thing that's come up.

                                    In terms of the load balanced ones, I did run a quick test for the load balanced servers.
                                    As soon as I disabled the rule, traffic stopped to the web servers.
                                    I monitored two of them and both stopped receiving connections.

                                    As mentioned, I have an alias that contains the three web servers LAN IPs.
                                    That's what you see in the rule, FoxHAP
                                    2023-12-19_09-17.png

                                    Now, I'm moving back to the tests I've been doing using just one server.
                                    This is the rule when I enable it;
                                    tynatrule.png

                                    Then the front end of HAProxy;
                                    frontend.png

                                    Then the back end;
                                    backend.png

                                    The load balanced servers aren't using ACME, they share their SSL certs over a SAN share.

                                    What I'd like to accomplish is to see the single server I initially wanted to learn from working.
                                    So, that one server has a virtual IP.
                                    I have HAProxy set up with frontend and backend.
                                    The ACME certificate is installed on pfsense. I also have them installed on the web server itself. I enable/disable while testing.

                                    On the frontend, I have the external IP (virtual IP) selected and I've tried 80 and 443, SSL offloading on/off.
                                    In the ACL, I have a unique name, I've set to 'Host matches' and the value is the fqdn.
                                    In the actions, table, I have s unique ACL name I created.
                                    In the Default Backend, I have the backend name.
                                    On the backend, I do have the client certificate selected.
                                    In Additional certificates, I have the ACME cert selected for this domain. and 'Add ACL for certificate subject alternative names' selected.

                                    These are from what ever I've been reading that keeps changing. One article says this, another says that, nothing has been consistent enough to get this working.

                                    That's why I posted asking if someone might have a current, up to date, known to work article/doc that I could following because after a while, it becomes a bunch of mixed up settings that will never work, which is where I'm at now. Again, not thinking about the load balanced servers yet.

                                    1 Reply Last reply Reply Quote 0
                                    • L
                                      lewis
                                      last edited by lewis Dec 19, 2023, 5:04 PM Dec 19, 2023, 5:03 PM

                                      @stephenw10 said in SSL certs handling and HAproxy:

                                      Make sure there are firewall pass rules to allow the incoming traffic to access HAProxy.

                                      Wait, do you mean some other kind of rule and not NAT rule?

                                      Check the HAProxy status to see what's happening.

                                      What am I looking for here, there's a lot of information.

                                      K 1 Reply Last reply Dec 19, 2023, 5:07 PM Reply Quote 0
                                      • L
                                        lewis
                                        last edited by Dec 19, 2023, 5:06 PM

                                        I was going to edit the previous but my extra notes might be missed.

                                        Yes, I do see haproxy status page.
                                        I also notice that other than many columns, there's little data in there which makes me think haproxy is in fact being bypassed.
                                        I'm not sure what part of that page would be useful to share.

                                        1 Reply Last reply Reply Quote 0
                                        • K
                                          kiokoman LAYER 8 @lewis
                                          last edited by Dec 19, 2023, 5:07 PM

                                          @lewis
                                          normal firewall rules on WAN. not a NAT
                                          destination should be "this firewall (self)" if you want haproxy to intercept the traffic
                                          like my print screen
                                          ae8202be-8c56-44f5-95fc-717ccc35670b-image.png

                                          ̿' ̿'\̵͇̿̿\з=(◕_◕)=ε/̵͇̿̿/'̿'̿ ̿
                                          Please do not use chat/PM to ask for help
                                          we must focus on silencing this @guest character. we must make up lies and alter the copyrights !
                                          Don't forget to Upvote with the 👍 button for any post you find to be helpful.

                                          1 Reply Last reply Reply Quote 0
                                          28 out of 136
                                          • First post
                                            28/136
                                            Last post
                                          Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.
                                            This community forum collects and processes your personal information.
                                            consent.not_received