Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    SSL certs handling and HAproxy

    Scheduled Pinned Locked Moved General pfSense Questions
    136 Posts 3 Posters 26.2k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • L
      lewis
      last edited by

      Just for giggles, I swapped out the rule from NAT to WAN for the web servers and again, the traffic to each web server just died.
      wan-rule.png
      I had to re-anable the NAT rule again.

      Back to the single server I was testing, no, I cannot connect to it using http or https.

      stephenw10S 1 Reply Last reply Reply Quote 0
      • stephenw10S
        stephenw10 Netgate Administrator
        last edited by

        Isn't that a node without HAProxy though? Without anything to proxy the traffic you'd need the NAT to reach the servers.

        So what error do you see when you try to connect through HAProxy?

        Do you see firewall states opened for that traffic?

        L 1 Reply Last reply Reply Quote 0
        • L
          lewis @stephenw10
          last edited by

          @stephenw10 said in SSL certs handling and HAproxy:

          Isn't that a node without HAProxy though? Without anything to proxy the traffic you'd need the NAT to reach the servers.

          Sorry, I'm not following this. A node without proxy?
          I was told here that the NAT rule was the wrong one for haproxy, that I should create a wan rule which I did.
          I've read that one I used a wan rule instead, the proxy would just know to handle traffic for that public IP.
          The traffic keeps coming in, even while I'm messing around with these changes.

          So what error do you see when you try to connect through HAProxy?

          I don't see any errors as I would post them. The traffic to the web servers simply stops, It doesn't get to the web servers once I'm using a wan rule.

          Do you see firewall states opened for that traffic?

          I didn't check so I'll do that next time I can disrupt the traffic.

          1 Reply Last reply Reply Quote 0
          • stephenw10S
            stephenw10 Netgate Administrator @lewis
            last edited by

            @lewis said in SSL certs handling and HAproxy:

            Just for giggles, I swapped out the rule from NAT to WAN for the web servers and again, the traffic to each web server just died.
            I had to re-anable the NAT rule again.

            I assumed you did that on some other pfSense instance where HAProxy is not present?

            To use HAProxy you must have firewall pass rules on WAN and you must not have any NAT rules.

            @lewis said in SSL certs handling and HAproxy:

            I don't see any errors as I would post them.

            So the browser just timesout?

            1 Reply Last reply Reply Quote 0
            • L
              lewis
              last edited by

              Some things are getting mixed up in the thread. That's why I suggested working on this one stand alone server first. Once I see how that is working, that will help me to understand how things need to be set up.

              First, I'm only working on one firewall, not multiples in trying to solve this.

              I have two different things going on.

              1: I thought haproxy was handling the three web servers I've talked about and we've now found that this was not the case.
              These servers have their own SSL certificates, not using ACME on the firewall.
              When I swap from a NAT rule to a WAN rule, all traffic to all three servers stop.

              2: I've set up a test web server, just one server, to see if I can use the SSL certificate on pfsense instead of the web server, to learn about doing this using pfsense, ACME and haproxy.
              In this case, I have ACME set up with a valid cert on pfsense and wanted to learn about this by having pfsense handle the ssl cert for the web server.
              So far, nothing has worked. In this case, I've also shared images of the rules, setup etc.
              The best I've been able to get in terms of reaching the web service is getting an error in the browser, complaining about HSTS which is odd because HSTS is disabled on the web server.

              This seems to imply that things are set up ok on pfsense but something on the web server is not.

              kiokomanK 1 Reply Last reply Reply Quote 0
              • stephenw10S
                stephenw10 Netgate Administrator
                last edited by

                That implies that you're hitting HAProxy with http and it's only configured for https.

                Try connecting to to with https. Do you see the expected states opened in the firewall. Do you see the expected cert on the client?

                1 Reply Last reply Reply Quote 0
                • kiokomanK
                  kiokoman LAYER 8 @lewis
                  last edited by

                  @lewis
                  about the port he was talking about this settings

                  42084d23-46a9-42b7-bf48-66576a0af6e1-image.png

                  the default is 80 or 443 for https, if you don't chage this port the pfsense gui take control of the traffic instead of haproxy

                  if you saw the pfsense cert you are near the solution, change that setting and you should see the cert of haproxy

                  ̿' ̿'\̵͇̿̿\з=(◕_◕)=ε/̵͇̿̿/'̿'̿ ̿
                  Please do not use chat/PM to ask for help
                  we must focus on silencing this @guest character. we must make up lies and alter the copyrights !
                  Don't forget to Upvote with the 👍 button for any post you find to be helpful.

                  1 Reply Last reply Reply Quote 0
                  • L
                    lewis
                    last edited by

                    One thing I mentioned and will again is that the pfsense GUI is set to a custom port without an SSL certificate.
                    Nothing will hit the pfsense GUI other than if someone got the correct port set in the Advanced/Admin Access/TCP port settings.
                    That port is also blocked and allowed to only one or two IPs so no one can get to the GUI.

                    Unless I'm missing something again?

                    Also, I double checked and there are no services on port 80 or 443 for the primary pfsense IP, all services are using ports with their own VIPs.

                    1 Reply Last reply Reply Quote 0
                    • stephenw10S
                      stephenw10 Netgate Administrator
                      last edited by

                      Great so that can't be an issue then.

                      So do you see states? Do you see the ACME cert on the client?

                      L 1 Reply Last reply Reply Quote 0
                      • L
                        lewis @stephenw10
                        last edited by lewis

                        @stephenw10
                        That's the problem with posts when they get so long. I had mentioned that once or twice :).

                        I can check the states but we're talking about the single server that has the cert on pfsense ACME now right?
                        As mentioned, I can get back to the load balanced stuff later. It would be nice to see this working first.

                        On that web server, I've now added a self signed cert so that it can respond to https requests without needing the real FQDN cert on the server.
                        That cert is on pfsense ACME.

                        The server accepts port 80 and 443, no redirect enabled right now so I can see any traffic coming to it.

                        When I run an ssl test, I see the the correct ACME cert details showing up but I also see;
                        Trusted: We were unable to verify this certificate
                        In the browser, I see; Error code: SEC_ERROR_UNKNOWN_ISSUER

                        Oddly, I do see some connections getting to the server;

                        www.domain.com 10.0.0.1 - - [20/Dec/2023:14:36:19 -0700] "GET /action/social/login/facebook?ossn_ts=1702991375&ossn_token=xxx HTTP/1.1" 301 20 159565 159325 "-" "Mozilla/5.0 (compatible; MJ12bot/v1.4.8; http://mj12bot.com/)"
                        and
                        www.domain.com 10.0.0.1 - - [20/Dec/2023:14:37:25 -0700] "GET / HTTP/1.1" 301 4691 181606 181062 "-" "Mozilla/5.0 (compatible; MJ12bot/v1.4.8; http://mj12bot.com/)"

                        Notice those connections are coming from 10.0.0.1 which in this case, is the firewall so I think that's a good thing. It means it's intercepting the traffic or haproxy is, right?

                        Next, looking at states while trying to connect to said web server, I see nothing what so ever.

                        1 Reply Last reply Reply Quote 0
                        • L
                          lewis
                          last edited by

                          I'm not sure why I didn't notice before or maybe there weren't any when I looked but;
                          I do see connections but they are all port 80 no matter if I use http or https from a remote browser/client.

                          8977630b-4708-4c9f-a685-51b97b7dc420-image.png

                          1 Reply Last reply Reply Quote 0
                          • L
                            lewis
                            last edited by lewis

                            Using TOR, I always get an HSTS notice but using other browser, I only get a cert warning.
                            As mentioned, I do not have HSTS enabled on this server at the moment.

                            I'm confused on why some traffic is showing up however.

                            1 Reply Last reply Reply Quote 0
                            • stephenw10S
                              stephenw10 Netgate Administrator
                              last edited by

                              Ok that mostly looks good. The LAN side states should be http if HAProxy is configured to use that. It's independent of whatever the front end connections are.

                              I expect to see states on WAN too though, from the client to HAProxy on the VIP?

                              That cert error look like it might just be a cert problem. Unknown issuer is odd though. ACME setup not quite right? Still using the testing CA maybe?

                              L 1 Reply Last reply Reply Quote 0
                              • L
                                lewis @stephenw10
                                last edited by

                                Ok that mostly looks good. The LAN side states should be http if HAProxy is configured to use that. It's independent of
                                whatever the front end connections are.

                                The haproxy front end is set to both 443 and 80. I shared an image above of some of the settings.

                                I expect to see states on WAN too though, from the client to HAProxy on the VIP?

                                None, all LAN in the states view.

                                That cert error look like it might just be a cert problem. Unknown issuer is odd though. ACME setup not quite right? Still
                                using the testing CA maybe?

                                I don't see any options for live or testing. How can I check that?

                                1 Reply Last reply Reply Quote 0
                                • L
                                  lewis
                                  last edited by

                                  offloading.png

                                  1 Reply Last reply Reply Quote 0
                                  • stephenw10S
                                    stephenw10 Netgate Administrator
                                    last edited by

                                    It's ACME that has options for production and testing.

                                    You are seeing no WAN side states at all?

                                    If so are you seeing blocked traffic on WAN?

                                    L 1 Reply Last reply Reply Quote 0
                                    • L
                                      lewis @stephenw10
                                      last edited by

                                      @stephenw10 said in SSL certs handling and HAproxy:

                                      It's ACME that has options for production and testing.

                                      I looked though the settings but don't see anything about testing.
                                      The only options are enable or disable in a couple of sections.

                                      You are seeing no WAN side states at all?
                                      If so are you seeing blocked traffic on WAN?

                                      There's lots of WAN traffic, just nothing to that server.

                                      1 Reply Last reply Reply Quote 0
                                      • stephenw10S
                                        stephenw10 Netgate Administrator
                                        last edited by

                                        Hmm, no states on WAN is an issue. How are you testing?

                                        Do you see states or bytes on the WAN firewall rue that should be passing that?

                                        The acme testing option is here:
                                        Screenshot from 2023-12-20 22-55-46.png

                                        But of nothing is even reaching WAN that's probably not the issue

                                        L 1 Reply Last reply Reply Quote 0
                                        • L
                                          lewis @stephenw10
                                          last edited by lewis

                                          @stephenw10 said in SSL certs handling and HAproxy:

                                          Hmm, no states on WAN is an issue. How are you testing?

                                          Do you see states or bytes on the WAN firewall rue that should be passing that?

                                          The acme testing option is here:
                                          Screenshot from 2023-12-20 22-55-46.png

                                          But of nothing is even reaching WAN that's probably not the issue

                                          The WAN seems to have a lot of traffic, just not to this web server.
                                          Thanks for the image, it turns out it is in testing mode.
                                          I've changed it to LE production but still no go.

                                          1 Reply Last reply Reply Quote 0
                                          • L
                                            lewis
                                            last edited by

                                            Just to confirm, I should be seeing traffic on WAN to the web server, in this case, 10.0.0.180.

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.