Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    CARP Setup with HAProxy and VIPs

    HA/CARP/VIPs
    3
    13
    1.5k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • D
      ddepaolis
      last edited by

      Greetings all, I've attually a PFSense configured with HAProxy (to do load balancing) and with some VIPs as IP alias: some on WAN interface, some on LAN interface (to manage load balancing through HAProxy). We're planning to add a second PFSense to gain full High Availability on our infrastructure. Of course this scenario is a bit complex. Then my first questions to begin with:

      1. Using CARP is it necessary (or anyway highly suggested) adding a net for syncronization between two nodes ?
      2. IP aliases I already set on WAN and LAN interface are compatible with this new HA environment through CARP ? I explain better: actually connections go to IP alias on WAN or LAN interface, then they go to be balanced through HA Proxy. Adding a VIP CARP two nodes can continue having both same VIP aliases or should it become different ?
      3. HA Proxy can continue to manage connections as it's doing now (http tunnel/Least Connections) or should I do some mods ?

      Of course thanks for any advice !

      planedropP 1 Reply Last reply Reply Quote 0
      • planedropP
        planedrop @ddepaolis
        last edited by

        @ddepaolis I'll help as best I can here, very experienced with HA but NOT with HAProxy so I may miss some things.

        1. Yes, you really do need a sync interface setup, this is require for proper HA functionality and should not be skipped.
        2. As far as VIPs, assuming I'm understanding right, you should be able to still use them.
        3. I may not be able to answer this one, but I think you can leave it as is now.

        Now I feel like I need to go build something similar in my lab just to test it out lol, want to learn more about HAProxy in association with HA environments, cool stuff. Not sure when/if I will have time to test this in a lab but if I do while this thread is still active I'll give some pointers.

        D 1 Reply Last reply Reply Quote 0
        • D
          ddepaolis @planedrop
          last edited by ddepaolis

          @planedrop Thanks so much for your reply; of course I'm planning a lab so do I.
          Let me explain better my load balancing configuration through HA proxy.

          I set a public alias VIP on WAN interface (1.2.3.4/28) and a private alias VIP on LAN interface (192.168.1.50/24). Alias VIP on WAN interface is configured with a NAT port forward rule for inbound connections, I explain later. Alias VIP on LAN interface is configured as "external address" in HA Proxy (frontend). HA Proxy, in turn, has two server configured in the backend

          Connections from inside (LAN) to those two servers go, of course, directly to alias VIP 192.168.1.50/24 in frontend HAProxy configuration.
          Connections from outside (WAN) to those two servers, follow this NAT Port Forward rule:
          Interface: WAN - Protocol: TCP - Source IP: Any - Source port: 1024-65535 - Destination address: Alias VIP 1.2.3.4/28 - Destionation Port: 443 - NAT IP: Alias VIP 192.168.1.50/24 - Nat Port: 443

          Then my main question: in a High Availability scenario using CARP, the slave PFSense node may have same Alias VIPs on WAN and LAN interfaces (1.2.3.4 / 192.168.1.50) or other ones ?

          With a single PFSense node my load balancing configuration through HAProxy works really good. But adding a new layer of High Availabilty with a second PFSense node will it continue working so fine ?

          V 1 Reply Last reply Reply Quote 0
          • V
            viragomann @ddepaolis
            last edited by

            @ddepaolis
            You can do exactly the same with an HA setup.
            However, consider that you need one IP in each network segment for each node and additionally a CARP VIP. So you need 3 IPs in each subnet.

            After configuring CARP HA, you have to hook up the other VIPs on the CARP VIP, so that they are available on both nodes and fail-over with the CARP.
            But you can also use the CARP VIP itself for a HAproxy frontend.

            D 1 Reply Last reply Reply Quote 0
            • D
              ddepaolis @viragomann
              last edited by

              @viragomann Ok, so the actual Alias VIP 1.2.3.4/28 on WAN interface should become a CARP VIP. Then I need two other Alias VIPs on WAN interface: 1.2.3.5/28 on master PFSense node and 1.2.3.6/28 on the slave PFSense node (they're like "physical IP").

              Same way, the actual Alias IP 192.168.1.50/24 configured in frondend HAProxy should become a CARP VIP. And I should continue using it in my HAProxy configuration.

              So is it all correct ? (I hope so).
              Thanks !

              V 1 Reply Last reply Reply Quote 0
              • V
                viragomann @ddepaolis
                last edited by

                @ddepaolis said in CARP Setup with HAProxy and VIPs:

                so the actual Alias VIP 1.2.3.4/28 on WAN interface should become a CARP VIP. Then I need two other Alias VIPs on WAN interface: 1.2.3.5/28 on master PFSense node and 1.2.3.6/28 on the slave PFSense node (they're like "physical IP").

                You need to set physical IPs indeed on the NICs of both nodes in Interfaces > Assignments.
                So on the primary you configure the WAN interface with a static IPv4 1.2.3.5/28 and the proper gateway. And on the secondary 1.2.3.6/28 with proper gateway.

                Ensure you have configured a sync network. A separate NIC is not really necessary, but it's highly recommended. Add a firewall rule to allow access on the sync interface.

                Then add the CARP VIP on the primary. If you use an existing VIP you have to remove this before.

                Also on the LAN you have to configure physical IPs on the interfaces and a VIP, which is used as default gateway on the the LAN devices then.

                See also High Availability Configuration Example in the pfSense docs.

                D 1 Reply Last reply Reply Quote 0
                • D
                  ddepaolis @viragomann
                  last edited by

                  @viragomann Yep. that's all clear in a normal scenario. However on my real enviroment, my actual PFsense node has the "main" public IP of subnet /28 set in the WAN NIC. Then all the other ones in that /28 subnet, are set as alias IPs on same WAN interface. So to explain better:

                  1.2.3.4/28 WAN NIC
                  1.2.3.5/28 Alias IP WAN
                  1.2.3.6/28 Alias IP WAN
                  1.2.3.7/28 Alias IP WAN
                  etc...

                  Splitting this situation on a High Availability scenario, the first 3 public IPs become 1.2.3.4/28 VIP CARP, 1.2.3.5/28 PFsense1 WAN NIC, 1.2.3.6/28 PFSense2 WAN NIC. The other following public IPs can remain as Alias IP on WAN interface ? Then Alias IPs can set the same on each PFSense node, without having addresses' conflict ?
                  Of course thanks so much for all infos I'm now collecting to develop my new enviroment !

                  V 1 Reply Last reply Reply Quote 0
                  • V
                    viragomann @ddepaolis
                    last edited by viragomann

                    @ddepaolis said in CARP Setup with HAProxy and VIPs:

                    Splitting this situation on a High Availability scenario, the first 3 public IPs become 1.2.3.4/28 VIP CARP, 1.2.3.5/28 PFsense1 WAN NIC, 1.2.3.6/28 PFSense2 WAN NIC.

                    Yes, for instance. Though it doesn't matter which IP out of your public range you assign to the interfaces.

                    The other following public IPs can remain as Alias IP on WAN interface ?

                    No, as managened, they VIPs have to hook up on the CARP VIP.
                    After assigning the CARP VIP, on the primary go to the virtual IP settings and change the interface to the CARP VIP.

                    Then Alias IPs can set the same on each PFSense node, without having addresses' conflict ?

                    The only you have to do on the secondary is to assign the interface IPs for all your network segments (WAN, LAN, VLANs) and configure the sync interface and allow access on it.
                    If the sync is working all VIP settings are synced from the primary.

                    Edit:
                    Should add, on the primary your have to configure the sync in System > High Availavility.

                    D 1 Reply Last reply Reply Quote 0
                    • D
                      ddepaolis @viragomann
                      last edited by

                      @viragomann Ok, now I'm setting my lab test enviroment. In this case I'm using, for easier mode of couse, private IPs on WAN and LAN interfaces. So This's my actual scenario:

                      PFsense-1 WAN IP: 192.168.41.20/24 (VHID 80 - Advertising frequency 1 base; 0 skew)
                      PFsense-2 WAN IP: 192.168.41.21/24 (VHID 80 - Advertising frequency 1 base; 100 skew)
                      PFSense WAN VIP (CARP): 192.168.41.22/24

                      PFSense-1 LAN IP: 192.168.42.20/24 (VHID 56 - Advertising frequency 1 base; 0 skew)
                      PFSense-2 LAN IP: 192.168.42.21/24 (VHID 56 - Advertising frequency 1 base; 100 skew)
                      PFSense LAN VIP (CARP): 192.168.42.22/24

                      Replica from master to slave goes fine. Now I aspected to be able to ping the WAN VIP 192.168.41.22 through a client from same subnet 192.168.41.0/24 but this doesn't happen: why ? Both PFSense nodes have a rule to permit ICMP "any to any" on WAN interface.
                      I also already set NAT Outbound in hybrid mode, as indicated in howto page. And I added an explicit rule to allow connections from LAN network through my WAN VIP as "traslated" address.

                      V 1 Reply Last reply Reply Quote 0
                      • V
                        viragomann @ddepaolis
                        last edited by

                        @ddepaolis
                        Since you access from private IPs to the WAN VIP now, did you remove the check "Block private networks" in the WAN interface settings on both nodes?

                        D 1 Reply Last reply Reply Quote 0
                        • D
                          ddepaolis @viragomann
                          last edited by

                          @viragomann Yes I confirm, both options "Block private networks and loopback addresses" and "Block bogon networks" are uncheched on both WAN and LAN NICs on both firewalls. But any ICMP echo reply comes from VIPs on WAN or on LAN. I noticed another unusual behavior. Immediatly after I set CARP VIP on LAN interface on master firewall, it goes immediately in "master" status also in the backup node. As in the pic I added. This behavior always happens when I replicate this action. Actually my lab is inside a VMWare VSphere server, so both PFsense firewalls are virtual machines. I don't know if it could have any effect about this situation.

                          PS: syncronization now goes on a third NIC /28.

                          screenshot_cluster_pfsense.png

                          V 1 Reply Last reply Reply Quote 0
                          • V
                            viragomann @ddepaolis
                            last edited by

                            @ddepaolis said in CARP Setup with HAProxy and VIPs:

                            Actually my lab is inside a VMWare VSphere server, so both PFsense firewalls are virtual machines. I don't know if it could have any effect about this situation.

                            Yes, it could.

                            Read Troubleshooting High Availability Clusters in Virtual Environments.

                            D 1 Reply Last reply Reply Quote 0
                            • D
                              ddepaolis @viragomann
                              last edited by

                              @viragomann I see, and this could be the explanation because I haven't ICMP echo replies on VIP CARP too ?
                              Then I plan another lab in another different virtualization environment.
                              Always thanks and regards !

                              1 Reply Last reply Reply Quote 0
                              • First post
                                Last post
                              Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.