Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Dual PfSense boxes, dual Internet connections, CARP, failover HELP!!

    Scheduled Pinned Locked Moved HA/CARP/VIPs
    15 Posts 3 Posters 5.5k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • J Offline
      jasonlitka
      last edited by

      Off the top of my head:

      Setup a gateway on LAN for your first box which points at the second pfSense system.  Add your WAN gateway & that second gateway to a gateway group as Tier 1 & Tier 2 respectively.  Add that gateway group your your LAN Firewall rules.

      I can't remember if gateways & groups automatically sync over pfsync, so you may have to manually create the GW & group on the second box so that the rule doesn't break when you failover.

      I can break anything.

      1 Reply Last reply Reply Quote 0
      • S Offline
        SquiJee
        last edited by

        Hey there Jason,

        Sorry for the slow reply but I did not have much time to invest on that lately.

        Yesterday I did what you said:

        • Added a new gateway on Pfsense1 with my Pfsense2 LAN address (192.168.100.2)
        • Created a new gateway group with Pfsense1's WAN as tier1 (192.168.255.1) and my newly added gateway as tier2 (192.168.100.2)
        • I checked on my Pfsense2 and the configs done were synched already.
        • Also have an allow all any/any in LAN rules, for testing purposes

        Then I unplugged my Pfsense1's WAN and…. no failover :(

        I'll try to specify the gateway group on the any/any LAN rule and see how it goes in a few hours and let you know the result edit:(well, did not work either). Other than that… Is there anything I missed? =/

        Thanks!

        1 Reply Last reply Reply Quote 0
        • J Offline
          jasonlitka
          last edited by

          Take a screenshot of the rules on your LAN interface.

          I can break anything.

          1 Reply Last reply Reply Quote 0
          • S Offline
            SquiJee
            last edited by

            Hi Jason,

            Here:
            http://imgur.com/fadQogt

            1 Reply Last reply Reply Quote 0
            • J Offline
              jasonlitka
              last edited by

              You haven't told your rule to actually use the gateway group.

              I can break anything.

              1 Reply Last reply Reply Quote 0
              • S Offline
                SquiJee
                last edited by

                Hi Jason,

                Yes I did but when I tested, it was still not working so I changed it back to "*"

                I changed that for the gateway group again now, tested again and still no failover. Here's how it looks now:
                http://imgur.com/Hqykcmg

                Also, would'nt it work even if I don't specify the group? I mean, isn't the "*" a catch all?

                Thanks!

                1 Reply Last reply Reply Quote 0
                • J Offline
                  jasonlitka
                  last edited by

                  What does that failover group look like?  What is the gateway status for each?

                  No, if you do not specify a gateway on your rule then it will use the system default.

                  Also, try specifying the gateway you setup for your second pfSense box on that rule.  It will either send your traffic that way or you won't have any connectivity.

                  I can break anything.

                  1 Reply Last reply Reply Quote 0
                  • S Offline
                    SquiJee
                    last edited by

                    Hi Jason,

                    I went in the gateway status menu (I didn't know it existed) and found out that, on my second box, the WAN of my first box was not reachable and therefore, offline on this side. I added a static route on my second box and now both boxes show the gateways online in the gateway status menu. I also sent a ping from my second box to my first box's WAN and it works.

                    Now the weird thing is that, maybe 15 minutes later, I went again in the gateway status menu on my first box and second box and  here's how it looks now:

                    http://imgur.com/mcRv2LH

                    It is really strange because I did not change anything! I can also still ping my first box's WAN from my second box just fine, but in the gateway group it show offline?! Also, on my first box, my second box shows as "Gathering data".

                    I also tried to unplug the first box's WAN and still no failover… Any idea why it's doing that?

                    Thanks again!

                    1 Reply Last reply Reply Quote 0
                    • J Offline
                      jasonlitka
                      last edited by

                      What have you setup for the failover gateway on each box?  It sound like you used the WAN IP of the other system.  You should be using the LAN IP if that is the shared network.

                      I can break anything.

                      1 Reply Last reply Reply Quote 0
                      • S Offline
                        SquiJee
                        last edited by

                        Hi Jason,

                        Not sure what you mean by "You should be using the LAN IP if that is the shared network" (edit: did you mean the CARP LAN vip (192.168.100.10), instead of the LAN IP of the boxes?) , but here's how the gateway groups are configured on each boxes:

                        First box:
                        Tier1: Pfsense1's WAN (192.168.255.1)
                        Tier2: Pfsense2's LAN (192.168.100.2)
                        never: Pfsense1's LAN (192.168.100.1)

                        Second box:
                        Tier1: Pfsense1's WAN (192.168.255.1)
                        Tier2: Pfsense2's LAN (192.168.100.2)
                        never: Pfsense1's LAN (192.168.100.1)
                        never: Pfsense2's WAN (DHCP)

                        That first box's WAN (192.168.255.1) shows offline in the gateway status of the second box. Also, I removed my static route and I can still ping it, but still shows offline in gateway status.

                        Thanks!

                        1 Reply Last reply Reply Quote 0
                        • J Offline
                          jasonlitka
                          last edited by

                          I'm going to use an example with slightly different IPs to make it more clear.

                          Interface IPs
                          Box 1

                          • WAN - 10.0.0.2
                          • LAN - 192.168.1.2

                          Box 2

                          • WAN - 172.16.0.2
                          • LAN - 192.168.1.3

                          CARP

                          • LAN - 192.168.1.1

                          Gateways
                          Box 1

                          • GW_WAN - WAN - 10.0.0.1
                          • GW_PF2 - LAN - 192.168.1.3

                          Box 2

                          • GW_WAN - WAN - 172.16.0.1
                          • GW_PF1 - LAN - 192.168.1.2

                          Gateway Groups
                          Box 1

                          • "Failover"
                              - Tier 1 - GW_WAN
                              - Tier 2 - GW_PF2

                          Box 2

                          • "Failover"
                              - Tier 1 - GW_WAN
                              - Tier 2 - GW_PF1

                          Rules
                          Apply the gateway group "Failover" to all the LAN rules you want to switch to the other box.

                          I can break anything.

                          1 Reply Last reply Reply Quote 0
                          • S Offline
                            SquiJee
                            last edited by

                            Hi Jason,

                            Thanks for clarifying this to me, had some stuff wrong on the second box. Got that all fixed up now as you described, but still, the setup does not failover yet.

                            There is 2 things I noticed, I don't know if it will tell you something but anyway:

                            1- I checked the CARP status while the first box's WAN was unplugged and the first box was still the "master". I guess that make sense in a way, since the LAN address still work fine. Do I need to add something in the CARP setting so that it checks the first box's WAN also?

                            2- When I go in the gateway status on the first box, the "GW_PF2 - LAN - 192.168.1.3" (If I take your example) always switch between "Online" and "Gathering data". What I mean there is that, if I keep refreshing the page, it always switch between the 2 modes.

                            Any other idea?

                            Thanks again for your time and your support!

                            1 Reply Last reply Reply Quote 0
                            • First post
                              Last post
                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.