Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Testing Virtualized HA in ESXi. When WAN is lost, only WAN VIP fails over, LAN interfaces erroneously stay on the master.

    Scheduled Pinned Locked Moved Virtualization
    11 Posts 2 Posters 1.2k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • DerelictD
      Derelict LAYER 8 Netgate
      last edited by

      Is the VM seeing lost link? That's usually pretty difficult to simulate in a virtual environment.

      Chattanooga, Tennessee, USA
      A comprehensive network diagram is worth 10,000 words and 15 conference calls.
      DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
      Do Not Chat For Help! NO_WAN_EGRESS(TM)

      1 Reply Last reply Reply Quote 0
      • cmcdonaldC
        cmcdonald Netgate Developer
        last edited by

        I'm guessing that the VM is not seeing a lost link, because the vNIC isn't technically losing link, just the phy uplink to the vSwitch.

        I guess what I don't understand is why preemption doesn't seem to work in this case. If I pull the phy WAN link, the WAN VIP moves over to the secondary. Why wouldn't this be enough to cause the LAN VIPS to demote themselves too?

        Need help fast? https://www.netgate.com/support

        1 Reply Last reply Reply Quote 0
        • DerelictD
          Derelict LAYER 8 Netgate
          last edited by

          Behause gateway failover is not the same as interface down. HA detects interface down which should never happen in a virtual environment. If that is what you want you will probably need to pass physical NICs through to the pfSense instances. I do not know if there is a way to tell ESXi to pass interface down to the associated VMs. Moving to Virtualization to get the right eyes on this.

          Chattanooga, Tennessee, USA
          A comprehensive network diagram is worth 10,000 words and 15 conference calls.
          DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
          Do Not Chat For Help! NO_WAN_EGRESS(TM)

          1 Reply Last reply Reply Quote 0
          • DerelictD
            Derelict LAYER 8 Netgate
            last edited by

            @vbman213 said in Testing Virtualized HA in ESXi. When WAN is lost, only WAN VIP fails over, LAN interfaces erroneously stay on the master.:

            Why wouldn't this be enough to cause the LAN VIPS to demote themselves too?

            Because the primary has no idea anything is wrong. See the sticky a the top of the HA category.

            Chattanooga, Tennessee, USA
            A comprehensive network diagram is worth 10,000 words and 15 conference calls.
            DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
            Do Not Chat For Help! NO_WAN_EGRESS(TM)

            cmcdonaldC 1 Reply Last reply Reply Quote 0
            • cmcdonaldC
              cmcdonald Netgate Developer @Derelict
              last edited by

              @Derelict said in Testing Virtualized HA in ESXi. When WAN is lost, only WAN VIP fails over, LAN interfaces erroneously stay on the master.:

              @vbman213 said in Testing Virtualized HA in ESXi. When WAN is lost, only WAN VIP fails over, LAN interfaces erroneously stay on the master.:

              Why wouldn't this be enough to cause the LAN VIPS to demote themselves too?

              Because the primary has no idea anything is wrong. See the sticky a the top of the HA category.

              But it certainly demotes it's own WAN interface (which then causes the secondary to take master status on the WAN VIP). So I guess either 1) I don't fully understand CARP or 2) I disagree with your assertion that "the primary has no idea anything is wrong"

              Need help fast? https://www.netgate.com/support

              1 Reply Last reply Reply Quote 0
              • DerelictD
                Derelict LAYER 8 Netgate
                last edited by

                It doesn't. It will stay MASTER. The backup stops seeing advertisements so it assumes MASTER too. The ONLY way it will assume BACKUP is if it sees the advertisements from the other node that are more preferable than its own settings. It can't see them with the cable unplugged.

                I explained all of this in that sticky.

                Chattanooga, Tennessee, USA
                A comprehensive network diagram is worth 10,000 words and 15 conference calls.
                DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
                Do Not Chat For Help! NO_WAN_EGRESS(TM)

                cmcdonaldC 1 Reply Last reply Reply Quote 0
                • cmcdonaldC
                  cmcdonald Netgate Developer @Derelict
                  last edited by cmcdonald

                  @Derelict But that's the thing, I do see BACKUP status on the primary's WAN VIP. So the primary is somehow aware that it isn't MASTER anymore. This is why I'm confused (too). Everything you're saying makes sense.

                  Need help fast? https://www.netgate.com/support

                  1 Reply Last reply Reply Quote 0
                  • DerelictD
                    Derelict LAYER 8 Netgate
                    last edited by

                    You will have to dig into the system logs to see what is happening.

                    HA generally needs a link down event to demote itself.

                    Chattanooga, Tennessee, USA
                    A comprehensive network diagram is worth 10,000 words and 15 conference calls.
                    DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
                    Do Not Chat For Help! NO_WAN_EGRESS(TM)

                    1 Reply Last reply Reply Quote 1
                    • cmcdonaldC
                      cmcdonald Netgate Developer
                      last edited by

                      I haven't had a chance to digest the system logs... but I had an additional comment.

                      It would seem that this problem could be avoided if I brought my WAN into my ESXi hosts over the "LAN" uplink on a VLAN. So one physical link for every CARP'ed interface within the pfSense VM.

                      Need help fast? https://www.netgate.com/support

                      1 Reply Last reply Reply Quote 0
                      • DerelictD
                        Derelict LAYER 8 Netgate
                        last edited by

                        pfSense just needs to see link down by whatever method your hypervisor allows.

                        Chattanooga, Tennessee, USA
                        A comprehensive network diagram is worth 10,000 words and 15 conference calls.
                        DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
                        Do Not Chat For Help! NO_WAN_EGRESS(TM)

                        1 Reply Last reply Reply Quote 0
                        • First post
                          Last post
                        Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.