Testing Virtualized HA in ESXi. When WAN is lost, only WAN VIP fails over, LAN interfaces erroneously stay on the master.
-
I have two ESXi 6.7U2 hosts each with three vSwitches: WAN, LAN, and SYNC. These three vSwitches each have one physical NIC uplink. WAN to the CPE, LAN to physical switch, SYNC direct connection between hosts.
Failover works fine if the entire primary goes down. However, if I pull the WAN uplink NIC on the master, on,y the WAN VIP fails over to the secondary. The LAN VIPs stay on the master and cause a service disruption.
Any ideas?
-
Is the VM seeing lost link? That's usually pretty difficult to simulate in a virtual environment.
-
I'm guessing that the VM is not seeing a lost link, because the vNIC isn't technically losing link, just the phy uplink to the vSwitch.
I guess what I don't understand is why preemption doesn't seem to work in this case. If I pull the phy WAN link, the WAN VIP moves over to the secondary. Why wouldn't this be enough to cause the LAN VIPS to demote themselves too?
-
Behause gateway failover is not the same as interface down. HA detects interface down which should never happen in a virtual environment. If that is what you want you will probably need to pass physical NICs through to the pfSense instances. I do not know if there is a way to tell ESXi to pass interface down to the associated VMs. Moving to Virtualization to get the right eyes on this.
-
@vbman213 said in Testing Virtualized HA in ESXi. When WAN is lost, only WAN VIP fails over, LAN interfaces erroneously stay on the master.:
Why wouldn't this be enough to cause the LAN VIPS to demote themselves too?
Because the primary has no idea anything is wrong. See the sticky a the top of the HA category.
Chattanooga, Tennessee, USA
The pfSense Book is free of charge!
DO NOT set a source port in a port forward or firewall rule unless you KNOW you need it!
Do Not Chat For Help! NO_WAN_EGRESS(TM) -
@Derelict said in Testing Virtualized HA in ESXi. When WAN is lost, only WAN VIP fails over, LAN interfaces erroneously stay on the master.:
@vbman213 said in Testing Virtualized HA in ESXi. When WAN is lost, only WAN VIP fails over, LAN interfaces erroneously stay on the master.:
Why wouldn't this be enough to cause the LAN VIPS to demote themselves too?
Because the primary has no idea anything is wrong. See the sticky a the top of the HA category.
But it certainly demotes it's own WAN interface (which then causes the secondary to take master status on the WAN VIP). So I guess either 1) I don't fully understand CARP or 2) I disagree with your assertion that "the primary has no idea anything is wrong"
-
It doesn't. It will stay MASTER. The backup stops seeing advertisements so it assumes MASTER too. The ONLY way it will assume BACKUP is if it sees the advertisements from the other node that are more preferable than its own settings. It can't see them with the cable unplugged.
I explained all of this in that sticky.
Chattanooga, Tennessee, USA
The pfSense Book is free of charge!
DO NOT set a source port in a port forward or firewall rule unless you KNOW you need it!
Do Not Chat For Help! NO_WAN_EGRESS(TM) -
@Derelict But that's the thing, I do see BACKUP status on the primary's WAN VIP. So the primary is somehow aware that it isn't MASTER anymore. This is why I'm confused (too). Everything you're saying makes sense.
-
You will have to dig into the system logs to see what is happening.
HA generally needs a link down event to demote itself.
-
I haven't had a chance to digest the system logs... but I had an additional comment.
It would seem that this problem could be avoided if I brought my WAN into my ESXi hosts over the "LAN" uplink on a VLAN. So one physical link for every CARP'ed interface within the pfSense VM.
-
pfSense just needs to see link down by whatever method your hypervisor allows.
