[SOLVED] CARP - Two Masters (was: Redundancy Network Layout)

[gdi2k]

Edit: See second post.

We run pfsense as VMs on two separate servers configured with CARP to avoid failure issues due to a server going down. WANs are directly connected to the switches via VLANs.

Now we would like to prevent outages that may be caused by a switch failure. I've scribbled a network layout and attached it, would this be a reasonable approach?

Does LAG mode 1 on the VM host (active / backup) work well for connecting a single pfsense instance to two switches, or is it better (simpler is sometimes better) just to let the first pfsense box become inaccessible when the first switch fails and let the second pfsense box on the second switch take over completely? If so, would the carp sync be better off running on a VLAN between the switches, rather than directly between the two VM hosts, (otherwise a switch failure would not be detected by carp)?

I have had all sorts of issues getting CARP to work properly - yesterday we all went offline because the gateway VIP became unpingable for reasons unknown. Are these sorts of issues common? Does it really work reliably or is it more trouble than it's worth? Are there any special switch settings which need to be set to make it work well?

[gdi2k]

So I simplified things quite a bit. See attached.

Now I have pfsense1 and pfsense2 (VMs) and connected to main switches 1 and 2 respectively. The idea is that I can survive a either a switch going down or a firewall.

Both main switches are connected the the other switches (to which user PCs are connected) with RSTP is enabled. That's all working fine.

But when CARP is enabled, both firewalls want to become master, and I can't wrap my head around why this would be.

Can I not do it this way?

EDIT: Forgot to mention that WAN1 and WAN2 are also connected directly to main switches 1 and 2 respectively, and link to pfsense via VLANs. That's all working nicely too. Just CARP is the issue now…

[gdi2k]

Bump.

Cannot get CARP working. Both are masters for all WANs.

Is this such an unusual setup?

[cmb]

Not an unusual setup at all.

When both are master, they can't communicate between each other (or at least CARP's multicast advertisements aren't getting between them). Find why they can't talk between each other and fix it and it'll likely all work at that point.

[gdi2k]

Thanks, your comment about multicast put me on the right track.

We run pfsense on KVM and used the passthrough NIC setting on the guest rather than just using a bridged set up. For reasons beyond me, this does not allow multicast traffic to pass to the network. Changing to a bridged network config for the pfsense VMs solved the issue. omping for the win.

My other issues are best placed in another topic, so marking this solved. Thanks!