Eliminating Switch as Single Point of Failure
Say I have a simple setup with a LAN and WAN. However, on the LAN side, I would like two switches, and each of my end devices will plug into a port on each switch (multi-port NICs).
If I setup multiple firewalls with CARP, each plugging into one switch, what would happen if the switch the primary firewall is plugged into switch lost?
The reason I ask is: It's just the LAN switch that is lost. Technically, both firewalls are fine and can still communicate with each other through the pfsync connection.
Would the system be smart enough to fail over to the backup firewall, even though technically both firewalls are ok?
CARP works by sending a heartbeat on the wire on each interface with a CARP IP. If this heartbeat is not seen on any interface, a backup will take over as master.
So in your situation, the backup would still take over, as it wouldn't see the heartbeat on the LAN side.
CARP is capable of only failing over certain interfaces, but iirc pfSense has a sysctl set that fails them all if any one interface fails.
Nice, exactly what I needed to know. Thank you.
to increase availability you can add more to it
if you use two NICs in each pfSense in a LAG going to a switch stack that can have a LAG across/between switches, you would then have a system that can tolerate a switch failure without failing over.
expanding across this, from this switch pair you would use the same technique to go to each other switch.
Thanks for the suggestion. I looked into LAGG but it didn't seem like it was supported in any meaningful way in 1.2.x, and since it's a production environment I couldn't risk running 2.x where it does seem to be supported.
If anyone cares, I did test using CARP/pfsync for switch redundancy and it does work, just as jimp indicated.