CARP over Bridge, is it necessary? And also about STP

[andre.paiz]

Hi everybody.

I need to bridge my WAN/LAN interfaces to preserve IP addresses, since we use public addresses in our university. And I also have 2 redundant firewalls. So I will deliver the main router network directly to my internal switch core and insert the transparent firewalls between them.

Both servers have 3 NICs (LAN, WAN and Management).

I have created the bridge interface with LAN and WAN with no IP addresses, since I use the Management IP to control the firewalls. Everything works fine.

Since I'm using 2 firewalls, I had to enable STP on both bridge interfaces to block layer 2 looping, and it also worked. Is this the correct scenario? Should I enable STP on both bridge interfaces (one for each firewall) and also on the switches in the LAN and WAN networks?

And the main question: Since STP is performing redundancy, should I use CARP? If yes, the virtual IP should be created in the management network as the other interfaces doesn't have IP addresses?

The only reason I see to use CARP is to perform state and configuration synchonization. Is it possible to do that without CARP?
I believe that STP will do redundancy in case of interface/server failure. Is that correct?

Any other recommendations for my scenario?

BTW, my PFSense version is 2.3.1.

Thanks a lot.

[jimp]

Bridging and HA are awful together. It's ugly, and as you've seen, requires STP to cooperate to shut down the loop.

Whether or not you need CARP is impossible to say without more information. If the firewall still needs IP-level redundancy for any reason, you still need CARP. For example if you need to use port forwards, outbound NAT, inbound routing, VPNs shared between the two nodes, etc.

XMLRPC Config sync and state synchronization can work without CARP.

[andre.paiz]

Hi Jimp. Thank you very much for the reply.

My WAN/LAN interfaces that will be in the bridge won't have IP addresses.
I also won't use NAT, since here we work only with public IP addresses.

My environment is:

L3 Switch With OSPF on WAN. Both WAN and LAN have public IPs and do not perform NAT.

Firewalls WAN connected to the LAN interface of the L3 switch
Firewalls LAN connected to my inside core router
Firewalls LAN and WAN are bridge
The firewalls have no IP addresses, only in a separate MGMT interface, outside the bridge.

Inside core router with IP address using the L3 border switch as default gateway

Is there a less "ugly" way to do it?
I need to use bridge to preserve public addresses.

[jimp]

It may be functional in that case without CARP then. There is no non-ugly way to have an HA firewall with a bridge – on anything, not just pfSense. The whole concept is ugly :-)

Redesigning the network to use proper routing and not a bridge is the only way forward that wouldn't be a potential source of problems.

[andre.paiz]

Thanks a lot for your help.

[jimp]

To clarify: The above isn't mean to be rude, but a statement of experience. At my previous job I ran an HA pair for years that was bridged and it was a never-ending nightmare of babysitting switches, some things not working during a primary failure, mysterious network issues, etc. I bit the bullet and redesigned the entire network to use routing and that same setup has had zero problems since, other than an unrelated hardware failure.