CARP+IPSEC+0.0.0.0/0

[npiersma]

I've run into a likely problem that is easy to reproduce so I wanted to share with you all. I am not sure it is an issue, it may be by design, but it has got me moving from hyper-v to VMware and back again ;).

I've set up two pfsense boxes and updated them to the latest version.

Assigned External IP [WAN] of 10.4.8.132/24 and 10.4.8.133/24
Assigned internal IP [LAN] of 192.168.1.2/24 and 192.168.1.3/24
Assigned internal pfsync [OPT1] IP of 192.168.199.1/30 and 192.168.199.2/30

Made sure all switches allowed spoofing/mac address switching etc.

Set up HA, verified config is replicating by setting up an alias on the master.

Created shared IP addresses for the WAN and the LAN. Set up NAT correctly as well as the DHCP server.
Clients work fine, no problem at all.

So rock-solid fail-over, HA working like a charm.

Now comes the funny part;
I've set up an IPSEC-tunnel between my two HA-pfsense boxes and one pfsense box at a cloud provider. The goal is to tunnel ALL local traffic through the tunnel and have it enter the big internet via my cloud provider.
When I use 0.0.0.0/0 as the remote network, both my HA-pfsense-LAN interface becomes MASTER in the CARP status view as soon as the Tunnel is active. When I use anything else like /24 or /16 subnet in phase 2 for the IPSEC-tunnel, the CARP-status is fine.

The WAN interface is unaffected by this setting.

It looks to me that the VRRP traffic is "snooped" away and send into the tunnel. But that's my interpretation.

The question here is, does it matter that both interfaces are master, in this particular case? Is this by design? Or is CARP and 0.0.0.0/0 over IPSEC mutually exclusive? --> found the answer, YES, it does.

The IPSEC-tunnel is not building predictably, sometimes the [WAN]MASTER/[LAN]MASTER node is building the connection, sometimes the [WAN]BACKUP/LAN[MASTER] (node is building the connection. But it is not stable.

As it is quite simple to reproduce, I would have expected more people running into this as well.

Hoping someone can shed some light on if this should work (or not).

Niels