CARP crashes with two LAN sub networks on the same WAN network

[Philippe Rachas]

Hello,

I have a COGENT WAN network xxx.xxx.xxx.0/24. COGENT gateway xxx.xxx.xxx.1. COGENT router xxx.xxx.xxx.2
On this WAN, I have 2 sub LAN networks LAN behind 2x2 pfsense 2.5.2 with HA.
Here a diagram with a fake WAN 88.88.88.0 / 24

EDIT : error in diagram, CARP IPs on PFS 2.2 are BACKUP, not MASTER

Everything work fine. Connection, NAT, HA. From outside (WWW), I can access to all servers, and to all Pfsense with real or virtual IP. Network 1 or 2, everything is OK
I can also PING any server from any other server, even on different network.

But I can't access to a NETWORK 2 IP service that need response (like web page) from a NETWORK 1 server.

If I disable CARP on NETWORK 1 routers (enabled on NETWORK 2)
Source : (NETWORK 1) 192.168.10.101 => DEST : (NETWORK 2) https://88.88.88.21
No problem. Work fine.

If CARP is enabled on NETWORK 1 et NETWORK 2 routers
Source : (NETWORK 1) 192.168.10.101 => DEST : (NETWORK 2) https://88.88.88.21
Unable to connect.

• Web page is not laoded
• CARP crashes and connection is broken
• All WAN CARP IPs from Pfsense 1.1 are in INIT Status
• CARP says that Demotion Status is incorrect.
• But, if I try just to PING instead loading page, no problem.

If I close the page, and reset CARP Demotion Satus, everything returns to normal state a few seconds later but with a new pfsync node in CARP status. A new node each time.

So I think there is a conflit between CARP and my two networks.
But I have unique VHID and pfsync peer IP forced on a LAN IP (no multicast).

So, I really don't know what I missed ...
Anyone can help me ?

Thanks
Regards