WAN2 to WAN1 failover not working
I have two firewalls running CARP, one master, one slave. Forget about the slave for now.
The master has several interfaces:
LAN is connected to a switch that is connected to the LAN.
WAN1 is connected to a DSL PPPOE model.
WAN2 is connected to a wireless radio connected to an ISP with a static IP
CARP is connected to the other slave pfsense firewall
I have set up CARP VIP on LAN and DHCP op both firewalls.
Focussing on the issue on hand, the master firewall has two gateways set up in a group. WAN2 is primary, WAN1 is the fallback. WAN1 is set to monitor the gateway IP. WAN2 too. WAN2 GW is Tier1, WAN1 GW is Tier 2.
I have two DNS servers for WAN1GW, and two for WAN2GW (Set up in General Setup).
Outbound NAT is set to manual. I have an entry for WAN1 Source LAN subnet all ports all destination ports NAT address WAN1 address NAT port * Static port NO.
I have an entry for WAN2 Source LAN subnet all ports all destination ports NAT address WAN2 address NAT port * Static port NO.
squid and squid filter are installed in transparent mode.
Firewall rules use Wanfailover gateway (i.e. the gateway group) as advanced gateway to enable it to route to the gateway not offline. I have not enabled a rule for port 80 because I want all HTTP traffic to flow through squid.
When both ISPs (WAN1 & WAN2) are up, everything works great. When I yank the network cable for WAN2 (primary), traffic flows for a little while through WAN1 (1 minute) until everything stops. I cannot even access the firewall on WAN1 IP. Plugging WAN2 cable back in resolves everything - I can access all sites.
What am I doing wrong?