High Availabilty Multi WAN VIP Gateway failover fault

[msudak]

Hi,

This is an intermittent problem
I have a pfSense HA config with dual WAN static public IP's as such:
pfSense1 WAN1 x.x.x.11/28, pfSense2 WAN1 x.x.x.12/28, Carp VIP WAN1 x.x.x.13/28
pfSense1 WAN2 x.x.x.156/29, pfSense2 WAN2, x.x.x.157/29, Carp VIP WAN2 x.x.x.158/29

System > Routing > Gateways:
Gateway WAN1 x.x.x.1
Gateway WAN2 x.x.x.153
Default gateway IPv4 = "WAN_FAILOVER", a Gateway Group for the above gateways where Gateway Priority Virtual IP is set to use the respective VIP (I don't know what this setting is for actually so if anyone has any info please advise.)

This configuration works for both internet and system failover.

However, when configuring two outbound NAT rules to use the VIP's (so my outbound IP will stay the same when in failover mode,) internet failover stops working! I'm using Hybrid Outbound NAT mode.

I discovered that if I set only one of the outbound NAT rules to use the Interface Address, all is well. Disabling the interface of the failed internet connection also corrects the problem.

Routes look correct and pfSense itself does have internet access, it's just NAT that is not working. A basic packet capture and states reveal that traffic is trying to go out using the NAT VIP but no connections become established.

Thanks for helping.