CARP gateway packet loss (but it works?)

[arduino]

I have pfSense deployed as a guest within Hyper-V.

I am using pulling two DHCP leases for my two WAN addresses (same ISP, different gateways).

I have five /29 static CARP IP's and setup the default gateway to be in the same network.

I am using Manual Outbound NAT for all local networks and the firewall, pointing to the CARP addresses.

Everything is working except the host does not have access to the internet. The default gateway, since it does not lay in the same network as the DHCP leased WAN, shows 20-40% packet loss and well over 1000ms RTT.

If I switch outbound NAT for the firewall to use a CARP, it does work for a short time, but eventually falls back into showing packet loss and high RTT times.

CARP failover DOES work fine at the moment.

Is there a solution to this? I am of the understanding that this isn't a supported setup, but I do not want to lose my static IPs on the supported setup.