I had to open a support ticket to get this fixed. Here is the reply from the technician:
–-----
Upon my initial reading here is what I think is happening:
Inbound connection arrives on pf2:WAN3
pf2 forwards the connection to the internal host
The internal host replies but its default gateway should be the LAN interface's CARP VIP which is currently on pf1
pf1 does not know what to do with the traffic so it is dropped.
The typical work around for this would be an outbound NAT entry on LAN so all traffic going to the inside host appears to come from the interface address on LAN. That will make the reply traffic same-subnet so the default gateway in the target host will not need to be used.
The downside is you lose the ability to see the actual outside source addresses in the logs/connections on the inside host. This might or might not be important to you.
This turned out to be exactly the problem. Adding an "outbound NAT" entry solved this.