Mobile IPsec works only on default WAN?
I have two WAN links, both work fine. Both WAN interfaces have a gateway assigned. I have set up a mobile IPSEC profile, which works fine. However, it only works if it is bound to the WAN interface with the default route for PFSENSE. More specifically:
WAN1 has gateway GW1
WAN2 has gateway GW2
If the IPSEC profile is bound to WAN1 and GW1 is the default gateway set for the PFSENSE system, the IPSEC works.
If the IPSEC profile is bound to WAN2 and GW2 is the default gateway set for the PFSENSE system, the IPSEC works.
If the IPSEC profile is bound to WAN1 and GW2 is the default gateway set for the PFSENSE system, the IPSEC does not work.
If the IPSEC profile is bound to WAN2 and GW1 is the default gateway set for the PFSENSE system, the IPSEC does not work.
I have several other services using both WAN addresses and they all work fine. A tcpdump on the external interface shows the incoming packets, but no replies. Racoon receives the phase 1 request, then complains that is has to resend the response (resend phase1 packet), then gives up (phase1 negotiation failed due to time up). It looks like racoon does not know how to send the reply back to the client, unless the default gateway is assigned via the interface Racoon is bound to.
Some more information:
This is not related to Racoon. I have enabled SSH for remote access, and I see exactly the same. The packets come in on the correct interface, but the replies go out on the interface with the default route, although the source address is the correct one (of the interface they came in to). If I manually add a route to the remote destination via the correct (non-default) gateway, then the replies go out on the correct interface. The conclusion is that the system does not reply via the same interface that the packet came through.
Routed packets are processed just fine - the reply goes back on the correct interface, the one that has originally received the packet. Only local PFSENSE services are affected.