Strongswan/IPSec routing issue with work around
-
I've found that with the 2 IPsec/Strongswan tunnels I have configured on my home PfSense router, that there's a routing issue between each of the 2 pairs that results in network connectivity through the respective tunnels are failing in only one direction.
Interestingly, for each pair, the networks of each side are able to communicate just fine (as the firewall rules permit). The issue is that the router to router communication is one sided. Here's the setup:
Home PfSense <--> Linux server (Debian Buster):
Linux server can reach home PfSense router
Home PfSense router cannot Linux ServerHome Pfsense <--> Remote PfSense
Home Pfsense can reach remote Pfsense
Remote Pfsense cannot reach remote home PfSense
This is the reverse of the aboveIn both cases, I'm talking about the internal addresses. Of course, both pairs can talk to each other over their public addresses and both tunnels are established.
This is not a firewall rule issue. This persists when I've opened up the firewall rules on both ends. What I see is a routing problem that strictly occurs with router to router traffic.
Via tcpdump, I observed the following:
Home PfSense <--> Linux server (Debian Buster):
Home PfSense router to Linux server traffic is attempting to go through the WAN interface of Home PfSense router.
In the reverse case, Linux server traffic goes through the Linux server's WAN interface, but this succeeds.Home Pfsense <--> Remote PfSense
Home PfSense to remote PfSense traffic goes through enc0 on the home PfSense router and succeeds.
Remote PfSense traffic attempts to go through the remote PfSense router's WAN interface and fails.So far, traffic has always succeeded and failed between each router pair in the same direction. But maybe there's a race condition and this could fail in the opposite direction? I haven't tried tearing down and bringing the tunnels back up a ton of times at this stage.
Searching around, this is a problem that does seem to occur at least on occasion with FreeBSD and Strongswan. The potential fix I found is to set 'route_via_internal' to 'yes'. I tried this by editing the config files on disk via the shells and then restarting the tunnels. This didn't help.
Instead, what you can do is add the LAN address as a gateway and then add a static route where the router's LAN address is the route for the remote network. While this is only necessary to add to the "broken side", I add it to both PfSense routers with the assumption that this can and will happen on both sides. I haven't added anything equivalent to the Linux server as I've seen nothing to indicate that Linux is affected at this point.
To clarify what I'm saying, here are the equivalent shell commands to achieve what I'm talking about:
On the router who's LAN IP is 192.168.0.1.
route add -net 10.0.0.0/24 192.168.0.1
On the other router whose LAN IP is 10.0.0.1.
route add -net 192.168.0.0.0/24 10.0.0.1
The above illustrates why the networks on each end works. The hosts on each network already know to use their respective router as the gateway for external networks.
And while this works fairly well, I'd like to see a real fix. Either something is wrong with my IPSec configurations (unlikely as the IPSec configs and firewall rules on both PfSense routers are pretty identical), there's a bug in PfSense's generated Strongswan config, or there's a bug in Strongswan itself.
Does anyone else have any idea on how to make this work without my work around? Does it sound like a bug? Like a pebcak? Are both of the routers for your IPSec pair table to communicate through the tunnel in both directions as expected?
Depending on the feedback I get or if I get no feedback, I may file a bug report.