21.02p1 policy based routing not passing replies on ipsec-VTI

ddbnj

There has been a few similar posts in multiWAN and ipsec but no solutions I can find to the problem above.

I have a site to site setup working with tunnels but switched to VTI in order to try policy based routing. The tunnels are configured using @jimp netgate hangout video as guidance. If I set a simple static route from one firewall to the other across the transfer subnet, I can ping from firewall to firewall:
Local site:
static route:
172.20.1.1/32 to siteb_vti 10.8.222.2

remote site
static route:
172.20.0.1/32 to sitea_vti 10.8.222.1

If I try to use a policy based route for the LAN, the pings from a server behind local lan to a server behind remote lan aren't coming back?

packet capture on VTI remote:

01:29:06.437445 IP 172.20.0.3 > 172.20.1.3: ICMP echo request, id 9, seq 518, length 64
01:29:06.519489 IP 172.20.1.3 > 172.20.0.3: ICMP echo request, id 31, seq 551, length 64
01:29:07.461410 IP 172.20.0.3 > 172.20.1.3: ICMP echo request, id 9, seq 519, length 64
01:29:07.543519 IP 172.20.1.3 > 172.20.0.3: ICMP echo request, id 31, seq 552, length 64
01:29:08.485408 IP 172.20.0.3 > 172.20.1.3: ICMP echo request, id 9, seq 520, length 64
01:29:08.567528 IP 172.20.1.3 > 172.20.0.3: ICMP echo request, id 31, seq 553, length 64

packet capture on VTI local:

01:28:11.146133 IP 172.20.0.3 > 172.20.1.3: ICMP echo request, id 9, seq 464, length 64
01:28:11.236848 IP 172.20.1.3 > 172.20.0.3: ICMP echo request, id 31, seq 497, length 64
01:28:12.170103 IP 172.20.0.3 > 172.20.1.3: ICMP echo request, id 9, seq 465, length 64
01:28:12.260839 IP 172.20.1.3 > 172.20.0.3: ICMP echo request, id 31, seq 498, length 64
01:28:13.194126 IP 172.20.0.3 > 172.20.1.3: ICMP echo request, id 9, seq 466, length 64
01:28:13.284834 IP 172.20.1.3 > 172.20.0.3: ICMP echo request, id 31, seq 499, length 64

If I add the static routes remote to local on both sites:

01:33:54.432732 IP 172.20.0.3 > 172.20.1.3: ICMP echo request, id 9, seq 801, length 64
01:33:54.446795 IP 172.20.1.3 > 172.20.0.3: ICMP echo reply, id 9, seq 801, length 64
01:33:55.004903 IP 172.20.1.3 > 172.20.0.3: ICMP echo request, id 32, seq 73, length 64
01:33:55.005231 IP 172.20.0.3 > 172.20.1.3: ICMP echo reply, id 32, seq 73, length 64
01:33:55.434185 IP 172.20.0.3 > 172.20.1.3: ICMP echo request, id 9, seq 802, length 64
01:33:55.445269 IP 172.20.1.3 > 172.20.0.3: ICMP echo reply, id 9, seq 802, length 64

01:35:23.563533 IP 172.20.0.3 > 172.20.1.3: ICMP echo request, id 9, seq 890, length 64
01:35:23.563856 IP 172.20.1.3 > 172.20.0.3: ICMP echo reply, id 9, seq 890, length 64
01:35:24.103532 IP 172.20.1.3 > 172.20.0.3: ICMP echo request, id 32, seq 162, length 64
01:35:24.114854 IP 172.20.0.3 > 172.20.1.3: ICMP echo reply, id 32, seq 162, length 64
01:35:24.568026 IP 172.20.0.3 > 172.20.1.3: ICMP echo request, id 9, seq 891, length 64
01:35:24.568353 IP 172.20.1.3 > 172.20.0.3: ICMP echo reply, id 9, seq 891, length 64

The firewall rule on the local LAN:

I suspect that either there is some other rule I can't find that is matching packets before my policy route rule but I can't find it.

jimp

You can't just do policy routing on VTI in both directions, you must use static routes. The way FreeBSD filters IPsec traffic it can't do reply-to or NAT on IPsec VTI interfaces with the default settings pfSense needs to filter policy-based (tunnel) IPsec traffic.

There is a workaround coming, but it involves breaking policy-based filtering to allow VTI filtering to work fully. See https://redmine.pfsense.org/issues/8686 and https://redmine.pfsense.org/issues/11395

ddbnj

@jimp

Does wireguard also need static routes to exist before creating policy based routing rules?

jimp

@ddbnj said in 21.02p1 policy based routing not passing replies on ipsec-VTI:

@jimp

Does wireguard also need static routes to exist before creating policy based routing rules?

If you place the rules to pass traffic on the assigned wireguard interface rule tabs, then it will work properly in both directions, just like it would work an appropriate OpenVPN setup.