Routed VTI Interface No Traffic On Other Side
-
I've been setting up a VTI connection in my lab to test a few things, specifically without changing the normal IPsec filter mode (this is to validate something in an environment where I have normal tunnels as well.
NAT is not being used, and traffic from the client I am pinging across this with does appear on both the LAN interface and the VTI interface itself, so the rules appear to be working. But on the second firewall, I don't see any traffic on the IPsec or VTI interfaces, so it's like it's being dropped before arriving over the tunnel, somehow.
From what I can tell, everything is setup as it should be, but no traffic reaches the other firewall. Both firewalls WAN connections are on the same private subnet.
I can do a pcap on the VTI interface on Firewall 1 and see the traffic as expected, so it appears the rules are working.
But, on Firewall 2 I can't see anything on the VTI interface when I do a pcap.
I was going to post photos of each bit but hit the rate limit soooooo that sucks lol.
If the VTI connection is up, and the traffic from Firewall 1 is shown in the pcap on said VTI interface, shouldn't the traffic show up on the IPsec or VTI interface on the second firewall?
-
@planedrop OK it seems to be specific to selecting a gateway for a policy route?
I adjusted all my rules to just allow everything since this is a lab, and pings started working fine. I see the traffic on the VTI interface pcap on the second firewall.
But if I change just the gateway in the allow rule, suddenly the traffic stops routing over the VTI?
I thought policy routing still worked relatively normally with VTI interfaces as long as NAT isn't being used for anything?
-
Yeah so adding the gateway for the VTI to the rule, in order to policy route traffic, results in the pings I am doing still showing up on the VTI interface on Firewall 1, but then nothing shows up on Firewall 2.
If I remove the gateway from the allow rule, then suddenly everything works as expected and I do see the traffic on the VTI interface pcap for Firewall 2.
-
According to the documentation below:
https://docs.netgate.com/pfsense/en/latest/vpn/ipsec/routed-vti.html#policy-routes
https://docs.netgate.com/pfsense/en/latest/vpn/ipsec/routed-vti.html#routed-ipsec-firewall-rules
https://docs.netgate.com/pfsense/en/latest/vpn/ipsec/routed-vti.html#vpn-ipsec-vti-firewall
https://docs.netgate.com/pfsense/en/latest/vpn/ipsec/advanced.html#advanced-ipsec-settingsIPsec Filter Mode should be set to Filter IPsec VTI and Transport on assigned interfaces, block all tunnel mode traffic in IPsec advanced options.
-
@mcury This can't be done if you want to use both VTI and tunnel based IPsec at the same time, since you can't put rules on the tunnel interfaces.
I did read all the documentation, and that specific page about 2 dozen times. (for what it's worth, I've also ready all of pfSense's docs cover to cover twice).
The issue here is that I need both VTI and normal tunnel IPsec, so I can't change the filter mode.
And again, the connection works, and traffic passes just fine if my rules are in place without selecting a gateway, it's only when a gateway is selected for the policy routing that the traffic doesn't appear on Firewall 2s VTI interface, only on Firewall 1s.
-
@planedrop said in Routed VTI Interface No Traffic On Other Side:
The issue here is that I need both VTI and normal tunnel IPsec, so I can't change the filter mode.
Ok.
And again, the connection works, and traffic passes just fine if my rules are in place without selecting a gateway, it's only when a gateway is selected for the policy routing that the traffic doesn't appear on Firewall 2s VTI interface, only on Firewall 1s.
If the documentation is saying that it should not work like that, but you want to keep trying, well, go ahead.
If you need to use tunnel and IPsec VTI, but can`t use policy route, use routing ipsec instead.
-
@mcury The documentation does not say that policy routing doesn't work without changing the IPsec Filter Mode.
It says:
To policy route traffic across a routed IPsec tunnel, use the assigned IPsec interface gateway in firewall rules as usual for policy routing.And:
This may not work as expected without NAT and/or reply-to, which require special settings. See Routed IPsec Firewall Rules for details.
I don't personally think may not work as expected tells me that it will not work.
Part of the reason for this is that I'd like to policy route to a gateway group, the production use case for this is connecting to a VPC that requires VTI to be used, but also requires 2 tunnels to be up. I only need to route a single subnet, so BGP would be more time than I'd like to spend, but was hoping to still benefit from the redundancy.
If I can't do that, I will just configure it for a single tunnel with no gateway selected in the Firewall Rules, but from what I am understanding in the docs, this should still work.
-
Perhaps it is me that didn`t understand the documentation correctly?
Note that English is not my native language..
I have been using BGP/OSPF with VTIs, but you must have two WANs at both sides.
I have a bunch of those configured, working perfectly.
Enable BFD and you will have a very fast convergence.. -
@mcury Yeah it's not very clear if I'm honest, so hard to say. Your english is great so I don't think it's that.
I will do more testing and see if I can get to the bottom of this. Worst case I can just either A. use static routes over 1 tunnel or B. use BGP instead, just for a single subnet it felt silly to configure BGP lol.
-
Yeah so as an update, it did work just fine when I changed the IPsec Filtering Mode, so it appears policy routing just doesn't work at all in the default mode.
-
Opened a Redmine about this since this either A. needs to be explained more clearly, or B. needs to be changed so the docs say "will" instead of "may".
https://redmine.pfsense.org/issues/16340
