IPSec with VTI not routing LAN traffic between 2 sites

BeDazzler · Sep 29, 2020, 4:08 PM

Hi All,

First time poster here, please be gentle :)

I am struggling to get IPSec with VTI working between 2 sites.

I've followed the pfSense help docs to configure site A with a VTI address 10.10.161.1/24 and site B with 10.10.175.1/24.

From the diagnostics menu they both see each other, however I cannot get any LAN traffic routed no matter what I do.

Static routes just timeout and go nowhere.

I've allowed IPv4 everything in IPSec firewall rules on both sides.

Is there something simple that I have missed or not doing ?

Many thanks

BeDazzler.

Derelict · Sep 29, 2020, 5:48 PM

@BeDazzler: site A with a VTI address 10.10.161.1/24 and site B with 10.10.175.1/24

What, exactly, does this mean.

The VTI address shoult be someting like 10.10.10.5/30 on one side and 10.10.10.6/30 on the other. Then there should be an interface assigned, and routes to the networks on the other side to the gateway on the other side.

https://docs.netgate.com/pfsense/en/latest/vpn/ipsec/routed-vti.html

BeDazzler · Sep 30, 2020, 12:44 AM

Thanks for the reply.

So I've done that too.

I have now configured site A with 10.10.161.1 and site B with 10.10.161.2 using /30.

The session between site A and site B is established and from the diagnostics menu I can ping from side to side fine, however I cannot route any traffic from site A to site B. Both site A and site B show each other's remote gateways as online.

As an example, on site A I've configured a static route for 8.8.8.0/24 to use the IPSEC VTI path 10.10.161.2, however no traffic routes if I ping 8.8.8.8 from a Windows machine connected to pfSense at site A.

When I remove the static route, it returns to working fine.

If I set the default gateway on site A to be the remote gateway on site B, that does not work either.

Is there somewhere else I need to configure pfSense to allow the static route to pass traffic from site A to site B ?

Many thanks

BeDazzler.

Derelict · Sep 30, 2020, 12:51 AM

You need a route for the reply traffic on the other side too.

https://docs.netgate.com/pfsense/en/latest/vpn/ipsec/routed-vti.html

BeDazzler · Sep 30, 2020, 1:45 AM

I have static routes on site A and site B.

I want to route all site A traffic out of site B.

I have configured a static route on site A for 0.0.0.0/24 to traverse 10.10.161.2 (Site B).

pfSense still routes that traffic out the local WAN interface on site A.

a tcpdump of ipsec1000 on site A pfSense shows there is no traffic going from pfSense on site A to site B, except for ICMP request and replies between 10.10.161.1 and 10.10.161.2.

I see this in the IPSec logs:
querying policy 0.0.0.0/0|/0 === 0.0.0.0/0|/0 in failed, not found
querying policy 0.0.0.0/0|/0 === 0.0.0.0/0|/0 out failed, not found

It's not making sense to me.

BeDazzler · Sep 30, 2020, 2:10 AM

OK I figured it out.

In the firewall rules for LAN I needed to allow traffic to and from IPSECVTI at both site A and site B.

I can now ping and route traffic from site A to site B and it works.

Thanks.

Derelict · Sep 30, 2020, 2:47 AM

You are going to need to outline everything you have done precisely. All routes, all firewall rules, any policy routing in place on the inside interfaces.

I see this in the IPSec logs:
querying policy 0.0.0.0/0|/0 === 0.0.0.0/0|/0 in failed, not found
querying policy 0.0.0.0/0|/0 === 0.0.0.0/0|/0 out failed, not found

Totally normal for logging on a VTI tunnel.

BeDazzler · Sep 30, 2020, 5:04 AM

Well, it was working so I rebooted the pfSense appliances and now it's not.

I've been able to get things working multiple times, but then after rebooting it stops working.

If I try to set the default gateway on site A to local and then reset it to IPSecVTI it starts working again - regardless of what static routes I have in place.

Site A:
IPSec VPN configured with IKEV2/AES256 connects up to site B fine.
P2 configured with local subnet 10.10.161.1/30, ESP, 256.
Session establishes, from diagnostics menu site A can ping site B.
No hosts on the LAN interface for site A can ping 10.10.161.2 at site B (this was previously working).

Site B:
IPSec VPN configured with IKEV2/AES256 connects up to site A fine.
P2 configured with local subnet 10.10.161.2/30, ESP, 256.
Session establishes, from diagnostics menu site B can ping site A.
No hosts on the LAN interface for site B can ping 10.10.161.1 at site B (this was previously working).

Firewall Rules - LAN on Site A:
IPv4 LAN net to ANY (allow)
IPv4 IPSECVTI net to ANY (allow)
IPv4 ANY to IPSECVTI (allow)
IPv4 ANY to ANY (allow)

Firewall Rules - IPSec on Site A:
IPv4 ANY to ANY (allow)

Firewall Rules - LAN on Site B:
IPv4 LAN net to ANY (allow)
IPv4 IPSECVTI net to ANY (allow)
IPv4 ANY to IPSECVTI (allow)
IPv4 ANY to ANY (allow)

Firewall Rules - IPSec on Site B:
IPv4 ANY to ANY (allow)

The pfSense appliance at site A has a local IP 172.25.9.252.
I have a Windows PC connected to the LAN interface at 172.25.9.99.
The Windows PC can ping 10.10.161.1 (site A VTI).

The pfSense appliance at site B has a local IP 10.10.75.1.
I have a Windows PC connected to the LAN interface at 10.10.75.52.
The Windows PC cannot ping 10.10.161.2 (site B VTI) no matter what rules I put in place.

It just doesn't work.

BeDazzler · Sep 30, 2020, 10:32 AM

Don't worry about this now, after 2 days trying to get this working I have dumped pfSense and moved onto something that works.

Derelict · Sep 30, 2020, 12:46 PM

@BeDazzler It works fine. You misconfigured it. When I asked you to show everything you have done, you didn't show creating any static routes across the VTI for the LAN subnets.

https://docs.netgate.com/pfsense/en/latest/vpn/ipsec/routed-vti.html#static-routes

BeDazzler · Sep 30, 2020, 3:21 PM

I created static routes.

It might work for someone else, however if I can't get it to work reliably there's no way I can support it, so it's dead and in the bin.

I ended up configuring a CentOS machine with Open VPN client and routing between 2 NICs.

Works. Simple and easy for anyone to maintain.

Took 14 hours less to setup.

jimp · Oct 1, 2020, 3:00 PM

If you are going to try routing Internet traffic like 8.8.8.8 across then you also need hybrid or manual outbound NAT rules to NAT traffic from a source of the subnet(s) across the VTI.

BeDazzler · Oct 2, 2020, 1:16 AM

Yep, tried that too.

To solve the problem, I just removed pfSense from site A and replaced it with a CentOS VM running OpenVPN client.

From there I have a VLAN on our Cisco LAN SW which has a DHCP scope to configure the LAN default gateway via OpenVPN.

Works perfectly.