VPN Clients cannot see internal network after 2.6 to 2.7 upgrade

uptown

@stephenw10 Yes that is the case. 2.6 gets the routes to the internal subnet and 2.7 is blocked to the internal subnet. We troubleshooted several different ways over the past week and just concluded that it is something in 2.7. We would have created a ticket with Netgate support but decided not to because 2.6 works without fail. And we'll just continue to work with that until possible 2.7.1/2.8 is the fix. But we are definitely open to any tips or something we may have overlooked.

stephenw10

@uptown said in VPN Clients cannot see internal network after 2.6 to 2.7 upgrade:

2.6 gets the routes to the internal subnet and 2.7 is blocked to the internal subnet.

There's a big difference between the client not getting a route and the client traffic being blocked. Really we need to know exactly what the difference at the client is between 2.6 and 2.7. Or that there is no difference at the client and the only thing different is at the firewall end.

uptown

@stephenw10 I'll explain it this way: We backed up the 2.6 server and then upgraded it to 2.7, with no configuration changes at all. When we discovered trouble with 2.7, we then switched over to the 2.6 backup we created which works well.

stephenw10

Yes I understand. But to diagnose the issue we need to know exactly what is failing. For example is the server failign to pass a route to the client at all? Or maybe it's passing it in some format the client is rejecting. Or maybe it's passing a route in exactly the same way, the client is using it but for some reason the firewall is blocking it. Or it could be a server end routing issue where hosts in the subnet behind the server cannot reply; though that seems unlikely.

uptown

@stephenw10 the pfSense 2.7 firewall is not passing the route through, on both ends when doing a trace the route dies at the 2.7 firewall.

stephenw10

Hmm, well if you traceroute to something on the LAN subnet from a VPN client and it's hitting pfSense at all then the client must have the correct route. Otherwise it would try to send it outside the VPN.

So that implies the firewall is either not routing it correctly or is blocking it. You don't see any blocked traffic though? Can I assume you don't see any passed traffic logged either?

NZ

I know this topic is a little old.
But we're also running into a similar issue, thought I add to it here.
Since we just started upgrading from v2.6 to v2.7.

Doesn't look like this is a blocking issue.
It seems to be routing related.

Problem is the same.
We have L2TP over IPSEC VPN setup.
On v2.6 when VPN is established anything on the LAN is accessible.
However after upgrading to v2.7 we can' see anything on the LAN expect the PFsense IP and L2TP server IP, plus the VPN client IP.

Somehow nothing is getting routed from the VPN client to LAN beyond PFsense.

We're still pulling our hair out trying to troubleshoot this.
But we did find a difference betwen v2.6 and v2.7 in the routes table.
The remote LAN is 192.168.1.0/24
PFsense is on 192.168.1.1
L2TP server IP: 192.168.1.247
VPN client IP: 192.168.1.248

In v2.6 the Gateway assignment looks like this (in the routes table):
192.168.1.1 Gateway: link#4
192.168.1.247 Gateway: link#9
192.168.1.248 Gateway: link#9

But on v2.7 it looks like this:
192.168.1.1 Gateway: link#4
192.168.1.247 Gateway: link#4
192.168.1.248 Gateway: link#9

You can see the gateway relation.
It seems to reversed.

Let me know if this sheds some more light on the issue.

stephenw10

Hmm, well I wouldn't expect that to make much difference. I also don't expect to see three gateways in the same subnet though.
https://docs.netgate.com/pfsense/en/latest/vpn/l2tp/configuration.html#ip-addressing

What does the routing table at the client look like after it connects?

NZ

The client end shows PFsense v2.7:

Network Destination        Netmask          Gateway       Interface  Metric
    192.168.1.0    255.255.255.0      192.168.1.247    192.168.1.248     26
    192.168.1.248  255.255.255.255         On-link     192.168.1.248    281

Even if we choose a completely different network for the VPN.
eg 10.10.10.1
Same thing.

So were out of ideas. Don't understand why it works fine like this in v2.6 but not in v2.7
As mentioned, the difference we found was in the way the gateways are structured.

Another thing I should add, is that we also have site-to-site IPSEC VPN tunnels setup between locations, and they can route to each other without any issue.
Infact, we can even ping the LAN on the remote IPSEC VPN from the client. But that same client cannot ping anything on the LAN it's initially VPN'd to.

So the issue seems to be with the Mobile VPN, since site-to-site is fine.
It's very odd. Doesn't make much sense.

stephenw10

Hmm, both ends of the L2TP are pfSense? I has assumed this was a remote access setup?

NZ

For L2TP were connecting with Windows built-in L2TP client.
So Windows connects to the PFsense L2TP via PFsense Mobile IPsec.

Basically we setup the PFsense L2TP server.
And the PFsense IPsec Mobile Client.
Then remotely connect with Windows to PFsense.

stephenw10

Ok, that's what I had assumed originally.

So is there a difference at the client routing table between 2.6 and 2.7?

NZ

We also checked that during our tests.
Routing table on client is same regardless 2.6 or 2.7
The only routing entries Windows gets from PFsense are the ones I listed above.

Also a tracert looks like this.
in v2.6:
Tracing route to 192.168.1.45 over a maximum of 30 hops
1 14 ms 13 ms 15 ms 192.168.1.247
2 17 ms 15 ms 16 ms 192.168.1.45

in 2.7:
Tracing route to 192.168.1.45 over a maximum of 30 hops
1 11 ms 4 ms 5 ms 192.168.1.247
2 * * * Request timed out.

.247 being the PFsense L2TP server IP.

This is why I think the the Gateway Link# assignments may have something to do with it. That's the only difference we've noticed so far.

NZ

@stephenw10

Any suggestions on what settings we can try?
Using Mobile IPSEC that is.
We don't want to use OpenVpn.

Unless this is a bug in the system that needs to be worked out?

stephenw10

You're able to use mobile IPSec dircetly? Without L2TP?

NZ

Only way to do that is ipsec to ipsec endpoints.
(eg Pfsense to Pfsense)
And yes that works fine.

We use Windows clients.
They need some kind of tunnel initiator like PPTP or L2TP.
I don't know of any way to IPSEC from Windows without that.

But you may be on to something, the L2TP server in Pfsense.
That's what creates those gateways in the route table.
But again it works fine in v2.6
So there must be an issue in v2.7 with L2TP server.

stephenw10

You can use IKEv2 mobile IPSec on Windows directly. It's just not as straight forward:
https://docs.netgate.com/pfsense/en/latest/recipes/ipsec-mobile-ikev2-client-windows.html

NZ

I will have to test it out.

But what do we do about the l2TP server issue?
Only other option is downgrading to v2.6

stephenw10

The only thing I can think of that might possibly be affected is the filtering change. Try setting 'IPsec Filter Mode' to assigned interfaces in the IPSec advanced settings.

However if that was the issue I'd expect to see blocked traffic in the firewall logs. Unless you have custom block rules without logging maybe?