Confusing behavior - push "route network subnet"?

CaptainElmo · Dec 16, 2015, 12:29 AM

Hello everyone!

I have an OpenVPN server on pfSense 2.2.5 in a road warrior configuration. Clients can connect (establish the tunnel) without issue.

If I have the redirect gateway option set on the server everything works as expected - clients can access all intended subnets through the tunnel.

If I uncheck the redirect gateway option (enabling split tunneling) and add the protected networks in the "IPv4 Local Network/s" box clients can no longer access any of the intended subnets through the tunnel. I have confirmed that the expected network routes are being added to the client, and can confirm in the firewall logs that pfSense is receiving the targeted traffic from the tunnel and the firewall is passing it. However, when in split tunnel mode the traffic dies somewhere inside pfSense once it gets past the firewall, with the result that clients cannot connect to anything through the tunnel.

I have discovered that adding the following line to the advanced options fixes the issue in split tunnel mode:
push "route network subnet";

What I don't understand is why? The routing table on the client appears to be exactly the same with or without this option, but without this option the client can access nothing through the split tunnel despite the intended traffic getting tunneled.

Would someone mind educating me on why this option is required for the split tunnel to function? Based on this behavior I'm concerned that I might have something incorrectly configured.

Thanks in advance!

viragomann · Dec 16, 2015, 2:05 PM

Hi!

@CaptainElmo:

I have discovered that adding the following line to the advanced options fixes the issue in split tunnel mode:
push "route network subnet";

What I don't understand is why? The routing table on the client appears to be exactly the same with or without this option, but without this option the client can access nothing through the split tunnel despite the intended traffic getting tunneled.

Would someone mind educating me on why this option is required for the split tunnel to function? Based on this behavior I'm concerned that I might have something incorrectly configured.

This option isn't necessary on a proper setup.

Since you don't let us see your server and client configuration, it's difficult to say, what's the cause issue.
At client you can check, if there is something different in interface parameters of the virtual adapter.

CaptainElmo · Dec 16, 2015, 3:54 PM

That is exactly what I am concerned about, and I'm not sure what could be wrong.

Attached are 2 screenshots of my server setup page, and below is the client config (with identifying info redacted) created by the client export utility. If there is anything else you would like to see please point me in the right direction on how to get it for you.


dev tun
persist-tun
persist-key
cipher AES-256-CBC
auth SHA256
tls-client
client
resolv-retry infinite
remote <wan ip="">1194 udp
lport 0
verify-x509-name "<anme>" name
auth-user-pass
pkcs12 <name>.p12
tls-auth <name>.key 1
ns-cert-type server</name></name></anme></wan>

viragomann · Dec 16, 2015, 5:44 PM

I cannot see any miss configuration.
Just, it's recommended to use compression.
And you've checked "Force DNS cache update", but you do not provide DNS. So what's the sense of this?

The proper way to pushing routes to clients is to enter the intended subnets in "Local Network/s" input field.
This works without any issue here.

Maybe it's a client issue. Is it Windows?
In Windows if you've "Redirect Gateway" checked, the VPN adapter is classified as domain network, otherwise it's unknown.

To check if it's on client, use "Packet Captuere" from Diagnostic menu on LAN and OpenVPN interface.

CaptainElmo · Dec 16, 2015, 11:01 PM

Yes, it's a windows client.

The packet capture is showing the expected packets:


15:54:49.040335 IP 192.168.5.14.49418 > 192.168.6.2.80: tcp 0
15:54:52.047850 IP 192.168.5.14.49418 > 192.168.6.2.80: tcp 0
15:54:58.069495 IP 192.168.5.14.49418 > 192.168.6.2.80: tcp 0

The firewall log also confirms that the firewall passed the traffic.

Yet the client is not connecting to the target machine in split tunnel mode. If I turn off split tunneling it works, but I don't want all of the client's other internet traffic flowing through me.

I'll keep tinkering with it, but I'm at a loss to understand what is happening.

viragomann · Dec 16, 2015, 11:36 PM

Is this packet capture taken from LAN or from VPN interface?

There are only seen requests from VPN client. If it looks like this at LAN, you get no response from LAN host.
If you get response on LAN check the routing table of pfSense.

CaptainElmo · Dec 16, 2015, 11:42 PM

The packet capture was on the VPN interface. Here is the packet capture on the target LAN interface:


16:39:26.732743 IP 192.168.5.14.49538 > 192.168.6.2.80: tcp 0
16:39:26.733033 ARP, Request who-has 192.168.6.1 tell 192.168.6.2, length 46
16:39:26.733042 ARP, Reply 192.168.6.1 is-at 00:08:a2:09:57:27, length 28
16:39:26.733217 IP 192.168.6.2.80 > 192.168.5.14.49538: tcp 0
16:39:26.733548 IP 192.168.5.14.49538 > 192.168.6.2.80: tcp 0
16:39:26.991870 IP 192.168.5.14.49539 > 192.168.6.2.80: tcp 0
16:39:26.992109 IP 192.168.6.2.80 > 192.168.5.14.49539: tcp 0
16:39:26.992395 IP 192.168.5.14.49539 > 192.168.6.2.80: tcp 0
16:39:29.740083 IP 192.168.5.14.49538 > 192.168.6.2.80: tcp 0
16:39:29.740318 IP 192.168.6.2.80 > 192.168.5.14.49538: tcp 0

CaptainElmo · Dec 16, 2015, 11:49 PM

Does OpenVPN play well on the same pfSense box when it has both client and server tunnels?

The OpenVPN server on the pfSense box is using UDP 1194, but I also have an OpenVPN client connection on the same box going out to an external IP on TCP 443 (not my call on the protocol for that tunnel) which has been stable for months.

So back to my troubleshooting:

For the OpenVPN server (UDP 1194) I tried assigning an interface to the server's network port (Interfaces->Assign). Doing this caused the road warrior client on UDP 1194 to work correctly, but when this tunnel is connected (and routed through an explicit interface) I can no longer access the TCP 443 tunnel. The error log shows:

openvpn[88415]: write TCPv4_CLIENT: Operation not permitted (code=1)

When I disconnect the UDP 1194 tunnel the TCP 443 tunnel starts working normally again. When I remove the explicitly-assigned interface from the UDP 1194 and just run it through the default ovpns network port the TCP 443 tunnel works in all cases but the UDP 1194 tunnel does not.

So are these two OpenVPN connections (one server, one client) conflicting with each other somehow? Is there something I need to do for them to play well together? They are shown as different services in pfsense and use different ports, but it appears they are not getting along somehow.

CaptainElmo · Dec 16, 2015, 11:54 PM

Here are screenshots for my OpenVPN client setup.

CaptainElmo · Dec 17, 2015, 1:44 AM

Well, I don't know how to explain this but it's working now. I manually re-keyed the "IPv4 Local Network/s" on the OpenVPN server setup screen and after saving it started working. The 7.0/24 subnet is on the other end of the OpenVPN client tunnel, so perhaps that was the commonality between tunnels causing them to interact? And my previous note about it working with split tunneling disabled also touches this since that field disappears when the "redirect gateway" option is checked.

It makes no sense to me at all, but so it is. After manually re-keying the subnets into that field everything is now working.