OpenVPN Site-to-Site Routes

[m4xm0rris]

Hi all,

Appreciate any help with this one as it is wrangling my brain. We have a primary office site (10.1.50.0/24) on a PPPoE WAN connection hosting a OpenVPN Peer to Peer Server with a tunnel network of 10.8.1.0/24. This pfSense instance is run as a VM on Proxmox.

Our secondary site (10.1.60.0/24) is at a serviced office location (Regus property if that helps at all) with a Netgate SG-1100. The WAN connection gets a DHCP lease on the buildings 192.168.8.0/22 network and the OpenVPN client on the Netgate connects perfectly.

Both pfSense instances are able to ping and connect to addresses on each others subnets, though no other devices on either network are able to route across the tunnel. I have put the corrrect subnets into the "IPv4 Remote Network(s)" section on both devices.

Only thing I've noticed that momentarily makes the tunnel rout-able is going into System > Routing > Static Routes and creating a static route across the VPN interface on each pfSense instance. This however only works until the tunnel is dropped, when it is reconnected, routing stops unless I manually recreate the static route.

Any help greatly appreciated!

[viragomann]

@m4xm0rris
The routes have to be managed by OpenVPN by adding the remote network into the "Remote networks" box in the settings, as you already tried.

So reenter the settings there, remove the static routes and check the routing table on both sites when the vpn is up.
If the routes aren't correct, check the OpenVPN logs for any issue regarding adding routes.

[m4xm0rris]

@viragomann @viragomann Thanks for your reply. The OpenVPN Logs have the following when connecting the tunnel (taken from the secondary site (10.1.60.0/24))

Feb 8 21:20:21	openvpn	50504	event_wait : Interrupted system call (code=4)
Feb 8 21:20:21	openvpn	50504	/usr/local/sbin/ovpn-linkdown ovpnc2 1500 1573 10.8.1.2 10.8.1.1 init
Feb 8 21:20:21	openvpn	50504	SIGTERM[hard,] received, process exiting
Feb 8 21:20:22	openvpn	67931	disabling NCP mode (--ncp-disable) because not in P2MP client or server mode
Feb 8 21:20:22	openvpn	67931	OpenVPN 2.4.9 aarch64-portbld-freebsd11.3 [SSL (OpenSSL)] [LZO] [LZ4] [MH/RECVDA] [AEAD] built on May 4 2020
Feb 8 21:20:22	openvpn	67931	library versions: OpenSSL 1.0.2u-freebsd 20 Dec 2019, LZO 2.10
Feb 8 21:20:22	openvpn	68231	NOTE: the current --script-security setting may allow this configuration to call user-defined scripts
Feb 8 21:20:22	openvpn	68231	Initializing OpenSSL support for engine 'cryptodev'
Feb 8 21:20:22	openvpn	68231	TUN/TAP device ovpnc2 exists previously, keep at program end
Feb 8 21:20:22	openvpn	68231	TUN/TAP device /dev/tun2 opened
Feb 8 21:20:22	openvpn	68231	/sbin/ifconfig ovpnc2 10.8.1.2 10.8.1.1 mtu 1500 netmask 255.255.255.255 up
Feb 8 21:20:22	openvpn	68231	/usr/local/sbin/ovpn-linkup ovpnc2 1500 1573 10.8.1.2 10.8.1.1 init
Feb 8 21:20:22	openvpn	68231	TCP/UDP: Preserving recently used remote address: [AF_INET]<Primary Site WAN IP>:2196
Feb 8 21:20:22	openvpn	68231	UDPv4 link local (bound): [AF_INET]192.168.8.21:0
Feb 8 21:20:22	openvpn	68231	UDPv4 link remote: [AF_INET]<Primary Site WAN IP>:2196
Feb 8 21:20:22	openvpn	68231	Peer Connection Initiated with [AF_INET]<Primary Site WAN IP>:2196
Feb 8 21:20:23	openvpn	68231	WARNING: this configuration may cache passwords in memory -- use the auth-nocache option to prevent this
Feb 8 21:20:23	openvpn	68231	Initialization Sequence Completed

The Server end of the tunnel has largely the same logs, with the exception of this message

ERROR: FreeBSD route add command failed: external program exited with error status: 1

Any ideas?

PS new on this forum so I apologise if there is a more preferred format for logs etc

[viragomann]

@m4xm0rris
The log shows your public IP. You'd better hide this.

But there is no entry of adding routes on the client and on the server it obviously failed.

For getting closer, please post the OpenVPN configurations of both site (with public IPs hidden) and the routing tables of both, when the VPN is down.

[m4xm0rris]

@viragomann said in OpenVPN Site-to-Site Routes:

ng closer, please post the OpenVPN configurations of both site

Thanks for the heads up, have yet to consume my 3rd coffee for the day

When you say "When the VPN is down", do you mean connected but unroutable (as it is now) or after disabling the connection's daemon?

[viragomann]

@m4xm0rris
When the connection is disabled, to see the origin routing table.

[m4xm0rris]

@viragomann Any black boxes indicate the Primary Sites WAN IP, as the Secondary Site has a WAN connection on another private subnet I have not censored this. Also I may have forgotten to mention, the Primary Site also has a OpenVPN Remote Access server on 10.1.1.0/24 so you may see this on the Route table. Let me know if you need anything else

Primary Site VPN:

Primary Site Routes:

Secondary Site VPN:

Secondary Site Routes:

[viragomann]

@m4xm0rris
The subnet 10.1.60.0/24 is defined on both sites.
On primary it is assigned to em0, the same as 10.1.50.0/24 and pfSense has the IP 10.1.60.60.

[m4xm0rris]

@viragomann Why this is appearing as this I have no idea. There are no interfaces, nor have there ever been any assisgnmnets on the 10.1.60.0/24 network on the Primary site. Only interfaces assigned at primary are the LAN (10.1.50.0/24), an IoT VLAN (10.1.51.0/24) and a Guest VLAN (10.1.52.0/24). Can you think as to why this would be the case?

[m4xm0rris]

@viragomann
I've just noticed in the Auto Rules in Outbound NAT on primary, there are listing of the 10.1.60.0/24 network. I have very little knowledge of how these work, does it help?

[viragomann]

@m4xm0rris
Its there, cause its assigned to an interface.
Check the virtual IPs and the interface settings.

[m4xm0rris]

@viragomann Well I'll admit I've rarely felt stupider than that. Indeed there was a Virtual IP setup for 10.1.60.60, think I must have created it at some weird point for some weird reason.
Having said that, I have deleted it now and restarted the OpenVPN service on both ends of the tunnel and Reset States, still the issue persists.

[m4xm0rris]

So after deleting the Virtual IP, clearing the "IPv4 Remote Network(s)" fields on both of the OpenVPN configs and adding in Static Routes for the remote subnets, it seems this is now working and the Static Route persists between tunnel reconnects. For some reason it still doesn't seem to work without defining a Static Route for the remote subnets to route over the VPN Interface gateway, but nonetheless, it works!

Would have never even considered to look in the Virtual IPs, thanks for your help @viragomann