[Site-2-Site] wrong gateway is set for remote networks

tim0ka

pfSence() OpenVPN serve is configured for site-2-site communication and assumes multiple remote networks are connected. there is config files that generated with GUI only and no custom options are used:

[24.03-RELEASE]/home: sudo cat /var/etc/openvpn/server2/config.ovpn
dev ovpns2
disable-dco
verb 3
dev-type tun
dev-node /dev/tun2
writepid /var/run/openvpn_server2.pid
#user nobody
#group nobody
script-security 3
daemon
keepalive 10 60
ping-timer-rem
persist-tun
persist-key
proto udp4
auth SHA256
up /usr/local/sbin/ovpn-linkup
down /usr/local/sbin/ovpn-linkdown
local XXX.XXX.XXX.XXX
tls-server
server 192.168.67.232 255.255.255.248
client-config-dir /var/etc/openvpn/server2/csc
ifconfig 192.168.67.233 192.168.67.234
tls-verify "/usr/local/sbin/ovpn_auth_verify tls 'timka-vpn.hopto.org' 1"
lport 1195
management /var/etc/openvpn/server2/sock unix
push "route 192.168.67.0 255.255.255.192"
route 192.168.67.128 255.255.255.224
route 192.168.67.160 255.255.255.224
capath /var/etc/openvpn/server2/ca
cert /var/etc/openvpn/server2/cert
key /var/etc/openvpn/server2/key
dh /etc/dh-parameters.2048
tls-auth /var/etc/openvpn/server2/tls-auth
data-ciphers AES-256-GCM:AES-128-GCM:CHACHA20-POLY1305
data-ciphers-fallback AES-256-GCM
allow-compression no
topology subnet

Example client custom override:

[24.03-RELEASE]/home: sudo catar/etc/openvpn/server2/csc/client1
iroute 192.168.67.160 255.255.255.224

Client1 is configured with IP:192.168.67.234 as the second host IP address within tunnel network and receives necessary route as expected:

PUSH: Received control message: 'PUSH_REPLY,route 192.168.67.0 255.255.255.192,route-gateway 192.168.67.233,topology subnet,ping 10,ping-restart 60,ifconfig 192.168.67.234 255.255.255.248,peer-id 0,cipher AES-256-GCM,protocol-flags cc-exit tls-ekm dyn-tls-crypt,tun-mtu 1500'

pfSense OpneVPN interface uses 192.168.67.233 as the first host IP address within the tunnel network. However, local routes added as following:

ip:192.168.67.160 netmask:255.255.255.248 gw:192.168.67.233

As a result, remote networks are not accessible from nether side. Am i missing anything in the configuration or this is some bug?

Workaround has been round actually:

remove remote networks definition from the server configuration
custom options are following

route 192.168.67.160 255.255.255.248 192.168.67.233
route 192.168.67.128 255.255.255.248 192.168.67.233

viragomann

@tim0ka
Seems you've set a different tunnel mask in the CSO than in the server settings.

Client1 is configured with IP:192.168.67.234 as the second host IP address within tunnel network

What?

192.168.67.234 is obviously the IP, which is assigned to the client.

pfSense OpneVPN interface uses 192.168.67.233 as the first host IP address within the tunnel network. However, local routes added as following:

ip:192.168.67.160 netmask:255.255.255.248 gw:192.168.67.233

That's expected. 192.168.67.233 is the servers virtual IP and the client adds route for the remote networks pointing to it.

Workaround has been round actually:

remove remote networks definition from the server configuration
custom options are following

route 192.168.67.160 255.255.255.248 192.168.67.233
route 192.168.67.128 255.255.255.248 192.168.67.233

Generally this does the same as setting the remote networks:
192.168.67.160/29,192.168.67.128/29
But obviously you did different settings before.

Not really clear, what you intend. What are the local networks on server and client site?

tim0ka

@viragomann
Sorry, some copy-paste error with last routes, but config is correct. Let me rephrase...
The intent to have many remote networks connected to the same OpenVPN server instance(site-to-site in subnet mode). Target topology is the following:

192.168.67.0/26 - pfSense - |
                            | - client1 (192.168.67.160/27)
                            | - client2 (192.168.67.128/27)  
                            | - ...
                            | - clientN

IPv4 Tunnel Network is set as 192.168.67.232/29.

Per my experiments:

pfSence gets 192.168.67.233
client1 gets 192.168.67.234 and route 192.168.67.0 255.255.255.192 192.168.67.233
client1 gets 192.168.67.235 and route 192.168.67.0 255.255.255.192 192.168.67.233
There is what i can see under for client1 connected:

All above is expected, but networks cannot interact. Under pfSence Diagnostics-Routes routes:

192.168.67.128/27	192.168.67.234	UGS	13	1500	ovpns2	
192.168.67.160/27	192.168.67.234	UGS	13	1500	ovpns2	
192.168.67.232/29	link#14	U	11	1500	ovpns2

in case if "IPv4 Remote network(s)" is set as "192.168.67.128/27,192.168.67.160/27" for OpenVPN server. Not sure if it is expected as 192.168.67.234 is client1 IP address. Per my work around:

IPv4 Remote network(s) is empty for OpenVPN server configuration
Custom options has two commands(there was type in previous post):

route 192.168.67.160 255.255.255.224 192.168.67.233
route 192.168.67.128 255.255.255.224 192.168.67.233

As a result local routing table(Diagnostics-Routes) like this:

192.168.67.128/27	192.168.67.233	UGS	13	1500	ovpns2	
192.168.67.160/27	192.168.67.233	UGS	13	1500	ovpns2
192.168.67.232/29	link#14	U	11	1500	ovpns2

And all works as expected

viragomann

@tim0ka said in [Site-2-Site] wrong gateway is set for remote networks:

All above is expected, but networks cannot interact. Under pfSence Diagnostics-Routes routes:

192.168.67.128/27 192.168.67.234 UGS 13 1500 ovpns2
192.168.67.160/27 192.168.67.234 UGS 13 1500 ovpns2
192.168.67.232/29 link#14 U 11 1500 ovpns2

in case if "IPv4 Remote network(s)" is set as "192.168.67.128/27,192.168.67.160/27" for OpenVPN server. Not sure if it is expected as 192.168.67.234 is client1 IP address. Per my work around:

Yes, this is expected.
You have to enter the client sites networks into the "Remote Networks" box in the server settings. This add the shown routes pointing to the virtual server IP.

The particular routing to the respective client is done inside OpenVPN then.
This requires that you configure a Client Specific Override for each you want to access the network behind.
In the CSO you have to state the particular client side network at "Remote Networks".

If you did configure the CSOs properly, but the routing does not work, verify if the CSO is applied. This needs the log verbosity level in the server settings to be set to at least 4, but don't set it higher to avoid noise. Then reconnect to client and check the log.

tim0ka

@viragomann
yes, I tried is before an just reproduced:

removed custom options from server configuration page
added IPv4 Remote Network/s as following

Client1 become inaccessible. There is what's under logs:

Jun 17 17:01:08 openvpn 84141   client1/XXX.XXX.XXX.XXX:54980 MULTI: Learn: 192.168.67.160/27 -> client1/XXX.XXX.XXX.XXX:54980
Jun 17 17:01:08 openvpn 84141   client1/XXX.XXX.XXX.XXX:54980 MULTI: internal route 192.168.67.160/27 -> client1/XXX.XXX.XXX.XXX:54980
Jun 17 17:01:08 openvpn 84141   client1/XXX.XXX.XXX.XXX:54980 MULTI: primary virtual IP for client1/XXX.XXX.XXX.XXX:54980: 192.168.67.234
Jun 17 17:01:08 openvpn 84141   client1/XXX.XXX.XXX.XXX:54980 MULTI: Learn: 192.168.67.234 -> client1/XXX.XXX.XXX.XXX:54980
Jun 17 17:01:08 openvpn 84141   client1/XXX.XXX.XXX.XXX:54980 OPTIONS IMPORT: reading client specific options from: /var/etc/openvpn/server2/csc/client1

Looks like CSO is processed correctly, but no route under Diagnostics - Routes on pfSense side. There is just a single one:

192.168.67.232/29	link#14	U	11	1500	ovpns2

viragomann

@tim0ka said in [Site-2-Site] wrong gateway is set for remote networks:

Looks like CSO is processed correctly, but no route under Diagnostics - Routes on pfSense side

As mentioned, the particular routes are only added inside OpenVPN as shown in the log.
In the pfSense routing table you only see routes pointing to the OpenVPN server IP.
This should work properly.

tim0ka

@viragomann
thank you for confirmation, but it does not work unless neither IPv4 Remote Network/s are set(cannot test with two client as for now, might be the issue) nor routes are added with custom options on OpneVPN server level