Routing Problem in Test Network

viragomann

Adding the route fails on the client as the log shows:

Wed Oct 11 16:26:53 2017 ERROR: Linux route add command failed: external program exited with error status: 2

The "push route" command in the custom options is not needed, since this is already set by the entry in "Local Network(s)". So delete it from the custom options field. Maybe this solve your issue.

Alejandro.Carbonara

@viragomann:

The "push route" command in the custom options is not needed, since this is already set by the entry in "Local Network(s)". So delete it from the custom options field. Maybe this solve your issue.

I made the change you suggested. It removed the add route error, thanks!

Wed Oct 11 17:21:53 2017 WARNING: file 'auth.txt' is group or others accessible
Wed Oct 11 17:21:53 2017 OpenVPN 2.4.3 x86_64-pc-linux-gnu [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [PKCS11] [MH/PKTINFO] [AEAD] built on Jun 30 2017
Wed Oct 11 17:21:53 2017 library versions: OpenSSL 1.0.2l 25 May 2017, LZO 2.08
Wed Oct 11 17:21:53 2017 TCP/UDP: Preserving recently used remote address: [AF_INET]192.168.57.3:1194
Wed Oct 11 17:21:53 2017 UDP link local (bound): [AF_INET][undef]:1194
Wed Oct 11 17:21:53 2017 UDP link remote: [AF_INET]192.168.57.3:1194
Wed Oct 11 17:21:53 2017 WARNING: this configuration may cache passwords in memory – use the auth-nocache option to prevent this
Wed Oct 11 17:21:53 2017 [server] Peer Connection Initiated with [AF_INET]192.168.57.3:1194
Wed Oct 11 17:22:04 2017 TUN/TAP device tun0 opened
Wed Oct 11 17:22:04 2017 do_ifconfig, tt->did_ifconfig_ipv6_setup=0
Wed Oct 11 17:22:04 2017 /sbin/ip link set dev tun0 up mtu 1500
Wed Oct 11 17:22:04 2017 /sbin/ip addr add dev tun0 192.168.80.2/24 broadcast 192.168.80.255
Wed Oct 11 17:22:04 2017 Initialization Sequence Completed

However, I am still failing to reach the internal machine, and the pfsense logs are still showing the similar GET INST BY VIRT [failed] error.

viragomann

Now you're missing the "route add" command on the client.
Please post your server and client config.

Alejandro.Carbonara

Okay, so taking from my config.xml:

_<openvpn><openvpn-server><vpnid>1</vpnid>
<mode>server_tls_user</mode>
<authmode>Local Database</authmode>
<protocol>UDP</protocol>
<dev_mode>tun</dev_mode>
<ipaddr></ipaddr>
<interface>wan</interface>
<local_port>1194</local_port>

<tls>-removed-

<local_network>192.168.58.0/24</local_network>
<local_networkv6></local_networkv6>
<maxclients>10</maxclients>

<passtos></passtos>
<client2client>yes</client2client>
<dynamic_ip>yes</dynamic_ip>
<pool_enable>yes</pool_enable>
<topology>subnet</topology>

<serverbridge_interface>none</serverbridge_interface>
<serverbridge_dhcp_start></serverbridge_dhcp_start>

<netbios_enable></netbios_enable>
<netbios_ntype>0</netbios_ntype>

<verbosity_level>10</verbosity_level></tls></openvpn-server></openvpn>
<dnshaper></dnshaper>
<cert><refid>52db1097e7e40</refid>

<crt>-removed-</crt></cert>
<cert><refid>59de7040dec5d</refid>

<type>server</type>
<caref>59de7040c414a</caref>
<crt>-removed-</crt>
<prv>-removed-</prv></cert>
<cert><refid>59de707095d08</refid>

<type>user</type>
<caref>59de7040c414a</caref>
<crt>-removed-</crt>
<prv>-removed-</prv></cert>

<gateways></gateways>
<ovpnserver><step1><type>local</type></step1>
<step6><certca>Cert-CA</certca>
<keylength>2048</keylength>
<lifetime>3650</lifetime>
<country>US</country>
<state>-removed-</state>
<city>-removed-</city>
<organization></organization>
<email>-removed-</email>
<uselist>on</uselist></step6>
<step9><certname>server</certname>
<keylength>2048</keylength>
<lifetime>3650</lifetime>
<country>US</country>
<state>-removed-</state>
<city>-removed-</city>
<organization></organization>
<email>-removed-</email>
<uselist>on</uselist></step9>
<step10><interface>wan</interface>
<protocol>UDP</protocol>
<localport>1194</localport>

<tlsauth>on</tlsauth>
<gentlskey>on</gentlskey>
<dhkey>2048</dhkey>
<crypto>AES-256-CBC</crypto>
<digest>SHA1</digest>
<engine>none</engine>
<tunnelnet>192.168.80.0/24</tunnelnet>
<localnet>192.168.58.0/24</localnet>
<concurrentcon>10</concurrentcon>
<interclient>on</interclient>
<dynip>on</dynip>
<addrpool>on</addrpool>
<topology>subnet</topology>
<nbttype>0</nbttype>
<advanced>push "route 192.168.58.0 255.255.255.0"</advanced></step10>
<step11><ovpnrule>on</ovpnrule>
<ovpnallow>on</ovpnallow></step11></ovpnserver>
<ca><refid>59de7040c414a</refid>

<crt>-removed-</crt>
<prv>-removed-</prv>
<serial>2</serial></ca>_

Taking from configFile.ovpn:

_dev tun
persist-tun
persist-key
cipher AES-256-CBC
auth SHA1
tls-client
client
resolv-retry infinite
remote 192.168.57.3 1194 udp
verify-x509-name "server" name
auth-user-pass
remote-cert-tls server
<ca>–---BEGIN CERTIFICATE-----
-removed-
-----END CERTIFICATE-----</ca>
<cert>-----BEGIN CERTIFICATE-----
-removed-
-----END CERTIFICATE-----</cert>
<key>-----BEGIN PRIVATE KEY-----
-removed-
-----END PRIVATE KEY-----</key>
<tls-auth>#

2048 bit OpenVPN static key

-----BEGIN OpenVPN Static key V1-----
-removed-
-----END OpenVPN Static key V1-----</tls-auth>
key-direction 1_

viragomann

XML-File :o
A screenshot from the server setting page is much easier to read. The whole config-file can be found in /var/etc/openvpn.

However, I cannot find any mistake in the config. Though the client doesn't set the routes. Maybe you're missing permissions for changing routes. However, this should be mentioned in the client log.

Alejandro.Carbonara

@viragomann:

Though the client doesn't set the routes. Maybe you're missing permissions for changing routes. However, this should be mentioned in the client log.

What do you mean the client doesn't set the routes? Is there a line missing?

When running the client VPN I am running as a user with passwordless sudo, using the ovpn generated by the openvpn exporter.

viragomann

If you have entered a network in the "Local Networks" field in the server setting, a route for this network should be pushed to the client.
After the client had connected you should see an entry for adding routes in the client log like

/sbin/ip route add 192.168.58.0/24 via 192.168.80.1

That line is missing in your log.

You may set the route manually to see if you have the necessary permissions.

Alejandro.Carbonara

I checked and was able to run that command, but I was getting "RTNETLINK answers: File exists" as a response.
To check, I looked at the IP routing tables:

Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
default 10.0.2.2 0.0.0.0 UG 0 0 0 eth0
10.0.2.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
192.168.57.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
192.168.58.0 192.168.80.1 255.255.255.0 UG 0 0 0 tun0
192.168.80.0 0.0.0.0 255.255.255.0 U 0 0 0 tun0

(Note that due to this being on vagrant, the first gateway is used to provision the machine.)

viragomann

So the necessary route is set fine as the routing table shows.
Has it been already set before you were running that command?

Alejandro.Carbonara

The route was set before running the command.

I've started and stopped the openVPN client several times on the machine, so it may have been set earlier.

johnpoz

"192.168.57.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1"

That route is not going down your tunnel.. So how would you expect to get their through the tunnel.. Seems your client is trying to go out eth1 to get there. With no gateway so it thinks that network is on its eth1 interface.

viragomann

I think, that's just the way how his OS prints networks connected directly to an interface. The line has only an U-flag, no G for a gateway.
It the same as
10.0.2.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0

viragomann

Now, as the route is set as it should be, why do you think, you have a routing problem?

Try to ping pfSense internal address and see if you get a response.

Alejandro.Carbonara

As of now, I can ping the internal ip of pfsense and get a positive response.

Attempting to ping the internal machine gets me no response whatsoever.

viragomann

Check the firewall of the internal machine.

Alejandro.Carbonara

The target machine is an ubuntu server VM.

sudo ufw status
Status: inactive

viragomann

And its default gateway is set to 192.168.58.2 (the pfSense internal address)?

johnpoz

Oh my bad.. I was reading that you wanted to get to 192.168.57… ooops..

If you can ping the internal IP address of pfsense then your tunnel is up and routing that network down the tunnel. Not being able to get to the machine on the network behind pfsense points to problem on the machine. As viragomann pointed out host firewall and or wrong gateway on the host not pointing back to pfsense are 2 very common problems.

Just because ufw is not running does not mean for example iptables is not running.. I run iptables on my ubuntu vms not ufw..

Alejandro.Carbonara

The routing table on the internal machine looks like:

Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
default 10.0.2.2 0.0.0.0 UG 0 0 0 eth0
10.0.2.0 * 255.255.255.0 U 0 0 0 eth0
192.168.58.0 * 255.255.255.0 U 0 0 0 eth1

Also, I checked and iptables is all accept rules.

johnpoz

Well how is that going to work??

Your sending unknown networks out 10.0.2.2

So how exactly would it get to the vpn clients ip on 192.168.80.0/24

Your going to have to create a host route on this machine pointing 192.168.80.0/24 to the 192.168.58 IP of pfsense.

Or you would have to source nat your vpn clients to look like they are coming from the 192.168.58 IP of pfsense so your host there knows how to talk to it.