OpenVPN fails to create route after upgrade to rc1
-
Hi folks,
after an upgrade from Beta-5 (early February I believe), to RC-1, I have a problem with the openvpn client connections.
The connection itself works just fine, but when pfsense tries to add a route to the vpn network, it logs this error:openvpn[766]: ERROR: FreeBSD route add command failed: external program exited with error status: 1
Naturally, without the route I cannot access the vpn network on the other side, so this is really a problem atm.
Does anyone have any suggestions on how to fix or debug this further?Thanks,
Marcus -
What is the command failing?
-
The actual command is not shown in the logs, just the error message posted above. Is there a way to up the verbosity of the openvpn client?
-
I was about to post the same thing. I just upgraded my main office from 1.2.3 to 2.0rc1, everything worked previously.
I'm using pre-shared key, site to site openvpn, main office is the server, remote office is client. My remote site (also 2.0 rc1) is fine, it can access everything in the main office without issue. However from the main office going to the remote site, I've got nothing. This error is in the openvpn log on the main office side:
openvpn[43883]: ERROR: FreeBSD route add command failed: external program exited with error status: 1
Now if I ssh into pfsense, I can ping the remote office fine, so somewhere the routing is correct. From any of my vlans though it fails, it appears it's routing straight out to the internet.
on the pfsense box, netstat -nr has a routing entry:
192.168.10.0/24 10.91.129.2 UGS 0 2424 ovpns2
Background info:
Local network is 10.90.0.0/19 (with a bunch of vlans)
remote site is 192.168.10.0/24In the openvpn config, I have the info above, tunnel network is 10.91.129.0/24 on both sides.
On the server side, I have:route 192.168.10.0 255.255.255.0;push "route 10.90.0.0 255.255.224.0";
in the advanced options.
-
The usual reason for FreeBSD to fail adding a route is that you already have a route to that subnet.
It's not 100% clear what exactly is where in your setup. Can you specify it a bit more in depth? At least distinguish exactly what subnets are at the main office and which are on the client side. Also, show the openvpn config and route table on both sides.
-
Ok, a little more testing and I was incorrect earlier, I can access the remote site from the other vlans, except 10.90.0.0/24 (vlan 100 - SAA). It figures this is the one I need though.
Let me know if you need more screenshots or info.Main office: 10.90.0.0/19 - OpenVPN server - Dual WAN (WAN_T1 & WAN_DSL, openvpn tied to WAN_TA)
subnets can be seen below in the routing table, but generally they are
10.90.0.0/24 VLAN 100
10.90.1.0/24 VLAN 101
etcRemote office: 192.168.10.0/24 - OpenVPN client
Routing tables -Main Office
Internet: Destination Gateway Flags Refs Use Netif Expire default 67.130.244.161 UGS 0 931024 em1 4.2.2.2 216.161.116.210 UGHS 0 784 pppoe0 4.2.2.4 67.130.244.161 UGHS 0 784 em1 10.90.0.0/24 link#9 U 0 1588250 em0_vl 10.90.0.1 link#9 UHS 0 0 lo0 10.90.1.0/24 link#10 U 0 30961 em0_vl 10.90.1.1 link#10 UHS 0 0 lo0 10.90.2.0/24 link#11 U 0 25691 em0_vl 10.90.2.1 link#11 UHS 0 0 lo0 10.90.3.0/24 link#12 U 0 5922 em0_vl 10.90.3.1 link#12 UHS 0 0 lo0 10.90.4.0/24 link#13 U 0 0 em0_vl 10.90.4.1 link#13 UHS 0 0 lo0 10.90.5.0/24 link#14 U 0 0 em0_vl 10.90.5.1 link#14 UHS 0 0 lo0 10.90.6.0/24 link#15 U 0 0 em0_vl 10.90.6.1 link#15 UHS 0 0 lo0 10.90.9.0/24 link#16 U 0 251035 em0_vl 10.90.9.1 link#16 UHS 0 0 lo0 10.90.10.0/24 link#17 U 0 35227 em0_vl 10.90.10.1 link#17 UHS 0 0 lo0 10.90.11.0/24 link#18 U 0 0 em0_vl 10.90.11.1 link#18 UHS 0 0 lo0 10.90.15.0/24 link#24 U 0 0 em0_vl 10.90.15.1 link#24 UHS 0 0 lo0 10.90.28.8/29 link#19 U 0 36129 em0_vl 10.90.28.9 link#19 UHS 0 0 lo0 10.90.28.16/29 link#20 U 0 31750 em0_vl 10.90.28.17 link#20 UHS 0 0 lo0 10.90.28.24/29 link#21 U 0 74373 em0_vl 10.90.28.25 link#21 UHS 0 0 lo0 10.90.28.32/29 link#22 U 0 19915 em0_vl 10.90.28.33 link#22 UHS 0 0 lo0 10.90.30.0/24 link#23 U 0 0 em0_vl 10.90.30.1 link#23 UHS 0 0 lo0 10.90.128.0/24 10.90.128.2 UGS 0 157 ovpns1 10.90.128.1 link#26 UHS 0 0 lo0 10.90.128.2 link#26 UH 0 0 ovpns1 10.91.129.1 link#27 UHS 0 0 lo0 10.91.129.2 link#27 UH 0 0 ovpns2 63.227.79.178 link#25 UHS 0 0 lo0 67.130.244.160/28 link#2 U 0 234 em1 67.130.244.165 link#2 UHS 0 0 lo0 127.0.0.1 link#7 UH 0 194 lo0 192.168.10.0/24 10.91.129.2 UGS 0 202444 ovpns2 205.171.2.65 216.161.116.210 UGHS 0 4468 pppoe0 205.171.3.65 216.161.116.210 UGHS 0 4498 pppoe0 208.67.220.220 67.130.244.161 UGHS 0 0 em1 208.67.222.222 216.161.116.210 UGHS 0 0 pppoe0 216.161.116.210 link#25 UH 0 0 pppoe0
Routing tables- Remote Office
Internet: Destination Gateway Flags Refs Use Netif Expire default 216.161.116.209 UGS 0 42180042 pppoe0 10.90.0.0/19 10.91.129.1 UGS 0 222473 ovpnc2 10.91.129.1 link#10 UH 0 788 ovpnc2 10.91.129.2 link#10 UHS 0 0 lo0 63.227.69.58 link#8 UHS 0 555 lo0 127.0.0.1 link#4 UH 0 97 lo0 192.168.0.0/24 192.168.0.2 US 0 0 em1 192.168.0.2 link#2 UHS 0 1808284 lo0 192.168.10.0/24 link#1 U 0 37595518 em0 192.168.10.1 link#1 UHS 0 0 lo0 192.168.190.0/24 192.168.190.2 UGS 0 23784 ovpns1 192.168.190.1 link#9 UHS 0 0 lo0 192.168.190.2 link#9 UH 0 0 ovpns1 216.161.116.209 link#8 UH 0 942669 pppoe0
-
Do you see anything in the firewall logs on either side?
Have you done packet captures on both routers to see if the traffic you expect to see is actually going into the VPN tunnel interface?
-
And doing some further testing, I'm not sure this is an openvpn thing. I also have a few ipsec tunnels to some other lesser used sites. The ipsec tunnels are created using the 10.90.0.0/19 subnet also, and again all vlans except for 100 can access the remote sites.
I'd guess there is a more basic network config problem, but I'm just not seeing it. All the main subnets/vlans are created with the same /24 scheme, with similar rules.
Nothing in the firewall logs on either side.
-
Ok, I've found the cause. My last rule in that vlans sends all non-local traffic (using an inverse) to my the 'failover' gateway group, vlan 100 is the only one that uses 'failover' as it's gateway, all the other vlans use the default. Is the gateway group bypassing the vpn tunnels when routing traffic?
Any thoughts on the best solution? (I can create a rule just before the inverse block allow the vpn traffic, didn't know if that is the cleanest solution).
-
That is the cleanest solution, make a rule above that that has a gateway of 'default'.