OVPN reports up, but cannot route between site-to-site
-
Hey folks,
I've been struggling with a site-to-site VPN ever since I had an ISP change and hardware failure about 4 months ago at Site1.Long story short, I have put attempts to make IPsec work on hold and am trying again with OpenVPN. The good news is that the logs on both sides report that the tunnel is up. The bad news is that I cannot route between the two sides.
I have created rules for each LAN subnet to allow traffic to the remote LAN network…should be redundant since I allow LAN traffic to go anywhere.
One note: Site2 is behind another gateway (cisco that I have no control over) which is wide open with 1:1 NAT (every port forwarded, nothing blocked). I have tried reversing the setup and making site1 the server and site2 the client ... no difference in the outcome. I'm fairly sure it should not matter since roadwarrior clients can connect on either side…but thats another thread.
Site1 (client)
Lan: 10.1.1.0/24
tun0: 10.1.60.2 --> 10.1.60.1Site2 (server)
Lan: 10.1.5.0/24
Pool: 10.1.60.0/24
tun0: 10.1.60.1 --> 10.1.60.2Site1 log:
May 14 19:55:31 openvpn[75285]: UDPv4 link remote: 204.152.xx.yy:8080 May 14 19:55:31 openvpn[75285]: UDPv4 link local (bound): [undef]:1194 May 14 19:55:31 openvpn[75285]: Preserving previous TUN/TAP instance: tun0 May 14 19:55:31 openvpn[75285]: LZO compression initialized May 14 19:55:31 openvpn[75285]: Re-using pre-shared static key May 14 19:55:29 openvpn[75285]: SIGUSR1[soft,ping-restart] received, process restarting May 14 19:55:29 openvpn[75285]: Inactivity timeout (--ping-restart), restarting May 14 19:54:29 openvpn[75285]: UDPv4 link remote: 204.152.xx.yy:8080 May 14 19:54:29 openvpn[75285]: UDPv4 link local (bound): [undef]:1194 May 14 19:54:29 openvpn[75285]: Preserving previous TUN/TAP instance: tun0 May 14 19:54:29 openvpn[75285]: LZO compression initialized May 14 19:54:29 openvpn[75285]: Re-using pre-shared static key May 14 19:54:27 openvpn[75285]: SIGUSR1[soft,ping-restart] received, process restarting May 14 19:54:27 openvpn[75285]: Inactivity timeout (--ping-restart), restartingSite2 log:
May 14 15:30:54 openvpn[23960]: UDPv4 link remote: [undef] May 14 15:30:54 openvpn[23960]: UDPv4 link local (bound): [undef]:8080 May 14 15:30:54 openvpn[23940]: ERROR: FreeBSD route add command failed: shell command exited with error status: 1 May 14 15:30:52 openvpn[23940]: /etc/rc.filter_configure tun0 1500 1561 10.1.60.1 10.1.60.2 init May 14 15:30:52 openvpn[23940]: /sbin/ifconfig tun0 10.1.60.1 10.1.60.2 mtu 1500 netmask 255.255.255.255 up May 14 15:30:52 openvpn[23940]: TUN/TAP device /dev/tun0 opened May 14 15:30:52 openvpn[23940]: gw 172.15.1.1 May 14 15:30:52 openvpn[23940]: LZO compression initialized May 14 15:30:52 openvpn[23940]: WARNING: file '/var/etc/openvpn_server0.secret' is group or others accessible May 14 15:30:52 openvpn[23940]: OpenVPN 2.0.6 i386-portbld-freebsd6.2 [SSL] [LZO] built on Sep 13 2007 May 14 15:30:51 openvpn[23256]: SIGTERM[hard,init_instance] received, process exiting May 14 15:30:50 openvpn[23256]: /etc/rc.filter_configure tun0 1500 1563 10.1.60.1 10.1.60.2 init May 14 15:28:18 openvpn[23256]: Listening for incoming TCP connection on [undef]:8080 May 14 15:28:18 openvpn[23243]: ERROR: FreeBSD route add command failed: shell command exited with error status: 1 May 14 15:28:16 openvpn[23243]: /etc/rc.filter_configure tun0 1500 1563 10.1.60.1 10.1.60.2 init May 14 15:28:16 openvpn[23243]: /sbin/ifconfig tun0 10.1.60.1 10.1.60.2 mtu 1500 netmask 255.255.255.255 up May 14 15:28:16 openvpn[23243]: TUN/TAP device /dev/tun0 opened May 14 15:28:16 openvpn[23243]: gw 172.15.1.1 -
How did you set the link up?
A shared key?
Did you fill in the "Remote network" field?What worries me is that in your log stands:
May 14 15:30:54 openvpn[23940]: ERROR: FreeBSD route add command failed: shell command exited with error status: 1
How did you add the route to the remote network?
-
How did you set the link up?
GruensFroeschli - the ironic part is that when I made the original post I was thinking "if GruensFroeschli reads this, how can I keep it short and succinct but also provide enough detail…"
Thank you for your help and interest!The short answer is: I am not pushing the routes to the local LANs, I thought that was why we fill that in during the OVPN cofig... I DID create a firewall rule for each local LAN that permits traffic from that subnet to the remote network
EG: site1 (LAN rules)... allow 10.1.1.0/24 --all traffic-->10.1.5.0/24 (and vice-versa) ...let me know if screen shots would helprather than describing, the best way I know is to use screenshots...Im hoping this is one of those "2nd set of eyes" answers...
Site1 - client (DHCP on wan, has never changed): client
Site2 - server (static WAN, behind NAT, no blocked ports): server
-
The config looks valid.
In your firewall rule.
Did you make sure that you allow traffic to the transfer subnet too?If you use the ping tool on pfSense itself.
Can you ping the opposite side of the tunnel?Could you post your routingtable here?
I still dont know what to make of that error.
Google doesnt yield any results besides a few other people had this problem too ^^" -
i had not added rules to allow the lan subnet to access the OpenVPN tunnel subnet…doing that allows me to ping the local side of each tunnel...but traffic still does not cross.
routing tables...
site1
Internet: Destination Gateway Flags Refs Use Netif Expire default 96.228.xx.yy UGS 0 48966650 dc0 10.1.1/24 link#6 UC 0 0 fxp1 router 00:30:48:41:01:35 UHLW 1 4 lo0 10.1.1.9 00:16:cb:c3:3b:45 UHLW 1 1 fxp1 368 10.1.1.15 00:0d:93:4e:68:5c UHLW 1 5409 fxp1 808 10.1.1.17 00:0d:93:4e:68:5c UHLW 1 164134 fxp1 549 10.1.1.27 00:16:cb:a8:1d:7f UHLW 1 3553 fxp1 1054 10.1.1.40 00:06:5b:99:34:ce UHLW 1 254224 fxp1 1150 10.1.1.65 00:1e:8c:91:8a:27 UHLW 1 29584724 fxp1 933 10.1.1.100 00:14:51:65:80:9e UHLW 1 2084222 fxp1 607 10.1.1.110 00:19:e3:03:93:a6 UHLW 1 293328 fxp1 202 10.1.1.121 00:14:51:1f:41:5a UHLW 1 23629 fxp1 687 10.1.1.122 00:0e:08:da:31:af UHLW 1 201 fxp1 438 10.1.1.136 00:02:b9:af:c9:80 UHLW 1 393 fxp1 1197 10.1.1.159 00:16:cb:a5:fe:24 UHLW 1 975374 fxp1 1049 10.1.1.167 00:16:cb:a5:e0:ea UHLW 1 1258 fxp1 334 10.1.2/24 link#2 UC 0 0 dc1 10.1.5/24 10.1.60.1 UGS 0 13 tun0 10.1.20/24 link#3 UC 0 0 dc2 10.1.20.109 00:18:01:30:c9:61 UHLW 1 1 dc2 309 10.1.60.1 10.1.60.2 UH 1 2 tun0 96.228.xx/24 link#1 UC 0 0 dc0 96.228.xx.yy 00:90:1a:a0:15:5b UHLW 2 19900 dc0 21 pool-96-228-xx-yy localhost UGHS 0 0 lo0 localhost localhost UH 1 0 lo0site2
Routing tables Internet: Destination Gateway Flags Refs Use Netif Expire default 172.15.1.1 UGS 0 6926420 dc1 10.1.1/24 router UGS 0 157 xl0 10.1.5/24 link#3 UC 0 0 xl0 router 00:06:5b:68:89:9b UHLW 2 33 lo0 10.1.5.5 00:14:51:6f:8b:16 UHLW 1 12 xl0 1184 den 00:17:f2:da:e0:b4 UHLW 1 742352 xl0 170 10.1.5.236 00:17:f2:fb:b0:15 UHLW 1 1 xl0 489 10.1.5.245 00:19:e3:da:3c:21 UHLW 1 3399 xl0 147 10.1.5.255 ff:ff:ff:ff:ff:ff UHLWb 1 2 xl0 10.1.6/24 link#1 UC 0 0 dc0 10.1.60.2 10.1.60.1 UH 0 9 tun0 localhost localhost UH 0 0 lo0 172.15 link#2 UC 0 0 dc1 172.15.1.1 00:04:dd:2d:49:20 UHLW 2 211285 dc1 192 172.15.1.2 00:14:bf:54:8a:7a UHLW 1 46 lo0 -
Did you add manually a static route for the 10.1.1.x/24 subnet on site 2?
Your routing table has an entry for this subnet but it points to the wrong destination:
Destination Gateway Flags Refs Use Netif Expire
10.1.1/24 router UGS 0 157 xl0What kind of gateway is "router"?
This should be 10.1.60.2 -
both sides are PFsense boxes …
I'm now curious what rule is causing that route to be present...
But wouldn't that just result in one side routing? -
As you have it right now you have one side routing.
Site 1 can send traffic to site 2.
But site 2 is not able to send a reply because it does no know the route back to the subnet on site 1.Maybe you could try to delete the openVPN config on the problematic side and take a look at the routing table again.
If It looks clean resetup the OpenVPN part on this box.I really dont know what could cause this.
-
Maybe you have a static route for 10.0.1.0/24 -network in the config of site2 (system->static routes) ?
-
Firstly, thank you all for taking the time to try and help - I really owe this community a lot!
I have been fighting VPN issues all night…all related to my hardware failure a few weeks ago...my road warriors (namely me, sitting in a hotel right now) cannot connect to my L2TP server which is behind the PFsense box at Site1....not to mention the problem in this thread that we are working on.
I have been pouring over the configs, deleting anything superfluous, recreating everything else (all though SSH tunnels) ... and for the life of me I cannot find what is creating the offending route.
So it has occured to me to ask, what is wrong with that route? Shouldn't I be trying to route traffice from site2 (10.1.5.0/24) to site1 (10.1.1.0/24) via the default gateway (router)?
Might it matter that both PFsense gateways hate the hostname of "router"
site1 - router.nsnet.com
site2 router.lynchburg.nsnet.comI have deleted the OVPN configs and rebuilt the with the exact same result..in fact when I delete the config, the routing table still shows the same thing...
I have been running netstat -r on the console of the PFsense boxes .... here is the result from the web UI of site2 (if it makes a difference)...note it does not say "router" but rather the LAN IP of the Site2 LAN (10.1.5.1)Destination Gateway Flags Refs Use Mtu Netif Expire
default 172.15.1.1 UGS 0 1672 1500 dc1
10.1.1/24 10.1.5.1 UGS 0 20934 1500 xl0 -
The problem with the route is that when the openvpn tunnel is up, traffic destined to the remote network should be going to tunX interface, not the normal gateway.
This is what I have on my pfsense box that is a client on a site-to-site tunnel, my local LAN is 192.168.13.0/24, remote LAN is 192.168.42.0/24, transfer net is
10.13.42.0/24.Destination Gateway Flags Refs Use Mtu Netif Expire
192.168.42 10.13.42.1 UGS 0 32133 1500 tun1
(tun1 because tun0 is used by another site-to-site tunnel)At the other end (the server):
192.168.13 10.13.42.2 UGS 0 1000282 1500 tun1
(tun1 in this case because the other end also has a server for roadwarriors at tun0)
