OVPN reports up, but cannot route between site-to-site

GruensFroeschli · May 14, 2008, 8:24 PM

How did you set the link up?
A shared key?
Did you fill in the "Remote network" field?

What worries me is that in your log stands:

May 14 15:30:54 openvpn[23940]: ERROR: FreeBSD route add command failed: shell command exited with error status: 1

How did you add the route to the remote network?

SpaceBass · May 15, 2008, 3:16 AM

How did you set the link up?

GruensFroeschli - the ironic part is that when I made the original post I was thinking "if GruensFroeschli reads this, how can I keep it short and succinct but also provide enough detail…"
Thank you for your help and interest!

The short answer is: I am not pushing the routes to the local LANs, I thought that was why we fill that in during the OVPN cofig... I DID create a firewall rule for each local LAN that permits traffic from that subnet to the remote network
EG: site1 (LAN rules)... allow 10.1.1.0/24 --all traffic-->10.1.5.0/24 (and vice-versa) ...let me know if screen shots would help

rather than describing, the best way I know is to use screenshots...Im hoping this is one of those "2nd set of eyes" answers...
Site1 - client (DHCP on wan, has never changed): client

Site2 - server (static WAN, behind NAT, no blocked ports): server

GruensFroeschli · May 15, 2008, 11:35 AM

The config looks valid.
In your firewall rule.
Did you make sure that you allow traffic to the transfer subnet too?

If you use the ping tool on pfSense itself.
Can you ping the opposite side of the tunnel?

Could you post your routingtable here?

I still dont know what to make of that error.
Google doesnt yield any results besides a few other people had this problem too ^^"

SpaceBass · May 15, 2008, 12:20 PM

i had not added rules to allow the lan subnet to access the OpenVPN tunnel subnet…doing that allows me to ping the local side of each tunnel...but traffic still does not cross.

routing tables...

site1

Internet:
Destination        Gateway            Flags    Refs      Use  Netif Expire
default            96.228.xx.yy        UGS         0 48966650    dc0
10.1.1/24          link#6             UC          0        0   fxp1
router             00:30:48:41:01:35  UHLW        1        4    lo0
10.1.1.9           00:16:cb:c3:3b:45  UHLW        1        1   fxp1    368
10.1.1.15          00:0d:93:4e:68:5c  UHLW        1     5409   fxp1    808
10.1.1.17          00:0d:93:4e:68:5c  UHLW        1   164134   fxp1    549
10.1.1.27          00:16:cb:a8:1d:7f  UHLW        1     3553   fxp1   1054
10.1.1.40          00:06:5b:99:34:ce  UHLW        1   254224   fxp1   1150
10.1.1.65          00:1e:8c:91:8a:27  UHLW        1 29584724   fxp1    933
10.1.1.100         00:14:51:65:80:9e  UHLW        1  2084222   fxp1    607
10.1.1.110         00:19:e3:03:93:a6  UHLW        1   293328   fxp1    202
10.1.1.121         00:14:51:1f:41:5a  UHLW        1    23629   fxp1    687
10.1.1.122         00:0e:08:da:31:af  UHLW        1      201   fxp1    438
10.1.1.136         00:02:b9:af:c9:80  UHLW        1      393   fxp1   1197
10.1.1.159         00:16:cb:a5:fe:24  UHLW        1   975374   fxp1   1049
10.1.1.167         00:16:cb:a5:e0:ea  UHLW        1     1258   fxp1    334
10.1.2/24          link#2             UC          0        0    dc1
10.1.5/24          10.1.60.1          UGS         0       13   tun0
10.1.20/24         link#3             UC          0        0    dc2
10.1.20.109        00:18:01:30:c9:61  UHLW        1        1    dc2    309
10.1.60.1          10.1.60.2          UH          1        2   tun0
96.228.xx/24       link#1             UC          0        0    dc0
96.228.xx.yy        00:90:1a:a0:15:5b  UHLW        2    19900    dc0     21
pool-96-228-xx-yy localhost          UGHS        0        0    lo0
localhost          localhost          UH          1        0    lo0

site2

Routing tables

Internet:
Destination        Gateway            Flags    Refs      Use  Netif Expire
default            172.15.1.1         UGS         0  6926420    dc1
10.1.1/24          router             UGS         0      157    xl0
10.1.5/24          link#3             UC          0        0    xl0
router             00:06:5b:68:89:9b  UHLW        2       33    lo0
10.1.5.5           00:14:51:6f:8b:16  UHLW        1       12    xl0   1184
den                00:17:f2:da:e0:b4  UHLW        1   742352    xl0    170
10.1.5.236         00:17:f2:fb:b0:15  UHLW        1        1    xl0    489
10.1.5.245         00:19:e3:da:3c:21  UHLW        1     3399    xl0    147
10.1.5.255         ff:ff:ff:ff:ff:ff  UHLWb       1        2    xl0
10.1.6/24          link#1             UC          0        0    dc0
10.1.60.2          10.1.60.1          UH          0        9   tun0
localhost          localhost          UH          0        0    lo0
172.15             link#2             UC          0        0    dc1
172.15.1.1         00:04:dd:2d:49:20  UHLW        2   211285    dc1    192
172.15.1.2         00:14:bf:54:8a:7a  UHLW        1       46    lo0

GruensFroeschli · May 15, 2008, 2:05 PM

Did you add manually a static route for the 10.1.1.x/24 subnet on site 2?

Your routing table has an entry for this subnet but it points to the wrong destination:

Destination Gateway Flags Refs Use Netif Expire
10.1.1/24 router UGS 0 157 xl0

What kind of gateway is "router"?
This should be 10.1.60.2

SpaceBass · May 15, 2008, 2:23 PM

both sides are PFsense boxes …
I'm now curious what rule is causing that route to be present...
But wouldn't that just result in one side routing?

GruensFroeschli · May 15, 2008, 2:35 PM

As you have it right now you have one side routing.

Site 1 can send traffic to site 2.
But site 2 is not able to send a reply because it does no know the route back to the subnet on site 1.

Maybe you could try to delete the openVPN config on the problematic side and take a look at the routing table again.
If It looks clean resetup the OpenVPN part on this box.

I really dont know what could cause this.

kpa · May 15, 2008, 8:56 PM

Maybe you have a static route for 10.0.1.0/24 -network in the config of site2 (system->static routes) ?

SpaceBass · May 16, 2008, 1:50 AM

Firstly, thank you all for taking the time to try and help - I really owe this community a lot!

I have been fighting VPN issues all night…all related to my hardware failure a few weeks ago...my road warriors (namely me, sitting in a hotel right now) cannot connect to my L2TP server which is behind the PFsense box at Site1....not to mention the problem in this thread that we are working on.

I have been pouring over the configs, deleting anything superfluous, recreating everything else (all though SSH tunnels) ... and for the life of me I cannot find what is creating the offending route.

So it has occured to me to ask, what is wrong with that route? Shouldn't I be trying to route traffice from site2 (10.1.5.0/24) to site1 (10.1.1.0/24) via the default gateway (router)?

Might it matter that both PFsense gateways hate the hostname of "router"
site1 - router.nsnet.com
site2 router.lynchburg.nsnet.com

I have deleted the OVPN configs and rebuilt the with the exact same result..in fact when I delete the config, the routing table still shows the same thing...
I have been running netstat -r on the console of the PFsense boxes .... here is the result from the web UI of site2 (if it makes a difference)...note it does not say "router" but rather the LAN IP of the Site2 LAN (10.1.5.1)

Destination Gateway Flags Refs Use Mtu Netif Expire
default 172.15.1.1 UGS 0 1672 1500 dc1
10.1.1/24 10.1.5.1 UGS 0 20934 1500 xl0

kpa · May 16, 2008, 2:59 AM

The problem with the route is that when the openvpn tunnel is up, traffic destined to the remote network should be going to tunX interface, not the normal gateway.

This is what I have on my pfsense box that is a client on a site-to-site tunnel, my local LAN is 192.168.13.0/24, remote LAN is 192.168.42.0/24, transfer net is
10.13.42.0/24.

Destination Gateway Flags Refs Use Mtu Netif Expire
192.168.42 10.13.42.1 UGS 0 32133 1500 tun1
(tun1 because tun0 is used by another site-to-site tunnel)

At the other end (the server):
192.168.13 10.13.42.2 UGS 0 1000282 1500 tun1
(tun1 in this case because the other end also has a server for roadwarriors at tun0)