OpenVPN and route issue - Remote LAN

[sallain]

Hi@pfSense experts :-)

I have a routing (???) issue with pfSense while using an OpenVPN connection.

2.1-RELEASE (amd64)
built on Wed Sep 11 18:17:48 EDT 2013
FreeBSD 8.3-RELEASE-p11

I am running the latest build of pfSense 2.1

pfSense Setup:
LAN1: 192.168.10.0/24
OpenVPN: 10.0.8.0/24

pfSense Gateway: 192.168.10.10/24 (CARP IP Adress)
Other Gateway: 192.168.10.250

LAN2 is available throught a second routeur (not the pfSense box) : 192.168.10.250
LAN2 : 192.168.3.0/24

pfSense static routes :
Network –-- Gateway    ----    Interface
LAN2    ---- 192.168.10.250  --- LAN

If I perform a traceroure from the LAN1 network, OK

# traceroute 192.168.3.33
traceroute to 192.168.3.33 (192.168.3.33), 30 hops max, 60 byte packets
 1  192.168.10.254 (192.168.10.254)  0.314 ms  0.334 ms  0.389 ms
 2  * 192.168.10.250 (192.168.10.250)  1.377 ms  1.382 ms
 3  10.255.240.1 (10.255.240.1)  9.823 ms  9.849 ms  9.834 ms
 4  10.34.158.2 (10.34.158.2)  21.809 ms  21.805 ms  21.787 ms
 5  192.168.3.33 (192.168.3.33)  21.771 ms  23.644 ms  23.610 ms

From my OpenVPN Tunnel, a traceroute to LAN1 is OK :

$traceroute 192.168.10.215
traceroute to 192.168.10.215 (192.168.10.215), 64 hops max, 52 byte packets
 1  10.0.8.1 (10.0.8.1)  70.540 ms  69.186 ms  69.403 ms
 2  192.168.10.215 (192.168.10.215)  69.615 ms  68.646 ms  71.234 ms

From my OpenVPN Tunnel, a traceroute to LAN2 fails :

$ traceroute 192.168.3.33
traceroute to 192.168.3.33 (192.168.3.33), 64 hops max, 52 byte packets
 1  10.0.8.1 (10.0.8.1)  80.744 ms  78.971 ms  69.645 ms
 2  * * *
 3  *

OpenVPN Server is setup with PUSH option :

push "route 192.168.3.0 255.255.255.0";

and also to provide IPV4 Local Networks as follow :

192.168.10.0/24,192.168.3.0/24

A traceroute test from pfSense (Diagnostics / Traceroute) is OK with "Any" but fails if I use the OpenVPN_Server source :

Traceroute output with ANY:
 1  192.168.10.250 (192.168.10.250)  0.518 ms  0.873 ms  0.426 ms
 2  10.255.240.1 (10.255.240.1)  8.974 ms  8.425 ms  8.004 ms
 3  10.34.158.2 (10.34.158.2)  19.960 ms  20.056 ms  19.996 ms
 4  192.168.3.33 (192.168.3.33)  21.988 ms  21.494 ms  19.973 ms

Is there something I missed ???

I've tried also to ask my OpenVPN Client (Viscosity Mac) to send all the traffic throught the VPN Tunnel : same troubles, can't reach 192.168.3.0/24 network.

I will appreciate a lot your opinions and/or solutions.

Thanks a lot for all.

Here is my routing table from the Client while the VPN Tunnel is opened :

Routing tables

Internet:
Destination        Gateway            Flags        Refs      Use   Netif Expire
0/1                10.0.8.5           UGSc           12        0    tun0
default            192.168.1.1        UGSc            8        0     en1
10.0.8.1/32        10.0.8.5           UGSc            0        0    tun0
10.0.8.5           10.0.8.6           UHr            24        0    tun0
127                127.0.0.1          UCS             0        0     lo0
127.0.0.1          127.0.0.1          UH              5   341905     lo0
128.0/1            10.0.8.5           UGSc            6        0    tun0
169.254            link#5             UCS             0        0     en1
192.168.1          link#5             UCS             2        0     en1
192.168.1.1        24:95:4:a9:6c:f0   UHLWIir         3       45     en1   1172
192.168.1.20       127.0.0.1          UHS             0        2     lo0
192.168.1.255      ff:ff:ff:ff:ff:ff  UHLWbI          0        6     en1
192.168.3          10.0.8.5           UGSc            0        4    tun0
192.168.10         10.0.8.5           UGSc            1        0    tun0
x.y.z.211/32  192.168.1.1        UGSc            1        0     en1

[phil.davis]

There is a bit of a path to 192.168.3.0/24 via the other router on your LAN (the static route) and a couple of hops. I would guess that the various intervening routers know a route back to your LAN (192.168.10.0/24) and thus can route echo replies back to the pfSense LAN IP. But on (or more) of them does not know about 10.0.8.0/24 - and so is ending the echo reply somewhere else, or dumping it.
Maybe start from a device in 192.168.3.0/24 and traceroute to 10.0.8.1 and see where it goes (or does not go).

[sallain]

Do you think if a CARP/VIP setup for both LAN and WAN could have a side effect ?

I'm not sure if the CARP + VIP used has an effect or nor while using the VPN, but anyway, my VPN Tunnel is working fine, I can reach without troubles any machines inside the LAN1…

As a reminder, a traceroute from a Linux Sever inside the LAN1 (192.168.10.0/24) is OK :

# traceroute 192.168.3.33
traceroute to 192.168.3.33 (192.168.3.33), 30 hops max, 60 byte packets
 1  192.168.10.254 (192.168.10.254)  0.285 ms  0.322 ms  0.373 ms
 2  192.168.10.250 (192.168.10.250)  0.961 ms  1.017 ms  0.997 ms
 3  10.255.240.1 (10.255.240.1)  11.347 ms  11.336 ms  11.315 ms
 4  10.34.158.2 (10.34.158.2)  23.280 ms  23.293 ms  23.272 ms
 5  192.168.3.33 (192.168.3.33)  23.252 ms  23.209 ms  23.185 ms

A ping check is also OK :

# ping 192.168.3.33
PING 192.168.3.33 (192.168.3.33) 56(84) bytes of data.
From 192.168.10.254: icmp_seq=1 Redirect Host(New nexthop: 192.168.10.250)
64 bytes from 192.168.3.33: icmp_seq=1 ttl=247 time=22.4 ms
From 192.168.10.254: icmp_seq=2 Redirect Host(New nexthop: 192.168.10.250)
64 bytes from 192.168.3.33: icmp_seq=2 ttl=247 time=20.7 ms
From 192.168.10.254: icmp_seq=3 Redirect Host(New nexthop: 192.168.10.250)
64 bytes from 192.168.3.33: icmp_seq=3 ttl=247 time=21.7 ms
^C

The routing table of this Linux server is OK :

# netstat -rn
Table de routage IP du noyau
Destination     Passerelle      Genmask         Indic   MSS Fenêtre irtt Iface
192.168.10.0    0.0.0.0         255.255.255.0   U         0 0          0 eth0
169.254.0.0     0.0.0.0         255.255.0.0     U         0 0          0 eth0
0.0.0.0         192.168.10.10   0.0.0.0         UG        0 0          0 eth0

So, this why I suppose routing is OK, despite the fact from VPN it doesn't…

[phil.davis]

You still do not know if 192.168.3.33 can correctly route back to 10.0.8.0/24.
From 192.168.3.33 do a "traceroute 10.0.8.1" and see how that goes. The path it takes and where it stops will help you find the device/s that do not know how to route to 10.0.8.0/24.

[sallain]

@phil.davis:

You still do not know if 192.168.3.33 can correctly route back to 10.0.8.0/24.
From 192.168.3.33 do a "traceroute 10.0.8.1" and see how that goes. The path it takes and where it stops will help you find the device/s that do not know how to route to 10.0.8.0/24.

OK, will be next week at the location and will be able to perform the test.

Thanks a lot for help, stay in touch for replies next week ;-)