Packets disappear between vpn client and vpn server
-
I have a hub and spoke site-to-site vpn setup, however – only in one direction and only on one spoke -- nodes on the client lan cannot reach nodes on the server lan, although they can reach the server itself. The opposite direction works fine (i.e. nodes on server lan can reach nodes on this client-lan.
The really odd thing is that the problem is not between the vpn server and the node on its lan. I know this because when I attempt to reach the server from the client-node, I can see the packets reach the server in the tcp dump, however if I try to get beyond the server to one of the nodes on the server-lan, the packets appear to vanish on the vpn client, never making it to the server.
Here is a diagram I made of the situation:
A ====> B ====> C ====> D ssh from node on server lan to node on client lan works, as does any portion of that A B <==== C <==== D ssh from node on client lan to vpn server works A <==== B <==== C D ssh from vpn client to node on server lan works A XXXXX B XXXXX C <==== D ssh from node on client lan to node on server lan fails!!! packets disappear between ovpnc and ovpns interfaces Server VPN VPN Client LAN Server Client LANWhile it would seem this has to be a firewall / NAT problem, I've been over my rules time and again, as well as logs (no blocks or rejections), and I've pfctl -d on both vpn server and client.
Note that another VPN client lan and vpn client node exists, E and F, which work fine in both directions.
Here is the routing table on the vpn server:
: netstat -r Routing tables Internet: Destination Gateway Flags Netif Expire default 65.145.227.209 UGS vtnet0 10.0.0.0/24 link#7 U bridge0 10.0.0.1 link#7 UHS lo0 10.5.0.0/24 10.0.0.100 UGS bridge0 10.10.0.0/24 10.0.0.101 UGS bridge0 10.20.0.0/24 10.0.0.102 UGS bridge0 63.141.227.208/29 link#1 U vtnet0 sangat.fortulite.n link#1 UHS lo0 localhost link#4 UH lo0… and on the vpn client ...
: netstat -r Routing tables Internet: Destination Gateway Flags Netif Expire default lo0-100.NWRKNJ-VFT UGS em1 10.0.0.0/24 link#8 U ovpnc1 10.0.0.102 link#8 UHS lo0 10.5.0.0/24 10.0.0.100 UGS ovpnc1 10.10.0.0/24 10.0.0.101 UGS ovpnc1 10.20.0.0/24 link#1 U em0 pfSense link#1 UHS lo0 10.250.0.0/24 10.255.255.1 UGS ovpnc2 10.250.0.0/16 10.255.255.1 UGS ovpnc2 10.255.255.0/24 10.255.255.1 UGS ovpnc2 10.255.255.1 link#9 UH ovpnc2 10.255.255.4 link#9 UHS lo0 nsphil01.verizon.n d6:1b:af:2b:e3:98 UHS em1 nsnwrk01.verizon.n d6:1b:af:2b:e3:98 UHS em1 93.232.81.0/24 link#2 U em1 pool-96-242-81-51\. link#2 UHS lo0 localhost link#3 UH lo0for reference, the is a vpn-client that does not have the problem:
netstat -r Routing tables Internet: Destination Gateway Flags Netif Expire default lo0-100.NWRKNJ-VFT UGS em1 10.0.0.0/24 link#8 U ovpnc1 10.0.0.100 link#8 UHS lo0 10.5.0.0/24 link#1 U em0 ads-120elmst-pfsen link#1 UHS lo0 10.10.0.0/24 10.0.0.101 UGS ovpnc1 10.20.0.0/24 10.0.0.102 UGS ovpnc1 10.30.0.0/24 10.0.0.103 UGS ovpnc1 10.250.0.0/24 10.255.255.1 UGS ovpnc2 10.250.0.0/16 10.255.255.1 UGS ovpnc2 10.255.255.0/24 10.255.255.1 UGS ovpnc2 10.255.255.1 link#9 UH ovpnc2 10.255.255.2 link#9 UHS lo0 nsphil01.verizon.n 48:5d:36:85:1e:48 UHS em1 nsnwrk01.verizon.n 48:5d:36:85:1e:48 UHS em1 93.242.143.0/24 link#2 U em1 pool-96-242-148-23 link#2 UHS lo0 localhost link#3 UH lo0The node on the client lan is set up to use the vpn-client as router/gateway:
# ip route list default via 10.20.0.1 dev vmbr0 10.20.0.0/24 dev vmbr0 proto kernel scope link src 10.20.0.251However, the node on the server lan has it's own wan, and also the routes must be statically set up (no way to push from client to server, afaik):
# ip route list default via 63.141.227.209 dev vmbr0 onlink 10.0.0.0/24 dev vmbr1 proto kernel scope link src 10.0.0.250 10.5.0.0/24 via 10.0.0.1 dev vmbr1 10.10.0.0/24 via 10.0.0.1 dev vmbr1 10.20.0.0/24 via 10.0.0.1 dev vmbr1 66.141.226.208/29 dev vmbr0 proto kernel scope link src 63.141.227.210I'm starting to lean toward a bug somewhere, rather than a mistake in my implementation. I'm at a loss, and any help is appreciated.