OpenVPN peer to peer - connects but won't pass traffic
-
Site A - OpenVPN server
pfSense 2.4.1
Open VPN server running on port 1195
Has 3 WAN interfaces. I'm trying to get the VPN running on WAN1.Site B - OpenVPN client
pfSense 2.3.4I've tripled checked the OpenVPN settings between client and server and they match. The status page indicates that the VPN is connected, but I am unable from a computer on the B network to ping anything on the A network. A computer on the B network can ping both sides of the private VPN network (10.0.27.1 and 10.0.27.2). I can ping from the pfSense B to various addresses on the A network.
B has an allow all firewall rule on the Open VPN tab.
A has an allow all firewall rule on the Open VPN tab.
A has an allow all firewall rule for port 1195 on the WAN1 tab. I've tried the destination as both "any" and "this firewall" with the same results.
A has an outbound NAT rule -
Interface:WAN1
Source: B's network/24
Source Port: *
Destination: *
Destination Port: *
NAT Address: WAN1 address
NAT Port: *There are other Outbound NAT rules for B's network/24 but they are for specific addresses (both pfSense boxes). These allow me to connect to the pfSense boxes when using IPSec VPN.
Any ideas on what I'm doing wrong? I suspect the NAT rule - mostly because I've never been quite able to wrap my mind around how those work.
-
but I am unable from a computer on the B network to ping anything on the A network. A computer on the B network can ping both sides of the private VPN network (10.0.27.1 and 10.0.27.2). I can ping from the pfSense B to various addresses on the A network.
That seems that the site B pfSense isn't the default gateway in the site B network. If it isn't you need a static route for the site A network on each device at site B you want to access or you do NAT on the VPN.
A has an outbound NAT rule -
Interface:WAN1
Source: B's network/24
Source Port: *
Destination: *
Destination Port: *
NAT Address: WAN1 address
NAT Port: *That rule only makes sense for accessing the internet from site B over the VPN. Don't know if that's what you want.
-
Here is the routing table for my computer on the B network.
Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 0.0.0.0 192.168.5.1 0.0.0.0 UG 100 0 0 eno1 169.254.0.0 0.0.0.0 255.255.0.0 U 1000 0 0 eno1 172.16.52.0 0.0.0.0 255.255.255.0 U 0 0 0 vmnet1 172.16.150.0 0.0.0.0 255.255.255.0 U 0 0 0 vmnet8 192.168.5.0 0.0.0.0 255.255.255.0 U 100 0 0 eno1The B network is 192.168.5.0/24 with the gateway as .1.
As far as I can tell, the routes on the B pfSense are correct:
Destination Gateway Flags Use Mtu Netif Expire default 47.186.30.1 UGS 6287086 1500 em0 10.0.27.1 link#8 UH 5 1500 ovpnc2 10.0.27.2 link#8 UHS 0 16384 lo0 10.1.16.0/24 10.1.16.2 UGS 0 1500 ovpns1 10.1.16.1 link#7 UHS 0 16384 lo0 10.1.16.2 link#7 UH 1544773 1500 ovpns1 47.186.30.0/24 link#1 U 1356533 1500 em0 47.186.30.3 link#1 UHS 0 16384 lo0 67.232.254.142 47.186.30.1 UGHS 126810 1500 em0 127.0.0.1 link#6 UH 197111 16384 lo0 172.16.0.0/24 10.0.27.1 UGS 181240 1500 ovpnc2 192.168.5.0/24 link#2 U 4555834 1500 em1 192.168.5.1 link#2 UHS 0 16384 lo0 207.177.38.109 47.186.30.1 UGHS 162037 1500 em0The A network is 172.16.0.0/24.
-
That rule only makes sense for accessing the internet from site B over the VPN. Don't know if that's what you want.
I don't really need to access the outside world over the VPN. I disabled all the outbound NAT rules for B's network, but it didn't help.
-
Traceroute from my computer on the B network below. Looks like the routing is correct, but something is not passing the traffic?
traceroute -n 172.16.0.212 traceroute to 172.16.0.212 (172.16.0.212), 30 hops max, 60 byte packets 1 192.168.5.1 0.453 ms 0.440 ms 0.560 ms 2 10.0.27.1 30.880 ms 30.886 ms 30.879 ms 3 * * * 4 * * * -
Do you run multiple OpenVPN instances on one site, either server or client?
-
Do you run multiple OpenVPN instances on one site, either server or client?
Yes, the server side (A) has another Open VPN instance on port 1194 for single users rather than peer to peer. I've been using that one to connect to network A since 2.4 messed up our IPSec VPNs.
The client pfSense (B) also runs an Open VPN instance for single users on port 1194.
-
So you should assign an interface to each OpenVPN instance. Have you done this?
-
So you should assign an interface to each OpenVPN instance. Have you done this?
I'm sure my ignorance will show, but I'm not quite sure what you mean. Are you saying that each OpenVPN server instance needs to be assigned to a different interface on the pfSense, even though they are listening on different ports?
-
No, you miss-understood. An interface has to be assigned to the VPN instance.
Interfaces > assign.At "available network ports" select the vpn instance (ovpnc1, ovpns1) and hit add, then click the new interface, enable it and set an appropriate name, no further settings to be made.
After that you should move the needed firewall rules to the new interfaces. "OpenVPN" is handled as interface group containing all OpenVPN instances. Rules on that tab effects all OpenVPN instances. -
I must still be missing something.
I assigned the interfaces as you suggested, on both the server and client sides. On the server side, ovpns1 is assigned to OPT5 for the original server that is used by various individuals. ovpns2 is assigned to OPT6 for the new peer-to-peer that is not yet working.
On the client side, ovpns1 is assigned to OPT1 for the server running there - I use this from my computer when traveling. ovpnc2 is assigned to OPT2 - the other side of the peer-to-peer that is not yet working.
Both client and server have an allow all rule for IPV4 on the OpenVPN tab.
The VPN shows connected on both sides.
I can still ping from the client pfSense to addresses on the server network if I use "Open VPN Client (perr-to-peer)" as the source address. I think this means that traffic can pass from the client pfSense over the VPN tunnel.
The routes on the client pfSense appear to me to be correct, as do the routes on the computers on the client network.
-
I can still ping from the client pfSense to addresses on the server network if I use "Open VPN Client (perr-to-peer)" as the source address. I think this means that traffic can pass from the client pfSense over the VPN tunnel.
And if you select the LAN address here it's not working?
-
Correct. I can ping (from pfSense) addresses on the server network from "Open VPN Client" but cannot ping from LAN.
The routes look correct as far as I can tell (see attached). There is a an allow all rule from the LAN network to any.
-
The routes on site A pfSense may be essential here. Check if there is a route for 192.168.5.0/24 pointing to the VPN clients IP.
-
Yes, there is a route on site A to B's internal network. See attached.
10.0.27.2 is the client end of the VPN tunnel network.
I can also ping from pfSense A to addresses on B's internal network if I use the OPT6 (VPN) source address. It fails using the LAN address.I forgot to mention earlier - the LAN, WAN1 and WAN2 interfaces on site A are CARP. The OpenVPN server is not running on the CARP virtual IP address but the real address of WAN1 on one of the pfSense boxes in site A.
-
So it seems the LAN computer does not response if the source address is one from LAN B network.
Hardly to believe, since it does though if the source address is out of the vpn tunnel.
Maybe the response goes up to another gateway?To see what's going on here, run a packet capture on site A LAN while you try to ping a LAN computer from site B LAN. Set to filter to ICMP to avoid too much noise.
Also run a capture while pinging the same computer from the site B pfSense with the default source. -
I am now more confused than ever.
Trying from a computer on the B network, a packet capture on site A LAN does not capture anything. A capture on the site A OpenVPN interface captures echo and reply messages between the 2 sides of the tunnel.
10:42:40.190119 AF IPv4 (2), length 32: (tos 0x0, ttl 64, id 3651, offset 0, flags [none], proto ICMP (1), length 28) 10.0.27.1 > 10.0.27.2: ICMP echo request, id 41407, seq 928, length 8 10:42:40.228986 AF IPv4 (2), length 32: (tos 0x0, ttl 64, id 29671, offset 0, flags [none], proto ICMP (1), length 28) 10.0.27.2 > 10.0.27.1: ICMP echo reply, id 41407, seq 928, length 8 10:42:40.595307 AF IPv4 (2), length 32: (tos 0x0, ttl 64, id 26687, offset 0, flags [none], proto ICMP (1), length 28) 10.0.27.2 > 10.0.27.1: ICMP echo request, id 39100, seq 931, length 8 10:42:40.595328 AF IPv4 (2), length 32: (tos 0x0, ttl 64, id 50986, offset 0, flags [none], proto ICMP (1), length 28) 10.0.27.1 > 10.0.27.2: ICMP echo reply, id 39100, seq 931, length 8From pfSense B default source, capturing on the A site Lan:
10:47:36.624118 0c:c4:7a:17:17:57 > ec:f4:bb:cf:2b:60, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 63, id 53205, offset 0, flags [none], proto ICMP (1), length 84) 10.0.27.2 > 172.16.0.212: ICMP echo request, id 39056, seq 0, length 64 10:47:36.624306 ec:f4:bb:cf:2b:60 > 00:00:5e:00:01:03, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 64, id 27802, offset 0, flags [none], proto ICMP (1), length 84) 172.16.0.212 > 10.0.27.2: ICMP echo reply, id 39056, seq 0, length 64From pfSense B default source, capturing on site A OpenVPN:
10:49:52.181762 AF IPv4 (2), length 32: (tos 0x0, ttl 64, id 4772, offset 0, flags [none], proto ICMP (1), length 28) 10.0.27.2 > 10.0.27.1: ICMP echo request, id 39100, seq 1771, length 8 10:49:52.181786 AF IPv4 (2), length 32: (tos 0x0, ttl 64, id 10633, offset 0, flags [none], proto ICMP (1), length 28) 10.0.27.1 > 10.0.27.2: ICMP echo reply, id 39100, seq 1771, length 8 10:49:52.278123 AF IPv4 (2), length 32: (tos 0x0, ttl 64, id 37252, offset 0, flags [none], proto ICMP (1), length 28) 10.0.27.1 > 10.0.27.2: ICMP echo request, id 41407, seq 1761, length 8 10:49:52.314909 AF IPv4 (2), length 32: (tos 0x0, ttl 64, id 19078, offset 0, flags [none], proto ICMP (1), length 28) 10.0.27.2 > 10.0.27.1: ICMP echo reply, id 41407, seq 1761, length 8 -
The captures on the OpenVPN interface only show ICMP packets between the OpenVPN client an server, not any packet destined to the LAN computer.
These seem to be the pings from dpinger on both sites. For Testing it will be better to turn off dpinger on the OpenVPN interfaces.Maybe packets from the site B LAN are never routed into the tunnel.
-
The packets from site B LAN are making it to and through the tunnel. Here is a packet trace on the Open VPN interface on site A (server):
10:47:34.720334 AF IPv4 (2), length 88: (tos 0x0, ttl 63, id 60640, offset 0, flags [none], proto ICMP (1), length 84) 192.168.5.5 > 172.16.0.212: ICMP echo request, id 51458, seq 0, length 64 10:47:35.724967 AF IPv4 (2), length 88: (tos 0x0, ttl 63, id 55095, offset 0, flags [none], proto ICMP (1), length 84) 192.168.5.5 > 172.16.0.212: ICMP echo request, id 51458, seq 1, length 64The same trace on the LAN interface of A does not see those packets. I suppose that means that either pfSense A does not know how to route those packets, or they are being blocked. pfSense A can ping an internal address from both the LAN and OpenVPN interfaces.
There are 2 OpenVPN tabs in the firewall -> rules page and both of them have an allow all rule.
Does this setup need an outbout NAT rule? I've tried both with and without one - see attached.
-
The outbound NAT should not be necessary if the routing work.
Beyond that, the rule source in the route would have to be the site B LAN network.What's your OpenVPN tunnel settings? At server it should be a /30 subnet, on client the tunnel field it should be empty.
