Site-to-site VPN only working with NAT [resolved]
-
Please help a netadmin out here with a frustrating problem.
I have two offices with pfSense 2.3.2 boxes. These act as the main gateway for the LAN at both sites. Each office has multiple VLANs (for LAN, voice and guest access) corresponding to separate VLAN-tagged interfaces in pfSense. I mention this for completeness, but it should have no impact on this setup.
I need to allow routed connectivity between the LAN interfaces in both offices such that devices on one LAN can communicate with devices on the other.
Office A LAN (pfSense server): 172.30.110.0/24
Office B LAN (pfSense client): 172.30.100.0/24
pfSense internal subnet: 10.0.8.0/24
See also screenshots below.I have ignored the default "OPENVPN" interface in the firewall rules and assigned a new interface to the VPN server/client at each end. On this interface I have added an IPv4* allow any/any rule. Both boxes use manual outbound NAT.
With this configuration, routes are correctly added to the routing tables at each end of the link. The pfSense boxes themselves can ping devices on each LAN from the other site, using their 10.0.8.1/10.0.8.2 VPN interface IPs to transmit these pings. Machines on each LAN can ping the VPN endpoint at the far end (e.g. office A can ping 10.0.8.2) and devices in office B can ping 10.0.8.1.
But the problem: devices on each LAN cannot access devices on the far LAN.
Checks and diagnoses made:
-
Ping from devices in one site other than pfSense to devices other than pfSense - no traffic.
-
Ping from pfSense to the OpenVPN interface on the far pfSense - works.
-
Ping from pfSense to devices on the far LAN subnet - works if the ping is set to 'auto source' (so it originates from the 10.0.8.x address). Does not work if the LAN interface of pfSense is used as the source.
-
Ping from site A client device to device in site B client device (e.g. 172.30.110.109 to 172.30.100.1) - packet capture on pfSense B shows the packet arrive (so routing working in site A). tcpdump on the target device shows no packet enters its NIC, hence no reply.
-
Ping from client device in either site to the OpenVPN interfaces in either site (10.0.8.x) - works in both directions
Tried with a mixture of devices on each end of the link - e.g. Linux boxes, managed switches etc. All devices have a gateway of the pfSense box interface in their local LAN, so no local routes should be necessary.
I have bodged the setup for now by adding a manual outbound NAT rule which NATs all traffic from the far site to the VPN interface's IP address. However, this is a rubbish configuration which I want to remove and have true routed connections between the devices in each site (along with appropriate restrictions on each firewall).
I consider myself generally competent with advanced networking topics, but cannot see the wood for the trees here. There's possibly some reply-to or gateway issue here causing the packets to go astray, but I've been staring at this for ~10 hours over three days and can't see it. Any ideas?
I attach screen captures of the server and client OpenVPN setups. If you require other screendumps please ask.
-
-
Unless you're trying to policy route specific traffic to the remote end and NAT it to the internet, you don't need the tunnel assigned to an interface, remove it on both ends.
Go back to Automatic Outbound NAT
Add and any/any rule to the OpenVPN tab on both ends, until basic connectivity is established end to end.
Add the subnets for each client-side VLAN to the IPv4 Remote network(s) box on the server config
Add the subnets for each server-side VLAN to the IPv4 Remote network(s) box on the client config
That's about it. Also, remember if you're testing windows PC's/servers, icmp echo reply is not enabled for traffic sourced outside of it's LAN by default.
-
Unless you're trying to policy route specific traffic to the remote end and NAT it to the internet, you don't need the tunnel assigned to an interface, remove it on both ends.
Go back to Automatic Outbound NAT
Add and any/any rule to the OpenVPN tab on both ends, until basic connectivity is established end to end.
Add the subnets for each client-side VLAN to the IPv4 Remote network(s) box on the server config
Add the subnets for each server-side VLAN to the IPv4 Remote network(s) box on the client config
That's about it. Also, remember if you're testing windows PC's/servers, icmp echo reply is not enabled for traffic sourced outside of it's LAN by default.
Hi.
Thanks for this. I tried all of this setup again - it was the most basic configuration I had when trying to get it to work initially, and eventually ended up at the more complex scenario I described.
In short, it didn't work. However, in desperation to get this work (others breathing down my neck) I took the nuclear option and rebooted the box at the client end. Everything suddenly sprung into life.
I can only assume there was a discrepancy between the web interface and the configuration as actually implemented, or perhaps I stumbled upon a bug or something. Anyway, getting it working became time critical, so I don't know the real root cause. I don't like 'reboot it' solutions but in this case it genuinely worked.
-
I've never dealt with assigning interfaces to VPN's, but in reading other posts over the years from people who have, the consensus seems to be that when making changes to interfaces involving VPN's, you need to bounce the service the VPN is on to get things working right.
It's possible that bouncing the service on your tunnel is all you needed vs. a full reboot, but who knows.
So, you're up and running with the simpler config?
-
I've never dealt with assigning interfaces to VPN's, but in reading other posts over the years from people who have, the consensus seems to be that when making changes to interfaces involving VPN's, you need to bounce the service the VPN is on to get things working right.
It's possible that bouncing the service on your tunnel is all you needed vs. a full reboot, but who knows.
So, you're up and running with the simpler config?
Ah, it might have worked if I had rebooted the services then. Anyway, yep, all working and the users are happy (phew!). Thanks so much for your help.
-
@marvosa said in Site-to-site VPN only working with NAT [resolved]:
IPv4 Remote network(s)
Same question, same problem, but on pfsense 2.4.4.
Bouncing openvpn / rebooting the nodes does not solve the problems
-
@paolone919191 Start a new thread, so we can get some specifics and offer some help.