IPSec using default gateway to reach remote endpoint when it shouldn't.
-
I've attached a basic network map to make things a bit simpler to understand. What we are trying to do is test PFSense to PFSense with CARP failover onsite. We've had a number of issues that we believe were caused because both PFSense groups were operating on the same network using similar VIDs with CARP. So we separated the networks and put a PFSense box in the middle that only routes between the two sites.
Anyway on the PFSense boxes at Location 1, the default gateway is A.B.C.97. It also has a gateway of A.B.C.102 that I route 172.16.1.0/24 to. I can ping 172.16.1.254 from Location1: PFSense-1 (Assuming it is the Master VIP for A.B.C.98).
On the PFSense boxes at Test Location2, the default gateway is 172.16.1.1 and they have no further gateways setup. I can ping from PFSense-1 at this location to the VIP at Location1.
So now I've tried to create an IPSec tunnel between the two locations. The tunnel never comes up (Or when it does, I can't send traffic over it). So I perform a packet capture on the WAN interface of Location1 PFSense and I'm seeing ICMP Destination Unreachable messages from A.B.C.61 (Our Provider's Router) for the destination address 172.16.1.254.
Additionally, when I go back and look at the packets that I captured for the IPSEC VPN, when the router at Location1 sends the ISAKMP packets to 172.16.1.254 (At Test Location2), the destination MAC address is for our Adtran router (Our default gateway) instead of the MAC Address for the PFSense Router.
It looks like the racoon is ignoring the static routes that I have in my config and is using the default gateway to reach the remote endpoint. Is there any way to fix this? Is there any reason it does this? Thanks!
-
Can someone move this to the appropriate forum (I guess the title needs changed too)? I've found this issue isn't specific to IPSEC as I tried to do an OpenVPN Site-to-Site Tunnel and it is doing the same thing. The weird thing is that ICMP and HTTP/HTTPS get routed to the proper gateway and OpenVPN/IPSec do not. I've attached screenshots from two packet captures. Notice the destinations are the same, but look at the MAC address they were sent to.
-
Update:
We think the PFSense boxes at location1 are ignoring new routes. I moved the PFSense Router internally so that we could just route traffic between the testlocation2 and location1. I can't access testlocation2 from location1 and vice versa unless I add a static route on the system on at location1 bypass the PFsense boxes.
At this point we are planning to reload the firewalls.
-
So we completely setup the firewall again from scratch. Anyway, everything was working except one rule wasn't queuing traffic properly. Anyway, I rebooted PFSense and it fixed the problem with traffic shaping, however it started doing this weird routing issue again. I definitely think this is a bug.
-
Without having closely examined all the details you're providing, I'd like to make a quick note that IPsec on FreeBSD doesn't use the system routing table to forward IP packets.
-
The problem isn't specific to IPsec. It also happened when I tried to setup a site-to-site OpenVPN tunnel. Does OpenVPN do that too?
-
Check Diagnostics > Routes - when you pick the interface for OpenVPN or IPsec, it adds a route to the peer's IP via that interface's gateway. Having two gateways on the same interface might be confusing that code.
OpenVPN you can set for an interface of "any" and then it won't add a route like that.
