IPSEC VTI with Dual-WAN configuration
-
I'm looking for guidance on the best way to set up IPSEC VTI connections for dual-wan sites. I have OSPF working with the FRR package to handle routing. If site 1 has WAN A and B, and site 2 has WAN C and D, how should the phase 1 connections be established?
In the past (without a routed VTI), we would set up WAN A to a host name and have a dynamic DNS app that would sense the current Internet connection and then update the DNS setting if the primary WAN connection went down. However, the failover wasn't nearly as fast as we'd like. We wanted to do away with dynamic DNS completely and let OSPF routing handle everything. I was hoping to just do a phase 1 from WAN A to WAN C, and another from WAN A to WAN D. Then I'd repeat both of those with WAN B, but I can't establish two phase 1 connections to the same remote IP.
-
Since you can't connect to the same peer over multiple WANs, your options are limited. For example you could do:
Tunnel 1: A-C
Tunnel 2: B-DThat's better than a single tunnel but if two of the wrong WANs happen to be down, it could still break.
If you did DynDNS on all four WANs combined with failover groups on the interfaces it might work, though. For example:
Tunnel 1: GWG Prefers WAN A and fails to B, connects to C
Tunnel 2: GWG Prefers WAN B and fails to A, connects to DAnd a mirror image of that on the other side (C-D connects to A, D-C connects to B)
Use unique identifiers and not local/peer IP address choices.
Even if some of those connections end up failing, there should still be at least one working combination up any given time if all goes well.
I haven't tried that, but it should work in theory.
-
Thanks for the response! That's what I was afraid of. We use that currently with traditional IPSEC and it works ok. Unfortunately, the dynamic DNS takes too long to update when an interface goes down, so we were hoping to avoid using dynamic DNS with the routed connections.
Unless you know of a dynamic DNS solution that updates very quickly?
I'll test this type of connection with the routed VTI and verify that it works as theorized.
-
The nature of DNS means it will always be slow, but by having two tunnels up and only one of them failing (hopefully) at a given point in time, it should still fail over fast and then within a few minutes, regain redundancy.
-
@jimp said in IPSEC VTI with Dual-WAN configuration:
Since you can't connect to the same peer over multiple WANs, your options are limited.
Hi,
that's exactly what I'm trying to do today, without success.
I was thinking to setup 4 tunnels and using source routing on both sides to achieve proper connection like this:
Tunnel1: A-C
Tunnel2: A-D
Tunnel3: B-C
Tunnel4: B-DFor now, my issue is the automatic rule that allows pfsense to get out ("let out anything IPv4 from firewall host itself"), since the traffic never reaches the routing rules.
If there wasn't that rule, is there anything else that would prevent to make the 4 tunnels work ?
-
When I tried it (before my post), pfSense won't accept the 2nd phase 1 configuration for the same endpoint. So if you have A-C already, then you can't save the configuration for B-C.
-
I just did some tests since it works in my lab and pfSense accepts the same endpoint if you use an alias (on both phase 1, it's not a trick to fool the system). If you use an ip, it's failing.
I don't know why.
-
@fbor said in IPSEC VTI with Dual-WAN configuration:
I just did some tests since it works in my lab and pfSense accepts the same endpoint if you use an alias (on both phase 1, it's not a trick to fool the system). If you use an ip, it's failing.
I don't know why.
Do you mean using other than IP Address/Peer IP Addres for the identifiers pair on P1 Proposal for Authentication??
-
No, I mean in the remote gateway field.
In the identifiers, I'm using KeyID Tag with different values between the two phases 1 (ikev2 in case it matters)
But anyway, there is still the source routing issue. I will try to manually insert those rules into /etc/inc/filter.inc to see if it would work.
-
Ok. So, in my setup, I have one pfsense with only one wan, and another pfsense with two wans.
I've managed to add the rules for the source policy routing in the one with two wans.
The two tunnels are up but only one seems to really work. There is traffic received on port 4500 from my ping but nothing on the vti interface. I don't see anything obvious for now.
-
so, I learned how to properly set routing with multiwan on pfsense and definitly, it's not working. When A - C or B - D are up, A - D or B - C are not functionnal (still, phase 2 in installed according to the logs).
I'll try next with the gwg failover.
-
pfSense adds a static route to the peer out the interface configured on the tunnel. You can't have more than one static route to the same destination, and you can't send traffic out one WAN using the address for the other (it will almost certainly get dropped by the ISP, or at best route asymmetrically).
So that particular type of configuration won't work, which is why it's prevented.
-
@jimp Many thanks for the explanation.
I also realized that it is not an alias you can set as endpoint on phase 1 but an hostname... (shame)
Anyway, I have tried to comment the phase 1 duplicate endoints control in vpn_ipsec_phase1.php, the other one in vpn.inc and the route_add_or_change calls in the same file. It then worked perfectly. All of the 4 tunnels are now up and running.
I will submit a PR with an option to properly do this since we would like to implement that kind of configuration in our pfSenses, and we may not be the only ones.
In short, in our situation:
A is an high-speed low-cost no SLA fiber
B is an lesser high-speed, higher cost, 24 hours SLA fiber
C is a high-speed low-cost no SLA fiber
D is a low-speed, high-cost 4 hours SLA SDSL
So we would like to avoid at most the configuration where the low-speed SDSL is used in case of failure.Regarding the PR, should it try to generate the rules for the source routing ? In that cast, it would be an option to choose between static routing and dynamic routing (route-to from interface to endpoint when the side of the tunnel is not just a responder). Otherwise it would just be disabling static routing and duplicates check.
Or I suppose we could just go to IPv6 to work around this, since we have now plenty of public IPv6 addresses.
-
so now, I realize there is no need for additional rules, just to correctly set the gateways on the wan interfaces. It seems too simple to be true.
-
@jimp Deploying the patch, I have encounter an issue when ipsec rules generation is not disabled, since the rules are taking all pfsense traffic (self) for the same destination. I will add into the patch a test in filter.inc to disable the ipsec rules generation just for those phases 1. The option would be, in summary, "allows you to use duplicate gateways but you are responsible for the routing settings".
If you prefer, I can also change the from clause from the generated rules to use the phase 1 interface address instead of (self) when this option is enabled.
