-
I have two sites, both with pfSense, and I have a working IPSec VTI tunnel between them. I have stood up BGP and routing is working as I expect; I can ping nodes across the VTI without any issue. Now, what I am trying to figure out is how to set one VLAN up with a default route in Site1, so that all it's traffic will go across the VTI when it's up, otherwise fail back to the local gateway when the VTI is down, which is why I want to use VTI and BGP to hopefully do all the routing heavy lifting for me. If I need to set it up so that all VLANs use Site1's WAN when the VTI is up, that's ok too.
The piece I can't seem to wrap my head around is how the heck to make the VLAN route across the VTI and use Site1 WAN when it can, otherwise use it's own WAN at Site2. I do have BGP set to Originate Default with Site1, and can see in the FRR Status that is the case, but when looking at the Routing tab under Diagnostics, I don't see that the default route is going across the VTI, so I'm a little confused on to what I'm missing.
Here's the diagram (using GNS3) of the setup with relevant info. I've mocked it in GNS3 because it's a long way to Site2, which is not a manned site, so reboot/console recovery is not always an option without a long drive. Besides, this should work in GNS3 the same, it's just forming the tunnel over an "external" IP space.
-
Any thoughts from anyone? I'm still struggling with this.
-
Still looking into this as I haven't been able to find a solution at all. I'm currently going down the path of not using BGP but rather start creating static routes, but I do not have hope there either.
-
A month on and still no luck with getting anything working. I’m really surprised at the lack of response from the forums on this to be honest.
-
Looks like a lot of folks are looking at this but not able or willing to answer. Unfortunately that makes me very disappointed, I've heard the pfSense community is a great community and is always willing to help, but not getting the feeling of that here.
-
You can route across the VTI and use Site1 WAN when it can, or use WAN at Site2 if vti fails . without BGP.
All you need, is to use Gateway Groups , put vti interface as tier 1 and its WAN as tier 2. then use Policy Routing on vlan interface.
https://docs.netgate.com/pfsense/en/latest/book/multiwan/policy-routing-configuration.html
https://docs.netgate.com/pfsense/en/latest/routing/multi-wan.html#firewall-rulesif IPsec VTi fails , the traffic will pass across its Wan.
also configure what VTI needs , like NAT and Static routes.
it works good here. -
@Zawi deceptively simple to say the least, and it took me a few times to see it in the documentation. I think I did try that before, but the key is that on my Site1 the Outbound NAT did not automatically include the subnet's from Site2, so once I put the Outbound NAT into Hybrid Mode and added the subnets, well things are now working as expected.
I am still using BGP though simply to avoid the static routes, I have a few subnets and am lazy. Couple of things I've learned also is under the Gateway entries, in Advanced you can define the thresholds for latency and packet loss for the gateway to be considered up/down, which is key here. Also, I had the VTI gateway set to disable monitoring, which in my testing also broke the failover, which was another key problem.