Problem with VTI / routed IPSEC + BGP in a hub and spoke arrangement
-
Config looks like this:
Hub - local lan is 10.0.0.0/16
Branch 1 - local lan is 10.1.0.0/16
Branch 2 - local lan is 10.2.0.0/16Since I'm using VTI / routed ipsec, I have to use a transit network. My understanding is that this should be a /30 network. I can configure vti connections for Hub <> Branch 1 and use something like 172.16.0.0/30, which gives me .0 through .3 as a network. I make Hub's address 172.16.0.1 and Branch 1's address 172.16.0.2
Setup BGP neighbors on each end, using the IP address in the transit network as the router ID. It works.
It falls apart when you add the third location. For Hub <> Branch 2 connection, I'm giving Hub a transit network address of 172.16.0.5 and Branch 2 172.16.0.6. This is the next /30 network up. Branch 2 can't talk to Hub at its actual router ID, which is 172.16.0.1 since they are separate networks. Hub also can't have two router IDs, one for each transit network.
I tried using a /24 transit network. It worked for some connections and not others, to the point it makes me think that pfSense treats the transit network as a /30 regardless of the addressing you give it.
I tried using the local LAN interface IP as the router ID. Did not work, obviously because no route exists to get traffic across the VPN connection to the lan interfaces on either end, which sure looks like a chicken and egg problem; I need BGP to give out a route to get to the LAN interfaces on either end, but BGP can't give me a route until I give it one statically, etc.
I am certain I could just setup static routing, but that defeats the entire purpose of having BGP in the first place. If I have to do that, I have no use for BGP. Plus I want to avoid having to manually manage routes.
-
I solved my own problem, and wanted to record the answer in case others have similar.
First, my initial assessment that the IP address of a Neighbor entry had to match the Router ID that was set on that neighbor's BGP service was wrong. It didn't work for me initially because I had some other things set wrong.
Here's how it's now working:
I setup the IPSEC connection between Hub and Branch 1 as described above. The transit network for this link is 172.16.0.0/30. Hub gets 172.16.0.1, Branch 1 get 172.16.0.2.
The connection between Hub and Branch 2 is similar. The transit network is set to 172.16.0.4/30. Hub gets 172.16.0.5, Branch 2 gets 172.16.0.6.
The router ID for all routers is set to the LAN IP address in BGP, NOT the transit network IP address.
Each Neighbor entry specifies the transit network IP. Ergo, Hub 1 has neigbor entries that specify 172.16.0.2 (Branch 1) and 172.16.0.6 (Branch 2).
The last setting, and this turned out to be the trick that made it work, is in the Neighbor settings in the Next Hop section. Took me a bit to find and identify this as the solution to my 'chicken and the egg' problem in my OP. In the Next Hop action, I set Next Hop Action to "Set (Peer Only)" and I set Peer to "Peer Address (set only)".
This setting is what got things going as expected.