Cross-Site routing of external IPs between two sites with BGP
-
Hi All
Sorry for the long post, but there's a bit of detail here…I have a situation where we have two physical locations (data centres) and two /24 ranges of external IPs that are BGP routed to both sites.
- I want to setup each site with a CARP failover/cluster of two pfSense boxes (so 4 in total).
- Each site has a different WAN carrier and provides a small subnet of public IPs, one of the IPs from each ISP will be the WAN side of the pfSense box (CARP IP) on that box's site.
- We also have dual redundant/geographically disperse dark fibre between the two sites trunking our 10gig switches together, so can present VLANs directly over those between the sites.
I think I have the design for the actual WAN/CARP and BGP side of things down pat and am just waiting for the carriers to setup BGP on their side to test all of that, but I keep coming back to one aspect of the design that I'm not 100% sure about.
Both /24 ranges will be routed to both datacentres, but I want one "Active" in each, i.e. range 1 will have an interface on site 1 and range 2 will have an interface on site 2, but if site 2 receives traffic for range 1 it should route that back over our dark fibre to to the site 1.
My first thought was to create a VLAN between the sites over our fibre and present a random subnet onto an interface on each router, then static route the range to the opposite site, but this means that traffic ingress will be on a different interface when coming cross-site verus coming direct from the local site's carrier which is not what I want (mainly due to firewall rule complexity).
What I really need is a solution that will allow me to route to the 'access' IP on each router which would then have the traffic originating on the WAN interface.
Now, I am doing things a bit differently with the WAN interfaces, I'm running pfSense 2.3 which allows me to use a virtual IP in a different subnet to what the physical interface is on, so I have 192.168.xxx.251/29 on the first box, 192.168.xxx.252/29 on the second and a VIP of 1.2.3.4 (as an example, in real life it's our public 'access' IP). This works brilliantly by the way and allows us to not waste the very small number of IPs provided by our ISP.
I could, theoretically, push a VLAN across both sites and throw the WAN connections on each site into the same VLAN, but then i'd have to present the carriers services into that VLAN too which is a risk I don't want to take (I want them kept separate), but if I was to do this i could route range 2 to the 192 address of router 1 and vice versa, but this is getting very messy in terms of having multiple subnets (the 192 range and two ISP ranges) within the same VLAN and I want to avoid that.
Is there a more elegant way that I can achieve this? I'm hoping I'm just being brain-dead and there is something really obvious.
Thanks for any help in advance :)
-
No takers on this one?
Is it that it's something that can't be achieved or is it just something I said haha.
-
I'm trying to do something sort of similar.
Ill post a new topic maybe itll help.