CARP with /31 and /29 WAN Address Blocks
-
Hello,
We have recently purchased two Netgate XG-1537 1U firewalls for use with an HA deployment.
At the moment we currently have two 1Gbps WAN connections from our ISP. Our ISP present each connection as /31 networks which is assigned to WAN1 and WAN2 of our existing firewall appliance (One for the interface addresses and another for the WAN Gateway addresses) They also provide us with a routed /29 block of addresses on each connection which is in a different subnet.
Each WAN connection currently goes from the ISP CPE equipment via Multi-Mode Fibre and terminates on our Layer 2 WAN switch which then is presented to our firewall WAN interfaces via RJ45 Ethernet.
When looking to move to HA using CARP we are wondering if the current address configuration assigned by our ISP would work for this? I seem to recall from reading somewhere previously that the primary WAN interfaces required a minimum of a /29 block with the gateways and CARP VIPs on the same subnet?
Can we have the primary firewall use the /31 networks for its WAN1 and WAN2 interfaces and the secondary firewall use the routed /29 blocks for its WAN1 and WAN2 interfaces? We would also need to use an address from the routed /29 blocks for the CARP VIPs on each WAN interface.
I can provide some more details and a network diagram if this helps.
Many thanks.
-
See the docs for a picture. Generally the shared WAN IP is in the same subnet as the two device WAN IPs. In that doc picture the .200 address would essentially move between the two routers. However I have actually gotten it to work with two pfSense routers behind a Comcast router, where the pfSense routers use the 10.1.10.x subnet for their WAN IPs and public IPs for Virtual IP/CARP. In that configuration the two routers can still independently get to the Internet (for updates etc.) because Comcast leaves NAT running on a "bridged" router.
Having two ISP connections share an address is what SD-WAN is for...the SD-WAN router is behind the (usually two different) ISP routers and the SD-WAN company takes care of routing traffic for their IP across one or both ISPs.
-
@steveits Thanks for that Steve.
Is there likely to be any problems associated with using addresses within the /29 blocks against the secondary WAN interfaces and for CARP VIPs?
The one issue I do see is that the secondary firewalls WAN interfaces would need to use the WAN gateway addresses specified on the /31 subnets - given that both of the /29 blocks are in a different subnet range the option "Use Non-Local Gateway" would need to be selected under the advanced gateway settings.
-
The CARP VIP for WAN needs to work on both connections so I'm not sure how that would work. The ISP would need to be routing traffic to one of the two WAN IPs (one of the two connections) which sounds more like SD-WAN to me.
Our data center for example has a /29 used across both routers using two addresses and the CARP VIP going to a switch, and there is of course just one WAN cable reaching into the data center. The HA benefits there are 1) if router1 fails no one notices, and 2) we can upgrade and reboot at our leisure, even during the day. We have a /25 in use on the LAN side, and it is being routed to the CARP VIP that is in the /29.
-
@mitchell-0 Tell your ISP you need three /29s. One for each WAN and one routed so you can run High Availability routers.
They should understand (though they might charge a little more.)
/31s are great when they are sufficient. In your case they are not.
-
@mitchell-0 said in CARP with /31 and /29 WAN Address Blocks:
Is there likely to be any problems associated with using addresses within the /29 blocks against the secondary WAN interfaces and for CARP VIPs?
Yes. The /29s will be routed to the /31 addresses on the other node.
-
@derelict Thanks, I'll look at contacting them today and explain the situation. We also have another site with the same configuration (Same ISP) so I suppose it will ultimately depend if they are willing to hand out that many addresses between 4 connections in total.
I know from previously we did have to fill out the required RIPE forms just to get the additional /29 blocks at the side of the /31 and going for anything larger had to be justified. I suppose it is understandable given the state of IPv4 addresses.
I wasn't sure if we could use a workaround in the event they cannot offer up the additional address blocks, although I suspect this would likely require a router or Layer 3 switch to sit behind both firewalls and using NAT with private addressing on the WAN interfaces. Not something I'm a fan of doing as it just adds a whole other area of complexity.
-
@mitchell-0 High Availability on the interfaces should be enough justification for /29s on the interfaces.
-
@derelict I have sent our requirements to our ISP and they seem happy to have a look at this for us.
Many thanks.
-
@mitchell-0 said in CARP with /31 and /29 WAN Address Blocks:
t our requirements to our ISP and they seem happy to have a look at this for us.
Many thanks.I guess you have Top Of Rack switches ? Right ?
Does you ISP use BGP ? If yes, i guess the /31 are 198. something. In that case, it's used for ISP BGP AS connection. Then, you could do a VRRP or HSRP with your /29 and add this /29 in the addressed network to your ISP.The idea is that pfsenses @ the end doesn't know your /31. /31 are just used for BGP sessions.
You TOR need to be L3 tor of course.
I could give you more info if you need.
-
@misterto We do have two Layer 3 Top of Rack switches that act as our collapsed core. These are Dell S4128F models configured with VLT and peer-routing for our internal VLANs. We aren't using any dynamic routing protocols at the moment just static routes present on pfSense for the VLAN subnets on the switches.
The /31s are 185. addresses.
I have been caught up with other projects recently so just waiting to hear back from our ISP. We have requested the /31 subnets on each WAN connection be changed to /29s and we will keep the existing routed /29 blocks that they have supplied us.
Many thanks.
-
Hi all,
Bumping this thread as we have finally gotten the IP subnet requirements from our ISP and just wanted to double check before we go ahead with the firewall configurations. Apologies as this had taken longer than we initially expected due to a lot of back and forth.
Our ISP has converted our existing WANs from /31s to /29s and have also retained our existing routed /29 subnets that were already in place, effectively giving us two /29s per physical WAN.
As the WAN subnets are in a different IP range from the additional routed /29 subnets what is the best way to add the additional address from the routed subnets to allow these to sync between firewalls without causing an IP conflict?
I’ve read about adding these as additional IP Aliases with the interface being the CARP VIP, however I was under the impression that this would only work if the addresses were in the same subnet as the CARP VIP?
Each WAN subnets addressing will cover each firewalls WAN1 and WAN2 interfaces plus shared CARP VIPs for outbound NAT while the addresses from additional routed /29 subnets will be used for external services (Web Servers, VDI access etc)
Many thanks.
-
@mitchell-0 said in CARP with /31 and /29 WAN Address Blocks:
Each WAN subnets addressin
Hi.
Do you have schema ?
-
WAN 1:
WAN Subnet: 161.12.60.232/29
ISP Gateway: 161.12.60.233
Routed Subnet: 161.12.51.32/29
Shared CARP VIP: 161.12.60.236WAN 2:
WAN Subnet: 161.12.60.240/29
ISP Gateway: 161.12.60.241
Routed Subnet: 161.12.51.40/29
Shared CARP VIP: 161.12.60.244Firewall 1:
WAN 1 Interface: 161.12.60.234
WAN 2 Interface: 161.12.60.242Firewall 2:
WAN 1 Interface: 161.12.60.235
WAN 2 Interface: 161.12.60.243
