Do you need multiple public IP's for basic failover functionality?

SteveITS

https://docs.netgate.com/pfsense/en/latest/highavailability/index.html#ip-address-requirements-for-carp
“It is technically possible to configure an interface with a CARP VIP as the only IP address in a given subnet, but it is not generally recommended. When used on a WAN, this type of configuration will only allow communication from the primary node to the WAN, which greatly complicates tasks such as updates, package installations, gateway monitoring, or anything that requires external connectivity from the secondary node. ”
I’ve done it where the Comcast router provides NAT even while in bridge mode, so both routers have a private IP and share the public IP.

Magoogle

This is in a datacenter, so I am limited to a single IP per client device and we dont have a device before the PFSense boxes that could handle any type of routing.

We dont want to allocate more than 1 IP to each client, unless they have multiple services using the same port because we are not using proxy currently.

SteveITS

@Magoogle Then you might be stuck. Unless you can try to emulate the NAT Comcast uses...something like:

router-outside:

WAN = public IP subnet 1
LAN = public IP subnet 2
LAN alias = 10.0.0.1/24

router1-client:

WAN = 10.0.0.2
WAN CARP alias = from public IP subnet 2
LAN CARP alias = 192.168.1.1
LAN = 192.168.1.2

router2-client:

WAN = 10.0.0.3
WAN CARP alias = from public IP subnet 2
LAN CARP alias = 192.168.1.1
LAN = 192.168.1.3

...just thinking out loud.

As noted in the doc page it can technically be done with router2 not having a working WAN but then to install anything on router2, or update router2, one has to fail over so router2 is live and then work on it.

Magoogle

@SteveITS

So I made it work, at least on my bench.

I used 10.0.0.1 for Primary WAN Interface, and 10.0.0.2 for Seconday WAN Interface with /30 subnet (This network goes no where)

I created a "Public" CARP WAN IP for the actual WAN, in this test its 10.1.25.250 (because its behind another firewall on the bench)

Setup my Sync interfaces, configured outbound nat to Hybrid and set it to use the CARP address as the NAT address.

Running ping tests to the internet from an interface behind these 2 virtual firewalls, I only see 1 packet drop when I emulate a firewall failure, by "turning off the power" to the Hyper-v VM

And powering it back up, it updates and takes over as Primary again without any issues. So It looks like this will do what I want without wasting public IP's. The only problem with this, is that the secondary firewall has no internet access of its own until it takes over as the Primary.

I guess I could create another virtual interface to act as a secondary WAN to allow it to talk to the internet outside of the "public" facing network.

SteveITS

@Magoogle said in Do you need multiple public IP's for basic failover functionality?:

create another virtual interface to act as a secondary WAN to allow it to talk to the internet outside of the "public" facing network

If you use a /29 instead of a /30, 10.0.0.3 goes on the router upstream from these two and hence is the gateway for 10.0.0.1 and 10.0.0.2. That gives those two Internet access over the NAT, or the shared/CARP IP.

Magoogle

@SteveITS The router upstream doesnt exist in the datacenter. Just a fiber handoff with 2 blocks of IP's

viragomann

@Magoogle said in Do you need multiple public IP's for basic failover functionality?:

The only problem with this, is that the secondary firewall has no internet access of its own until it takes over as the Primary.

By creating a failover group with the WAN gateway and the LAN of the primary, the secondary can go out to the internet through the primary node if it's in backup state.

You would have to configure the HA settings on the primary that this gets not overwritten.

Magoogle

@viragomann Where on the primary HA would that configuration be changed? I dont see in the offered selections to exclude that type of change?

viragomann

@Magoogle
You have to disable syncing of "Static Route configuration" in System > High Availability Sync.
This of course means then, that you have configure all static route and gateway groups on the secondary as well.

Magoogle

@viragomann I created a gateway group and a secondary gateway on the secondary firewall. I tried with WAN and the LAN as the interface and the IP's of the primary as the gateway. while the gateway shows online, the secondary is unable to ping out to the world.

viragomann

@Magoogle
On the secondary you need to add the primary's LAN address as a gateway in System > Routing > Gateways. I'll call it PrimLan

Then go to the gateway groups tab and add you new failover group:
WAN gateway > Tier1
PrimLan > Tier2

Got back to the gateway tab and state this group as default gateway.

Ensure that the monitoring is enabled on both gateways and that the WAN gateway state is offline, when the primary is the master.

Magoogle

@viragomann Thats how I set it up. But when using the console for pfsense, it cant ping out even though the Tier2 gateway shows as online.

viragomann

@Magoogle
Check Status > Gateways.
Is the tier2 the default now?