CARP with 1 IP

alxbob

I am trying to do something similar any help would be highly appreciated!

ITI

I too read the release notes on the carp ip address being outside the alias subnets.
Since public IP addresses are worth their weight in gold depending on your ISP I was very curious to see if it works.

So far my test setup works flawlessly.
Failover and Restore works perfectly.
Adding and updating rules remain in sync etc….

Note: Adding or Updating "Alias IP" in the "Virtual IPs" section won't add/update on the backup firewall, they have to be added to the backup firewall separately.
(I believe this is a bug and currently pursuing it in a separate thread.)

I started with the setup I am currently working on which is configured with 3 Public IP addresses.
Similar to what you had done I then converted the two public firewall addresses to private addresses in their own subnet (i.e. NOT on the same subnet as my LAN or DMZ).
I also have a dedicated sync on the opt2 interface.
The Public gateway should be marked as Default if it isn't already.

One thing that may be different from your set up is that I have redundant ISP connections so I have a switch in front of the firewalls connecting the firewall WANs to the ISP.
In other words a switch between the firewalls and the modem or whatever your connection might be.
Whether having this additional switch makes a difference or not I do not know but it is required for my setup.

The Remote Access also has to be addressed since you now only have 1 public ip address (excluding your alias ip's).
I'm using port forwarding rules and managing the firewalls on the sync interface as follows.
Dest.addr - Dest.ports - NAT IP - NAT Ports
PublicIP - 444 - Primary Sync IP - WebConfigurator Port
PublicIP - 445 - Backup Sync IP - WebConfigurator Port
Update: The page is a little slow when accessing the backup FW but I think this maybe related to conflicting session/cookies.

A better way if you have 3 ip's:
Dest.addr - Dest.ports - NAT IP - NAT Ports
PublicAliasIP-2 - 443 - Primary Sync IP - WebConfigurator Port
PublicAliasIP-3 - 443 - Backup Sync IP - WebConfigurator Port

That's it and so far it is working perfectly although there is likely areas for improvement.

Any and all feedback is welcome.

Update:
One thing I noticed is when the WAN cable is unplugged, the primary LAN (DMZ in my case) goes into backup status where as the WAN goes into "Init" status.
On the backup both the WAN and LAN go into Master status.
I don't know if this would be considered a flaw or what the consequences are while the WAN is in this "Init" status. Update: I now think this the normal behavior
However, FailOver and Recovery still work as expected.

alxbob

Can anyone post a small how to about carp with 1 ip? Thanks in advance

ITI

Update on my working single IP CARP setup.

First, the issue of adding additional Alias IPs not syncing to the backup was a configuration error I had made.
Alias IPs must added to the WAN CARP IP interface, not the WAN interface as I had initially done although it worked it probably shouldn't have!

I have also discovered that to add additional Alias IPs the CIDR mask (bits) of the private network must be the same as the public wan and gateway network.
i.e. If your public IP address and gateway are x.x.x.x/27 the private addresses used on your wan interfaces must also be /27.

gdi2k, change your private wan IPs to /29 instead of /24.

Everything is working as expected. My two previously assigned FW public IPs are now available for other uses and have them forwarding to 2 other test servers.

Derelict

Here's how I briefly tested it:

https://forum.pfsense.org/index.php?topic=88940.msg491787#msg491787

Note that you set a gateway to the ISP and mark it as the default. There are no gateways set on the actual WAN interfaces. Then all your traffic has to NAT from an actual, public CARP VIP.

In those examples, you should regard the 172.27.0.0/24 network as public. It's my test bench setup and that's its WAN network.

ITI

I think anybody that is following this thread is likely aware of the other.
I don't know why that thread was even started since this one already existed and was showing proof of success.

dotdash

Did some live testing on this today. The configuration was a simulation of a single WAN setup where you only have a single IP assigned and want to run a pair of firewalls for redundancy. I used a pair of firewalls with three interfaces running 2.2 x64. One LAN, one WAN, and SYNC.
I did a fairly standard CARP setup on them, the one difference being I used private ips on the WAN. I added a single CARP vip on the WAN with the public IP and real subnet. Others have mentioned matching the fake WAN subnet to the real WAN subnet- in my case it didn't make a difference, perhaps because I wasn't stacking alias vips on the CARP. I added the real gateway from system, routing and set it as default. I then added the gateway on the WAN interface. It didn't seem to make a functional difference, but I think it would in a multi-WAN setup. Failover and failback seemed to work fine. The one issue I had was that the secondary was not accessible from the outside. The primary worked normally once I created an access rule allowing https traffic to the public vip. I tried various port-forward tricks, as suggested by ITI, but couldn't get them to work.
My production uses of this will probably involve setups with a larger subnet routed via a /30 transport and multiple WANs, so I'll have to see how those factors change the config, but the test worked well for a single WAN with one public ip.

dotdash

Ok, I'm back at it trying to do failover with a /30 circuit. It works, but I can't figure out how to get a gateway group to work with this.
This is multiwan, other WAN has enough publics to put on each firewall. Having to mark the gateway up all the time is not much better than having the line only work on the primary. Anyone figure out how to get a monitor working if the interface is private with a public CARP? Tried adding a nat from private wan, but no joy.

Derelict

Without being able to select the CARP VIP as the interface in the gateway config it's going to be a challenge.

dotdash

Perhaps it will require a feature request to work properly…

CyberTiVo

This seems to fit the topic. Since the same subnet restrictions have been removed from CARP, I want to just use 1 IP also. However, Comcast here charges an extra $40 (20 static & 20 for business) a month to get a static IP. Since my IP has not changed in 2 years, I want to use the IP I get from DHCP as my VIP. Easy to do until the lease runs out: Use WAN NIC to get DHCP IP, assign the IP to the CARP VIP, go in WAN interface and spoof MAC address. Now everything should run fine until the lease expires because there is no place to put the MAC in CARP VIP GUI page. How does pfSense decide which MAC to assign to the VIP? Is there a way for the VIP to get a DHCP address that it would continue to renew, so as to avoid problems? If I am way off on this please point me in the right direction.

bbrendon

@dotdash:

Ok, I'm back at it trying to do failover with a /30 circuit. It works, but I can't figure out how to get a gateway group to work with this.
This is multiwan, other WAN has enough publics to put on each firewall. Having to mark the gateway up all the time is not much better than having the line only work on the primary. Anyone figure out how to get a monitor working if the interface is private with a public CARP? Tried adding a nat from private wan, but no joy.

I think this is a GUI issue. From what I've seen if you're using a single WAN IP with CARP, the GUI doesn't have enough options to properly configure apinger. I looked in /var/etc/apinger.conf and it appears the WAN IP is forced as the source IP but unfortunately when doing CARP with one public IP, you have to assign the WAN interface a private IP.

Where can we get a feature request to update the GUI so we can pick the CARP WAN IP or the WAN interface as the source?

tia.

dotdash

I worked around this with the help of another poster. I was able to get the gateway status/failover working by adding an outbound nat rule: WAN 'this firewall' * * * (CARP IP) * NO
Make sure you restart apinger after adding the rule.

bbrendon

Thanks. There is already a feature request here for it : https://redmine.pfsense.org/issues/4597

thesurf

I tryed this out. Carp with 1 IP on interface with private IPs.
As other Mentioned you need to modify your outbond nat because default rule uses the Interfaces Address. This needs to be changed to the carp IP address.

But some things I came accros:

DHCP Proxy will not work, since it use the Interface IP and DHCP will not find a IP Range. There is no setting to change the CARP of the interface
I have also Issues with async routing on another setup. There I had the interfaces and the carp in the same network (classig 3 IP setup). To switch over to the 1 IP carp setup didn't resolve it.

k-bull

I, myself, was trying to do the same exact thing. I found this:

https://b3n.org/pfsense-firewall-ha-failover-cluster/

It worked for me.

I did it with my first machine being hardware and my second being virtual with a managed switch.

Just trying to help here.