@bp81 said in Carp Failover not smooth....:

@Kevin-S-Pare said in Carp Failover not smooth....:

I have carp setup, states syncing etc.

but when I do a traceroute to the natted IP, it hits the router ip first. so when I do a carp failover it doesn't use the states and it drops a few packets.

So what am I missing here? at this point we stopped syncing states becuase there really isnt much point the failover works all the same.

from an external machine, you can see the ip change for the hop just before, which is the wan interface of the router.

I missing something but not sure what!

I'm a little unclear on your exact configuration here. Your first statement was when doing a traceroute (I assume you are doing this from a host outside the router's LAN network, ie, across an internet connection) you're seeing the packets go to the WAN address of the router (the WAN address, NOT the CARP address) then the final hop after the router's WAN address is to the CARP address. Is my perception here correct?

@Kevin-S-Pare said in Carp Failover not smooth....:

@SteveITS One router is a Netgate 8200, the other is a Netgate 6100.

Makes no difference if I sync states or not when I ping the ip of the webserver, the packet loss is the same.
Ironically, when I fail back, it comes back smoothly.

Both routers are running 24.11

We host alot of citrix connections and thats where we notice it most.

Typically it is not recommended to run two different pieces of hardware in a CARP/HA cluster. Sometimes you can get away with it, sometimes you can't, and sometimes it kind of works but has issues.

Its primarily two different network cards. but they are the same card. I've tested it with identical devices and the same issue happens.

anyway, big picture, we walked away from this. in the event of a router failure we will have packet loss anyway and it will failover. the only advantage was to fail the router live during the day for maintenance. We really dont need to do this. the goal of a redundant router has been fulfilled. if we really need to do a firmware upgrade we can just failover at night when usage/imact is low. not a big deal.

@Xando said in Incorrect definition of CARP roles:

What is your version of pfsense?

It does run on 2.7.2 CE, I really suspect Hyper-V - QEMU combination.

Do you have the patients and/or time to setup the backup node on Hyper-V (export the config of the backup node, import on another Hyper-V machine)?

Add: Or a package capture, although I haven't done that for CARP and don't know what to expect.

@jason0 I am just facing the same issue! Any tricks to get GIF over CARP?

@wickeren can share pictures of the CARP interface configs, the sync interface and the relevant firewall rules?

@awebster

You must uncheck Synchronize Vitual IPs in the System -> High Avail. Sync, otherwise the MASTER will keep overwriting the ADVBASE value.
This also means you must manually configure the VIP address on each box, initially check the Synchronize Vitual IPs when you do the setup, then uncheck it to go into production, and never check it again.

In the year 2025 I found this comment which resolved my problem!

@michmoor said in CARP alternative:

if the secondary firewall needs to install patches/packages, is that when you just flip it to Master (One WAN IP being shared).

It needs to have packages and updates at all times, not just when it's master. Otherwise you'd have to fail over to it to do any sort of maintenance, which defeats the idea of HA to reduce disruptions.

@bp81 I ended up performing the addition using the first method you mentioned, plus the firewall configuration, and then it worked like a charm. It even added all of the packages to the new node.

Now, I'm dealing with an IPSec speed issue, but that is a whole separate issue and one I have already opened another thread on.

Thank you for taking the time to reply; it is much appreciated, and now if someone else is looking for the same thing, they have some really good options!

Have an excellent day!

TSoF

@Nyxtorm said in Different CARP LAN - WAN unplug behavior:

I'm replying to myself in case anyone has the same case.

On my WAN interfaces, I had a static IPv4 (on the local subnet of my Livebox (French ISP router)), and an IPv6 in DHCP6.

Once I've also set the IPv6, and the gateway v6 to static, the behavior is fine when I disconnect the WAN: the WAN interface goes to INIT, the others to BACKUP on the primary, and the secondary recovers MASTER status on all interfaces.

I believe that this behavior is what you would typically see if one of your WAN interfaces (your IPv6 gateway in this case) is set to DHCP instead of static addressing. CARP/HA doesn't tend to work that well with WANs using DHCP instead of static addressing.

We are also seeing this since we upgraded to 24.11 with all patches applied.
"A communications error occurred while attempting XMLRPC sync." on primary node.

Accessing the webgui on secondary node hangs the firewall after 5-10 seconds.
If we access the CLI everything seems fine and no hangs unless we initiate a reboot, Then the secondary hangs and we need to pull the power to recover.

This happens usually after 5-14 days of uptime of secondary node.

Thanks man, that seems to work very well.

Had to adapt some bits, becaus i only have a single WAN setup and also using Wireguard besides OpenVPN. But it seems to work so far.

Had to add the disable/enable commands for Wireguard, but did not test yet if that works out of the box. But seems like to work, since my phone has a always on 0.0.0.0/0 connection to my firewall and after testing failover my phone still has internet...

It's helpful thanks for sharing.

I am seeing a similar problem on pfSense+ 24.11 (patches applied).

The ADMIN group being REMOVED from user rights assignments on secondary/backup HA cluster members any time the password is changed on the primary member.

I am having to logon to the secondary members and manually add the user(s) back to the ADMIN group.

This is not desired behavior, and I confirmed it is not happening on CE 2.7.2 (patches applied).

@mike_vc

I used to have this issue too, so on every new firewall I setup, I always make sure to add the following values under System, Advanced, System Tunables:

net.inet.carp.preempt 1
net.inet.carp.ifdown_demotion_factor 240

Also, make sure that the primary firewall's CARP skew is 0, and the backup firewall's CARP skew is 100.