HA CARP setup, WAN not working

pecker88

I followed this guide verbatim:
https://vorkbaard.nl/how-to-set-up-pfsense-high-availability-hardware-redundancy

I have 5 static IPs, (/24 block) and dedicated hardware for both pfSense boxes. Each WAN interface has its own static IP. I'm able to reach the internet on both boxes, no issues. SYNC interface works fine, LAN CARP VIP works fine also.

primary FW:
1 static IP on WAN interface
1 CARP VIP on WAN interface
2 IP Aliases on CARP VIP

When I enter CARP maintenance mode on primary FW, all CARP interfaces show as "Master" on secondary FW, as expected. LAN communication continues to work. But, continuous WAN pings to google DNS IP on a client fails. The secondary FW is not able to reach the internet via the outbound NAT CARP VIP.

I enabled firewall logging on inbound rules that fwd traffic to various web/application servers. And, those firewall log entries only appear on the primary FW. I'm not seeing incoming traffic for those rules on the secondary FW. It's like they are still being passed to the primary FW, even though it's CARP status is "backup"?

I did a packet capture on WAN, primary/secondary firewalls and didn't see any duplicate VHIDs.

I don't know what else to try?

Derelict

I would suggest following this:

https://docs.netgate.com/pfsense/en/latest/solutions/reference/highavailability/index.html

CARP issues are pretty invariably a misconfiguration or a layer 2 switching problem.

awebster

As @Derelict suggests, consult the docs, and to that I would add that it is important to remember that at the base of Ethernet communication, it is between two mac addresses, layer 3 comes after, consequently, you need to make sure that the layer 2, mac address visibility is as you expect it to be on the devices facing the CARP cluster, both upstream and downstream, and as well as on pfSense boxes.
Packet captures (detail level full) are a great way to check this, and pay particular attention to the ethernet addresses.