For posterity, the problem was initially solved by changing the switch between the two CARP members.

Apparently the Aruba-flavoured HP switches call home ( to activate.arubanetworks.com ), and while I'm not sure why this wrecks things for a multi-WAN CARP setup, once the feature was disabled on the switch it ceased causing STP problems.

We only found out because we had to replace the (replaced) CARP switch in a hurry (next planned setup will have redundant switches too).
The guy who arrived first grabbed the first unattended switch he could find (it was the HP), without knowing the problem it initially caused, and was just happy to have it already configured with the correct VLAN groups. Asking around, it turned out the only change in configuration was the mentioned call home feature being disabled.

Hope this helps someone. It had me banging my head against the rack for way too long.

Works best if you do.

In general, if it is worth HA it is worth doing right. Especially if you are responsible for moving multi-gigabits of traffic around.

Okay seemd 18.9 ist the 2.4.4._p2 and 19.1 ist 2.4.4_p3.
Both system have installed the p3 release according to /conf/upgrade_log

[16/16] Upgrading pfSense from 2.4.4_2 to 2.4.4_3...
[16/16] Extracting pfSense-2.4.4_3: ..... done

@Mode: Please move to install / Update category

Problem "solved".
I have monitoring on my wan gw and both on my core router.
I have disabled the monitoring on my wan gw and the error gone. So if you only have 1 public ip the gw monitoring should be off. Not the best solution but this workes only.

Sounds like your ISP is not compatible with CARP. Some aren't.

https://forum.netgate.com/topic/134297/cox-and-the-carp-mac/
https://forum.netgate.com/topic/146254/carp-outgoing-traffic-black-hole/

Thank you for your response. We are going to use CARP.
We will also build LAGGs for upstream and downstream links so the probability for failure should be pretty low.
We thought about using BGP because our upstream devices can handle that and because it would mean less cabling / ports.

Just FYI I got an answer to this, just not the one I wanted. See my response in https://forum.netgate.com/topic/134297/cox-and-the-carp-mac/17

Yeah that's too bad. Thanks for pursuing it further and reporting back.

@zimmy6996 said in CARP/HA VMWARE ESXi 6.0.0 - Breaking HA after latest ESX patching ....:

Net.ReverseProsCheck

Hey there Zimmy, i am setting up a similar setup within my vmware environment for HA, i have a couple questions for you. 1. where can i find the Net.ReverseProsCheck setting on my host? 2. on your secondary (slave) pfsense vm node did you configure all of the interfaces with an ip or only config the lan and carp interfaces. for example my primary pfsense has about 8 different networks: lan/wifi/wan/sonos/etc do i need to recreate all of these interfaces and set them with a static ip on the secondary box? Thank you in advanced any bit of guidance you can provide would be greatly appreciated.

Thanks for the insight.

The other traffic on that physical interface will be negletible (only management data), so we will go for redundancy with VLAN.

OK, so I didn’t manage to work out what was specifically causing the problem. I was using a relatively old version of pfsense (2.3.3). I downloaded the latest version and redid the setup from scratch, and it just worked!

@Yathus said in Additionnal Subnet /29 over a PPPOE Connection:

I tried with /32 one by one, not working too.

Finally i re-add all IP from my block, one by one, /32 over Locahost interface and now i can ping !

Over WAN interface it's not working...

I captured on all interfaces (not only the one connected to ISP) and both firewalls. Only my firewalls advertised VHIDs. Could not spot any foreign MACs...
Moreover beside VHID=3, also 5 caused the same issue.

If the advskew is 254 it is almost certainly in maintenance mode. The unit will not fail over unless it loses an interface on link down. It will not fail over on "OpenVPN service, VM or WAN failure." I am not sure what that means exactly.

@Pavel88 You know that this Screenshot is about OPNsense, not pfSense and that you're probably in the wrong forums?

Besides that, deleting any automatically created NAT rules and replacing them with "any" isn't recommended on either platform. Why did you remove them? It has a reason, why we differentiate localhost 127.0.0.1 and the LAN network when doing outbound NAT as @viragomann is absolutely right above. Without the right outbound NAT for 127.0.0.1 (-> has to be WAN address) there will be no internet on the standby node as it can't translate to your WAN VIP without being master.