I'm not familiar with HA on Hyper-V, but I don't think disabling one of the interfaces is a valid failover test. I'm not sure how one of the VMs is going to lose link without the other if your hosts are plumbed properly.

What you see is by design. Loss of link is considered a physical failure. A gateway failure would still have link but lose connectivity. If you don't want a modem restart to cause a transition, place a switch between the firewalls and modem(s) (but be sure not to create another single point of failure).

Hello, Make sure that your LANs can talk to each other.(as in LAN 1 on box 1 can talk to LAN 1 on box 2) I know with ESXi, to make pfsense do the VLANing i had to set VLAN ID in the ESXi Switch properties->Virtual machine port group -> General tab -> VLAN ID to All(4095) Hopefully this helps, jammcla

-disable IGMP on switch(smart.L2,L3?) -change skew on secondary(ex skew 101)

So you've only set up synchronisation, but not CARP fialover. Follow this guide: https://doc.pfsense.org/index.php/Configuring_pfSense_Hardware_Redundancy_%28CARP%29

What are you using to balance between the servers? The built-in relayd load balancer? HAProxy? For management purposes you'd always have to connect directly to each individual backend server to query them. You should never attempt to manage anything HA using the failover address, it doesn't matter if it's pfSense or something else. Always address the units individually for management.

If it was the exact same issue I had then the real problem ended up being my testing method.  Although I never understood why yanking a cable isn't a valid test.  When I simply unplug POWER from the primary router the failover is nearly instantaneous.  I never posted a follow up on simulating a switch failure but that also failed over as expected. On another note, I have had to enter maintenance mode 5 times recently so that I could change out some other equipment inline with these.  Three out of five the switchover was nearly 30seconds and the WebInterface on both routers locks up completely which switching over.  Looking at top via ssh during the switchover doesn't show anything locked up nor maxing out RAM/CPU so not sure what the problem was there if it is related at all.

Is looks normal to me, MASTER on first node, Backup on second. After shutdown of the MASTER, the second becomes MASTER To my big surprise, in about 33% of the pings to the CARP VIP I get a reply now, the others time out…..

Hi hackce, what do the logs on the second PFSense say? Perhaps you are using the wrong password or the firewall prevents the synchronization. Did you try to add an "allow all" firewall rule to your sync network for testing? Cheers, Jesper

The clustered hypervisor would protect against node failure by transferring the VM to another running node, but it would not protect against software failure from within the VM. If, for some reason, a CARP'd pfSense instance were to freeze/fail, in this scenario the failover CARP pfSense node should take over. Frankly, I'm not sure which is the more likely scenario: a failing hypervisor node or a failing pfsense instance. In my particular configuration, I'm not clustering my Hyper-V nodes but instead as stand-alone nodes.

In case anyone else needs an answer to a similar problem, while searching for something entirely unrelated, I came across this link https://forum.pfsense.org/index.php?topic=42532.0 In the final post was the solution to my problem which I have cut and pasted from there to here for ease of reference and added some notes of my own relating to the key points. The following also pertained to my situation… "The router had already been in production for a while and had some NAT port forwards configured" "I assumed those rules would carry right over to the CARP setup because the destination was WAN." The following is what got me on the right track .... "I went to make a new rule for some reason or another and noticed that there was a new destination choice called WAN CARP (what I had named that VIP).  When I realized the firewall was discriminating between real IPs and virtual IPs, I had my answer." This bit summed up my situation perfectly too ... "I guess I just assumed that my rules were all per-interface, but they're actually more granular than that.  Changed all my regular stuff to the CARP destination" When I did the above - it worked.  As the original poster sad, it is worth noting that the NAT rules are quite so granular ...

Build all the interfaces first. Make them exactly match the primary, in the same order, but with a different interface address, obviously.

Sorry, my mistake. I missed one thing clearly written on the ufficial guide: the states syncronization MUST be enabled on the slave node too! After enabling this everything workey, now my OpenVPN/SSH connections remain up&running even if I shutdown the primary node, pretty impressive :) Thanks all for your help!

We use pfSense version '2.3.2_1' and pfBlockerNG '2.1.1_4', we don't experience this issue, check you're on the latest version. If you're using a dedicated NIC for XMLRPC (recommended) you should use a cross-over cable and set the speed\duplex to 100base TX with static IPs. Enter the following settings for pfBlockerNG -> Sync (might be an issue with it using the generic settings): Enable Sync: Sync to hosts defined below Protocol: HTTPS Target IP/Hostname: backup pfSense IP Target Port: 443 Target Password: your admin password We had a few problems when we initially built our environment on Broadcom NICs, we now use Intel. Hope this helps.

That ticket was referring to the keep-alive pinger process, which is what I already mentioned. The two systems check heartbeats but that's at a completely different level than IPsec. For 99.9 of people it works fine as-is.  For someone with a misconfigured network it'll have a problem, like you had, but there is so little benefit to "solving" this corner case it's just not worth doing. It could negatively impact cases that are working fine now.

Thank you viragomann, it make sense :D, I'll start from CARP. SenseRider

Yes, you can drive 4 pfSense in CARP mode and sync configuration from one to the other, but not over internet, this doesn't make any sense anyway. For CARP all interfaces sharing the same VIP has to be connected to the same switch. On each box you have to set a different skew value for the VIP, that one with the lower skew has the higher priority. For syncing you can only sync from one the another, so you can sync from the first (master) to the second and from the second to the third and so on.