Hello David, many thanks for answering! It's still a little bit abstract for me, so I think I will 1st configure the existing firewall to also have LTE access fallback and then look into the failover. I will probably follow up with some more specific questions. Best

Good morning forum, I'm just suffering the same question as Andrew M. Robinson. The schema he's proposing seems to be the best one when HA is required both at filtering level (pfSense) and routing level (switches behind pfSense, L3 maybe?). After looking at this thread, it seems that it's posible to create a LAGG link (2 links from pfSense box1 to switch box1, 1 link from pfSense box1 to switch box2 - and same for pfSense box2, 2 connections from pfSense box2 to switch box2 and another one to switch box1), but apparently you would need to have stacking kit between those Catalyst. Question is: is really stacking kit needed here or is it possible to do cross-stack LAGG by just creating an LACP trunk link between the switches? (simulating the stacking kit). Thank you very much, kind regards David

One Interface can only have one MAC address. All VIPs except CARP hooking up on it have the same MAC. But since it is a virtualized pfSense, you may add multiple virtual WAN Interfaces to it. On each one you can set another MAC after.

After reading Documentation found out myself. Enter on both systems  "sysctl net.inet.carp.preempt=0"  in Command Prompt (Web Interface) But be sure about your routing! Maybe nothing will work on one fail. sysctl net.inet.carp.preempt=1  can enable it again

You don't need an alias to use CARP VIPs in other subnets on recent versions. You should be fine deleting the alias IPs, as to why you can't, I don't know. Try deleting the jail carp first, then delete the alias, then re-create your jail carp. Perhaps it is incorrectly referencing the alias ip, you could edit the properties and look.

@dotdash: The interface is WAN. You change the destination from 'WAN address' to your vip via the dropdown. Ahh. I found it. Thanks

Sorry, did not meant to cause offense.  I had not considered the bot issue because responses always seem to come back so quickly.  Your point well made.  I was merely expressing surprise not complaining or anything else but I can see how my meaning was easily misconstrued.

I'm not familiar with HA on Hyper-V, but I don't think disabling one of the interfaces is a valid failover test. I'm not sure how one of the VMs is going to lose link without the other if your hosts are plumbed properly.

What you see is by design. Loss of link is considered a physical failure. A gateway failure would still have link but lose connectivity. If you don't want a modem restart to cause a transition, place a switch between the firewalls and modem(s) (but be sure not to create another single point of failure).

Hello, Make sure that your LANs can talk to each other.(as in LAN 1 on box 1 can talk to LAN 1 on box 2) I know with ESXi, to make pfsense do the VLANing i had to set VLAN ID in the ESXi Switch properties->Virtual machine port group -> General tab -> VLAN ID to All(4095) Hopefully this helps, jammcla

-disable IGMP on switch(smart.L2,L3?) -change skew on secondary(ex skew 101)

So you've only set up synchronisation, but not CARP fialover. Follow this guide: https://doc.pfsense.org/index.php/Configuring_pfSense_Hardware_Redundancy_%28CARP%29

What are you using to balance between the servers? The built-in relayd load balancer? HAProxy? For management purposes you'd always have to connect directly to each individual backend server to query them. You should never attempt to manage anything HA using the failover address, it doesn't matter if it's pfSense or something else. Always address the units individually for management.

If it was the exact same issue I had then the real problem ended up being my testing method.  Although I never understood why yanking a cable isn't a valid test.  When I simply unplug POWER from the primary router the failover is nearly instantaneous.  I never posted a follow up on simulating a switch failure but that also failed over as expected. On another note, I have had to enter maintenance mode 5 times recently so that I could change out some other equipment inline with these.  Three out of five the switchover was nearly 30seconds and the WebInterface on both routers locks up completely which switching over.  Looking at top via ssh during the switchover doesn't show anything locked up nor maxing out RAM/CPU so not sure what the problem was there if it is related at all.

Is looks normal to me, MASTER on first node, Backup on second. After shutdown of the MASTER, the second becomes MASTER To my big surprise, in about 33% of the pings to the CARP VIP I get a reply now, the others time out…..

Hi hackce, what do the logs on the second PFSense say? Perhaps you are using the wrong password or the firewall prevents the synchronization. Did you try to add an "allow all" firewall rule to your sync network for testing? Cheers, Jesper

The clustered hypervisor would protect against node failure by transferring the VM to another running node, but it would not protect against software failure from within the VM. If, for some reason, a CARP'd pfSense instance were to freeze/fail, in this scenario the failover CARP pfSense node should take over. Frankly, I'm not sure which is the more likely scenario: a failing hypervisor node or a failing pfsense instance. In my particular configuration, I'm not clustering my Hyper-V nodes but instead as stand-alone nodes.

In case anyone else needs an answer to a similar problem, while searching for something entirely unrelated, I came across this link https://forum.pfsense.org/index.php?topic=42532.0 In the final post was the solution to my problem which I have cut and pasted from there to here for ease of reference and added some notes of my own relating to the key points. The following also pertained to my situation… "The router had already been in production for a while and had some NAT port forwards configured" "I assumed those rules would carry right over to the CARP setup because the destination was WAN." The following is what got me on the right track .... "I went to make a new rule for some reason or another and noticed that there was a new destination choice called WAN CARP (what I had named that VIP).  When I realized the firewall was discriminating between real IPs and virtual IPs, I had my answer." This bit summed up my situation perfectly too ... "I guess I just assumed that my rules were all per-interface, but they're actually more granular than that.  Changed all my regular stuff to the CARP destination" When I did the above - it worked.  As the original poster sad, it is worth noting that the NAT rules are quite so granular ...