Dear Christopher, Thank you very much!! Adding the tunable did solve the problem. I rebooted eight times and I experienced no more split brain situations. As with 2.1.5, the machine designated as CARP master was master for all networks after all reboots as long it was on. Before adding the tunable, I needed to reboot about eight times to end up without a split brain situation. I did make two more observations which may be relevant: One of my pair of firewalls is connected to a stacked switch. Of the LAGG with three members, two cables are connected to one switch in the stack and one to the other switch. In that setting, CARP issues did occur more frequently without the tunable. Maybe, the switch interfaces are coming up and down slightly slower due to stack coordination. At the other pair of my firewalls, all three LAGG member cables go to the same switch, as there is only one due to rack space limitations. There, split brain situations did occur without the tunable, but less frequently. After adding the tunable, starting quagga did not work on the backup switch one time, but without practical consequences. Other than that, also starting and stopping quagga does work again after adding the tunable. In general, I feel that a human readable text about CARP changes in 2.2 similar to the examples in the draft 2.1 book would be very helpful. For example, I am still banging my head to get captive portal running on a CARP / LAGG interface again after upgrading to 2.2 (https://forum.pfsense.org/index.php?topic=87991.msg495896#msg495896). Without understanding the changes, that is hard to do. Regards, Michael

and now, the states table for the server : WAN tcp 192.168.1.131:8622 <- 94.23.250.17:35042 SYN_SENT:ESTABLISHED LAN         tcp 94.23.250.17:35042 -> 192.168.1.131:8622 ESTABLISHED:SYN_SENT I do not understand the second line. Why LAN intf and arrow indicating connection from a wan address to the server's address ? But indeed, arrow from server to wan is nowhere to be seen

Issue Resolved, Reinstalled Both firewalls and now everythings working fine.

I'm sorry.  I see CARP and I think CARP.  Now I get it.

I can access the web GUI of my bridged Netgear CG3100D at that same IP address with no rules or other pfSense configuration. To be honest, though, I've never figured out how or why this works.  Pretty sure the old SB 5100 used to as well. I did have to follow the guide to get to my TP-link ADSL modem.

Hi well this setup is an inherited matter so not really sure where to start. I am learning as I go and work on what we've have got at present. So when you say wan carp ip im assuming you are referring to this shared ip that is being used by both firewalls? I would also assume that that particular rule that you have mentioned has been set as we have not been having any reports of it. I will check though. Any other tips of what to check? Also noticed that the ntp time sources on both firewalls are different? Should they be the same or is once suppose to be relying on the other machine? Just logically thinking as if a server time is not syncing, you can't rdp to it. Not sure if that is a contributing issue?

Oh okay, I didn't know it was a known issue. As I said, no big deal at all…

@cmb: IP alias not on a CARP parent Thank you cmb, I see the error. I was adding the IP Alias to the "WAN interface" rather than the "WAN Carp interface". It really wasn't' that obvious when adding the IP Aliases since the default drop down value was already displaying WAN as a selectable item. Once I clicked to see the full list of choices the error was obvious. I had of course read and used the document dotdash refered to as a reference but since it isn't current and contains errors it isn't a document I would refer anyone to. FYI, the LAN ip addresses in the text portion don't correctly reference the ip's in the drawing.  To new people this can be confusing as it was to me at first. If the page ever gets updated (which I assume it will to add the new features of 2.2), maybe it wouldn't hurt to mention that additional IP Alias need to be added to the WAN CARP IP. Thanks for your time, much appreciated.

Since my hoster is also using KVM i found this thread https://forums.freebsd.org/threads/issues-with-carp-under-qemu.22398/ I tried to set sysctl net.inet.carp.drop_echoed but it is not available. Was it renamed or is it just not part of the pfsense kernel anymore?

@ptt: The static ip from my isp is XXX.XXX.334.35 334 ??? are you sure Extremely sure! https://www.youtube.com/watch?v=uHkRda6w-ik ;D ;D ;D

I've set up replicating SANs before and I've always just used a crossover cable for the synchronization interface.  I don't trust another piece of electronic equipment in the mix for no reason, and I've never had a problem with direct-connected interfaces.

@doubledgedboard: So, I'm fine setting up the network aspects, but from my initial research it looks like single-wan-ip CARP doesn't work as well with DHCP WAN IPs. From what I've read, it can be done, but I have to manually assign the gateway on the slave when a failover occurs? Is this true, or is there a better way to handle DHCP WAN? As Jims said - https://forum.pfsense.org/index.php?topic=63319.msg342542#msg342542 so I don't think it would work out of box with DHCP WAN IP, but with single-wan-ip it should work.

Thanks -  if I curl the WAN IP it's returning the internal address, I don't have any rules setup to allow the webconfigurator on the WAN port. Port forwarding + NAT Proxy appears to have worked, I didn't realise there is a difference with 1:1 NAT Thanks for your help!

Then my setup will not work.hmm ugly

@cmb: Likely from those IPs not working in general on the secondary, assuming they're CARP IPs or IP aliases with a CARP parent. While failed over if you go to Diag>Ping on the secondary, source from one of the affected IPs, and ping out to something on the Internet, does it work? Are these physical boxes, or VMs? Most common reason that comes to mind is VMware without appropriate vswitch config to allow the CARP virtual MACs to be used on the secondary system. These are physical boxes.  I haven't actually tried the Diag>Ping on the secondary when the failover occurs.  I'll do that next time it fails over.  But at least right now, I can ping from an external source both WANs of both pfsense boxes, in addition to the CARP VIP shared between them on each WAN.  If it were a problem from the IPs not working in general, would I not be able to ping the secondary's? For reference, the IPs are set up like so (and as of right now, I can ping all of them externally): BR network: pfsense01:  208.xxx.xxx.171  (NIC's actual address) pfsense02:  208.xxx.xxx.172  (NIC's actual address) BR VIP:        208.xxx.xxx.170  (CARP VIP shared between the two IPs above) CH network: pfsense01:  71.xxx.xxx.19  (NIC's actual address) pfsense02:  71.xxx.xxx.20  (NIC's actual address) BR VIP:        71.xxx.xxx.18  (CARP VIP shared between the two IPs above) @nikkon: @cmb: What type of VPNs? What traffic no longer works? Specific to certain NATed IPs, or? OpenVPN only. In our case, we don't use OpenVPN currently, our site-to-sites are IPSec.

Outbound NAT rule maps all LAN connections to the WAN CARP IP: 172.16.0.1