Please start a new thread as that is a completely different issue.

To add closure to this issue, the problem went away by resetting sysctl net.inet.carp.demotion from 240 to 0 with: sysctl net.inet.carp.demotion=-240 sysctl net.inet.carp.demotion is essentially a penalty against the advskew settings. Returning this to 0 made the VIPs stable and removed the warning from the CARP status page, though it would recur following a reboot. According to https://forum.pfsense.org/index.php?topic=89132.msg496865#msg496865 the problem is caused when using CARP on a LAGG. When the LAGG is initialised it loses some CARP advertisements and causes net.inet.carp.demotion to be increased by the value of net.inet.carp.senderr_demotion_factor (240). Setting: net.inet.carp.senderr_demotion_factor=0 means that this issue no longer occurs when at boot time and is therefore resolved permanently.

If the settings were truly identical, it would have worked. Something must not have matched. There are many CARP+LAGG systems working OK on pfSense 2.2 (and 2.1).

Dear Christopher, Thank you very much!! Adding the tunable did solve the problem. I rebooted eight times and I experienced no more split brain situations. As with 2.1.5, the machine designated as CARP master was master for all networks after all reboots as long it was on. Before adding the tunable, I needed to reboot about eight times to end up without a split brain situation. I did make two more observations which may be relevant: One of my pair of firewalls is connected to a stacked switch. Of the LAGG with three members, two cables are connected to one switch in the stack and one to the other switch. In that setting, CARP issues did occur more frequently without the tunable. Maybe, the switch interfaces are coming up and down slightly slower due to stack coordination. At the other pair of my firewalls, all three LAGG member cables go to the same switch, as there is only one due to rack space limitations. There, split brain situations did occur without the tunable, but less frequently. After adding the tunable, starting quagga did not work on the backup switch one time, but without practical consequences. Other than that, also starting and stopping quagga does work again after adding the tunable. In general, I feel that a human readable text about CARP changes in 2.2 similar to the examples in the draft 2.1 book would be very helpful. For example, I am still banging my head to get captive portal running on a CARP / LAGG interface again after upgrading to 2.2 (https://forum.pfsense.org/index.php?topic=87991.msg495896#msg495896). Without understanding the changes, that is hard to do. Regards, Michael

and now, the states table for the server : WAN tcp 192.168.1.131:8622 <- 94.23.250.17:35042 SYN_SENT:ESTABLISHED LAN         tcp 94.23.250.17:35042 -> 192.168.1.131:8622 ESTABLISHED:SYN_SENT I do not understand the second line. Why LAN intf and arrow indicating connection from a wan address to the server's address ? But indeed, arrow from server to wan is nowhere to be seen

Issue Resolved, Reinstalled Both firewalls and now everythings working fine.

I'm sorry.  I see CARP and I think CARP.  Now I get it.

I can access the web GUI of my bridged Netgear CG3100D at that same IP address with no rules or other pfSense configuration. To be honest, though, I've never figured out how or why this works.  Pretty sure the old SB 5100 used to as well. I did have to follow the guide to get to my TP-link ADSL modem.

Hi well this setup is an inherited matter so not really sure where to start. I am learning as I go and work on what we've have got at present. So when you say wan carp ip im assuming you are referring to this shared ip that is being used by both firewalls? I would also assume that that particular rule that you have mentioned has been set as we have not been having any reports of it. I will check though. Any other tips of what to check? Also noticed that the ntp time sources on both firewalls are different? Should they be the same or is once suppose to be relying on the other machine? Just logically thinking as if a server time is not syncing, you can't rdp to it. Not sure if that is a contributing issue?

Oh okay, I didn't know it was a known issue. As I said, no big deal at all…

@cmb: IP alias not on a CARP parent Thank you cmb, I see the error. I was adding the IP Alias to the "WAN interface" rather than the "WAN Carp interface". It really wasn't' that obvious when adding the IP Aliases since the default drop down value was already displaying WAN as a selectable item. Once I clicked to see the full list of choices the error was obvious. I had of course read and used the document dotdash refered to as a reference but since it isn't current and contains errors it isn't a document I would refer anyone to. FYI, the LAN ip addresses in the text portion don't correctly reference the ip's in the drawing.  To new people this can be confusing as it was to me at first. If the page ever gets updated (which I assume it will to add the new features of 2.2), maybe it wouldn't hurt to mention that additional IP Alias need to be added to the WAN CARP IP. Thanks for your time, much appreciated.

Since my hoster is also using KVM i found this thread https://forums.freebsd.org/threads/issues-with-carp-under-qemu.22398/ I tried to set sysctl net.inet.carp.drop_echoed but it is not available. Was it renamed or is it just not part of the pfsense kernel anymore?

@ptt: The static ip from my isp is XXX.XXX.334.35 334 ??? are you sure Extremely sure! https://www.youtube.com/watch?v=uHkRda6w-ik ;D ;D ;D

I've set up replicating SANs before and I've always just used a crossover cable for the synchronization interface.  I don't trust another piece of electronic equipment in the mix for no reason, and I've never had a problem with direct-connected interfaces.

@doubledgedboard: So, I'm fine setting up the network aspects, but from my initial research it looks like single-wan-ip CARP doesn't work as well with DHCP WAN IPs. From what I've read, it can be done, but I have to manually assign the gateway on the slave when a failover occurs? Is this true, or is there a better way to handle DHCP WAN? As Jims said - https://forum.pfsense.org/index.php?topic=63319.msg342542#msg342542 so I don't think it would work out of box with DHCP WAN IP, but with single-wan-ip it should work.

Thanks -  if I curl the WAN IP it's returning the internal address, I don't have any rules setup to allow the webconfigurator on the WAN port. Port forwarding + NAT Proxy appears to have worked, I didn't realise there is a difference with 1:1 NAT Thanks for your help!

Then my setup will not work.hmm ugly