Interface Names & Order are identical in Status > Interfaces. I've tried both CARP and interface addresses and neither the CARP nor the Master responds to pings or is in the ARP cache. Nothing on the consoles either, other than login events. Your last paragraph mentions something that puzzles me too. The backup doesn't take over or detect the master is down. However when tested by disabling CARP on the Master the backup does take over. After rebooting the Master this morning I see a reconfiguration occurred on Saturday afternoon. That's the only entry between the reboots on Friday morning and this morning. I'll try disabling the firewall rule sync to see if that helps.

I am also experiencing this issue after I upgraded 4 boxes running various levels of 2.3.1 up to 2.3.2. Same environment: pfsense on ESXi 5.5/6.0 hosts. On my end though, It might be an issue with how Suricata is using the XMLRPC sync? Appears to be breaking once a rules refresh is complete. Suricata is setup to NOT ask the target slave to refresh their own rules. Haven't had any XMLRPC errors from the other pair of 2.3.2 VMs that DO NOT have Suricata Sep 7 00:31:49 php-fpm 7073 /rc.filter_synchronize: New alert found: A communications error occurred while attempting XMLRPC sync with username admin https://192.168.254.2:443. Sep 7 00:31:49 php-fpm 7073 /rc.filter_synchronize: A communications error occurred while attempting XMLRPC sync with username admin https://192.168.254.2:443. Sep 7 00:31:49 php-fpm 7073 /rc.filter_synchronize: XML_RPC_Client: Connection to RPC server 192.168.254.2:443 failed. Operation timed out 103 Sep 7 00:30:33 check_reload_status Syncing firewall Sep 7 00:30:33 php-cgi suricata_check_for_rule_updates.php: [Suricata] The Rules update has finished. Sep 7 00:30:29 php-cgi suricata_check_for_rule_updates.php: [Suricata] Snort GPLv2 Community Rules are up to date... Sep 7 00:30:28 php-cgi suricata_check_for_rule_updates.php: [Suricata] Snort VRT rules file update downloaded successfully. Sep 7 00:30:09 php-cgi suricata_check_for_rule_updates.php: [Suricata] There is a new set of Snort VRT rules posted. Downloading snortrules-snapshot-2983.tar.gz... Sep 7 00:30:08 php-cgi suricata_check_for_rule_updates.php: [Suricata] Emerging Threats Open rules file update downloaded successfully. Sep 7 00:30:06 php-cgi suricata_check_for_rule_updates.php: [Suricata] There is a new set of Emerging Threats Open rules posted. Downloading emerging.rules.tar.gz... Sep 6 17:59:01 lonrogfw-bluesteel.voyageurtransportation.ca nginx: 2016/09/06 17:59:01 [error] 57076#100061: *2654 upstream timed out (60: Operation timed out) while reading response header from upstream, client: 10.1.100.7, server: , request: "POST /widgets/widgets/ipsec.widget.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php-fpm.socket", host: "10.1.100.30", referrer: "https://10.1.100.30/" Sep 6 17:27:55 php-cgi rc.restart_webgui: Creating rrd update script Sep 6 17:27:53 rc.php-fpm_restart 54531 >>> Restarting php-fpm Sep 6 17:27:50 lonrogfw-bluesteel.voyageurtransportation.ca nginx: 2016/09/06 17:27:50 [alert] 28036#100081: *38216 kevent() reported about an closed connection (53: Software caused connection abort) while reading response header from upstream, client: 10.1.100.7, server: , request: "GET / HTTP/1.1", upstream: "fastcgi://unix:/var/run/php-fpm.socket:", host: "10.1.100.30" Sep 6 17:27:47 login login on ttyv0 as root Sep 6 17:26:50 php-fpm 44517 /pkg_edit.php: [suricata] XMLRPC sync sending auto-SID conf files to https://192.168.254.2:443. Sep 6 17:26:50 php-fpm 44517 /pkg_edit.php: New alert found: A communications error occurred while attempting Suricata XMLRPC sync with https://192.168.254.2:443\. Failed to transfer file: enablesid-sample.conf Sep 6 17:26:50 php-fpm 44517 /pkg_edit.php: A communications error occurred while attempting Suricata XMLRPC sync with https://192.168.254.2:443\. Failed to transfer file: enablesid-sample.conf Sep 6 17:26:50 php-fpm 44517 /pkg_edit.php: XML_RPC_Client: Connection to RPC server 192.168.254.2:443 failed. Operation timed out 103 Sep 6 17:26:05 lonrogfw-bluesteel.voyageurtransportation.ca nginx: 2016/09/06 17:26:05 [error] 28036#100081: *38216 upstream timed out (60: Operation timed out) while reading response header from upstream, client: 10.1.100.7, server: , request: "POST /pkg_edit.php?xml=suricata/suricata_sync.xml HTTP/1.1", upstream: "fastcgi://unix:/var/run/php-fpm.socket", host: "10.1.100.30", referrer: "https://10.1.100.30/pkg_edit.php?xml=suricata/suricata_sync.xml" Sep 6 17:25:35 php-fpm 44517 /pkg_edit.php: [suricata] XMLRPC sync sending auto-SID conf files to https://192.168.254.2:443. Sep 6 17:25:35 php-fpm 44517 /pkg_edit.php: New alert found: A communications error occurred while attempting Suricata XMLRPC sync with https://192.168.254.2:443\. Failed to transfer file: dropsid-sample.conf Sep 6 17:25:35 php-fpm 44517 /pkg_edit.php: A communications error occurred while attempting Suricata XMLRPC sync with https://192.168.254.2:443\. Failed to transfer file: dropsid-sample.conf Sep 6 17:25:35 php-fpm 44517 /pkg_edit.php: XML_RPC_Client: Connection to RPC server 192.168.254.2:443 failed. Operation timed out 103 Sep 6 17:24:20 php-fpm 44517 /pkg_edit.php: [suricata] XMLRPC sync sending auto-SID conf files to https://192.168.254.2:443. Sep 6 17:24:20 php-fpm 44517 /pkg_edit.php: New alert found: A communications error occurred while attempting Suricata XMLRPC sync with https://192.168.254.2:443\. Failed to transfer file: disablesid-sample.conf Sep 6 17:24:20 php-fpm 44517 /pkg_edit.php: A communications error occurred while attempting Suricata XMLRPC sync with https://192.168.254.2:443\. Failed to transfer file: disablesid-sample.conf Sep 6 17:24:20 php-fpm 44517 /pkg_edit.php: XML_RPC_Client: Connection to RPC server 192.168.254.2:443 failed. Operation timed out 103 Sep 6 17:23:05 php-fpm 44517 /pkg_edit.php: [suricata] XMLRPC sync sending auto-SID conf files to https://192.168.254.2:443. Sep 6 17:23:05 php-fpm 44517 /pkg_edit.php: [suricata] XMLRPC sync is starting. Sep 6 17:23:05 check_reload_status Syncing firewall Sep 6 17:23:05 check_reload_status Syncing firewall Sep 6 12:31:22 php-fpm 49586 /rc.filter_synchronize: New alert found: A communications error occurred while attempting XMLRPC sync with username admin https://192.168.254.2:443. Sep 6 12:31:22 php-fpm 49586 /rc.filter_synchronize: A communications error occurred while attempting XMLRPC sync with username admin https://192.168.254.2:443. Sep 6 12:31:22 php-fpm 49586 /rc.filter_synchronize: XML_RPC_Client: Connection to RPC server 192.168.254.2:443 failed. Operation timed out 103 Sep 6 12:30:06 check_reload_status Syncing firewall Sep 6 12:30:06 php-cgi suricata_check_for_rule_updates.php: [Suricata] The Rules update has finished. Sep 6 12:30:06 php-cgi suricata_check_for_rule_updates.php: [Suricata] Snort GPLv2 Community Rules are up to date... Sep 6 12:30:06 php-cgi suricata_check_for_rule_updates.php: [Suricata] Snort VRT rules are up to date... Sep 6 12:30:04 php-cgi suricata_check_for_rule_updates.php: [Suricata] Emerging Threats Open rules are up to date... Sep 6 12:16:07 lonrogfw-bluesteel.voyageurtransportation.ca nginx: 2016/09/06 12:16:07 [error] 28036#100081: *12636 upstream timed out (60: Operation timed out) while reading response header from upstream, client: 10.1.100.7, server: , request: "POST /widgets/widgets/ipsec.widget.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php-fpm.socket", host: "10.1.100.30", referrer: "https://10.1.100.30/"

Got it debugged by the switch provider and they made a new firmware for the switches were it works :)

No, I am not complaining about ISC, I know the server is widely used in many Linux/Unix environments. I just wanted to know if this was a missconfiguration on my side or normal behaviour.

Because that's the way ISC DHCPD works in failover mode.

Thank you VIragomann. I will try this as soon as the company opens again after summer break ;)

Fixed. I had dual connections active/active from VMWare to the 3750 switch.  I had to setup a channel group on the switch and set the vSwitch to Route based IP hash on top of the security settings.  Not sure why it was working with the similar setup on the Primary server, but both are now setup with the correct load balance settings.

You can't have WAN and LAN in the same subnet. And if that is a bridge, then you do not want IP addresses on both WAN and LAN. But I hope it's not a bridge, since bridge+carp = big mistake.

Thanks for your help ! I finally got it to work but honestly not really sure what was the issue. On my pfsense2 i changed the LAN ip and the WAN IP. In the NAT rule i changed several times back and forth the translation address from interface address to 192.168.0.2, rebooted the Humax modem and it worked finally. When i turn off the pfsense1 i will keep having internet with pfsense2, when pfsense 1 is back online it is still working as well… I think this is solved. Thanks a lot for your help and sorry i bothered you with this !

Is there any way to work around this issue?  Perhaps some script that I can modify which gets called any time dhcpv6.conf and radvd.conf is written?

That is expected behavior. When the two units have properly configured DHCP servers, they both hand out leases and they share lease information. They each agree on a portion of the address space to serve. The above assumes you have filled in the "Failover Peer IP" on the master (and that it has been copied to the secondary during config sync). If you left that out, then filling it in will correct the problem.

As long as you can get a link from node to node it should work. Otherwise you might have to run it through a switch on an isolated VLAN. At heart, not that much different than any other NIC except for the physical connections.

To clarify: The above isn't mean to be rude, but a statement of experience. At my previous job I ran an HA pair for years that was bridged and it was a never-ending nightmare of babysitting switches, some things not working during a primary failure, mysterious network issues, etc. I bit the bullet and redesigned the entire network to use routing and that same setup has had zero problems since, other than an unrelated hardware failure.

That is expected behavior. When the two units have properly configured DHCP servers, they both hand out leases and they share lease information. They each agree on a portion of the address space to serve. The above assumes you have filled in the "Failover Peer IP" on the master (and that it has been copied to the secondary during config sync). If you left that out, then filling it in will correct the problem.