If your settings are not syncing, check Firewall > Virtual IPs, on the CARP Settings tab. On 2.0 the whole lower 2/3 of the page is for config sync. Put in the IP, password, and check all of the boxes on the master. Leave that whole section empty on the slave.

I forgot to add the Pfsense box is a virtual machine on our hypervisor and currently has 2 network cards. One network card is called LAN and the other is WAN.

You can only properly do CARP failover with static IP WANs, it wouldn't fail over properly and keep connections alive with a PPPoE WAN. Also I don't believe the second WAN can be on-demand when used with multi-wan. pfSense constantly pings the gateways of all WANs to ensure they are usable, and there isn't a mechanism currently to handle on being left down until needed. This is in the works, though, because it would be useful for 3G connections where bandwidth is expensive.

If they're routed to you, you don't need anything at layer 2 ("Other" VIPs will suffice). To use that with CARP, have the ISP route the IP blocks to a CARP IP on your WAN subnet.

CARP IPs must be within the subnet of the interface's IP. That's not going to change in the near future.

Yep, you got it there at the end. You have one CARP VIP that has the VHID - announcements happen there, and the other IP Alias type VIPs sit on top of that CARP VIP. They all fail in a group with the CARP VIP. Actually faster than they would individually because they don't need <x>advertisements per second, where <x>is your number of IPs, just the one of the CARP VIP. Less headache, less VHIDs, less network traffic. It's a very nice way to do it.</x></x>

Did the update from last night and now it works great > "Reboot" and "Halt System" on both pfsense :)

The hardware will work as long as you have the same number of interfaces and they are assigned in the same order. Doesn't matter if they are different drivers or types. On each interface, both boxes need an IP and then the shared IP, so at least three IPs in every subnet.

Hi Jim, thanks for the answer, I agree. I'll take a look at implementing this if no one beats me to it.

Thank you very much, I'll give it a try.

Ok, how I resolved it. First I tried created a new router from the 1.2.3 VM, and restored the configuration from the original router.  This router had the same behavior as the failed router. Second I copied the running secondary router, made a few adjustments to the config so that it was the primary and it worked. It has now been running for 12+ hours with no problem. I would guess, that I had some config issues that I missed. User error again!

If the /29 is routed to your CARP VIP, and not the WAN IP, it should follow from one box to the other. I'm not sure about the load balancer, I thought that failed over as well but I haven't tried it myself.

Never heard of anything like that, I setup at least a couple CARP installs every week with VoIP behind them, and have done some for VoIP providers with thousands of simultaneous calls going through. You have to uninstall open-vm-tools package before upgrading or you'll end up in a panic loop. Could be a problem with a conflicting VHID, or any number of other things, hard to say without digging into packet captures.

Can your pbx use stun to let the server determine the wan ip? If you need a stun server setting you can use stun.sipgate.net:10000 although your itsp may have their own or there may be one closer to you.

A few questions. "every time I restart one of the boxes, I need to get into the carp settings and click save for the Sync states so it returns to work." Q1. Which box do you have to log into, the box you restarted or the box that is currently running? Q2. Can you post the settings you have enabled on the "Carp Settings" tab? Q3. How are you checking the states, just visually? I ask the last question because my states are set to sync and they do, however they do not line up line for line.  Just wondering if that's what you were expecting.

What kind of detail do you want? There is just shy of a full page in the book that covers the VIP types and how they work. It's section 6.8, page 119 in the print edition. There is a little more info in the doc wiki as well. Given what you've shown there, either Proxy ARP or CARP type VIPs should work for you. You can forward the ports like you describe in #1 just fine. You can't do scenario #2 unless you bridge an interface to WAN. Doable, but bridging can get ugly.

Hi, Many thanks for the answer. I'll try it soon.