I have fixed my weird web interface time outs now. It seems just one was being particularly slow, the BACKUP CARP device, the MASTER was OK. I took the compact flash card out of the BACKUP ALIX board and put it in a third board of the exact same spec, still slow. I then wiped and re-flashed the card, placed it back into it's original BACKUP ALIX board, and restored the config from which I had backed up to my laptop. Now its as fast as the MASTER and everything is fine, so perhaps it was a bad flash. That may seem a bit obvious, and a waist of time posting here, but we have 40+ ALIX & WRAP boards, this has happened before. Hopefully that will help someone else! Cheers, JamesB.

It's best to have both ISPs connected to both units, that way you get the benefit of Multi-WAN redundancy and CARP. If both IPs are dynamic, that's not necessarily fatal if you're willing to put up with double NAT. You can put the ISP modems into router mode, setup a "DMZ" in them to point all traffic at the CARP address you make for that WAN, and so long as easy ISP modem/router is using a different subnet and a separate switch/VLAN, and you set a monitor IP on each to something on the Internet somewhere, it can still work. Not as pretty as having a /29 to use on each WAN, but it would get the job done. You could use a separate ISP on each CARP node, but you wouldn't get Multi-WAN failover. If the ISP on WAN1 failed, it wouldn't make the cluster fail to the secondary node unless you power off the modem there manually.

Nearly all of those are cases of settings that do not sync. Only the specific settings listed in the CARP/HA sync options will sync, and that does not include anything in System > General, System > Advanced, interface settings, and so on. Make sure you have all of the areas checked that you want to sync, or they won't sync.

@nicolas010: Hi all, I have this problem, I already have 2 pfSense in CARP with 4 interfaces each machine, but I need to apply CARP to other 4 pfSense with 3 interfaces each, but they are not phisically near the first 2. And these other 4 should be master as well […] If CARP interfaces don't match in same network segments it make no sense to "carp" over both networks ? I can guess what you want - as our 2 office setup it would be nice to configure with 1 Master "configuration" machine both places with each master/slave failover. But on the other hand it's better to have both places their own CARP regulation because you can have only up to 256 CARP addresses per Setup => on /24 and your finished ;) I've found here in forum some weeks ago a hint that there is a plan (sometimes in this century? ;)) to deploy a "cluster"-configuration tool for pfsense so that many pfsense clusters can managed at one place… that would be great ;)

@cmb: 1:1 NAT doesn't expose everything, only what your firewall rules permit. With CARP or IP alias VIPs you can let the firewall respond to pings on them. :o PERFECT ! Works great. ::) I just assumed that 1:1 opened everything…  ::) Thank You so much... :D

Thanks jimp, seems so simple when it's written down so well  ;)

It probably doesn't work on all ISP, my ISP runs proxy ARP for the whole subnet that I'm in. So you would have to filter that on the device between your firewall and your ISP I think. Not very clean, if you ask me. But besides that, yes this can work if you can separate out what traffic is sent for CARP and the internet, because you would need traffic going to the multicast IP for CARP to not be coming from the VIP, of course. should be possible with rules like: src: 127.0.0.0/8 dst: 224.0.0.0/8 uses IP: real IP src: 127.0.0.0/8 dst: !224.0.0.0/8 uses IP: VIP (you probably need some more, and I don't think pfSense currently allows this)

This is what it looks like on my side: 3 MACs with an IP each: 192.168.3.203 08:00:27:fe:07:7d v-pfSense1.home.xxxx.net WAN < "physical" box1  192.168.3.204 08:00:27:68:d9:26 v-pfSense2.home.xxxx.net WAN < "physical" box2 192.168.3.201 00:00:5e:00:01:02  << Virtual IP (as seen from other device on WAN) you don't need to mess with MAC addresses, it uses the physical ones and creates one for the virtual IP. Are you doing NAT or classical routing on the WAN? What is the gateway for the device on the WAN interface in case of classical routing (should be the VIP) A layout/IP plan would help if further help is needed.

is the WAN2 connected to the ISP equipment directly? they might be running some settings that aren't compatible with carp (proxy arp? multicast filtering?) can you try with just a (dumb)switch as WAN2?

You can find the settings: Hyper-V manager, right click virtual machine/settings Open Network card (click + sign) Advanced Features, Enable MAC adres spoofing. i'll have try'd it but it is still nog working…. Please help.

@jnex26: Well I've partly answered my own question, Carpdev does not seem to have been implemented yet on pfsense So how do you configure an ordered failover of based upon bgp sessions ? which interface won't work? ;) [2.1-BETA1][root@gw1.zws8.local]/root(32): ifconfig -g carp pfsync0 wan_vip211 wan_vip212 lan_vip213 lan_vip214 opt2_vip215 wan_vip216 wan_vip217 looks good. I guess you haven't found this (I searched long time to find it): [2.1-BETA1][root@gw1.zws8.local]/root(33): sysctl -a | grep carp … net.inet.ip.same_prefix_carp_only: 0 net.inet.carp.allow: 1 net.inet.carp.preempt: 1              <<=== this option must be set under Advanced =>  System Tunables net.inet.carp.log: 1 net.inet.carp.arpbalance: 0 net.inet.carp.suppress_preempt: 0 net.link.ether.inet.carp_mac: 0

Okay.. VIP ok. NAT rule, not okay…. Http redirect to Https is OHN!  (Oh hell no!) Rules changed to reflect HTTPS on VIP and to HTTPS on LAN IP  ...  Works now on both LAN and connecting from outside. Jits.

RTFM ;) http://doc.pfsense.org/index.php/CARP_Configuration_Troubleshooting#VMware_ESX.2FESXi_Users seems you forgotten If you have multiple physical ports on the same vswitch, you must enable the Net.ReversePathFwdCheckPromisc option to work around a vswitch bug where multicast traffic will loop back to the host, causing CARP to not function with "link states coalesced" messages. (See below) with perhaps("(see below)" the need to switch off/on promiscous mode on every vhost to enable this.

@IcePick: After making no headway with the ebgp/carp master issues we stopped trying to set the next hop to the carp IP in the announcement from the pfsense cluster. We are now setting the next hop with a filter on the upstream router. yes, was our  first solution here, too till I found out why it happened on my side. Problem was that even the read in output didn't helped much to understand why it won't work the "logical" way: bgpd -v -n -f /var/etc/openbgpd/bgpd.conf I guess there is a very special order of filtering rules but they are not officially explained (or I haven't them found)… But Setting HOP on peer side should be good enough ;) Bests

this thread's been hijacked enough, please start new threads. Locking this. OP if you want to follow up to this please PM me and I'll be glad to unlock. The rest of you, you need your own thread where people can help you troubleshoot without making a mess of someone else's thread.

Thanks for a good news! :)

The workaround: Since trying to find a way to get CARP to startup after Quagga OSPF seemed a bit futile, and obviously FreeBSD has no metrics to use for static routes, I needed another solution. It finally occurred to me that if CARP just wouldn't steal the IP back, then the solution would be pretty obvious.  Leave the IP with whoever has it at that time.  Turns out this works pretty well: Added sysctl: net.inet.carp.preempt = 0 Adjust CARP entries (which in my case included turning off synchronization) to have the advertising base/skew set the same. So, now when my router fails, it's just a matter of a second or two while routing tables update their states to stop trying to send traffic through the failed device.

If you're expecting points (green) 2 & 3 to support VRRP, then no.  There's no VRRP support in pfSense. If you're expecting them to use CARP to each pretend to be (green) .1, then it should work.  Just note that what's past them in the perimeter-net might affect how they work in terms of actually doing failover.  Data has to make it back in, after all, and this is determined by the routing tables, and whether or not the routes from the perimeter-net know that it can take either path (long as it's up) to get back to your subnet. I imagine though, by this time you've either tried it, or abandoned it, since it's been more than a month.  I just happened to be searching for something similar to what I'm trying at work to see if anyone has gotten through a hurdle I'm running into, but that's a post for another topic.