generally that's because you're NATing the secondary firewall's Internet traffic to a CARP IP. Not clear from that screenshot which rule would be doing that given I'm not sure what the WAN IP is, but just make sure you're not NATing traffic sourced from either firewall's WAN IP.

The docs recommend setting (or, rather, not changing) the source away from the specific networks.

Using "any" as a source for outbound NAT is almost always going to have bad and/or unintended results, with or without CARP, but especially with CARP.

If you want to get away with only using one outbound NAT rule, make an alias containing your local/internal networks and use that as the source, not 'any'.

@galphanet:

BUT this give the active gateway access to Internet, not the secondary one.
[/quote
correct, because the secondary doesn't have the CARP addresss. You would need internet access for the secondary addresses if you need both firewalls on the internet all the time.

I have fixed my weird web interface time outs now. It seems just one was being particularly slow, the BACKUP CARP device, the MASTER was OK.

I took the compact flash card out of the BACKUP ALIX board and put it in a third board of the exact same spec, still slow. I then wiped and re-flashed the card, placed it back into it's original BACKUP ALIX board, and restored the config from which I had backed up to my laptop. Now its as fast as the MASTER and everything is fine, so perhaps it was a bad flash.

That may seem a bit obvious, and a waist of time posting here, but we have 40+ ALIX & WRAP boards, this has happened before.

Hopefully that will help someone else!

Cheers,
JamesB.

It's best to have both ISPs connected to both units, that way you get the benefit of Multi-WAN redundancy and CARP. If both IPs are dynamic, that's not necessarily fatal if you're willing to put up with double NAT. You can put the ISP modems into router mode, setup a "DMZ" in them to point all traffic at the CARP address you make for that WAN, and so long as easy ISP modem/router is using a different subnet and a separate switch/VLAN, and you set a monitor IP on each to something on the Internet somewhere, it can still work.

Not as pretty as having a /29 to use on each WAN, but it would get the job done.

You could use a separate ISP on each CARP node, but you wouldn't get Multi-WAN failover. If the ISP on WAN1 failed, it wouldn't make the cluster fail to the secondary node unless you power off the modem there manually.

Nearly all of those are cases of settings that do not sync.

Only the specific settings listed in the CARP/HA sync options will sync, and that does not include anything in System > General, System > Advanced, interface settings, and so on.

Make sure you have all of the areas checked that you want to sync, or they won't sync.

@nicolas010:

Hi all, I have this problem, I already have 2 pfSense in CARP with 4 interfaces each machine, but I need to apply CARP to other 4 pfSense with 3 interfaces each, but they are not phisically near the first 2. And these other 4 should be master as well […]

If CARP interfaces don't match in same network segments it make no sense to "carp" over both networks ?

I can guess what you want - as our 2 office setup it would be nice to configure with 1 Master "configuration" machine both places with each master/slave failover.
But on the other hand it's better to have both places their own CARP regulation because you can have only up to 256 CARP addresses per Setup => on /24 and your finished ;)

I've found here in forum some weeks ago a hint that there is a plan (sometimes in this century? ;)) to deploy a "cluster"-configuration tool for pfsense so that many pfsense clusters can managed at one place… that would be great ;)

@cmb:

1:1 NAT doesn't expose everything, only what your firewall rules permit. With CARP or IP alias VIPs you can let the firewall respond to pings on them.

:o
PERFECT !

Works great.
::) I just assumed that 1:1 opened everything…  ::)

Thank You so much...
:D

Thanks jimp, seems so simple when it's written down so well  ;)

It probably doesn't work on all ISP, my ISP runs proxy ARP for the whole subnet that I'm in. So you would have to filter that on the device between your firewall and your ISP I think. Not very clean, if you ask me.

But besides that, yes this can work if you can separate out what traffic is sent for CARP and the internet, because you would need traffic going to the multicast IP for CARP to not be coming from the VIP, of course.

should be possible with rules like:
src: 127.0.0.0/8 dst: 224.0.0.0/8 uses IP: real IP
src: 127.0.0.0/8 dst: !224.0.0.0/8 uses IP: VIP

(you probably need some more, and I don't think pfSense currently allows this)

This is what it looks like on my side:

3 MACs with an IP each:

192.168.3.203 08:00:27:fe:07:7d v-pfSense1.home.xxxx.net WAN < "physical" box1 
192.168.3.204 08:00:27:68:d9:26 v-pfSense2.home.xxxx.net WAN < "physical" box2
192.168.3.201 00:00:5e:00:01:02  << Virtual IP (as seen from other device on WAN)

you don't need to mess with MAC addresses, it uses the physical ones and creates one for the virtual IP.

Are you doing NAT or classical routing on the WAN?
What is the gateway for the device on the WAN interface in case of classical routing (should be the VIP)

A layout/IP plan would help if further help is needed.

is the WAN2 connected to the ISP equipment directly? they might be running some settings that aren't compatible with carp (proxy arp? multicast filtering?)

can you try with just a (dumb)switch as WAN2?

You can find the settings:
Hyper-V manager, right click virtual machine/settings
Open Network card (click + sign)
Advanced Features, Enable MAC adres spoofing.

i'll have try'd it but it is still nog working….
Please help.

@jnex26:

Well I've partly answered my own question,

Carpdev does not seem to have been implemented yet on pfsense

So how do you configure an ordered failover of based upon bgp sessions ?

which interface won't work? ;)

[2.1-BETA1][root@gw1.zws8.local]/root(32): ifconfig -g carp
pfsync0
wan_vip211
wan_vip212
lan_vip213
lan_vip214
opt2_vip215
wan_vip216
wan_vip217

looks good. I guess you haven't found this (I searched long time to find it):

[2.1-BETA1][root@gw1.zws8.local]/root(33): sysctl -a | grep carp
…
net.inet.ip.same_prefix_carp_only: 0
net.inet.carp.allow: 1
net.inet.carp.preempt: 1              <<=== this option must be set under Advanced =>  System Tunables
net.inet.carp.log: 1
net.inet.carp.arpbalance: 0
net.inet.carp.suppress_preempt: 0
net.link.ether.inet.carp_mac: 0

Okay..

VIP ok.

NAT rule, not okay….

Http redirect to Https is OHN!  (Oh hell no!)

Rules changed to reflect HTTPS on VIP and to HTTPS on LAN IP  ...  Works now on both LAN and connecting from outside.

Jits.

RTFM ;)

http://doc.pfsense.org/index.php/CARP_Configuration_Troubleshooting#VMware_ESX.2FESXi_Users

seems you forgotten

If you have multiple physical ports on the same vswitch, you must enable the Net.ReversePathFwdCheckPromisc option to work around a vswitch bug where multicast traffic will loop back to the host, causing CARP to not function with "link states coalesced" messages. (See below)

with perhaps("(see below)" the need to switch off/on promiscous mode on every vhost to enable this.

@IcePick:

After making no headway with the ebgp/carp master issues we stopped trying to set the next hop to the carp IP in the announcement from the pfsense cluster.
We are now setting the next hop with a filter on the upstream router.

yes, was our  first solution here, too till I found out why it happened on my side.

Problem was that even the read in output didn't helped much to understand why it won't work the "logical" way:

bgpd -v -n -f /var/etc/openbgpd/bgpd.conf

I guess there is a very special order of filtering rules but they are not officially explained (or I haven't them found)…

But Setting HOP on peer side should be good enough ;)

Bests