Got busy and haven't had time to tcpdump yet. Any other ideas out there about anything else that can be looked at in the interim?

I have no blade-related experience, but did you put the vswitch ports that connect to your pfsense vm's, and which are to take part in CARP, into a port group that has promiscous mode enabled? I usually create a duplicate port group (same vlan, same vswitch), which has promiscous mode enabled, and put the pfsense interfaces into that port group, and all vm's that use the pfsense as a gateway into the port group with promiscous disabled.

For me it's easier to have only one failover, the setup is so that the slave doesn't have all features (no backup wan connection) so only 1 network doesn't have the failover when there is a network fail. If all networks will switch in depended, I still can switch the master down, all networks will go down and the slave would take over all networks. I have created a stable situation again, I found out when there is an open network (both pfsense are set to init, the network becomes unstable in a couple of hours) But still I want to failover independent, I don't get why the option has been taken out.

Also see thread http://forum.pfsense.org/index.php/topic,68439.0.html, if the suggestions in this thread don't help as they may be similar issues.

So, I turn on spanning tree protocol on both sides of the bridge, to include the wifi, vlan, pfsync and bridge interface…and it all works.

@jimp: Yes. Perfecto! Thanks Jim!

ok thanks for the help.

As I see it, the only practical reason to use separate interface for sync is to avoid blocking it by firewall mistakes or overloading the NIC with traffic. So your topology seems more than fine by me :)

Changing VHID requires restart of CARP - disable, then enable in status>carp. Other than that, CARP uses shared password which should match on both ends.

Okay i have this all up and running - the issue was that BT had not setup the bloody service despite telling me several times they had! So here is how to setup BT Business infinity with 5 IPs on PFsense: WAN: Have this setup on PPPoE as usual with the correct user name and password which was provided to you. N.B. both the user name and password are case sensitive so make sure you get it right! You'll then pick up a random dynamic IP on your WAN interface for general internet access. VIPs (your 5 static IPs) All you need to do here is on the web gui go: Firewall > Virtual IPs Then depending on what kind of VIP you want just create 1 VIP for each static IP you have. my settings: Type: IP Alias Interface: WAN IP Address(es): type: Single address, Address: x.x.x.x / 29 Press save and you are done! Now you can play around and NAT things 1:1 or just port forward all you want. Enjoy!

Well, it seems this saved the day: [2.1-RELEASE][admin@master]/root(20): pkg_add -r http://files.pfsense.org/packages/amd64/8/All/libevent-1.4.14b_2.tbz Definitely this is bug.

I took a tcpdump of both interfaces does this look normal? FW1 00:00:00.000000 IP 10.1.0.2 > 224.0.0.18: VRRPv2, Advertisement, vrid 10, prio 0, authtype none, intvl 2s, length 36 00:00:02.001079 IP 10.1.0.2 > 224.0.0.18: VRRPv2, Advertisement, vrid 10, prio 0, authtype none, intvl 2s, length 36 00:00:02.001082 IP 10.1.0.2 > 224.0.0.18: VRRPv2, Advertisement, vrid 10, prio 0, authtype none, intvl 2s, length 36 00:00:02.001087 IP 10.1.0.2 > 224.0.0.18: VRRPv2, Advertisement, vrid 10, prio 0, authtype none, intvl 2s, length 36 00:00:02.001082 IP 10.1.0.2 > 224.0.0.18: VRRPv2, Advertisement, vrid 10, prio 0, authtype none, intvl 2s, length 36 00:00:02.001081 IP 10.1.0.2 > 224.0.0.18: VRRPv2, Advertisement, vrid 10, prio 0, authtype none, intvl 2s, length 36 00:00:02.001085 IP 10.1.0.2 > 224.0.0.18: VRRPv2, Advertisement, vrid 10, prio 0, authtype none, intvl 2s, length 36 FW2 tcpdump -i bce1_vlan101 -ttt -n proto CARP tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on bce1_vlan101, link-type EN10MB (Ethernet), capture size 96 bytes 00:00:00.000000 IP 10.1.0.3 > 224.0.0.18: VRRPv2, Advertisement, vrid 10, prio 100, authtype none, intvl 2s, length 36 00:00:02.392089 IP 10.1.0.3 > 224.0.0.18: VRRPv2, Advertisement, vrid 10, prio 100, authtype none, intvl 2s, length 36 00:00:02.392086 IP 10.1.0.3 > 224.0.0.18: VRRPv2, Advertisement, vrid 10, prio 100, authtype none, intvl 2s, length 36 00:00:02.392088 IP 10.1.0.3 > 224.0.0.18: VRRPv2, Advertisement, vrid 10, prio 100, authtype none, intvl 2s, length 36 00:00:02.392089 IP 10.1.0.3 > 224.0.0.18: VRRPv2, Advertisement, vrid 10, prio 100, authtype none, intvl 2s, length 36 00:00:02.392089 IP 10.1.0.3 > 224.0.0.18: VRRPv2, Advertisement, vrid 10, prio 100, authtype none, intvl 2s, length 36 00:00:02.392093 IP 10.1.0.3 > 224.0.0.18: VRRPv2, Advertisement, vrid 10, prio 100, authtype none, intvl 2s, length 36 00:00:02.392085 IP 10.1.0.3 > 224.0.0.18: VRRPv2, Advertisement, vrid 10, prio 100, authtype none, intvl 2s, length 36 00:00:02.392089 IP 10.1.0.3 > 224.0.0.18: VRRPv2, Advertisement, vrid 10, prio 100, authtype none, intvl 2s, length 36

I figured this out, apparently if you make a gateway from the interfaces tab it can gain priority and cause your routing to break? Probably user error on my end but once I removed the gateway I was able to route traffic again. Thanks for your help.

It really is necessary to have the pfSense versions match up, otherwise you will have configuration sync and/or state sync issues. Otherwise, as has been stated, you don't need identical hardware or NICs, just an identical number of interfaces and to have them assigned in the exact same order on both.

Read this thread http://forum.pfsense.org/index.php/topic,67045.0.html

quote>–-So that part is working until I get to the public IP addresses that were assigned to me. 209.37.20.65/27 ( made up the ip addresses but they are similar) I cannot figure out how or what I am doing at this point. this circuit I has a block of IP addresses in the class c range that are supposed to be routed to the other Public Ip in the A class that they gave me. I tried putting in the 12.x.x.x ip address as the DG and using the first IP address in the C class as the IP on the WAN interface. I then put the remaining IP addresses as IP Aliases under CARP settings. I can ping the public IP addresses from the LAN side but cannot ping from a different circuit on the WAN side. When I do trace route it is one hop. In review. When I use the 12.x.x.x settings the circuit will start routing and work. Once I told the ATT rep that that was working he went ahead and assigned me the block of IP addresses in the class C range. So I assumed they use the 12.x.x.x to provision the circuit, test it out and then assign the real IP addresses but using the Class C settings as I have for many years results in no surfing. The only thing I could get from the tech was I need to put the 12.x.x.x IP address as the DG. The graphic below is a sample config from a Cisco  

have you tried searching the forums? believe it or not, you are not the first one to have these problems. :) http://forum.pfsense.org/index.php?topic=44529.0 CMB said this in the thread above: Microsoft finally dropped some code to provide proper FreeBSD support, which we'll integrate when we get to a base version that supports it (2.2). In the mean time, hyper-v isn't a great option.

Hi ssheikh, thanks a lot for your swift reply! I'll ask to enable the promiscuous mode only for WAN, LAN and OPT1 (DMZ) interfaces, than. Kind regards, Luigi