I would recommend the Mikrotik RB260GS switch. Can mirror multiple ports to one sensor port and supports vlans. http://wiki.mikrotik.com/wiki/SwOS

CARP VIPs are always single host addresses. The subnet mask on a CARP VIP must match the parent subnet. So if you WAN is x.x.x.a/28, then your CARP VIP must be (for example) x.x.x.b/28 – it's still just one IP. It's not like proxy ARP where it makes a bunch of IPs if you pick a larger mask.

I believe this is where i should be looking Load balancer (hinted from here http://forum.pfsense.org/index.php/topic,68769.0.html)

The way that I have been able to get this working is to Create a VIP on a separate subnet, then I have a Rule that allows any traffic to that Subnet to the VIP. Since the LAN Traffic is on the same private space interface the traffic can flow between the two. Just make sure you allow ANY ANY Traffic between the two DMZ's  then any Server/Client on DMZ-1 can talk to the VIP on DMZ-2 that is a Load Balancer that points to Servers in DMZ-1 Subnet… seems a bit wonky but it works.. the biggest problem with allowing traffic to flow out the same interface and then back in.. NO Load balancer will allow this.. so you have to create another subnet for it to route to... even if the servers behind the Load Balancer are on the same subnet. Good luck!

It does but you need to manually handle the VIP settings because they can't sync automatically. Also the DHCP failover setup doesn't support 2+ peers IIRC.

@craggy: How long should it take for a secondary firewall to take over the CARP VIP and start passing traffic again when the master is shutdown or rebooted? We have set up a secondary pfSense cluster in a new Data Center and the failover times are about 10 seconds with 5 to 10 ping drops and a very obvious pause during the switchover phase whereas at our primary site our pfSense cluster fails over in about 2 seconds with usually only a single ping dropped and is generally unnoticed. Both sites are identical, VMware esxi 5.1, hp bl460c blades, same HP VC networking etc. Only difference is the data centre. My boxes are instant; I've got a failing box and it needs a reboot every few days. If you're seeing 10 seconds then it might be something about being virtualized or maybe you've changed the base & skew settings.  For base & skew I use 0/1 on the primary and 0/100 on the secondary.

Nevermind…was a cabling problem - all is working

Got busy and haven't had time to tcpdump yet. Any other ideas out there about anything else that can be looked at in the interim?

I have no blade-related experience, but did you put the vswitch ports that connect to your pfsense vm's, and which are to take part in CARP, into a port group that has promiscous mode enabled? I usually create a duplicate port group (same vlan, same vswitch), which has promiscous mode enabled, and put the pfsense interfaces into that port group, and all vm's that use the pfsense as a gateway into the port group with promiscous disabled.

For me it's easier to have only one failover, the setup is so that the slave doesn't have all features (no backup wan connection) so only 1 network doesn't have the failover when there is a network fail. If all networks will switch in depended, I still can switch the master down, all networks will go down and the slave would take over all networks. I have created a stable situation again, I found out when there is an open network (both pfsense are set to init, the network becomes unstable in a couple of hours) But still I want to failover independent, I don't get why the option has been taken out.

Also see thread http://forum.pfsense.org/index.php/topic,68439.0.html, if the suggestions in this thread don't help as they may be similar issues.

So, I turn on spanning tree protocol on both sides of the bridge, to include the wifi, vlan, pfsync and bridge interface…and it all works.

@jimp: Yes. Perfecto! Thanks Jim!

ok thanks for the help.

As I see it, the only practical reason to use separate interface for sync is to avoid blocking it by firewall mistakes or overloading the NIC with traffic. So your topology seems more than fine by me :)

Changing VHID requires restart of CARP - disable, then enable in status>carp. Other than that, CARP uses shared password which should match on both ends.

Okay i have this all up and running - the issue was that BT had not setup the bloody service despite telling me several times they had! So here is how to setup BT Business infinity with 5 IPs on PFsense: WAN: Have this setup on PPPoE as usual with the correct user name and password which was provided to you. N.B. both the user name and password are case sensitive so make sure you get it right! You'll then pick up a random dynamic IP on your WAN interface for general internet access. VIPs (your 5 static IPs) All you need to do here is on the web gui go: Firewall > Virtual IPs Then depending on what kind of VIP you want just create 1 VIP for each static IP you have. my settings: Type: IP Alias Interface: WAN IP Address(es): type: Single address, Address: x.x.x.x / 29 Press save and you are done! Now you can play around and NAT things 1:1 or just port forward all you want. Enjoy!