First of all:

Do not forget to backup config on both boxes before and after config changes.

Answers:
1)2)3)If you have two boxes, use only carps for fail-over. Configure a full redundant firewall configuration.

all ips on each interface(vlan or real) must be on same subnet too. The vhid must be unique for each virtual ip and it is used to check health between boxes on each interface(vlan or real) with carp enabled.

If it is a layer2 switch, you will not be able to route between vlans. you will need a layer3 switch.
Create vlans as much as you need at pfsense, vmware and switch.
create carps on each interface for fail-over between firewalls.
Set these virtual ips as gateways on each vlan/virtual machine.

Considerations:
Do not forget to configure a sync interface between boxes.

After all carp settings done, use firewall rules do block/permit what you need(ping, www, ssh,etc).

Read this if you want to setup a first level DOS prevention on your network.
http://forum.pfsense.org/index.php?topic=38273.0

If you have carps, the best way(in my opinion) is to separate these gateways on pfsense.
If you can't enable another interface on pfsense, create some vlans on your switch and configure it on pfsense.

If both of your links are internet links, this will be very hard to set two 'default' gateways on same interface.

att,
Marcello Coutinho

If you have only one ip on each subnet, you can do it combining forces with you router(s). ;D

see (http://forum.pfsense.org/index.php/topic,35281.msg200865.html#msg200865) for a detailed explanation of how to do this.

after nat on router, at pfsense:

You can't have two subnets on same interface, you need to create a interface for each subnet.

the minimun amount of ips for it will be 4 on the same subnet.
1 for the router
1 for pfsense1
1 for pfsense2
1 to be published as a carp ip between two pfsenses. (this can be as much as you need 1, 2,…10 ips)

If you plan to have each pfsense pluged into different switches, you will must have a dedicated interface between both for sync.
My suggestion is to plug all interfaces of each firewall in only one swtich(using vlans), this prevents some carp mistakes between master and slave when not all interfaces are offline.

FIREWALL1 <-> SWITCH 1
FIREWALL2 <-> SWITCH 2

FIREWALL1 <-CROSSOVER-> FIREWALL2

If you have two gigabit interfaces on each firewall you can do everything. one for sync and other with a lot of vlans.

att,
Marcello Coutinho

i anxiously followed that carpdev porting idea a while back. I used to have multiple IPs from my provider and a sweet pfsense failover setup, but not anymore.

What I've done as a poor mans substitute is to configure my DSL modem(ancient westell 6100) nat feature to "static nat" traffic to a single 192.168.x.y ip. I then use that ip as the carp/vip IP and then assign 192.168.x.y+1 and 192.168.y+2 to the wan ports on the A and B pfsense machine respectively.  the westell still seems to allow other 192.168.x.z ips to talk and not hit the static nat rule, which allows both pfsense boxes to talk to the internet doing normal NAT, and anything unknown coming in, it sends to the CARP ip.

with the carp ip active on a machine it should be used as the src ip, so that traffic will come back to the same machine or in the event of a failover, the other pfsense box will take over  the carp ip and get the traffic.

i was worried that the mac/ip changing at failover would confuse the modem but it seems to handle it in my testing.

in the end it has worked out fairly well for my purposes although i dont do any of the problematic type applications like video conferenceing, voip etc and i have no incoming services other than openvpn tunnels.

the openvpn tunnels are just setup on the dsl modem as specific nat rules that map different UDP ports to each firewall.
the openvpn client has two remote ip entries, oen with each port and it will rotate through them if one goes down… but that means it goes down and reconnects. but i'm just doing simple road-warrior type vpn tunnels so it's not a problem.

anyway, i've been happy with it.

as you mentioned that your block is in same subnet you may use carp version of vip and that allows you to use those vip's in firewall

There probably isn't a good way to do that in the GUI for lo0. You could install the shellcmd package and then add a shellcmd in there, those get run at bootup and it may do what you need. In the shellcmd just add the ifconfig command you would normally run by hand.

[FYI- don't address a forum post to anyone specific - let everyone in on the fun]

If you have one of those IPs as your WAN, and their device as the gateway, then you can use either CARP or Proxy ARP (or on 2.0, IP Alias) VIPs. They would all work to use the additional IPs.

Thanks for the reply!

I have been testing this out for some time now and it will not work for me.
Have found out that the the packets sent to and from the server to IP 239.255.255.250 are SSDP protocol over UDP.
Those are non routeble protocol.

I got the IGMP broadcast through with the proxy as you said but I could not get the SSDP through.

I think the only solution to this is to connect a second NIC on the Mediaserver and connect it to DMZ.

/illern.

The testing continued today and finally I got so desperate that I decided to take my very simple Sitecom 5 port switch and connected both LAN interfaces of the firewalls to that. To my total surprise: it worked!
So there has got to be something in the configuration of my DLINK DGS-3024 switches.

This topic can be closed with a wise lesson for everyone: never underestimate the power of your switches, they can *** up everything…

Wow.  Thanks!  Ok, I am happy!  Thank you for your information!

YAAA you fixed me!  Thanks a bunch!

Guess I celebrated too fast. It worked yesterday, but after coming back to work today it doesn't work anymore. Will do some more testing later on.

EDIT: I updated both machines to 2.0RC3. After the subsequent reboot it's working again, even after coming back to work the next morning. Let's see how it goes.

I have 2.0 installed here at home but just for a little while longer I am going to wait to install 2.0 in a production environment. Thanks for all your advice I'll read about Proxy ARP I could of swore I remembered reading something about trouble with FTP.

have you tried to add computername\username format or just username?

If your settings are not syncing, check Firewall > Virtual IPs, on the CARP Settings tab. On 2.0 the whole lower 2/3 of the page is for config sync. Put in the IP, password, and check all of the boxes on the master. Leave that whole section empty on the slave.

I forgot to add the Pfsense box is a virtual machine on our hypervisor and currently has 2 network cards. One network card is called LAN and the other is WAN.

You can only properly do CARP failover with static IP WANs, it wouldn't fail over properly and keep connections alive with a PPPoE WAN.

Also I don't believe the second WAN can be on-demand when used with multi-wan. pfSense constantly pings the gateways of all WANs to ensure they are usable, and there isn't a mechanism currently to handle on being left down until needed. This is in the works, though, because it would be useful for 3G connections where bandwidth is expensive.

If they're routed to you, you don't need anything at layer 2 ("Other" VIPs will suffice). To use that with CARP, have the ISP route the IP blocks to a CARP IP on your WAN subnet.

CARP IPs must be within the subnet of the interface's IP. That's not going to change in the near future.