It doesn't work as I would want.
I spent quiet a long time on pf docs and suppose this setting should keep the real server IP during this timeout setting. I am not sure i understood , but a round robin translation rule and a sticky setting should keep sources IP to the same destination server in the next connection…

I can see the src nodes (and my own IP) in the pfs statistics, but I get several ones with my own IP to different web real server (behind the carp VIP) and I still get loadbalanced on the 3 www when the states are expired (before the 4 minutes defined). My max src nodes are under the 10000 limit - 2000 - so I think this is something else.
I will continue to read pf howtos etc
Any piece of advice would be appreciated,

I had something like this with my cluster. (but not using CP I didn't think that worked with CARP has this been fixed?)

After running CARP for ages with no problems I decided to unplug the KVM from the slave to use on another machine. So I unplugged it and rebooted the slave and up it came all fine so went back to the WebGUI to check and after a few mins of fiddling the slave became master on the LAN on its own. So rebooted and same prob so I plugged the kvm back in and no problem.

It seemed to be having some issue sharing IRQs for the nics with no kvm attached. In the end I fiddled with the IRQ settings changing them from auto to fixed and it has been fine ever since.

I cant remember what the message was but it would pop up on the console

So might be worth a look

A carp cluster failover should be similar to a normal failover setup. Did you change the gateway to your Wan1Fail pool on the default firewall run on the LAN tab? You also may need to add a route to provider 2's DNS server via the OPT2 interface (if clients use pfSense for DNS).
The two pools are simply so you can use policy routing with failover. For example, you could add a rule sending http out wan2 with Wan2Fail as the gateway. If you used the WAN2 gateway instead of the pool, http would break if wan2 went down.

That is what I see as well.  VRRP frames from upstream with a length of 20.  So is there a way to keep this from filling the system log.  I created a firewall rule that filters out CARP (the same protocol number as VRRP) from the upstream IPs, but it does not seem to have any effect.

You must hone your search-fu, grasshopper. It is currently weak. I'm too lazy to link you to relevant threads, so I'll give you some quick setup tips. As a side note, those IP's look a bit odd. Providers usually assign /29 blocks to customers with only three or four servers, and those numbers only make sense in a /27.
Anyway:
I'm assuming you have the WAN of the firewall set to .206 and the LAN set to .1
You would then go to firewall, virtual IPs, and add 207, 208, and 209 as Virtual Ips. I would use Proxy-arp, single-address for each of them.
If you have only a few ports to open, then go firewall, NAT, port-forward. Pick the correct external ip, internal ip and port, leave the 'auto-create firewall rule' box checked. Repeat for other services.
You could also add 1-1 NATs, then open the required ports under firewall, rules, WAN using the Internal IP of the server.

…any set of IPs in the same subnet...
@sullrich:

CARP is multicast.  Unless your ISP is blocking this traffic you could be stepping on an upstream VRRP host or even another CARP host.

On re-reading I really didn't say that very well!  :/
  What I meant was any set of IPs that were on their own subnet, but separate from the existing public IP.  E.g. the public IP could 1.2.3.4 but the CARP stuff could all take place on 10.1.1.1, 10.1.1.2 and 10.1.1.3 which the ISP shouldn't care about. 
  Since the CARP functionality is intended to detect and recover from hardware failures it really shouldn't matter what IPs it's using behind the scenes, right? 
  (And upon some research it looks like this capability is actually being added to CARP right now - would be very nice to have in pfSense!  :)

I found the problem.

The WAN interface on Server 2 was plugged into the wrong VLAN on the switch.  Not sure why that would cause this problem, but all is working as it should now.

Thanks.
I wiped the configuration and started from scratch again.
Second time around, CARP setup is ok.

1.0.1 is not the recommended version, try 1.2RC3.

is it possible that the embadded image does not support the
"redir" command and also does not support the "pkg_add" command?
I just tried to do it like it is described in the other thread, but the shell only returns

# pkg_add pkg_add: Command not found. # redir redir: Command not found. #

so at the moment, I have a IP-address (configured with ifconfig) on my WAN interface, and from the shell, I can ping my modem. But of course, I can't apply anny rules (nat, fw) to this ip address.

Thanks for you advice,

This installation is for 2 companies hosted in the same place. Right now we share the same Internet connection with a signle basic pfSense server, but now, we want to separate networks and WANs. However we still want to have either one to be able to fail over on the other one in case its WAN access fails.

It is better automated, but if this needs to be operated manually, it's not really an issue.

Well from what I have read, you cannot have two tunnels to the same subnet on different isp's(go to the dual wan/routing section and tons of people have asked how to do a failover vpn but everyone says it is currently not possible), so in order for me to handle an isp fail I wanted to have isp1 on pfsense1 and isp2 on pfsense2 and monitor the other end of the tunnel so if the isp or the pfsense goes down it will fail to pfsense2 and the backup isp.  If there is a way to do a failover vpn, I suggest you go into the dual wan/routing section and let everyone know.
Thanks

Thanks cmb.
After I posted I was thinking it through during the day and realized it wouldn't work and shifting the terminating ip to the same as the range of  ip's I have a might make more sense.
My main focus is being able to run 1 to 1 and Squid.
My experience is showing me that squid is worth running as it improves the experience of my customers.

I have another issue posted as another topic I don't know if you can answer if you happen to glance back.

I have 2 gateways.
2 seperate company's providing. (one is the 10 Mb fiber with the ip allocation)
I'm using one or the other  but I'd like to use both.
There is almost no info in the forum about load balancing .

the two gateways are on the same subnet connected by an 8km wireless link.
Can I set them up to load balance?
and can I set them up to failover using a single link ?
I'm sorry about effectivly double posting but I think you might know and I haven't found even the outline of how to set up load balancing.
Once again I'm in your debt
Cheers

Thanks for the help, I will just ignore the messages for now….

Ah I see, cheers.  Makes things simpler

Yep 1.2-RC3 is very stable (better than 1.0.1) and I would use it in production. The RELENG_1 path is the next 1.3 edition in its very early stages (it is a slightly confusing numbering system maybe RELENG_1_3 would be better).

I suppose I should mention how we route to the blocks internal to our network.

We have a /29 assigned to the WAN interface, and carp running between the two firewalls.

We then have our upstream statically routing blocks to the CARP IP of the firewall, and internally have these blocks assigned to interfaces directly connected on the firewalls.  No RFC1918 IP space is used, only public address space.

Okay.  So I got this working by throwing in a NAT rule.