I have found out the root causes.  It's not the problem with the switch.  It was b/c of the firewall rules.  When the FW outbound LAN rule got change to use the Failover pool, the default route is no long effective.  When master & slave send out the broadcast message of VRRP to 244.0.0.18, it used the Failover for it routing table and Failover pool is only routed to either WAN1 or WAN2 which doesn't know the route of the internal LAN subnet.  That's why stage MASTER/MASTER were on both machines.  Once I create the new rule for LAN subnet to allow traffic to 244.0.0.18 using the default gateway, then it fixed the problem.

It's working great now.  Disable on master –> switch over to salve.  Enable back --> fallback to master.

Exactly

thanks for yours help

This was a bug.  Just fixed it.

Thank you

To answer the first of my two questions, I just came up with this:

http://kerneltrap.org/node/1021

Which basically means you can't bind scripts to it, just monitor link status from cron and act upon it (MASTER -> dial out)

Which brings back the second question.

What's the proper way of initiating PPPoE dialout from shell?

@jmhoms:

when you talk about adding firewall rules that utilize the pool as gateway, these rules must be added in the LAN interface, isnt it ?

That is right, you have to make the rule on the interface the traffic comes in from.

@jmhoms:

and the gateway IPs in the load balacer configuration must be the WAN and WAN2 CARP addresses, isnt it ?

No, you use their gateways (it's a gateway pool). If you use the latest snapshot you'll have these as pulldown options so there is no footshooting with this setting anymore.
Don't forget to set your firewall>nat, outbound to advanced outbound nat to utilize your CARP VIPs.

@jmhoms:

If so, i'm trying to make this setup working in a wmare test environment, no luck for now, i'll keep trying.

I have heard that CARP is not happy inside vmware. Haven't tried it myself though.

@jmhoms:

I supose it will work, so , do you think that if i have 2 offices with this configuration, will be possible to do IPSEC between the WANs CARP addresses ?

Yes, just have a look at the failover tab at vpn>ipsec. I have a setup running in this configuration.

whats the way around that ?

would it just simply be

if destination is in this range and from WAN forward out DMZ interface
and back out again
If destination is from DMZ servers range to the net forward out WAN interface ?

That way you wouldnt need NAT or Bridging ?

http://faq.pfsense.com/index.php?action=artikel&cat=1&id=167&artlang=en&highlight=arp

I'm trying the same that jpinder70, but with 2 adsl connections (and later will try to setup a redundant balaced ipsec meshed network).

It seems obvious that each pfsense system must have a wan ip of each of the adsl/t1 connections in order to have a carp address for each connection. I only have 1 public static ip per adsl, and will belong to carp interface, because the traffic must go out with this ip, cos is the only routed to my connection by my isp. That way, as the wan adresses must be in the same subnet as the carp address, i will take 2+2 ip that not really belong to me, and i assume that my natted networks never will get to the real ips (anyway these probably doent have any public service that must be directly accessed by my users).

Actually i only have 3 nic in each pfsense. So i'm trying some setups to see if they work without need of 4rt nic, hope to hear your feedback.

I connected both adsl routers, and both wan of pfsenses to the same ethernet segment.
My pfsense1 sync to pfsense2. I tried also to activate that pfsense2 sync to pfsense1. It seems to work, but there is some delay when apply changes, maybe there is some kind of cyclic action :? i don't know if it's ok that setup.
Actually my  wan of pfsense1 have the adsl1 public ip, and wan of pfsense2 have the adsl2 public ip. I setup a carp address for adsl1 subnet in pfsense1, and a carp address for adsl2 suvnet in pfsense2. I was expecting for an error in sync, because pfsense1 doesnt know about adsl2 subnet, and pfsense2 neither of adsl1 net. Pfsense system have sync and now i have the carp adresses in both pfsenses. Maybe is not necessary that both pfsesne to be in both wan subnets ?¿¿ i think that yes it's mandatory, because don't seem to work (no error in frontend anyway).

Assuming that both subnets are mandatory, i would like to know if it's possible to setup a wan interface with the two wan ips (1 per each adsl conn).  Maybe with proxy arp virtual ip ?? i don't see any aliasing option to assign multiple ip to an interface in the frontend (like in rc.conf _alias method in freebsd). I read somewhere that is not recommended, anyone have any hint with this ? maybe this will be an issue in the way the traffic wil go out ??¿ maybe the balacer will not work properly ?

i keep monitoring this thread to see if the jpinder70 setup works.

Thanks.

Demonstration of what it does:
http://pfsense.com/mirror.php?section=tutorials/carp/carp_failoversim.htm

How to set it up:
http://pfsense.com/mirror.php?section=tutorials/carp/carp-cluster-new.htm
http://doc.pfsense.org/index.php/Setting_up_CARP_with_pfSense

Technical Info:
http://www.openbsd.org/faq/pf/carp.html
http://www.countersiege.com/doc/pfsync-carp/

any help or comment?

Thanks,
Hans

so I have to set wan /16 and vip /16 ?

Seems to be bad luck in that case. You'll have to wait for the next major version (most likely 2.0) which has support for another type of alias that should be able to handle this condition.

okay this is what i found if you have a address of *x.x.x.25 and you also have a address of x.x.x.2 you will ned up  with the same carp numbers. I have fixed it but still does not solve my problem of not being able to download files or email attachments. I have tried every suggestions mentioned on the forum. So i am open to new ideas

http://faq.pfsense.com/index.php?action=artikel&cat=1&id=167&artlang=en&highlight=bad%20gateway

We enable preepmtion by default now, that's why the box is missing (the tutorial was not updated regarding this). I'll have a lokk at the doc if it can be made more clear or more easy to understand.

Oops, thats wrong :)  I'll get it updated.  Thanks

See http://www.countersiege.com/doc/pfsync-carp/ for how it works.

Thanks for the quick response.  Flipped everything over to Proxy-ARP and it works.