CARP not working as expected



  • Hi all
    I have a small setup that looks like this:

    • 1 ISP  FC switch 4 ports in which i have 2 wan ip's
    • 2x cisco SG-200 8Gbit ports (sw1 and sw2)
    • 2x PfSense 2.2 boxes (wan, lan, carp physical interfaces / pf1 & pf2)
    • application server (2x Gbit lan in TeamLB mode)

    The setup looks like this:
    Both cisco sw are configured in mirror (different ip 10.45.1.251/252):
    -port 1+port 2 configured as LAGG
    -port 3+port 4 are reserved for ISP/wan: both are connected from port 3 to ISP FC sw
    -port 5 is connected to wan on pf's (sw1p5-> pf1 wan and  sw2p5 -> pf2 wan)
    -port 6 is connected to lan on pf's (sw1p6 -> pf1 lan and sw2p6->pf2 lan)
    -port 7 is connected to server interface (sw1p7 -> serverLan1 and sw2p7-> serverLan2)
    -port 8 mngt
    On both port 6 & 7 i have created a Vlan (vlanID 1111)
    On the pf boxes i have created a Vlan interface with VlanID 1111 and assigned to Lan interface mac.
    On the appServer both interfaces are in LB Team mode with vlan set as vlanid 1111
    I use the same wan ip on both pf-wan interfaces!

    Lan netwrok : 10.45.1.0/24  -> pf1  .1.251  / pf2  .1.252
    Vlan network : 10.45.3.0/24 -> pf1  .3.252  / pf2  .3.252
    Carp network: 10.45.2.0/24  -> pf1  .2.1    /  pf2    .2.2
    virtual ip: Lan .1.200 , Wan (from the same wan net) , vlan .3.250

    I have created on the primary (.251) the CARP setup (pfsync and virtual ip's + fw rules) using carp interface for wan, lan, and vlan sync.From primary to bkp have sync (in logs the config looks perfect) i have tested some config replication and works (dhcp setup, vpn setup, users are all replicated to the bkp pf)
    The problem i have is that both pf's are seen as Master. I belive this is because i use the same WAN ip.If i connect the pf2wan the gateway on primary becomes offline and online on the primary. => master on wlan if i get it right.Still after ~ 30 seconds internet start working but i have no more access to the other pf.If i disconnect and connect on the other side is the same.I'm not able to make them work in the same time.after a switch like this even on lan or vlan (where i need failover) i have no more access and no more wan ping.
    I guess i missed something…but i can't identify the issue.

    Any suggetions?



  • anyone?


  • Netgate

    You need 3 WAN IPs.  One for each interface and one for the CARP VIP.



  • ok…i only use the real and single wan ip that i have.
    i need to have a second one for pf2 ? real or virtual?


  • Netgate

    You need 3 WAN IPs.  One for each interface and one for the CARP VIP.



  • @Derelict:

    You need 3 WAN IPs.  One for each interface and one for the CARP VIP.

    On with one should I use the real wan ip?



  • You need three real public ip addresses. One for fw1, one for fw2, and one for the virtual IP.
    If your WAN is a /30, it is not going to work.



  • Then my setup will not work.hmm ugly