CARP VIP at single pfSense (1.2.3) fails to BACKUP constantly



  • Hi all,

    We've problem with pfSense VM on VMware. We're using this scenario for a log time with no problem, until now. When I create CARP VIP, it fails to BACKUP state immediately (sometimes one or two ICMP echo replies come).

    I'm really messed up and do not know where to start investigate, all settings on the others pfSense boxes (physical or in VMware) seems to be the same…

    I'll appreciate any hint...
    -tt-

    Some config informations follow:

    
    # ifconfig -a
    le0: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
    	options=8 <vlan_mtu>ether 00:50:56:8e:49:d1
    	inet 192.168.1.1 netmask 0xffffff00 broadcast 192.168.1.255
    	inet6 fe80::250:56ff:fe8e:49d1%le0 prefixlen 64 scopeid 0x1 
    	media: Ethernet autoselect
    	status: active
    le1: flags=8943 <up,broadcast,running,promisc,simplex,multicast>metric 0 mtu 1500
    	options=8 <vlan_mtu>ether 00:50:56:8e:2e:49
    	inet x.y.58.61 netmask 0xffffffe0 broadcast x.y.58.63
    	inet6 fe80::250:56ff:fe8e:2e49%le1 prefixlen 64 scopeid 0x2 
    	media: Ethernet autoselect
    	status: active
    le2: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
    	options=8 <vlan_mtu>ether 00:50:56:8e:1a:e7
    	inet 192.168.254.1 netmask 0xffffff00 broadcast 192.168.254.255
    	inet6 fe80::250:56ff:fe8e:1ae7%le2 prefixlen 64 scopeid 0x3 
    	media: Ethernet autoselect
    	status: active
    lo0: flags=8049 <up,loopback,running,multicast>metric 0 mtu 16384
    	inet 127.0.0.1 netmask 0xff000000 
    	inet6 ::1 prefixlen 128 
    	inet6 fe80::1%lo0 prefixlen 64 scopeid 0x4 
    enc0: flags=41 <up,running>metric 0 mtu 1536
    pfsync0: flags=41 <up,running>metric 0 mtu 1460
    	pfsync: syncdev: lo0 syncpeer: 224.0.0.240 maxupd: 128
    pflog0: flags=100 <promisc>metric 0 mtu 33204
    carp0: flags=8 <loopback>metric 0 mtu 1500
    	carp: INIT vhid 57 advbase 1 advskew 0</loopback></promisc></up,running></up,running></up,loopback,running,multicast></vlan_mtu></up,broadcast,running,simplex,multicast></vlan_mtu></up,broadcast,running,promisc,simplex,multicast></vlan_mtu></up,broadcast,running,simplex,multicast> 
    

    (le0 is LAN iface, le1 is WAN iface, le2 is OPT1 iface, i'm trying to create VIP on WAN iface)

    
    net.inet.ip.same_prefix_carp_only: 0
    net.inet.carp.allow: 1
    net.inet.carp.preempt: 1
    net.inet.carp.log: 2
    net.inet.carp.arpbalance: 0
    net.inet.carp.drop_echoed: 0
    net.inet.carp.suppress_preempt: 0
    
    

    When I create the VIP, in the Log I can see this (log entries are reversed):

    
    Mar 28 17:23:17 	kernel: carp0: link state changed to DOWN
    Mar 28 17:23:17 	kernel: carp0: 2 link states coalesced
    Mar 28 17:23:17 	kernel: carp0: MASTER -> BACKUP (more frequent advertisement received)
    Mar 28 17:23:17 	check_reload_status: reloading filter
    Mar 28 17:23:14 	kernel: carp0: link state changed to DOWN
    Mar 28 17:23:14 	kernel: carp0: 2 link states coalesced
    Mar 28 17:23:14 	kernel: carp0: MASTER -> BACKUP (more frequent advertisement received)
    Mar 28 17:23:14 	kernel: carp0: INIT -> MASTER (preempting)
    Mar 28 17:23:14 	kernel: carp0: link state changed to DOWN
    Mar 28 17:23:14 	kernel: carp0: link state changed to DOWN
    Mar 28 17:23:14 	kernel: carp0: 2 link states coalesced
    Mar 28 17:23:14 	kernel: carp0: MASTER -> BACKUP (more frequent advertisement received)
    Mar 28 17:23:14 	kernel: carp0: INIT -> MASTER (preempting)
    Mar 28 17:23:14 	kernel: carp0: link state changed to DOWN
    Mar 28 17:23:14 	kernel: carp0: link state changed to DOWN
    Mar 28 17:23:14 	kernel: carp0: 2 link states coalesced
    Mar 28 17:23:14 	kernel: carp0: MASTER -> BACKUP (more frequent advertisement received)
    Mar 28 17:23:14 	kernel: carp0: INIT -> MASTER (preempting)
    Mar 28 17:23:14 	kernel: le1: promiscuous mode enabled
    
    

    When I disable and re-enable CARP, I can see this in the Log

    
    Mar 28 17:50:00 	kernel: carp0: link state changed to DOWN
    Mar 28 17:50:00 	kernel: carp0: 2 link states coalesced
    Mar 28 17:50:00 	kernel: carp0: MASTER -> BACKUP (more frequent advertisement received)
    Mar 28 17:49:57 	kernel: carp0: link state changed to DOWN
    Mar 28 17:49:57 	kernel: carp0: MASTER -> BACKUP (more frequent advertisement received)
    Mar 28 17:49:55 	kernel: carp0: link state changed to UP
    
    

    This is the tcpdump of vrrp messages:

    
    # tcpdump -en -i le1 'vrrp'
    tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
    listening on le1, link-type EN10MB (Ethernet), capture size 96 bytes
    17:55:22.672646 00:00:5e:00:01:39 > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 70: x.y.58.61 > 224.0.0.18: VRRPv2, Advertisement, vrid 57, prio 0, authtype none, intvl 1s, length 36
    17:55:22.672769 00:00:5e:00:01:39 > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 70: x.y.58.61 > 224.0.0.18: VRRPv2, Advertisement, vrid 57, prio 0, authtype none, intvl 1s, length 36
    17:55:25.682679 00:00:5e:00:01:39 > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 70: x.y.58.61 > 224.0.0.18: VRRPv2, Advertisement, vrid 57, prio 0, authtype none, intvl 1s, length 36
    17:55:25.682806 00:00:5e:00:01:39 > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 70: x.y.58.61 > 224.0.0.18: VRRPv2, Advertisement, vrid 57, prio 0, authtype none, intvl 1s, length 36
    
    # tcpdump -en -i carp0 'vrrp'
    tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
    listening on carp0, link-type NULL (BSD loopback), capture size 96 bytes
    17:55:34.714166 AF IPv4 (2), length 60: x.y.58.61 > 224.0.0.18: VRRPv2, Advertisement, vrid 57, prio 0, authtype none, intvl 1s, length 36
    17:55:37.724078 AF IPv4 (2), length 60: x.y.58.61 > 224.0.0.18: VRRPv2, Advertisement, vrid 57, prio 0, authtype none, intvl 1s, length 36
    
    


  • Whoa, I knew it's not problem of pfSense. My co-worker had done mistake in ESX advanced configuration - the 'Net.ReversePathFwdCheckPromisc' parameter must have the value of '1'.


Locked