Two VIPs showing as MASTER on both servers, all others are working fine.
-
I have two pfSense (2.0.1) boxes running carp with 5 virtual IPs. Of the five VIPs, two are showing as master on both boxes.
Both boxes are DL380 G3s, they're connected through two Cisco 3750G's that are trunked together. I've putting the sync interface on it's own dumb switch, and that didn't help.
The two VIPs that are failing are for VLANs, though other VLANs are working fine. Below is the ifconfig from both boxes. I appreciate any help or pointers that anyone can offer.
Master
bge0: flags=8943 <up,broadcast,running,promisc,simplex,multicast>metric 0 mtu 1500
options=8009b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,linkstate>ether 00:11:85:bc:a1:3d
inet6 fe80::211:85ff:febc:a13d%bge0 prefixlen 64 scopeid 0x1
inet 10.0.0.156 netmask 0xffffff00 broadcast 10.0.0.255
nd6 options=3 <performnud,accept_rtadv>media: Ethernet autoselect (100baseTX <full-duplex>)
status: active
bge1: flags=8943 <up,broadcast,running,promisc,simplex,multicast>metric 0 mtu 1500
options=8009b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,linkstate>ether 00:11:85:bc:a1:3c
inet6 fe80::211:85ff:febc:a13c%bge1 prefixlen 64 scopeid 0x2
inet 10.3.0.2 netmask 0xffffff00 broadcast 10.3.0.255
nd6 options=3 <performnud,accept_rtadv>media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
em0: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
options=209b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,wol_magic>ether 00:02:a5:4e:a0:e8
inet 10.3.253.1 netmask 0xfffffffc broadcast 10.3.253.3
inet6 fe80::202:a5ff:fe4e:a0e8%em0 prefixlen 64 scopeid 0x3
nd6 options=3 <performnud,accept_rtadv>media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
em1: flags=8802 <broadcast,simplex,multicast>metric 0 mtu 1500
options=209b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,wol_magic>ether 00:02:a5:4e:a0:e9
media: Ethernet autoselect
status: no carrier
lo0: flags=8049 <up,loopback,running,multicast>metric 0 mtu 16384
options=3 <rxcsum,txcsum>inet 127.0.0.1 netmask 0xff000000
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5
nd6 options=3 <performnud,accept_rtadv>pfsync0: flags=41 <up,running>metric 0 mtu 1460
pfsync: syncdev: em0 syncpeer: 10.3.253.2 maxupd: 128 syncok: 1
pflog0: flags=100 <promisc>metric 0 mtu 33200
enc0: flags=0<> metric 0 mtu 1536
bge1_vlan2: flags=8943 <up,broadcast,running,promisc,simplex,multicast>metric 0 mtu 1500
options=3 <rxcsum,txcsum>ether 00:11:85:bc:a1:3c
inet6 fe80::211:85ff:febc:a13d%bge1_vlan2 prefixlen 64 scopeid 0x9
inet 10.3.2.2 netmask 0xffffff00 broadcast 10.3.2.255
nd6 options=3 <performnud,accept_rtadv>media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
vlan: 2 parent interface: bge1
bge1_vlan3: flags=8943 <up,broadcast,running,promisc,simplex,multicast>metric 0 mtu 1500
options=3 <rxcsum,txcsum>ether 00:11:85:bc:a1:3c
inet6 fe80::211:85ff:febc:a13d%bge1_vlan3 prefixlen 64 scopeid 0xa
inet 10.3.3.2 netmask 0xffffff00 broadcast 10.3.3.255
nd6 options=3 <performnud,accept_rtadv>media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
vlan: 3 parent interface: bge1
bge0_vlan254: flags=8943 <up,broadcast,running,promisc,simplex,multicast>metric 0 mtu 1500
options=3 <rxcsum,txcsum>ether 00:11:85:bc:a1:3d
inet6 fe80::211:85ff:febc:a13d%bge0_vlan254 prefixlen 64 scopeid 0xb
inet 10.3.254.2 netmask 0xffffff00 broadcast 10.3.254.255
nd6 options=3 <performnud,accept_rtadv>media: Ethernet autoselect (100baseTX <full-duplex>)
status: active
vlan: 254 parent interface: bge0
vip1: flags=49 <up,loopback,running>metric 0 mtu 1500
inet 10.0.0.155 netmask 0xffffff00
carp: MASTER vhid 1 advbase 1 advskew 0
vip2: flags=49 <up,loopback,running>metric 0 mtu 1500
inet 10.3.0.1 netmask 0xffffff00
carp: MASTER vhid 2 advbase 1 advskew 0
vip3: flags=49 <up,loopback,running>metric 0 mtu 1500
inet 10.3.2.1 netmask 0xffffff00
carp: MASTER vhid 3 advbase 1 advskew 0
vip4: flags=49 <up,loopback,running>metric 0 mtu 1500
inet 10.3.3.1 netmask 0xffffff00
carp: MASTER vhid 4 advbase 1 advskew 0
vip5: flags=49 <up,loopback,running>metric 0 mtu 1500
inet 10.3.254.1 netmask 0xffffff00
carp: MASTER vhid 5 advbase 1 advskew 0Slave
bge0: flags=8943 <up,broadcast,running,promisc,simplex,multicast>metric 0 mtu 1500
options=8009b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,linkstate>ether 00:0b:cd:9d:20:f3
inet6 fe80::20b:cdff:fe9d:20f3%bge0 prefixlen 64 scopeid 0x1
inet 10.0.0.157 netmask 0xffffff00 broadcast 10.0.0.255
nd6 options=3 <performnud,accept_rtadv>media: Ethernet autoselect (100baseTX <full-duplex>)
status: active
bge1: flags=8943 <up,broadcast,running,promisc,simplex,multicast>metric 0 mtu 1500
options=8009b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,linkstate>ether 00:0b:cd:9d:20:f2
inet6 fe80::20b:cdff:fe9d:20f2%bge1 prefixlen 64 scopeid 0x2
inet 10.3.0.3 netmask 0xffffff00 broadcast 10.3.0.255
nd6 options=3 <performnud,accept_rtadv>media: Ethernet 1000baseT <full-duplex>status: active
em0: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
options=209b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,wol_magic>ether 00:11:0a:56:08:80
inet 10.3.253.2 netmask 0xfffffffc broadcast 10.3.253.3
inet6 fe80::211:aff:fe56:880%em0 prefixlen 64 scopeid 0x3
nd6 options=3 <performnud,accept_rtadv>media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
em1: flags=8802 <broadcast,simplex,multicast>metric 0 mtu 1500
options=209b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,wol_magic>ether 00:11:0a:56:08:81
media: Ethernet autoselect
status: no carrier
lo0: flags=8049 <up,loopback,running,multicast>metric 0 mtu 16384
options=3 <rxcsum,txcsum>inet 127.0.0.1 netmask 0xff000000
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5
nd6 options=3 <performnud,accept_rtadv>pfsync0: flags=41 <up,running>metric 0 mtu 1460
pfsync: syncdev: em0 syncpeer: 224.0.0.240 maxupd: 128 syncok: 1
pflog0: flags=100 <promisc>metric 0 mtu 33200
enc0: flags=0<> metric 0 mtu 1536
bge1_vlan2: flags=8943 <up,broadcast,running,promisc,simplex,multicast>metric 0 mtu 1500
options=3 <rxcsum,txcsum>ether 00:0b:cd:9d:20:f2
inet6 fe80::20b:cdff:fe9d:20f3%bge1_vlan2 prefixlen 64 scopeid 0x9
inet 10.3.2.3 netmask 0xffffff00 broadcast 10.3.2.255
nd6 options=3 <performnud,accept_rtadv>media: Ethernet 1000baseT <full-duplex>status: active
vlan: 2 parent interface: bge1
bge0_vlan3: flags=8943 <up,broadcast,running,promisc,simplex,multicast>metric 0 mtu 1500
options=3 <rxcsum,txcsum>ether 00:0b:cd:9d:20:f3
inet6 fe80::20b:cdff:fe9d:20f3%bge0_vlan3 prefixlen 64 scopeid 0xa
inet 10.3.3.3 netmask 0xffffff00 broadcast 10.3.3.255
nd6 options=3 <performnud,accept_rtadv>media: Ethernet autoselect (100baseTX <full-duplex>)
status: active
vlan: 3 parent interface: bge0
bge0_vlan254: flags=8943 <up,broadcast,running,promisc,simplex,multicast>metric 0 mtu 1500
options=3 <rxcsum,txcsum>ether 00:0b:cd:9d:20:f3
inet6 fe80::20b:cdff:fe9d:20f3%bge0_vlan254 prefixlen 64 scopeid 0xb
inet 10.3.254.3 netmask 0xffffff00 broadcast 10.3.254.255
nd6 options=3 <performnud,accept_rtadv>media: Ethernet autoselect (100baseTX <full-duplex>)
status: active
vlan: 254 parent interface: bge0
vip1: flags=49 <up,loopback,running>metric 0 mtu 1500
inet 10.0.0.155 netmask 0xffffff00
carp: BACKUP vhid 1 advbase 1 advskew 100
vip2: flags=49 <up,loopback,running>metric 0 mtu 1500
inet 10.3.0.1 netmask 0xffffff00
carp: BACKUP vhid 2 advbase 1 advskew 100
vip3: flags=49 <up,loopback,running>metric 0 mtu 1500
inet 10.3.2.1 netmask 0xffffff00
carp: MASTER vhid 3 advbase 1 advskew 100
vip4: flags=49 <up,loopback,running>metric 0 mtu 1500
inet 10.3.3.1 netmask 0xffffff00
carp: MASTER vhid 4 advbase 1 advskew 100
vip5: flags=49 <up,loopback,running>metric 0 mtu 1500
inet 10.3.254.1 netmask 0xffffff00
carp: BACKUP vhid 5 advbase 1 advskew 100</up,loopback,running></up,loopback,running></up,loopback,running></up,loopback,running></up,loopback,running></full-duplex></performnud,accept_rtadv></rxcsum,txcsum></up,broadcast,running,promisc,simplex,multicast></full-duplex></performnud,accept_rtadv></rxcsum,txcsum></up,broadcast,running,promisc,simplex,multicast></full-duplex></performnud,accept_rtadv></rxcsum,txcsum></up,broadcast,running,promisc,simplex,multicast></promisc></up,running></performnud,accept_rtadv></rxcsum,txcsum></up,loopback,running,multicast></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,wol_magic></broadcast,simplex,multicast></full-duplex></performnud,accept_rtadv></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,wol_magic></up,broadcast,running,simplex,multicast></full-duplex></performnud,accept_rtadv></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,linkstate></up,broadcast,running,promisc,simplex,multicast></full-duplex></performnud,accept_rtadv></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,linkstate></up,broadcast,running,promisc,simplex,multicast></up,loopback,running></up,loopback,running></up,loopback,running></up,loopback,running></up,loopback,running></full-duplex></performnud,accept_rtadv></rxcsum,txcsum></up,broadcast,running,promisc,simplex,multicast></full-duplex></performnud,accept_rtadv></rxcsum,txcsum></up,broadcast,running,promisc,simplex,multicast></full-duplex></performnud,accept_rtadv></rxcsum,txcsum></up,broadcast,running,promisc,simplex,multicast></promisc></up,running></performnud,accept_rtadv></rxcsum,txcsum></up,loopback,running,multicast></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,wol_magic></broadcast,simplex,multicast></full-duplex></performnud,accept_rtadv></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,wol_magic></up,broadcast,running,simplex,multicast></full-duplex></performnud,accept_rtadv></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,linkstate></up,broadcast,running,promisc,simplex,multicast></full-duplex></performnud,accept_rtadv></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,linkstate></up,broadcast,running,promisc,simplex,multicast> -
Make sure you have permit access to both firewall on sync interface to avoid config sync problems.
The carp check is done in interface you choose to use carp.
The only situation when both sides belives they are master is when you do not have communication between them.Are you using vmware or physical machines?
-
Thanks for the reply. I thought CARP and VIPs were using just the pfSync interface, didn't realize that they're using the actual interfaces as well (makes sense)
I checked and for the VIPs that work, I'm able to ping across from interface to interface (for example: 10.3.0.2 to 10.3.0.3), for the two that are not working, I'm not able to ping across. Looks like that'd be my issue. Strange, firewalls are open on both boxes, and I'm not seeing anything logged. I'll start looking at the switches.
To answer your other questions, I have an allow any:any on firewall for the pfSync interface and both boxes are physical.
Thanks for your help with this, I think I'm on the right track now.
-
Taking a second look through everything, turns out I had the problematic vlans assigned to the wrong interface in pfSense. Once I got that straightened out, everything started working.
facepalm
Thanks again for the help.