Carp work fine on all interfaces but one
-
hello,
we have installed 2 pfSense 2, that are connected via private interface.
all is working fine, we have 13 interfaces (one wan, two lans, a lot of DMZ, one for each subnet).
configuring the interface work correctly and put the second firewall as backup.the problem is that if we reboot the backup firewall, all the interfaces but one goes up in backup state inside carp status.
of course we checked for differences between the configuration of the VIP, of the carp, inside the switch (vlans) and so on…
the only difference is the ip of published VRRP packets, see next.deleting the carp and re-creating it again will set again correctly the backup firewall as backup in carp status.
seems like the secondary does not get the primary VRRPv2 packet that mark it "online".the network is XXX.XXX.30.240/28,
the first router is 30.241,
the second router is 30.242,
the gateway for the subnet (the vip) is 30.243.show filter log on secondary:
00:00:00.000000 rule 11/0(match): block in on igb1_vlan102: XXX.XXX.30.243 > 224.0.0.18: VRRPv2, Advertisement, vrid 14, prio 0, authtype none, intvl 1s, length 36
00:00:01.000978 rule 11/0(match): block in on igb1_vlan102: XXX.XXX.30.243 > 224.0.0.18: VRRPv2, Advertisement, vrid 14, prio 0, authtype none, intvl 1s, length 36
00:00:01.001000 rule 11/0(match): block in on igb1_vlan102: XXX.XXX.30.243 > 224.0.0.18: VRRPv2, Advertisement, vrid 14, prio 0, authtype none, intvl 1s, length 36
00:00:01.000968 rule 11/0(match): block in on igb1_vlan102: XXX.XXX.30.243 > 224.0.0.18: VRRPv2, Advertisement, vrid 14, prio 0, authtype none, intvl 1s, length 36
00:00:01.000973 rule 11/0(match): block in on igb1_vlan102: XXX.XXX.30.243 > 224.0.0.18: VRRPv2, Advertisement, vrid 14, prio 0, authtype none, intvl 1s, length 36
00:00:01.000980 rule 11/0(match): block in on igb1_vlan102: XXX.XXX.30.243 > 224.0.0.18: VRRPv2, Advertisement, vrid 14, prio 0, authtype none, intvl 1s, length 36
00:00:01.000981 rule 11/0(match): block in on igb1_vlan102: XXX.XXX.30.243 > 224.0.0.18: VRRPv2, Advertisement, vrid 14, prio 0, authtype none, intvl 1s, length 36
00:00:01.000980 rule 11/0(match): block in on igb1_vlan102: XXX.XXX.30.243 > 224.0.0.18: VRRPv2, Advertisement, vrid 14, prio 0, authtype none, intvl 1s, length 36tcpdump on the interface on secondary
12:01:11.027030 IP XXX.XXX.30.243 > 224.0.0.18: VRRPv2, Advertisement, vrid 5, prio 0, authtype none, intvl 1s, length 36
12:01:11.510254 IP XXX.XXX.30.242 > 224.0.0.18: VRRPv2, Advertisement, vrid 5, prio 100, authtype none, intvl 1s, length 36
12:01:12.027503 IP XXX.XXX.30.243 > 224.0.0.18: VRRPv2, Advertisement, vrid 5, prio 0, authtype none, intvl 1s, length 36
12:01:12.901548 IP XXX.XXX.30.242 > 224.0.0.18: VRRPv2, Advertisement, vrid 5, prio 100, authtype none, intvl 1s, length 36
12:01:13.028009 IP XXX.XXX.30.243 > 224.0.0.18: VRRPv2, Advertisement, vrid 5, prio 0, authtype none, intvl 1s, length 36
12:01:14.028468 IP XXX.XXX.30.243 > 224.0.0.18: VRRPv2, Advertisement, vrid 5, prio 0, authtype none, intvl 1s, length 36
12:01:14.292843 IP XXX.XXX.30.242 > 224.0.0.18: VRRPv2, Advertisement, vrid 5, prio 100, authtype none, intvl 1s, length 36
12:01:15.028943 IP XXX.XXX.30.243 > 224.0.0.18: VRRPv2, Advertisement, vrid 5, prio 0, authtype none, intvl 1s, length 36
12:01:15.684141 IP XXX.XXX.30.242 > 224.0.0.18: VRRPv2, Advertisement, vrid 5, prio 100, authtype none, intvl 1s, length 36
12:01:16.029423 IP XXX.XXX.30.243 > 224.0.0.18: VRRPv2, Advertisement, vrid 5, prio 0, authtype none, intvl 1s, length 36
12:01:17.029901 IP XXX.XXX.30.243 > 224.0.0.18: VRRPv2, Advertisement, vrid 5, prio 0, authtype none, intvl 1s, length 36
12:01:17.075435 IP XXX.XXX.30.242 > 224.0.0.18: VRRPv2, Advertisement, vrid 5, prio 100, authtype none, intvl 1s, length 36tcpdump on primary:
12:01:11.153204 IP XXX.XXX.30.242 > 224.0.0.18: VRRPv2, Advertisement, vrid 5, prio 100, authtype none, intvl 1s, length 36
12:01:11.670669 IP XXX.XXX.30.243 > 224.0.0.18: VRRPv2, Advertisement, vrid 5, prio 0, authtype none, intvl 1s, length 36
12:01:11.712192 IP XXX.XXX.30.244 > 173.194.35.19: ICMP echo request, id 60206, seq 7086, length 64
12:01:11.712562 IP 173.194.35.19 > XXX.XXX.30.244: ICMP echo reply, id 60206, seq 7086, length 64
12:01:11.713491 IP XXX.XXX.30.244.2273 > 93.95.210.20.64436: Flags [P.], ack 0, win 250, options [nop,nop,TS val 256114177 ecr 1762791], length 128
12:01:11.781887 IP 93.95.210.20.64436 > XXX.XXX.30.244.2273: Flags [.], ack 18961, win 1002, options [nop,nop,TS val 1763046 ecr 256114177], length 0
12:01:11.781958 IP 93.95.210.20.64436 > XXX.XXX.30.244.2273: Flags [P.], ack 18961, win 1002, options [nop,nop,TS val 1763046 ecr 256114177], length 48
12:01:11.782110 IP XXX.XXX.30.244.2273 > 93.95.210.20.64436: Flags [.], ack 48, win 250, options [nop,nop,TS val 256114194 ecr 1763046], length 0
–-As you can see in the logs, the prio0 is announced through 30.243 and not 30.241 as expected (and as it work in all the other interfaces, using the real ip of the device and not the virtual ip).
we didn't rebooted the first firewall, as we are in the middle of a migration and we would like to do it when we are in server farm (300 Km away)...
Any hints apart from reboot the first firewall?
thanks,
d.