CARP WAN Gateway Up for Box 1 but down for Box 2
-
Hi, I have a carp with two boxes.
My WAN Subnet is 172.16.0.16/29172.16.0.17 is DSL Modem making NAT to Internet as any dsl modem does.
172.16.0.18 is CARP VIP
172.16.0.19 is WAN IP Box1
172.16.0.20 is WAN IP Box2Outbound NAT is set to manual so I have only one rule:
IF Protocol Source Destination Translation
WAN Any 172.16.0.0/22 Any CARP VIP (172.16.0.18)I have configured the two boxes to monitor the Wan Gateway that is 10.68.226.213. This is a host on the ISP network so I don't have any kind of access to it, but can get echo reply so I use this to monitor my internet connection.
At Box 1 WAN Gateway shows as green and up.
At Box 2 WAN Gateway shows as red and down.Only at Box 2 in the firewall log I get a lot of:
Blocked source 10.68.226.213 destination 172.16.0.20 ICMPAfter making some packet capturing I noticed this:
Packet Capture for Box 1:
IP 10.68.226.213 > 172.16.0.18: ICMP echo reply, id 32455, seq 44288, length 44
IP 10.68.226.213 > 172.16.0.20: ICMP echo reply, id 47308, seq 44288, length 44Packet Capture for Box 2:
IP 172.16.0.18 > 10.68.226.213: ICMP echo request, id 32455, seq 44288, length 44
IP 10.68.226.213 > 172.16.0.20: ICMP echo reply, id 47308, seq 44288, length 44I understanf from this that Box 2 sends an echo request using the VIP 172.16.0.18 as source IP but get echo reply with WAN IP as destination and for that reason drops the packet and mark the WAN Gateway as failed.
I'm not sure why this is happening. I don't know where the real IP of WAN is taked for the echo reply.I created an outbound NAT rule that says:
IF Protocol Source Destination Translation
WAN ICMP 172.16.0.16/29 10.68.226.213/32 Interface AddressAfter that the two boxes shows Wan Gateway as UP and no more block logs at the Box 2.
Why is this necesary with CARP? Maybe is my mistake?
If its not my mistake this must be documented if it is not already. -
Why is this necesary with CARP? Maybe is my mistake?
If its not my mistake this must be documented if it is not already.This is required because normal traffic going out should come from the CARP address and monitoring traffic should come from the real address.
I think this would be a good addition to the docs. Maybe it's in the book (I still don't have it, sorry).
http://doc.pfsense.org/index.php/Configuring_pfSense_Hardware_Redundancy_%28CARP%29 -
The docs recommend setting (or, rather, not changing) the source away from the specific networks.
Using "any" as a source for outbound NAT is almost always going to have bad and/or unintended results, with or without CARP, but especially with CARP.
If you want to get away with only using one outbound NAT rule, make an alias containing your local/internal networks and use that as the source, not 'any'.