CARP WAN Gateway Up for Box 1 but down for Box 2



  • Hi, I have a carp with two boxes.
    My WAN Subnet is 172.16.0.16/29

    172.16.0.17 is DSL Modem making NAT to Internet as any dsl modem does.
    172.16.0.18 is CARP VIP
    172.16.0.19 is WAN IP Box1
    172.16.0.20 is WAN IP Box2

    Outbound NAT is set to manual so I have only one rule:
    IF        Protocol  Source          Destination  Translation
    WAN  Any        172.16.0.0/22  Any              CARP VIP (172.16.0.18)

    I have configured the two boxes to monitor the Wan Gateway that is 10.68.226.213. This is a host on the ISP network so I don't have any kind of access to it, but can get echo reply so I use this to monitor my internet connection.

    At Box 1 WAN Gateway shows as green and up.
    At Box 2 WAN Gateway shows as red and down.

    Only at Box 2 in the firewall log I get a lot of:
    Blocked source 10.68.226.213 destination 172.16.0.20 ICMP

    After making some packet capturing I noticed this:

    Packet Capture for Box 1:
    IP 10.68.226.213 > 172.16.0.18: ICMP echo reply, id 32455, seq 44288, length 44
    IP 10.68.226.213 > 172.16.0.20: ICMP echo reply, id 47308, seq 44288, length 44

    Packet Capture for Box 2:
    IP 172.16.0.18 > 10.68.226.213: ICMP echo request, id 32455, seq 44288, length 44
    IP 10.68.226.213 > 172.16.0.20: ICMP echo reply, id 47308, seq 44288, length 44

    I understanf from this that Box 2 sends an echo request using the VIP 172.16.0.18 as source IP but get echo reply with WAN IP as destination and for that reason drops the packet and mark the WAN Gateway as failed.
    I'm not sure why this is happening. I don't know where the real IP of WAN is taked for the echo reply.

    I created an outbound NAT rule that says:
    IF        Protocol  Source              Destination            Translation
    WAN  ICMP      172.16.0.16/29  10.68.226.213/32  Interface Address

    After that the two boxes shows Wan Gateway as UP and no more block logs at the Box 2.
    Why is this necesary with CARP? Maybe is my mistake?
    If its not my mistake this must be documented if it is not already.



  • @gabrielpc1190:

    Why is this necesary with CARP? Maybe is my mistake?
    If its not my mistake this must be documented if it is not already.

    This is required because normal traffic going out should come from the CARP address and monitoring traffic should come from the real address.
    I think this would be a good addition to the docs. Maybe it's in the book (I still don't have it, sorry).
    http://doc.pfsense.org/index.php/Configuring_pfSense_Hardware_Redundancy_(CARP)


  • Rebel Alliance Developer Netgate

    The docs recommend setting (or, rather, not changing) the source away from the specific networks.

    Using "any" as a source for outbound NAT is almost always going to have bad and/or unintended results, with or without CARP, but especially with CARP.

    If you want to get away with only using one outbound NAT rule, make an alias containing your local/internal networks and use that as the source, not 'any'.


Locked