Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    CARP WAN Gateway Up for Box 1 but down for Box 2

    HA/CARP/VIPs
    3
    3
    2.0k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • G
      gabrielpc1190
      last edited by

      Hi, I have a carp with two boxes.
      My WAN Subnet is 172.16.0.16/29

      172.16.0.17 is DSL Modem making NAT to Internet as any dsl modem does.
      172.16.0.18 is CARP VIP
      172.16.0.19 is WAN IP Box1
      172.16.0.20 is WAN IP Box2

      Outbound NAT is set to manual so I have only one rule:
      IF        Protocol  Source          Destination  Translation
      WAN  Any        172.16.0.0/22  Any              CARP VIP (172.16.0.18)

      I have configured the two boxes to monitor the Wan Gateway that is 10.68.226.213. This is a host on the ISP network so I don't have any kind of access to it, but can get echo reply so I use this to monitor my internet connection.

      At Box 1 WAN Gateway shows as green and up.
      At Box 2 WAN Gateway shows as red and down.

      Only at Box 2 in the firewall log I get a lot of:
      Blocked source 10.68.226.213 destination 172.16.0.20 ICMP

      After making some packet capturing I noticed this:

      Packet Capture for Box 1:
      IP 10.68.226.213 > 172.16.0.18: ICMP echo reply, id 32455, seq 44288, length 44
      IP 10.68.226.213 > 172.16.0.20: ICMP echo reply, id 47308, seq 44288, length 44

      Packet Capture for Box 2:
      IP 172.16.0.18 > 10.68.226.213: ICMP echo request, id 32455, seq 44288, length 44
      IP 10.68.226.213 > 172.16.0.20: ICMP echo reply, id 47308, seq 44288, length 44

      I understanf from this that Box 2 sends an echo request using the VIP 172.16.0.18 as source IP but get echo reply with WAN IP as destination and for that reason drops the packet and mark the WAN Gateway as failed.
      I'm not sure why this is happening. I don't know where the real IP of WAN is taked for the echo reply.

      I created an outbound NAT rule that says:
      IF        Protocol  Source              Destination            Translation
      WAN  ICMP      172.16.0.16/29  10.68.226.213/32  Interface Address

      After that the two boxes shows Wan Gateway as UP and no more block logs at the Box 2.
      Why is this necesary with CARP? Maybe is my mistake?
      If its not my mistake this must be documented if it is not already.

      1 Reply Last reply Reply Quote 0
      • S
        SeventhSon
        last edited by

        @gabrielpc1190:

        Why is this necesary with CARP? Maybe is my mistake?
        If its not my mistake this must be documented if it is not already.

        This is required because normal traffic going out should come from the CARP address and monitoring traffic should come from the real address.
        I think this would be a good addition to the docs. Maybe it's in the book (I still don't have it, sorry).
        http://doc.pfsense.org/index.php/Configuring_pfSense_Hardware_Redundancy_%28CARP%29

        1 Reply Last reply Reply Quote 0
        • jimpJ
          jimp Rebel Alliance Developer Netgate
          last edited by

          The docs recommend setting (or, rather, not changing) the source away from the specific networks.

          Using "any" as a source for outbound NAT is almost always going to have bad and/or unintended results, with or without CARP, but especially with CARP.

          If you want to get away with only using one outbound NAT rule, make an alias containing your local/internal networks and use that as the source, not 'any'.

          Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

          Need help fast? Netgate Global Support!

          Do not Chat/PM for help!

          1 Reply Last reply Reply Quote 0
          • First post
            Last post
          Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.