Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Backup CARP router unable to access internet

    Scheduled Pinned Locked Moved
    HA/CARP/VIPs
    3
    7
    1.4k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • S
      samelmore
      last edited by

      I currently have an HA setup between two pfSense 2.3.4 routers.  CARP failover works just fine.  However, the backup CARP router cannot access the internet.  I have advanced/manual outbound NAT rules setup for each of my subnets using the external CARP address, so all traffic is coming from the VIP's from those networks (going out the WAN).  For the 127.0.0.0/8 network outbound NAT (WAN interface), I have tried NO NAT, I have tried the VIP WAN IP, as well as the Interface Address.  None seem to allow the backup CARP router to access the internet.  This is a problem because, as one example, I cannot check for updates from the backup router without switching it to master first.

      To troubleshoot, I have verified that when pinging from the backup CARP router to the internet, the packet goes out using the WAN VIP.  Therefore, if I packet capture on the master CARP router, it sees the ICMP packets returned to it, so they're ultimately not getting back to the backup router.

      What outbound NAT rule do I need to enable this to work (to be able to ping/access from backup router to internet addresses)?  Are the outbound rules even the problem in this case?

      Thanks,
      Sam

      1 Reply Last reply Reply Quote 0
      • DerelictD
        Derelict LAYER 8 Netgate
        last edited by

        Do you have a public IP address for the primary interface, secondary interface, and the CARP VIP on WAN?

        I have verified that when pinging from the backup CARP router to the internet, the packet goes out using the WAN VIP.

        Traffic from the firewall itself should go out sourced from the interface address, not the CARP VIP.

        If the secondary is sending traffic from the firewall out WAN and translating the source address to the CARP VIP, your outbound NAT rules are wrong.

        Source any there is not what you want. Neither is source WAN net or WAN address.

        Chattanooga, Tennessee, USA
        A comprehensive network diagram is worth 10,000 words and 15 conference calls.
        DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
        Do Not Chat For Help! NO_WAN_EGRESS(TM)

        1 Reply Last reply Reply Quote 0
        • S
          samelmore
          last edited by

          Yes, I understand I don't want packets from the pfSense to go out as the WAN VIP.  I have tried using the outbound rule as follows, which still fails:

          Interface: WAN
          Source: 127.0.0.0/8
          Destination: any
          NAT Address: Interface Address

          This still results in ping packets originating from the backup pfSense router (using Diagnostics -> Ping) coming from the WAN VIP.  I'm unsure how to troubleshoot further, as the above outbound NAT rule seems like it should work correctly.

          Thanks,
          Sam

          1 Reply Last reply Reply Quote 0
          • DerelictD
            Derelict LAYER 8 Netgate
            last edited by

            Yeah. Post your outbound NAT rules on the primary and secondary.

            Chattanooga, Tennessee, USA
            A comprehensive network diagram is worth 10,000 words and 15 conference calls.
            DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
            Do Not Chat For Help! NO_WAN_EGRESS(TM)

            1 Reply Last reply Reply Quote 0
            • S
              samelmore
              last edited by

              Outbound NAT rules:

              Iface WAN - src 127.0.0.0/8 - dst * - NAT: WAN Address
              Iface WAN - src 192.168.187.0/24 - dst * - NAT: WAN VIP
              Iface WAN - src 192.168.179.0/24 - dst * - NAT: WAN VIP
              Iface WAN - src 192.168.21.0/24 - dst * - NAT: WAN VIP

              1 Reply Last reply Reply Quote 0
              • DerelictD
                Derelict LAYER 8 Netgate
                last edited by

                Outbound connections from the secondary are not sourced from 127.0.0.1. They are sourced from the interface address. There is no outbound NAT. In fact, it is normal to change those rules to use the CARP VIP if sourced from 127.0.0.0/8, not WAN address. Again, what you do not want to NAT is sourcing from the WAN subnet.

                I cannot think of anything off the top of my head to tell you to check.

                I'd need to see a lot more information about your firewall's config. The normal CARP config does not do what you are saying it is doing.

                You might need to get over not providing WAN address information so we can actually see what is going on there.

                pcaps on WAN interfaces for icmp on my lab HA pair:

                Primary:
                04:36:17.542757 IP 172.25.228.18 > 172.25.228.1: ICMP echo request, id 58300, seq 6383, length 8
                04:36:17.542897 IP 172.25.228.1 > 172.25.228.18: ICMP echo reply, id 58300, seq 6383, length 8
                04:36:18.070377 IP 172.25.228.18 > 172.25.228.1: ICMP echo request, id 58300, seq 6384, length 8
                04:36:18.070492 IP 172.25.228.1 > 172.25.228.18: ICMP echo reply, id 58300, seq 6384, length 8
                04:36:18.602750 IP 172.25.228.18 > 172.25.228.1: ICMP echo request, id 58300, seq 6385, length 8
                04:36:18.602878 IP 172.25.228.1 > 172.25.228.18: ICMP echo reply, id 58300, seq 6385, length 8

                Secondary:
                04:34:26.972616 IP 172.25.228.19 > 172.25.228.1: ICMP echo request, id 35112, seq 17494, length 8
                04:34:26.972759 IP 172.25.228.1 > 172.25.228.19: ICMP echo reply, id 35112, seq 17494, length 8
                04:34:27.502601 IP 172.25.228.19 > 172.25.228.1: ICMP echo request, id 35112, seq 17495, length 8
                04:34:27.502752 IP 172.25.228.1 > 172.25.228.19: ICMP echo reply, id 35112, seq 17495, length 8
                04:34:28.032678 IP 172.25.228.19 > 172.25.228.1: ICMP echo request, id 35112, seq 17496, length 8
                04:34:28.032828 IP 172.25.228.1 > 172.25.228.19: ICMP echo reply, id 35112, seq 17496, length 8

                The CARP VIP is 172.26.228.17

                Outbound NAT rules for WAN are attached.

                ![Screen Shot 2017-08-25 at 9.37.51 PM.png](/public/imported_attachments/1/Screen Shot 2017-08-25 at 9.37.51 PM.png)
                ![Screen Shot 2017-08-25 at 9.37.51 PM.png_thumb](/public/imported_attachments/1/Screen Shot 2017-08-25 at 9.37.51 PM.png_thumb)

                Chattanooga, Tennessee, USA
                A comprehensive network diagram is worth 10,000 words and 15 conference calls.
                DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
                Do Not Chat For Help! NO_WAN_EGRESS(TM)

                1 Reply Last reply Reply Quote 0
                • E
                  EdCazini
                  last edited by

                  Hi, in my case (v 2.4.2-RELEASE (amd64) ), I solved it with the following settings:

                  • Outbound NAT Mode = Hybrid Outbound NAT rule generation
                  • Putting below as the 1st rule in Firewall/NAT/Outbound (in primary, will replicate to backup), see pic below
                  Iface=WAN - src=This Firewall - dst=* - NAT=WAN Address

                  P.S.

                  • been using pfSense since 2006
                  • but this is my 1st post. edit: oops my 2nd post actually, the 1st was way back 2015  :D
                  • so I beg you tolerance if I woke up a sleeping thread or broke some forum rules
                  • just wanted to help

                  Ed

                  1 Reply Last reply Reply Quote 1
                  • NobleKangarooN NobleKangaroo referenced this topic on
                  • First post
                    Last post