Navigation

    Netgate Discussion Forum
    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search

    Multi-WAN IPv4 failover with BGP + NAT without dropping connections

    Routing and Multi WAN
    3
    4
    90
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • D
      davidbalbert last edited by

      tl;dr: is it possible to have seamless failover (no dropped connections) using v4 with BGP if you have to NAT because you don't have enough public v4 addresses?

      The details

      I'm in the process of planning a multi-WAN failover setup. We'll have two connections, a dedicated 1Gbps fiber as a primary, and a cable modem as a backup. We'll have BGP sessions on both connections (technically, our cable ISP won't speak BGP with us, but all traffic on the cable modem will be routed over a tunnel to an ISP that will).

      It's important that our users experience zero downtime when our primary connection goes down (no connection drops, etc.). This is easy with v6: when the primary connection goes down, we'll announce our v6 address space over our backup, and traffic will keep flowing (in this scenario, each LAN user has a publicly routable v6 address).

      For v4 this seems more complicated. We'll have a /24, but this isn't enough addresses for all the users on our network. This means we'll have to NAT our LAN. So here's the problem:

      • Assume our /24 is 3.2.1.0/24.
      • ISP1 gives us an address of 1.1.1.2 for WAN1 (the primary)
      • ISP2 gives us an address of 2.2.2.2 for WAN2 (the backup)

      Under normal operation, LAN traffic leaving WAN1 will have its source address set to 1.1.1.2 by NAT. When failover happens, LAN traffic will start having its source address set to 2.2.2.2 when it leaves on WAN2. This is a problem. Any traffic that's connection oriented will fail when the remote recipient (or a firewall/NAT on the path to the recipient) sees a source address that it hasn't seen before, and the connection will have to be reestablished.

      Is there a way to work around this problem?

      The only thing I can think of is having some sort of virtual interface with a public IP address from our /24 (let's say 3.2.1.1) sitting between the LAN interface and the WAN interfaces. NAT would then happen on this virtual interface instead of the WAN interfaces. This way, the source address of all packets leaving our network would be 3.2.1.1, regardless of whether they left on WAN1 or WAN2.

      Questions:

      • Is this possible on pfSense? It requires some sort of custom topology within the router than I'm not sure how to put together.
      • Is this the right solution to the problem?
      • How would you set this up?

      Unrelated BGP question:

      • Is it possible to use addresses from our /24 for WAN1 and WAN2? I think the answer is "no" because that would require announcing a subnet smaller than a /24 to each of our ISPs, but I want to double check my thinking.

      Thanks,
      Dave

      dotdash 1 Reply Last reply Reply Quote 0
      • dotdash
        dotdash @davidbalbert last edited by

        I would set this up with a separate box doing the BGP connection to ISP1 and ISP2. This box would present your 3.2.1.0/24 network on it's 'LAN' side, which would be the gateway for the firewall, which would have a 3.2.1.x WAN. My 2c, YMMV.

        1 Reply Last reply Reply Quote 0
        • D
          davidbalbert last edited by

          That makes sense, and it's a reasonable suggestion.

          I'd like to do it with one box if I can, though. I'm the only person in our (small) org who does network administration, and it's only a small part of my job. The cost in complexity of having another box in our network closet is something I'd like to avoid.

          1 Reply Last reply Reply Quote 0
          • Derelict
            Derelict LAYER 8 Netgate last edited by

            Yup. BGP in that case should be handled by a router that doesn't care what interface a packet arrives on because it is not maintaining firewall states.

            ISP1    ISP2
             +        +
             +        +
             BGP ROUTER
                 +
                 |
                 |
                 +
              FIREWALL
            
            1 Reply Last reply Reply Quote 0
            • First post
              Last post

            Products

            • Platform Overview
            • TNSR
            • pfSense
            • Appliances

            Services

            • Training
            • Professional Services

            Support

            • Subscription Plans
            • Contact Support
            • Product Lifecycle
            • Documentation

            News

            • Media Coverage
            • Press
            • Events

            Resources

            • Blog
            • FAQ
            • Find a Partner
            • Resource Library
            • Security Information

            Company

            • About Us
            • Careers
            • Partners
            • Contact Us
            • Legal
            Our Mission

            We provide leading-edge network security at a fair price - regardless of organizational size or network sophistication. We believe that an open-source security model offers disruptive pricing along with the agility required to quickly address emerging threats.

            Subscribe to our Newsletter

            Product information, software announcements, and special offers. See our newsletter archive to sign up for future newsletters and to read past announcements.

            © 2021 Rubicon Communications, LLC | Privacy Policy