Failover from routed to natted carrier

  • Can someone please provide general guidance on how to failover from a routed internet connection to a natted connection? DHCP is also a concern. Obviously want to avoid reassigning all the CPE's from public to private IP's upon failover. Three PfSense boxes are employed at different locations, one at each carrier and one for fail-over. Wan1 will be Tier 1 so any compromise to achieve co-existence would be preferred on the Wan2 side of the failover.


    N --routable------PfSense routed--Wan1--|
    T  /30 transit                                        |
    E  /24 useable                                      |--PfSense failover--250 natted CPE's
    R                                                          |
    N --nonroutable--PfSense natted--Wan2--|
    E  /29 transit
    T    & useable

  • No ideas?

  • Netgate

    Looks like a regular failover group with WAN1 as tier 1 and WAN2 as tier 2 should be fine.

    You just want to make sure that outbound NAT rules are present to map your /24 to address(es) on WAN2 when traffic is going out that interface.

    You can probably start with automatic outbound NAT, convert to manual, and disable all the rules for WAN1. That should get you pretty close.

    Things that work with no NAT on WAN1 might not work switching to being NATted on WAN2. That all depends on the application.

  • Thanks Derelict for your ideas. PfSense natted is already setup and operating. The other two routers are built and will be dropped in as soon as the fiber is lit and routed.

    My toughest challenge is what to do with all the natted CPE's (customer radios) that currently have a private Wan address but will have a public Wan address on Wan1 (fiber). When it fails over I don't think I can have reserved all the CPE IP's in the natted router (Wan2) as public IP's on the Lan, PfSense would probably balk at that.  The only other idea I have is to reassign all the CPE public IP's back to private IP's upon failover which I would expect to be slow and clumsy.

    What would be the cleanest fail-over method in terms of how to re-config the natted router?

  • Netgate

    What other two routers? That is not one pfSense with two WANs?

    You're probably going to need a better diagram with at least a representation of what IP addresses are where and more details about your specific issues.

    See my .sig for an example diagram with the type of information that is helpful.

  • Ya, I don't have it drawn up in Visio yet but plan too. All three routers will have one Lan and one Wan port. Below I numbered the PfSense routers that will be needed for this modified topology. The public IP addresses are listed between the routers and the internet. The fiber connected router #1 will be routed not 1:1 nat providing an individual public IP to each CPE at the customer's home.  The existing in-service natted router #2 is at the bottom of the diagram currently serving all CPE's.  Router #3 is only for fail-over and route modification so all CPE's can either go to the natted router or the "routed only" router.

    I'm not enthusiastic about natting router #1 just so it fails over cleaner to router #2 when router #1's carrier is routed so each CPE can have a public IP and eliminates double-natting that has been a PITA at router #2.

    I                        Router#1
    N –routable------PfSense routed--Wan1--|
    T  /30 transit                                        |  Router #3
    E  /24 useable                                      |--PfSense failover--250 natted CPE's
    R                                                          |
    N --nonroutable--PfSense natted--Wan2--|
    E  /29 transit    Router#2
    T    & useable

  • Netgate

    Sorry I still don't see why that is not one router with two WAN ports directly connected to the ISPs. You're doing multi-wan either way. Why the extra layer?

    You don't have to NAT router #1 for WAN1 if it is 2 WAN 1 LAN. You can set it to only NAT when the traffic is sent out WAN2.

    No matter what, the public IP addresses for the CPEs will not be available via WAN2 when WAN1 is down. The end user addresses will inevitably be double-nat when that happens.

  • It can't be all done on one router because the CPE's all merge over backbone at point A, the new carrier's fiber is at point B, and the natted router (non-fiber) is at point C because the existing carrier service is located at point C.  And points A, B and C are a few miles apart, not co-located.  I plan to make the existing carrier the fail-over tier 2.

    I didn't think public IP addresses could be made available on the LAN of router #2.  So what is the best solution short of not having any fail-over at all?  What configuration for router #2 would be best to make the fail-over happen the smoothest.  This isnt' a CARP solution because Wan1 and Wan2 are not identical topology.  Short of a better answer I suppose the only one is to have modem #2 reassign all the CPE's from public to private IP's via DHCP, again a clumsy solution.  I expect failover, which normally takes 10 seconds, may take upwards of a minute.  And I may have to somehow force DHCP to happen quickly, rather than rely on default timings on the server and client side.  Currently using the resolver on router #2.

  • Netgate

    NAT on router #2. Don't on router #1. There isn't going to be anything resembling "seamless" failover. All existing firewall states on Router #1 will be useless.

    No inbound connections will be possible into router 2.