Routing configuration to minimize downtime during hardware swap.
I've been juggling an idea around for a way to minimize downtime when swapping out my pfSense router. I'm mostly interested in this for educational purposes. What I'm thinking is connect the old router's WAN port to an extra port on the new router, move the internet connection over to the WAN port on the new router, and reconfigure as necessary to make it so that either router can be used by hosts at the default gateway (the configurations are all static). This allows all hosts to function, regardless of whether they have their original or updated IP configurations (specifically, default gateway).
Here's a sketch I made up of what this network might look like:
If I'm understanding this right, I could preconfigure the new router completely, relocate the WAN connection to the new router, take the old router's WAN port and connect it to the new router. The IP configuration of the old router's WAN interface would need to be changed to use the new 172.16.2.200 network, but that interface is already set as the default route. Hosts with their old configuration will send to default gateway 10.0.0.254, which will then get passed on to the new router. Hosts with updated configurations will send straight out through 10.0.0.1.
Would be great to know if I'm missing something that would make this not work as I'm picturing.
Also, I'd like to confirm my understanding that in this configuration, inbound traffic won't ever go to the old router - It'll just get sent straight out to the switch by the new router, even for hosts which are using their original configuration. This shouldn't cause any issues.
I swapped my new router in this afternoon, and was ultimately able to get the previously described configuration working. In case anyone else reading this finds it interesting/useful, here's what I needed to do to make it happen:
Preconfigure the new router - This can all be done without interrupting the original network at all: Setup one interface on the existing LAN and connect it. Setup one interface as WAN. Setup a third interface OPT1 (or whatever you want to call it), give it a static IP in a new, separate network.
In the new router, configure a firewall rule for OPT1 interface to allow all traffic for all protocols. Without this, the new router's firewall will block all traffic coming from the old router.
Swap the cables: Move the cable providing your internet over to the new router's WAN port. Then with another cable, connect the WAN port of the old router to the OPT1 port of the new router.
Set IP configuration of the WAN interface to be part of the network that OPT1 is configured for.
That's it - the network will now work and function like in the diagram above. Hosts will have connectivity, whether they are using the original configuration, or have been updated to use the new router as the default gateway.
Note: There are many things this will break that aren't detailed here - VPN's/Web Caching/IDS/etc. The configurations in your old router need to be moved to the new one before they'll do anything, so if you have any mission-critical stuff going on in your router, plan accordingly.