Dual ISP with a twist



  • Hi,

    Although I have extensive Cisco networking background, I consider myself a noob on pfSense. The task I'm about to do, I've done on a Cisco router without any problems. But since it's getting old and can't pretty much do some of the things I want, I switched to pfSense.

    So here it is…. kindly refer to the diagrams. The diagrams are a simplified representation of what my home network backbone looks like.

    Figure 1: Routing scenario when all ISPs are UP

    Suffice to say, User Group A gets preferential treatment and in case the primary ISP (PLDT) goes down (Figure 2), they should still have internet access routed to a secondary ISP on another building. All of the cabling and equipment are already working. All I need to do is to configure pfSense to 'automagically' route internet traffic to Router 2 when ISP PLDT becomes unreachable.

    Figure 2: Routing scenario when the primary ISP is DOWN

    But here lies the twist.

    From pfSense's perspective, it only has one (1) WAN connection, and routing Internet-bound traffic to Router 2 would technically be coming inbound to pfSense's LAN then outbound again through this same interface, before heading for Router 2. I've tested this config by adding Router 2 to the list of gateways and configuring it as the default gateway. But in doing so, no traffic goes through the primary ISP unless I put a check on "Default Gateway" for the primary route.

    Is there anyway to make this automatic?

    Any help will be appreciated.



  • System > Routing

    Add a new entry to the Gateways tab (eg. "Globe") with interface LAN and gateway IP of 192.168.1.2.

    Add a new entry on the Groups tab (eg. "Failover") with "GW_WAN" as Tier 1, and "Globe" as Tier 2.  Trigger level should be "Member Down".

    Firewall > Rules

    Edit your LAN firewall rules to use the new gateway group.



  • ^ Thank you!

    All that was wrong was that I failed to assign the Gateways group to the outbound firewall rule. Everything's ok now and failover is working.



  • Just want to ask if it's normal behavior that routing doesn't go back immediately to the primary gateway when service resumes?

    Based on my observations, when ISP1 goes down, routing shifts to ISP2 after several seconds… which is fine. But when ISP1 goes back up, it takes forever to shift routing back to ISP1.

    ISP1 is already configured as Tier 1.



  • @oj88:

    Just want to ask if it's normal behavior that routing doesn't go back immediately to the primary gateway when service resumes?

    Based on my observations, when ISP1 goes down, routing shifts to ISP2 after several seconds… which is fine. But when ISP1 goes back up, it takes forever to shift routing back to ISP1.

    ISP1 is already configured as Tier 1.

    WAN1 is up and has active states.  WAN1 fails and all connections are now established on WAN2.  WAN1 comes back up and NEW connections use it but EXISTING connections keep using WAN2 until they fall idle and the states are closed.



  • Ahh.. that makes sense.

    This is sorta shot in the dark and may even sound funny but, is there an "override" switch for that behavior? :D I mean, will it be possible for pfSense to force the termination of existing sessions to WAN2 once WAN1 goes online?

    The reason I asked is because I have an OpenVPN router behind pfSense that I imagine will keep using WAN2 since it presumably needs to maintain sessions to keep the VPN tunnel up. And because WAN2 has a lower bandwidth, all traffic passing through the VPN slows to a crawl until I manually reset the tunnel.

    So simply, my preferred behavior would be, when WAN1 goes up, drop all sessions to WAN2 and re-establish them through WAN1.

    Thanks!



  • You could reset all the states.



  • You could set up two OpenVPN tunnels. One over each WAN.
    Then do the loadbalancing/failover between the two tunnels instead/additionally to the WAN.



  • @Jason:

    You could reset all the states.

    True. But I'd rather have this done automatically, if possible.

    @GruensFroeschli:

    You could set up two OpenVPN tunnels. One over each WAN.
    Then do the loadbalancing/failover between the two tunnels instead/additionally to the WAN.

    The OpenVPN thing is a subscription (to access Netflix, Hulu, Pandora, etc.) for our media devices. I would rather not create a second tunnel as it would entail an additional subscription and cost.

    Thanks!



  • you could try to enable default gateway switching (System: Advanced: Miscellaneous: Loadbalancing).
    it's possible that you'd get a faster responds this way, but it might have consequences elsewhere - be ware ;)