Multi-wan [dual] and policy based routing with failover



  • I'm using pfsense 2.0.3 and I have the following setup

    WAN1 4.4.4.4
    WAN2 5.5.5.5

    I have a gateway group setup for load balancing and failover such as:

    Tier 1: WAN1 & WAN2

    I setup a rule that says the following:

    If your destination is for the 9.9.9.9/24 network from the LAN then go out of gateway [WAN1].

    I've been able to setup the rule as explained above under the firewall rules, but I don't see an additional option to fail back to use [WAN2] if [WAN1] is down/unavailable.

    Is this something that is possible or does this happen automatically?


  • Rebel Alliance



  • Here's what you need to do, under system -> Routing -> Gateway Groups

    1. Create a first group with description name "BALANCE", And set Tier 1 for both "wan's" and Trigger level to "latency or packet loss" [this for load balance]"

    2. Create a second group, description name "Wan1 Fail Wan2 Use"  and priority set wan1 to Tier1 and wan2 to Tier2, set "Trigger level" to member down.

    3. Create a third group, description name "Wan2 Fail Wan1 use" and priority set wan1 to Tier2 and Wan2 to Tier1, set "Trigger level" to member down.

    Now Coming Firewall Rules –> LAN, you need to create a three new rules,

    LIKE 1) BALANCE RULE

    Interfaces: Lan
    Protocol: ANY
    Source: LAN SUBNET
    Destination ports: ANY
    Gateway;BALANCE

    1. FAILOVER RULE

    Interfaces: Lan
    Protocol: ANY
    Source Address: ANY
    Destination ports: ANY
    Gateway;Wan1 Fail Wan2 Use

    1. FAILOVER RULE

    Interfaces: Lan
    Protocol: ANY
    Source Address: ANY
    Destination ports: ANY
    Gateway;Wan2 Fail Wan1 use

    Make sure to place them on top of the lan rules!
    This is more them enough for fail-overs.



  • Thanks ppt, and srk.

    I've made the rules as such for the LAN connectivity, but do I create two additional rules copying failover rule 2) and failover rule 3) for my destination rule of reaching of 9.9.9.9/24 by selecting the gateway of "Wan1 Fail Wan2 Use", followed by the gateway of ""Wan2 Fail Wan1 use"



  • I don't really understand what srk3461 is trying to explain with the rules given.
    It is usually handy to have 3 gateway groups, as explained by srk3461, because many people have a requirement for:
    a) BALANCE - some traffic to be "randomly" load-balanced, and;
    b) WAN1PRIO - some particular traffic to go out WAN1 (failing over to WAN2), and;
    c) WAN2PRIO - some other particular traffic to go out WAN2 (failing over to WAN1).
    If you have a need for all 3 possibilities, then make all 3 gateway groups.
    Then add "policy-routing" rules directing traffic. If you don't add any of these, then all your internet traffic will flow out the default gateway, ignoring the gateway groups you made!
    Each rule needs to select the traffic you want, and then in the advanced section, select the gateway group you want the traffic to be fed into. e.g. on LAN rules:
    IPv4 all, Source LANnet, port *, destination 9.9.9.0/24, port *, gateway group WAN1PRIO

    First make rules selecting the specific traffic you want to direct to a special gateway group. Then (if required) you can make a general rule feeding all other traffic to one of the gateway groups.



  • @phil.davis; This is just a basic setup just for a fail-over and for balancing random traffic, B'cos shon never "specified" the kind of traffic he wants to be balanced or what kinda network he has…. !

    phil what if there's no default gateway!?

    @shon; take phil.davis points into consideration!
    I ain't no expert!



  • A default gateway is quite handy for the pfSense box to find its way to the internet to do upgrades, install packages… :)
    But I suspect that you could have policy-routing rules (and static routes if useful) that cover all your traffic, then the default route would never be used, and could therefore be removed from the routing table. But I don't really see a benefit in trying to do that - only for heck of it when you have nothing better to do at the weekend (hmm - Le Tour is starting, time to stop playing on computers)



  • I've setup the three gateway groups as such:

    1. load_balancing  - WAN1(Tier1), WAN2(Tier1) [Packet Loss or High Latency]
    2. failover_from_WAN1_to_WAN2 - WAN1(Tier1), WAN2(Tier2) [Member Down]
    3. failback_from_WAN2_to_WAN1 - WAN1(Tier2),WAN2(Tier1) [Member Down]

    These are my Firewall Rules

    1. Any Protocol/Port from LAN net to Any using load_balancing gateway
    2. Any Protocol/Port from LAN net to Any using failover_from_WAN1_to_WAN2 gateway
    3. Any Protocol/Port from LAN net to Any using failback_from_WAN2_to_WAN1 gateway
    4. Any Protocol/Port from LAN net to 9.9.9.9/24 using failover_from_WAN1_to_WAN2 gateway
    5. Any Protocol/Port from LAN net to 9.9.9.9/24 using failback_from_WAN2_to_WAN1 gateway



  • 1. Any Protocol/Port from LAN net to Any using load_balancing gateway
    2. Any Protocol/Port from LAN net to Any using failover_from_WAN1_to_WAN2 gateway
    3. Any Protocol/Port from LAN net to Any using failback_from_WAN2_to_WAN1 gateway
    4. Any Protocol/Port from LAN net to 9.9.9.9/24 using failover_from_WAN1_to_WAN2 gateway
    5. Any Protocol/Port from LAN net to 9.9.9.9/24 using failback_from_WAN2_to_WAN1 gateway

    a) You only want 1 rule for each from/to set of addresses - the system will use the first rule it matches, later rules will therefore not be matched/used.
    b) The more specific rules must be first in the list, then the specific traffic gets matched and the "other general crud" falls through to match the later general rule.
    e.g.
    1. Any Protocol/Port from LAN net to 1.2.3.0/24 using failover_from_WAN1_to_WAN2 gateway
    2. Any Protocol/Port from LAN net to 4.5.6.0/24 using failover_from_WAN2_to_WAN1 gateway
    3. Any Protocol/Port from LAN net to Any using load_balancing gateway

    This has some specific rules that direct some traffic out WAN1 first, and some traffic out WAN2 first, then load balance the rest between WAN1 and WAN2.

    In your rule destination IP address/network you can put Aliases - so make an alias for all the destinations you want to send to the same gateway group, then make 1 rule with the alias as the destination, and "using failover_from_WAN1_to_WAN2 gateway"…



  • So what your saying is that my rule sets should look like this:

    1. Any Protocol/Port from LAN net to 9.9.9.9/24 using failover_from_WAN1_to_WAN2 gateway
    2. Any Protocol/Port from LAN net to 9.9.9.9/24 using failback_from_WAN2_to_WAN1 gateway
    3. Any Protocol/Port from LAN net to Any using load_balancing gateway
    4. Any Protocol/Port from LAN net to Any using failover_from_WAN1_to_WAN2 gateway
    5. Any Protocol/Port from LAN net to Any using failback_from_WAN2_to_WAN1 gateway

    Are the rules 2, and 5 necessary?



  • Are the rules 2, and 5 necessary?

    Rule 1 will get matched and that traffic goes to failover_from_WAN1_to_WAN2 gateway - that provides the failover in the requirement
    Rule 2 is never matched (the packets that match it were already matched to rule 1)
    Rule 3 sends everything else to load_balancing gateway
    Everything has already been matched now, so rule 4 and 5 do nothing.

    Rules 1 and 3 are effective

    Rules 2, 4 and 5 are not used and not necessary.



  • I have the following rules set up now:

    1. Any Protocol/Port from LAN net to 9.9.9.9/24 using failover_from_WAN1_to_WAN2 gateway
    2. Any Protocol/Port from LAN net to Any using load_balancing gateway

    Thanks!



  • I know this thread is kinda old but still hoping to get a response. I've been using pfsense on and off since 2007 which had served me well when I used it. Hardware failures in between last 7-8 years prompted me to go for traditional off the shelf routers from Bestbuy.

    Recently, I got a need for dual wan setup, tried ddwrt and tomato but got a hold of a SG 2440 appliance and had it setup.

    My requirement is simple:

    I have a cable modem and a gigabit fiber modem.

    Gigabit Fiber Modem (Bell Canada):
    With gigabit fiber modem I have the ISP provided TV and Phone for my home. Their TV uses a difference subnet and I have to have the TV receivers connected to their modem's lan ports. If I try a different switch than the TVs doesn't seem to work. So I will simply get a LAN connection from this fiber optic modem to the OPT1 port on PfSense. Pfsense gets a Private IP from this modem.

    Cable Modem (Rogers):
    I have the modem setup as modem only, disabling the router / residendial gateway function in the modem which had worked well with netgear/linksys routers in the past. The modem is connected to the WAN port on pfsense. Pfsense gets a public IP.

    My requirement is, I will use one of these gateway's as my primary WAN connection. When 1 fails, I want the other to take over. When the primary returns, I want the backup to switch back to primary.

    In this example. Lets say my WAN is primary and my OPT1 is backup/failover.

    1. I have a gateway group setup - called dualwan (WAN being tier1 and OPT1 being tier2)
    2. I have the firewall rules under LAN I have LAN net using dualwan as gateway
    3. In general setup I have two public DNS defined for both gateways (8.8.8.8 for WAN and 8.8.4.4 for OPT1)
    4. Both WAN and OPT1 are configured to obtain IP addresses by means of DHCP.
    5. WAN shows a public IP and OPT1 shows a private IP from the fiber modem
    6. I have deafult gateway switching enabled in System/Advanced/Miscellaneous (If I don't have this checked the failover does not work)

    If I lets say disconnect the cable from WAN, I see that my gateway switches to OPT1 (what is my ip in google search shows OPT1 public ip).

    Upto here all works fine.

    The problem I am having is when I connect WAN cable bringing my primary gateway up, it does not switch back to the primary. I thought the gateway group setup with tier1 for WAN is supposed to switch it back.

    Am I missing anything?

    I do not need load balancing group created. I just need one to being primary and 2nd one to be failover.

    Thanks,

    KK

    –-------------
    switching back to primary seems to be working now. I found a post which basically says to create a floating rule in the firewall settings. Not sure if this is what required or doing a reboot of pfsense fixed it. Here is a screen shot of the floating rule:

    http://prntscr.com/esvxdx

    Here is the screenshot of the LAN rule:
    http://prntscr.com/esvy5q

    Thanks,

    KK



  • @kimkhan:

    6. I have default gateway switching enabled in System/Advanced/Miscellaneous (If I don't have this checked the failover does not work)

    It would seem that this is the only setting needed to do what you and I are attempting to accomplish. If you don't use this, the more advanced way would be to use the gateway groups and not set the automatic gateway switching. Examples show gateway groups are necessary if you are looking to balance load across one more more WANs. So far, I have not been able to get this to work well regardless of using the default gateway enabled setting or using the numerous multi-wan setup instructions posted in this forum and all over the internet. Another issue I have noticed is when either WAN goes down or down and back up, the user experience is high latency to internet connections and the pfSense GUI becomes unresponsive or incredibly slow. Restarting the firewall or restarting PHP-FPM seems to restore normal operation until the next time one of the WAN connections bounces or goes down.

    It's been about two months since you posted this as a possible solution, I would be interested to know how it's working out for you not that some time has passed?

    Thanks,

    Markn455