Multiple Gateways on a single WAN?



  • I'm not sure if this is in the right section as I only have 1 WAN port, or even if its possible with the equipment that I have here

    I have a Netgate SG-2220, which has 1 WAN and 1 LAN port.  I'd like failover on the WAN port which is plugged into a switch, which has 2 connections[gateways] available.

    Ive tried adding both gateways, setting gateway groups with tiers with the appropriate firewall rules, but I just cant get the second gateway to activate if the first one deactivates.

    For some context.  We're in Antarctica and we use 2 satellite connections.  One connection is only available for 12 hours a day (limited satellite visibility) and the other is 24 hours, but is much slower and more expensive (we're talking 128kbps here and thousands of dollars in connection fees).  I'd like the pfsense to use the 24 hour connection (Iridium) until the 12 hour connection becomes available (BGAN) and then automatically switch over to Iridium when the BGAN connection drops later in the day.

    Iridium is on 192.168.0.5 (and added as a gateway with a monitor IP of 10.20.20.30 - which is Iridium DNS)
    BGAN is on 192.168.0.7 (and added as a gateway with a monitor IP of 8.8.8.8)

    WAN is set to 192.168.0.8
    LAN is set to 192.168.2.1 (DHCP 192.168.2.100-250)
    I have 8.8.8.8 and 8.8.4.4 as DNS set in general setup, with 8.8.8.8 for the 0.5 gateway and 8.8.4.4 set for the 0.7 gateway

    A gateway group exists with both gateways added, with BGAN Tier1 and Iridium Tier2

    Under Interfaces > WAN > Static IPv4 config, I have IPv4 Upstream Gateway set to 192.168.0.7, otherwise the connection doesnt work.  I've tried the recommended none (On local area network interfaces the upstream gateway should be "none"), but the connection doesnt work.

    For firewall rules, I've added rules and choose the group for gateway.

    It all works fine if I manuually pick the gateway under Interfaces > WAN > IPv4 upstream Gateway, it's just annoying having to manually change it each time.

    Any ideas?


  • Netgate Administrator

    Antartica, nice.  ;D

    You should be able to do this as long as the two WAN gateways are different so it can route packets to each independently.

    It looks like you have a conflict there. You have 8.8.8.8 set as the monitor IP for BGAN but the DNS for Iridium. Each of those things sets a static route so they are in conflict. I suspect you have no DNS when on the Iridium gateway.
    Swap the DNS assignments if that is the case.

    By default pfSense using the DNS resolver in resolving mode with DNSSec enabled. That can only work with multiwan if you have dfault gateway switching enabled and it's better not to do that if you can.
    Switch the DNS Resolver to forwarding mode and disable DNSSec. It should work with both WANs then.

    You need to have a gateway set on the WAN interface to ensure outbound NAT is active in the default automatic mode. The same NAT rule will apply to both gateways here, that should be fine.

    Try setting a rule to route a specific destination via a specific gateway using an IP you know responds to ping (not Google DNS though as those are already routed). If you have two rules applying to each gateway you can test connectivity to those from a client behind the firewall.

    Steve



  • Wow nice.

    Besides the link being slow: do you have a limit on the amount of data you can transfer over each link?
    How stable are the links?

    You might want to increase the time of the probe interval.
    No use in sending lots and lots of pings on a slow line when you know it there/not there.



  • Thanks for the replies!

    I've configured the DNS as instructed here, and set different IPs for monitoring, but it looks like it uses the BGAN gateway regardless (which is the default gateway).  I can disconnect the Iridium from the switch and the monitor IP still responds, so its not routing them properly.

    General Settings
    DNS Server settings
    8.8.8.8 - 192.168.0.7
    208.67.220.220 - 192.168.0.5

    System/Routing/Gateways
    BGAN (default) 192.168.0.7 199.193.201.12 (monitor IP)
    Openport 192.168.0.5 208.67.220.220 (monitor IP)

    Gateway Groups
    BGAN Tier 1
    Openport Tier 2

    DNS Resolver
    Enable DNS resolver is TICKED
    DNSSEC is NOT TICKED
    DNS Query Forwarding Mode TICKED

    As for the links, the BGAN is on a 30gb month plan, and the Iridium on a 2gb month plan.  The BGAN gives us around 55k/sec download and an uptime of around 80%, whereas the Iridium (Openport) gives us around 8-10k/sec with a 90% uptime - not bad considering we're at the bottom of the world!



  • A quick update that might illicit a response

    No matter how many gateways I add or which ones I choose in the firewall rules, it always uses the default gateway set under interfaces>wan.

    Any ideas on how I can specify which gateway to use under rules when I only have 1 WAN port?  It always uses the one under interfaces regardless…



  • If your switch is capable of vlan tagging then you can trunk your WAN into two subinterfaces. Then you can have a separate subnet per gateway and configure the upstream gateway on the interface.
    I couldn't make it work without separate vlans.

    I don't really understand the idea behind upstream gateway per interface. Why should a L3 networking device with a manageable routing table have (only) one gateway per interface?


  • Netgate Administrator

    Hmm, yes the gateways should be observed even if they're on the same interface.

    The gateway monitoring to public IPs accessible via both gateways is more of an issue. You might be able to choose something internal to each providers network that can only be reached via the correct gateway but still indicates the link is up. A traceroute might show something you can use or the providers themselves might give you something. And, yes, you probably want to set the probe interval to something much longer. The default 500ms adds up to quite a lot of traffic.

    Can we see your routing table?

    Steve



  • Thanks for help guys

    Is this the routing table you need?  Or dod I need to type something at the command line?

    
    default	192.168.0.7	UGS	909	1500	igb0	
    8.8.8.8	192.168.0.7	UGHS	267666	1500	igb0	
    127.0.0.1	link#6	UH	391288	16384	lo0	
    192.168.0.0/24	link#1	U	8182	1500	igb0	
    192.168.0.8	link#1	UHS	0	16384	lo0	
    192.168.2.0/24	link#2	U	3033066	1500	igb1	
    192.168.2.1	link#2	UHS	0	16384	lo0	
    199.193.201.12	192.168.0.7	UGHS	175810	1500	igb0	
    208.67.220.220	192.168.0.5	UGHS	413247	1500	igb0	
    

Log in to reply