Forcing routing a destination network always to a specific gateway



  • Both my ISPs have route hops through private IP address space on the way to eventually getting to the real internet. I know the general address range that each happens to use (from observing traceroute). And I have got lucky and they are both using different chunks of private address space that is different to what I use myself. So I make an Alias like:
    SubisuInternal 10.10.0.0/16
    Subisu is on OPT1. So I make a rule on my LAN(s):
    Pass source LANaddress destination SubisuInternal gateway OPT1_DHCP.
    Gateway OPT1_DHCP has a monitor IP like 8.8.4.4 out on the real internet - so apinger can determine if the real internet is reachable through the gateway.

    When apinger thinks that OPT1 is up, it works. But when there is high packet loss, or latency, and apinger thus declares the link down, the pf rule set becomes like this:

    # Gateways
    GWWAN_DHCP = " route-to ( vr0 10.49.223.250 ) "
    GWOPT1_DHCP = "  "
    GWTestNets = " route-to ( vr1_vlan20 10.11.20.1 ) "
    GWVPNclients = "  route-to { ( vr0 10.49.223.250 )  }  "
    GWInetGeneral = "  route-to { ( vr0 10.49.223.250 )  }  "
    GWSubisu1 = "  route-to { ( vr0 10.49.223.250 )  }  "
    GWADSL1 = "  route-to { ( vr0 10.49.223.250 )  }  "
    
    

    All those GWInetGeneral, GWSubisu1, GWADSL1 are gateway groups and have been correctly routed to the GWWAN_DHCP (because OPT1_DHCP is down).

    But I really want to be able to still traceroute out OPT1_DHCP to addresses I know in the Subisu internal network. Then I can send them an email (via the other ISPs gateway!) to tell them at which hop in their internal network the packet loss or latency is being introduced.

    The actual physical link to the rooftop ISP device is up, and pfSense has received an IP address on OPT1 with a gateway. So it can physically get out that way to something (just not reliably to the internet as a whole).

    The problem is:

    GWOPT1_DHCP = "  "
    

    So my rule/s that specifically specify OPT1_DHCP GW end up with no "route-to" specified and thus go to the default gateway.
    If I check System->Advanced, Miscellaneous, Skip rules when gateway is down, then GWOPT1_DHCP does not even get included in the rule set, and the rules do not appear, and thus this SubisuInternal traffic goes out the default gateway.
    I would like a 3rd option - to keep the rule as is, even though the gateway is "down" according to apinger.

    Or should this be done just by putting an ordinary pass rule in for that destination and also having a static route that sends it to a particular gateway?
    (thus passing the traffic to the normal routing table and letting the static route do its thing)

    Hmmm - maybe I answered my own question there, but interested in others comments about this kind of situation.