Multiple WAN gateways same subnet and fw interface 2.3.1

  • I have a firewall cluster with two nodes (each 2.3.1-RELEASE-p1) and I'm using CARP for shared virtual IP addresses on each set of firewall interfaces such as (all static, no DHCP assignment of firewall interfaces).

    fw1 WAN, fw2 WAN, CARP
    fw1 DMZ, fw2 DMZ, CARP
    fw1 LAN, fw2 DMZ, CARP

    Rule, state and DHCP server synchronization between the two firewalls works well and CARP failovers work well.

    Now I'm trying to add a failover ISP link to the cluster and I'm having troubles with the Gateway Group.  I can add a second gateway of a lower tier via another physical interface on either of the cluster firewalls and the Gateway Group functions as expected for the firewall that the failover gateway is attached to - with the Gateway Group specified in the outbound firewall rules, when the tier 1 gateway disappears, traffic is directed out the tier 2 gateway - all good.

    When I try to share the second gateway between the two clustered firewalls via a single shared WAN subnet, the Gateway Group doesn't seem to work properly.  When the primary gateway (tier 1) is taken offline, both gateways go into failure mode instead of failing over to the secondary (tier 2) gateway.

    Firewall configuration that works:

    ISP1 –---|
    WANGW*            |-- WAN (re0) - fw1
    Tier 1
      |                |-- WAN2 (ue0) - fw1
    ISP2 ---|
    Tier 2

    *=default gateway for the firewall interface/subnet, upstream ping destinations shown under each gateway

    Firewall configuration I want to use so the tier 2 gateway is available to both cluster members:
    ISP1 –|
    WANGW*          |        |
    Tier 1          |-- WAN (re0) - fw1
      |            |
    GWGroup1      |-- CARP -
      |            |
      |            |-- WAN (re0) - fw2
      |            |
    ISP2 --|
    Tier 2

    • = default gateway for WAN interface/subnet, upstream ping destinations shown under each gateway

    I suspect the pinger process is using the default gateway (WANGW) for both the external check IP addresses versus forcing the ping out each respective gateway .. but I don't have any specific evidence to prove that yet.  I haven't updated yet to current (2.3.2) to see if there are more options or a resolution to this problem, although I have followed the conversations in the forum around possibly using multiple IP addresses / DNS queries for determining quality of the link.

    My question: Is there any way to have the Gateway Group work if the two gateways are on the same subnet (and thus on the same firewall interface)?

    Thanks for any thoughts/feedback .. apologies if I've missed some needed information.