Load balancing on an SG-2100 works but failover doesn't

atlassol

I need a fresh set of eyes to help me with this.

I'm configuring an SG-2100 in my Lab LAN (10.0.2.x). I have "sticky connections" turned off. The WAN and LAN 4 (WAN2) ports are configured to be a multi-WAN gateway group. They go through another network and out through a single ISP upstream firewall (An SG-1100) from the building. They are served DHCP from the upstream firewall.

I'm using speedtest.net for my load balance tests.

Load balancing between these 2 interfaces works very well with traffic being roughly balanced. Here is the gateway configuration for this option:

When I disconnect WAN (from the WAN port) traffic is routed to WAN2. However, when I disconnect WAN2 (from LAN 4), external traffic just stops. It makes no difference if I have the gateways at staggered tiers (Tier 1 and Tier2) or both at Tier 1.

The Gateway Error logs contain the following extract:

Apr 22 14:48:40 dpinger 36420 exiting on signal 15
Apr 22 14:48:40 dpinger 69737 send_interval 500ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 1 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% alarm_hold 10000ms dest_addr 10.0.2.1 bind_addr 10.0.2.106 identifier "WAN_DHCP "
Apr 22 14:48:42 dpinger 69737 WAN_DHCP 10.0.2.1: Alarm latency 0us stddev 0us loss 100%
Apr 22 14:52:56 dpinger 69737 exiting on signal 15
Apr 22 14:52:56 dpinger 80362 send_interval 500ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 1 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% alarm_hold 10000ms dest_addr 10.0.2.1 bind_addr 10.0.2.106 identifier "WAN_DHCP "
Apr 22 14:52:58 dpinger 80362 WAN_DHCP 10.0.2.1: Alarm latency 0us stddev 0us loss 100%
Apr 22 14:58:27 dpinger 80362 exiting on signal 15
Apr 22 14:58:27 dpinger 65901 send_interval 500ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 1 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% alarm_hold 10000ms dest_addr 10.0.2.1 bind_addr 10.0.2.117 identifier "WAN2_DHCP "
Apr 22 14:58:29 dpinger 65901 WAN2_DHCP 10.0.2.1: Alarm latency 0us stddev 0us loss 100%
Apr 22 14:59:27 dpinger 20721 send_interval 500ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 1 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% alarm_hold 10000ms dest_addr 10.0.2.1 bind_addr 10.0.2.106 identifier "WAN_DHCP "
Apr 22 14:59:27 dpinger 65901 exiting on signal 15
Apr 22 14:59:29 dpinger 20721 WAN_DHCP 10.0.2.1: Alarm latency 0us stddev 0us loss 100%
Apr 22 14:59:30 dpinger 20721 WAN_DHCP 10.0.2.1: sendto error: 64
Apr 22 14:59:31 dpinger 20721 WAN_DHCP 10.0.2.1: sendto error: 64
Apr 22 14:59:36 dpinger 20721 exiting on signal 15
Apr 22 14:59:36 dpinger 93478 send_interval 500ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 1 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% alarm_hold 10000ms dest_addr 10.0.2.1 bind_addr 10.0.2.106 identifier "WAN_DHCP "
Apr 22 14:59:38 dpinger 93478 WAN_DHCP 10.0.2.1: Alarm latency 0us stddev 0us loss 100%

Thoughts or suggestions?

Thx

tinfoilmatt

@atlassol This is not going to work since LAN 4 is, in fact, a switchport on the LAN backplane (together with ports LAN 1 - LAN 3)—not a router interface. You can configure only one non-LAN interface on the SG-2100 since there are only two router interfaces total.

~~To do what it seems like you're trying to do, you would need a third OPT interface. But even if you did, what would be the point of WAN and an OPT functioning as failover interfaces?~~

Also, that your "load balancing" configuration works at all is remarkable. But you should be aware that when configured this way and WAN is disconnected, any traffic passing through so-named "WAN2" is not firewalled.

EDIT: Apparently the SG-2100's hardware does allow this per the link that @SteveITS has posted below.

SteveITS

@tinfoilmatt I think OP has done https://docs.netgate.com/pfsense/en/latest/solutions/netgate-2100/opt-wan.html. Or should have.

@atlassol Your second pic has them with the same IP …? That’s not going to work on any configuration.

atlassol

@tinfoilmatt This should work (as SteveITS points out from the PfSense online documentation). I have firewall rules for the Gateway Group defined (of which WAN2) is a member, but you make a good point that I should check that the rules are properly applied in both a load balanced environment as well as a failover (if I get that working). Thx.

atlassol

@SteveITS the issue of duplicate IP#s is one that I didn't see (or that didn't strike me as odd for some reason, but you would be right). I need to set this aside for a couple of days and maybe pick it up after I get some higher priority issues taken care of. Thx.