Slow upload speed with failover WAN
-
For years I've been running with 1 WAN connected to my primary ISP's fiber connection. My default gateway.
Recently I've added a 2nd backup WAN via LTE modem connected by ethernet. Setup a new interface, created a gateway group and set that as the tier 2 interface above my primary ISP. Furthermore only using that gateway group via a firewall rule for specific devices in an alias. So very little (essential traffic) will actually failover.
Noticing my upload speeds have tanked since doing this. Disabling the LTE interface brings upload speeds back to expected.
Inbound and outbound traffic on the hosts i'm testing with clearly appear to be going through my primary ISP's fiber connection as evidenced by traceroutes and speedtest identifying my ISP correctly. Furthermore pfsense graphs on the dashboard show mere bytes of data going over the LTE connection (presumably a few pings to check uptime).
I do have Limiters setup for bufferbloat, they're set at 500Mb, so I don't think that's the issue. The speeds I'm seeing when this is happening are < 100Mb.
Anyone have a thought as to what might be the culprit?