Traffic on Tier2 Gateway w/out Failover Event
-
Netgate 6100 running v25.07.1.
The problem was also happening on prior version.Gateway Group in Failover mode, Packet Loss or Latency.
Starlink is Tier 2, primary ISP is Tier 1.
I am seeing significant traffic, 300k steady - 80Mbs burst, over Starlink without any failover events logged. I'm hitting our Starlink 50GB cap in 4-7 days.
There are no firewall rules that specify the Starlink GW. All fw rules either don't specify a gateway or specify the Tier 1 gateway.
Inbound WAN rules only allow traffic on the Tier1 interface so it should go out the same path it came in on.
Traffic monitor shows the gateway IP and the WAN2 interface as the source/destination.- How can I track down the source of the traffic?
- What could be causing traffic in the absence of a failover event?
- How can I prevent this?
Thank you for your help.
-
Also, there are 3 IPSEC tunnels on the WAN interface.
-
Tech support provided the help needed to get this resolved.
-
Set Kill State on Gateway Recovery. This setting is under Advanced-->Miscellaneous, not Gateway or Gateway Groups so it's easy to miss, as I did.
-
Use Diagnostics-->States to view what traffic was using the T2 Gateway (Starlink).
-
Set the Gateway Group as the specified Gateway on the firewall rules that were incorrectly using the T2 Gateway (Starlink).
-
I believe setting the Gateway Group in the rule, vs relying on the Default Route = Gateway Group, was necessary because of this issue specific to Starlink:
Classless static routes received on DHCP WAN can override chosen default gateway
-
-
@pfnewb2016 said in Traffic on Tier2 Gateway w/out Failover Event:
Tech support provided the help needed to get this resolved.
-
Set Kill State on Gateway Recovery fnaf
This setting is under Advanced-->Miscellaneous, not Gateway or Gateway Groups so it's easy to miss, as I did. -
Use Diagnostics-->States to view what traffic was using the T2 Gateway (Starlink).
-
Set the Gateway Group as the specified Gateway on the firewall rules that were incorrectly using the T2 Gateway (Starlink).
-
I believe setting the Gateway Group in the rule, vs relying on the Default Route = Gateway Group, was necessary because of this issue specific to Starlink:
Classless static routes received on DHCP WAN can override chosen default gateway
Despite configuring a default route, your network traffic was incorrectly utilizing the T2 Gateway (Starlink), necessitating the explicit setting of the Gateway Group on firewall rules rather than relying on the default route. What is the specific issue with Starlink's behavior that makes it necessary to explicitly define the Gateway Group in firewall rules, rather than relying on the default route, to prevent incorrect traffic routing?
-
-
@andrekyler
Check the Redmine link in #4.pfSense is overriding the Gateway Group as the DG with the classless static route that Starlink is including in their DHCP offer. Either Starlink needs to stop sending the static route, or pfSense needs to handle it differently. It feels more like an incompatibility than a bug, but way above my pay grade and I doubt Starlink is going to make a change.
Per Jim Pingle 3 yrs ago:
"It's unusual to get classless static routes from DHCP in most cases so the situation has likely never come up before.
From the notes in the source:
# RFC 3442: If the DHCP server returns both a Classless Static # Routes option and a Router option, the DHCP client MUST ignore # the Router option.So there should maybe be more logic there as well to treat the default as a gateway replacing the router option (e.g. ignore the route and set it as the value of $new_routers)"
A few related discussions show it's affecting other L3 devices:
https://www.reddit.com/r/Starlink/comments/msfy4k/dhcp_option_121_no_more_need_to_add_a_static/https://community.ui.com/questions/Gateway-target-0-0-0-0-and-not-routing-packets/8576234c-5d61-4ef7-81e0-27ac5b07d7e1
https://documentation.meraki.com/SASE_and_SD-WAN/MX/Design_and_Configure/Deployment_Guides/Meraki_and_Starlink_Deployment_Guide