Limiters don't always work?
-
Okay, here's my situation:
Multi-WAN, single router, running OpenVPN and Wireguard.One of my ISPs fails to throttle speeds at our plan max (500Mbps); when we pull anything more than that, they just drop the excess packets. Annoying.
So, I created two limiters @ 470Mbps. I applied these on the LAN interface and ran speed tests, but the limiters only worked in one direction (incoming). Fair enough, the interface rule is only in the 'in' direction. So I tagged LAN traffic with "Throttle_Outgoing", and created a floating match rule (WAN out) with the limiter applied, matching on that tag.
It seems to work some of the time. If I run a speedtest from a machine on the LAN, we don't exceed 430Mbps or so. If I set the limiters to say, 100 or 200 or 300, same thing. But there is another machine on the same LAN subnet that seems to be able to pull traffic faster than the limiter allows, but only by ~20Mbps.
When this happens, we start seeing packet loss either on our gateway, or on one of our VPNs.
So I guess my question is: what is the proper way to put a throttle in both directions on my internet connection, and have pfSense respect it? -
@ash-0 on the rule on LAN the directions are from the interface’s perspective. You need 470 in and 470 out.
No mask on the limiter if you want all devices to share the same limit.
You can’t limit what the web server sends but you can throttle it coming out of pfSense.
-
@SteveITS Yep, I did all that.
-
@ash-0 Not sure if it's still true, but when I first set up limiters I found multiple sources of information claiming that what you want are floating rules. And specifically, they must not be "quick" (i.e. they should be normal "last match" floating rules) and the action must be "match", not "pass".
I have such floating match rules for WAN out and WAN in and the behavior seems to be what I would expect, at least as far as I am aware.
-
@TheNarc The wizard makes traffic shaper rules floating by default. We have our office building set up with rules on LAN though. Each tenant has their own rule to allow outbound connections from their IP and set the limits for down+up. Per the docs "Limiters may be applied on normal interface rules, or on floating rules. On floating in the out direction, the In/Out selections are flipped conceptually."
I'll admit to not having tested every single one to see if any are just over the assigned limit, however. A limit/rule would apply to new connections, of course, not open states.
@ash-0 Any chance there is another rule matching, above the one with the limiter?
-
@SteveITS no, I don't think so; all I have are the floating match rules for limiting WAN in/out on the one gateway, and on the LAN I have some block rules, a rule to match the one host in question (with limiters), and a default-allow. I stopped doing the packet tagging.
And, ultimately I just moved my one chatty host on to the other gateway, because that one doesn't require me to do anything, it just doesn't allow me to exceed my bandwidth. But I'm concerned that on my primary WAN, the issue is going to return since I'm convinced at this point I'm not doing it correctly.
-
I noticed today that if I change the gateway on the rule on my host in question, the limiter works. So I disabled the floating match rules I had on WAN in/out that also had limiters, and the problem went away. WAN still spikes up to 50Mbps past my limiter, but LAN doesn't, so the additional traffic is coming from somewhere else. I guess the source of my problem was having limiter-enabled floating match rules on WAN as well as LAN rules. I don't know why yet.. maybe it makes the limiter additive, even though it was the same limiter.
-
@ash-0 You're using multiple WAN/policy routing? I thought there was a thread about that and limiters at some point relatively recently. Maybe I was thinking of https://forum.netgate.com/topic/197993/limiter-source-mask-now-after-nat-when-using-gateway-groups-2-8-change/ but that's about groups.
-
@SteveITS Hmm, yeah, I don't think that bug is applicable to my situation, beacuse they're using source masking to apply limiters to individual /32s, wheras I'm looking to throttle traffic collectively on a gateway, no source mask in play.