Limiters applied to multiple WAN gateways - second gateway egress throttled to 0.2-0.5Mbps!?
-
Running pfSense+ 22.05 here and I have two WAN connections, - one 350/40Mbits (up/down) and the other 70/20Mbits. I've got the two gateways in a fail-over group with the faster connection as primary. However I have a couple of rules on my LAN interface which do policy-based routing for some LAN hosts to send them out the secondary connection all the time.
I've created a set of 4 floating "match" rules for my traffic limiters (2 rules for each WAN interface, one "in" rule, one "out" rule) and those match rules funnel traffic to my 4 x limiters (each limiter has 1 queue each) for each interface/direction. E.g.
- WAN1 upload limiter queue
- WAN1 download limiter queue
- WAN2 upload limiter queue
- WAN2 download limiter queue
Each match rules use direction (in/out) and then the respective WAN interface and gateway as the only match criteria. I'm also aware of the in/out flip of the logical in/out pipes when assigning queues to an "out" rule versus an "in" rule.
My issue is exactly the one referenced at the start of this thread, whereby when I try and configure the traffic limiters on the secondary (slower) WAN connection, ingress (download) throttling works fine but egress on the secondary WAN interface gets completely obliterated. I struggle to get more than 0.2-0.5Mbps upload (despite the limiter being set to 20Mbps).
Quoting from the thread above describes my symptoms 100%:
The problem is if you set it up like this the download limiter works fine but the upload limiter does not - it blocks the traffic almost completely.
For example: a 3/5 Mbit limiter gives a 0,2/5Mbit internet access....Weirdly, if I increase the egress limiter bandwidth setting for my secondary WAN connection to say 100Mbits (5 times faster than it actually is), it does improve the egress speed (up to about 5-7Mbps), but the limiter status does not show any packages being dropped in either scenario and it makes no sense that I'm having to artificially inflate the egress bandwidth well beyond what it actually is.
This only happens on the secondary WAN connection. The limiter on the primary works perfectly. I've tried tail drop, CoDel and PIE queue management (as referenced in the above linked thread) but none of those make any difference to the symptoms.
The thread I linked to above hints at a bug, but nobody ever acknowledged that possibility in that thread and I've tried just about every combination but cannot get any form of limiter to work on the secondary gateway without killing egress speeds completely (without having to artificially inflating the bandwidth value). Surely this is some sort of bug as the behaviour just seems totally bizarre?
I know I could try and fudge something by putting limiters on the LAN side instead, but that is going to be messy and horrible and not really do what I want, especially during a fail-over scenario.
So any thoughts/wisdom/insight most appreciated as I've run out of ideas.
-
Just to note, I also found another thread referencing the issue, but again, no clear solution:
before the upgrade limiters worked, after upgrade the limiters (without having them changed) only worked only for downloading, not for uploading any more. Upload only lets pass very little data (e.g. some kbit/s to max 1 Mbit/s), most of the time it fails completely. Also ping packets get lost.
-
@tumbleweedcity are you using captive portal? There were several fixes in 23.01 for that and limiters:
https://docs.netgate.com/pfsense/en/latest/releases/23-01.html#captive-portal -
@steveits Nope not using the captive portal function at all.