QoS / Traffic Shaping / Limiters / FQ_CODEL on 22.05

zrnkkjyi

@luckman212 I stopped using this firewall for OpenWrt, because OpenWrt is better and has the best solution to fix the bufferbloat to date and it's called Qosify.

Qosify: new package for DSCP marking + cake - OpenWrt Forum

Destello

This post is deleted!

luckman212

Just adding some notes from redmine...

Currently this bug (#13026: Limiters do not work) appears to be blocked by the following 2 bugs:

12579 says "#12003 should be merged first" but even though progress is at 0%, it appears a patch has been merged. 13027 also has a merge request pending. Target on 13027 is 22.09—hope we don't have to wait that long to have functioning limiters again!

@jimp is there any movement going on with this (imo) important bug? Thanks

marcosm

@luckman212 It's being worked on.

luckman212

@marcos-ng Good to know. I just updated to 22.05.a.20220426.1313 and was going to test a bit, but I'll keep waiting for some news on redmine.

luckman212

Just reporting back here to wrap this up. I've been busy with other stuff but finally got around to retesting this. All working great on 22.05.r.20220604.1403. It's so nice to have this working again! Increased WAF factor by 10x.

betapc

@luckman212 Just one question. Did you use the same settings on post # 1 or did you change something?

Thanks.

luckman212

@betapc yes I'm using the same settings described in the guide.

Keylimesoda

@luckman212 I'm still seeing this issue in 22.05-REL?

No matter what limiter I'm putting on the WANDown it's still going full-bandwidth and bufferbloat is suffering.

Gertjan

@keylimesoda said in QoS / Traffic Shaping / Limiters / FQ_CODEL on 22.05:

I'm still seeing this issue in 22.05-REL?

That's why @luckman212 said :

@luckman212 said in QoS / Traffic Shaping / Limiters / FQ_CODEL on 22.05:

All working great on 22.05.r.20220604.1403.

Go from stock 22.05 to 22.05..r.20220604.1403 and re test ;)

netblues

@gertjan Are you sure?
Stock 22.05 was 22.05-RELEASE (amd64)
built on Wed Jun 22 18:56:13 UTC 2022
FreeBSD 12.3-STABLE

20220604 is older.
And as far as redmine says.
https://redmine.pfsense.org/issues/13026?tab=history
It has been resolved in 22.05

Gertjan

@netblues said in QoS / Traffic Shaping / Limiters / FQ_CODEL on 22.05:

It has been resolved in 22.05

I stand corrected I guess.

netblues

@keylimesoda said in QoS / Traffic Shaping / Limiters / FQ_CODEL on 22.05:

@luckman212 I'm still seeing this issue in 22.05-REL?

No matter what limiter I'm putting on the WANDown it's still going full-bandwidth and bufferbloat is suffering.

Which brings us back to the original question regarding wandown

I just checked it on 22.05 and it DOES work.
Do a speedtest and see what speed are you getting and if it is consistent.
(eg. wan links from wisp's tend to fluctuate in speed)
And do keep in mind that wan down can only be controlled indirectly, by dropping tcp packets and hoping that ack-window will take care the rest.
If your incoming traffic is e.g. udp and there is no mechanism in the app that utilises udp traffic to request fewer data, there is no way to stop flooding your download.

Keylimesoda

@netblues I'm getting different behavior on different speedtests which is odd. Could be the UDP issue?

On Ookla and FAST, the WAN down shows as scaling correctly. And in fact on FAST it shows almost no bufferbloat.

On Waveform test, it seems to ignore the download limiter, and shows all kinds of wonky behavior around buffer bloat on download (ranging from 20ms to 120ms). Upload is steadier.

In an informal test (watching ping times to google.com while running Ookla), I am seeing significant ping impact (from 16ms to 40-70ms) under load, which suggests that something is still off.

cwagz

@keylimesoda said in QoS / Traffic Shaping / Limiters / FQ_CODEL on 22.05:

@netblues I'm getting different behavior on different speedtests which is odd. Could be the UDP issue?

On Ookla and FAST, the WAN down shows as scaling correctly. And in fact on FAST it shows almost no bufferbloat.

On Waveform test, it seems to ignore the download limiter, and shows all kinds of wonky behavior around buffer bloat on download (ranging from 20ms to 120ms). Upload is steadier.

In an informal test (watching ping times to google.com while running Ookla), I am seeing significant ping impact (from 16ms to 40-70ms) under load, which suggests that something is still off.

I have been seeing really weird results on the Waveform test. It seems broken. I have a 1Gbps connection and it will show over 1200Mbps and then go all over the place showing increased latency. I have my limiter set at 940Mbps. Speedtest (Ookla) seems to respect the limiters.

netblues

@keylimesoda Try liming to a ridiculous slow rate, something like 10Mbit down., 10Mbit up and see what happens there

If the line suffers from great speed spikes limiters don't work well.
I see this on a install that has a stable ftth line, and a 5g
on the same box.
5g leaves a lot to be desired at C, while ftth is a+ on bufferbloat tests

karnaahai

I just went ahead and bought a TAC Pro sub. Order SO22-30515. Hope I can get some assistance next week.

mindwolf

@luckman212 said in QoS / Traffic Shaping / Limiters / FQ_CODEL on 22.05:

Firewall > Rules > Floating:

Action: Pass
Quick: checked
Interface: WAN1
Direction: out
Family: IPv4
Protocol: any #[Change this to TCP & UDP by ctrl + click each in drop-down menu]
Source: WAN1 address
Dest: Any
Gateway: WAN1
In/Out Pipe: WAN1UpQ / WAN1DownQ

Now traceroute and icmp will operate correctly without adding extra rules.

dennypage

@mindwolf said in QoS / Traffic Shaping / Limiters / FQ_CODEL on 22.05:

Now traceroute and icmp will operate correctly without adding extra rules.

You may still find value in the general approach. It's a personal preference, but I prefer to exclude ping request/reply from the limiters because I don't want them delayed or dropped under load. I have a separate floating rule for ICMP request/reply just before the limiter assignment rule(s) to achieve this.

mindwolf

@dennypage

There are many ways to approach this but my suggestion does take icmp and other protocols out of the equation. The firewall floating rule ONLY includes tcp and udp. I just installed pfsense the other night and curiously ran into the same issues running ping and traceroute with my windows laptop having the “repeating” issue along with dropped pings. This change resolved my issue and still controls bloat. Cake has this feature aimed towards a 11:1 or higher rate.

Finding a way to drop duplicate acks is another avenue worth exploring for extending the ingress bandwidth at the expense of more cpu usage. I started with openwrt and the sqm folks learning much over the years.