Playing with fq_codel in 2.4

fabrizior

@mind12 I drove myself nuts trying to tune the limiter & queue knobs at various limiter bandwidth settings on a 400/25 mbps service with and without load (100 sockets generating ~30+ MB/s download throughput).

What I’ve learned from folks here (thank you):

The only setting that induced definitive change in my test results was the bandwidth limit. The other settings recommended seem sufficiently high to allow proper functionality under load while varying the limiter bandwidth as suggested to identify the required headroom.

The general consensus seems to be that, once configured for appropriate headroom based on your provisioned rates, any variable and higher-than-desired latency results are likely induced somewhere upstream with the ISP’s equipment suffering from bufferbloat (or over-provisioning).

If speed tests and bufferbloat latency numbers stay fairly stable with and without significant load running in parallel with the test, then your side of things is well-tuned.

@pentangle said in Playing with fq_codel in 2.4:

@mind12 If the results don't change then you are within your bandwidth limits. The idea being that the limiters and FQ_CoDel need a certain amount of 'headroom' in order to operate (shuffling smaller packets to the front of the queue, etc), and so it'll operate well until it doesn't have enough 'headroom' to play with. You need to determine that headroom (bandwidth limit) and the easiest way to do it is to edge the bandwidth limits up until latency takes a nosedive at which point you know that your FQ_CoDel's efficiency is being impaired by the amount of headroom it has to play with, so you dial it back a notch and voila - the fastest throughput you could get whilst retaining low latency.
As regards other limits, I suggest you do a little reading on what those do for you. The settings I gave should be more than adequate for your connection - they work well with my 300/50 connection here.

MoonKnight

Got this test without playing with fq_codel
I do have 500/500 connection

Pentangle

@ciscox With a 500/500 connection, unless you're regularly maxing it out (unlikely) then you might not find shaping is necessary for you.

Pentangle

p.s. here's mine whilst watching a Youtube video at 1080p (because I couldn't be bothered to pause it:
https://www.waveform.com/tools/bufferbloat?test-id=1117b948-fafd-4eaa-9332-1b3a09c50819

thiasaef

I did everything according to the instructions in the reply to #815, but traceroute does not work. Any idea how to fix this? Adding the icmp exception rule to LAN is not an option for me.

q54e3w

@thiasaef said in Playing with fq_codel in 2.4:

Adding the icmp exception rule to LAN is not an option for me.

Curious why? It might help folks advise if we understand.

thiasaef

@q54e3w

I have multiple lan interfaces (so I thought it would be a bad idea to try that). And I don't understand why the guide does not work (I'm still on 2.4.5-RELEASE-p1).

bartkowski

@thiasaef I have mine on the Floating (with Quick checked) applied to WAN. May that can work for you?

thiasaef

@bartkowski, my floating rules look like this:

Traceroute output:

traceroute netgate.com
traceroute to netgate.com (208.123.73.73), 30 hops max, 60 byte packets
 1  _gateway (192.168.20.1)  0.098 ms  0.138 ms  0.080 ms
 2  208.123.73.73 (208.123.73.73)  2.615 ms  2.822 ms  4.052 ms
 3  * * * 
 4  208.123.73.73 (208.123.73.73)  22.185 ms  17.234 ms  17.226 ms
...
 8  208.123.73.73 (208.123.73.73)  18.800 ms  18.792 ms  21.285 ms
 9  * * * 
10  * * * 
11  208.123.73.73 (208.123.73.73)  167.760 ms  169.189 ms  169.182 ms
...
15  208.123.73.73 (208.123.73.73)  167.513 ms *  164.364 ms

thiasaef

I'm stupid ... all I had to do to make it work was to enable the --icmp option in traceroute, since traceroute uses udp by default on linux.

PS: Could someone explain me why fq_codel still works in both directions when I disable the 3rd floating rule (WAN-In FQ-CoDel queue).

mind12

@thiasaef Are you really sure that it works?
I made the same mistake before that the states were not cleared to the test IPs and resulted the same as before. Make sure to kill all states to the testing server before testing again.

thiasaef

@mind12 it definitely works if I add the -I flag to the traceroute command, but the 1st floating rule (policy routing traceroute workaround) seems to have nothing to do with it.

I logged the outgoing traceroute traffic both with and without the -I flag using Wireshark, but I could not find any packets of the ICMP subtype: Traceroute.

I would be glad if someone with more expertise than us would chime in on this.

mind12

@thiasaef I replied to this I'm sorry, not the traceroute part

"PS: Could someone explain me why fq_codel still works in both directions when I disable the 3rd floating rule (WAN-In FQ-CoDel queue)."

thiasaef

@mind12 if I disable the 3rd floating rule, reset the firewall state table and then run the waveform bufferbloat test, I get the following result: https://www.waveform.com/tools/bufferbloat?test-id=9cced6ab-e6a9-48b9-a7a5-c562dc7df528, my limiter is set to (96, 36).

tomashk

@thiasaef

I believe you need third rule (WAN in) only if somebody initialize connection from "internet" and connects to your WAN (so you also need some other rule to allow connection from "internet" to your WAN)

Your WAN out rule makes almost all work because it is working with traffic from your LAN going to internet and all responses (matching response is like the same traffic as your WAN out).

Because of that for fq_codel I have only one rule (but I also use different approach - when you have rule on "LAN in" there is no problem with ping and traceroute)

I hope it makes sense and I'm not wrong :)

thiasaef

Flent is amazing - simple yet incredibly powerful! The result after optimizing my VVDSL line:

Before:

https://www.waveform.com/tools/bufferbloat?test-id=9bee8331-5e7f-4503-a74d-d146ab511ab9

rrul-2021-05-07T200244.228034.vvdsl_native.flent.gz

After:

https://www.waveform.com/tools/bufferbloat?test-id=1ccd779a-c3f3-4b0f-be46-22638b446d96

rrul-2021-05-07T200823.580873.vvdsl_fq_codel.flent.gz

sikita

Hi, what about "hn ALTQ support" setting? Is it recommended enabled or disabled in 2.5.1?

thiasaef

Is there a recommended strategy for when the WAN speed fluctuates significantly, such as with a cellular connection (e.g., LTE)?

robnitro

@thiasaef
Not that I know of. If you can change the settings via a script, perhaps you can run a loop that runs a ping and when its high, drops the speed and checks again

Gargoyle router which is a version of openwrt does it well though, auto speed adjust.

user7364

Hi guys,

just want to ask how to add a 1mbit guarantee to a voice vlan on the limiter? I am a bit scared that voip does not work if someone in the network max out upload/download ... i already use QoS as best i can, but i think reserved bandwith for the vlan is maybe safer?