FQ_CODEL drops and speed limit
-
Hi all,
I have a basic FQ_CODEL setup, floating on WAN. No custom setting apart from speed limits.
Line is 210/21Mbit from VirginMedia UK.When my UP codel limit is 19Mbit I see drp (dropped) packages when doing speed test...While when limit is set to 25Mbit (more than what my line can handle) I see no drops when speedtesting.
Why is that?? It seems counterintuitive..
Thanks
-
It is how bandwidth is regulated, instead of having packets stored in a buffer causing bufferbloat fq-codel drops packets to keep latency low, nothing out of line there.
-
@bobbenheim should dpinger be low and without variation when CODEL works ok?
thanks
-
@daemonix monitoring show one minute averages so it might be easier to see the difference by pinging google, or something, while you are doing a speed test.
-
@bobbenheim i got some weird results. pinging 8.8.8.8 is ok while uploading + testing upload simultaneously (like two test from diff hosts) but goes completely dead "Request timeout for icmp_seq" when on full download test. (197ish mbit)
-
@daemonix that is because it's dropping ICMP packets, make a floating rule on Interface:WAN, Direction:out, Protocol:ICMP and choose your Gateway in advanced options. This topic has been discussed several times in the thread Playing with fq_codel in 2.4 but doesn't give a clear answer to why it does that, but omitting ICMP traffic from fq-codel is a fix that makes ICMP traffic work as intended and one i use myself.
-
@bobbenheim thanks!
-
I can recommend using Flent to test and tuning fq-codel as it utilizes download and upload at the same time. You can activate Windows subsystem for Linux and install the Ubuntu app and run it in windows if you are using Windows 10 like most might be doing :)
-
@bobbenheim perfect I didnt know about it! (No windows here :) )
-
@bobbenheim I have a question! Should under test load the RTT and standard deviation stay almost the same (or at least low) if fq_codel is configured correctly?
Mine seems to go from 10ms+-3ms to 70ms+-200ms
EDIT: I might have found my mistake! now it went up to 12ms+-7ms
EDIT2: probably not... :( -
@daemonix what speeds are you seeing during speed tests and what have you set in limiters?
You are most likely still hitting the buffer your ISP have set for your line and you need to set a lower bandwidth in your limiter. -
Im on 212ish/21ish Mbit (UK Virgin Media) and my limits are 200mbit and 19mbit just to be safe. (all other setting a the "default")
-
@daemonix try something lower like 200/15 and see if that works and then increase until you hit the buffer.
-
@bobbenheim I just did 100/10. download is perfect (RTT), upload really bad again. The limiter works as I get iperf3 10Mbit. RTT got to 77+-300ms!
Seems weird. Do I need so extra settings?
EDIT: 190 down is also affecting RTT.
-
@daemonix try lowering limit on upload to 800, if you are queuing to many packets you will still get high latency.
-
@bobbenheim is rfsense's RTT and SD values reliable? I also tested setting the monitor IP to 8.8.8.8. RTT reported by PF (on grafana/influxdb) goes crazy while pinging 8.8.8.8 from a Mac seems ok.
(I have added your quick pass rule for ICMP on WAN out before all tests) -
@daemonix you should see a difference in pfsense but i would use flent for testing just because it's easier and it utilizes download and upload at the same time.