Playing with fq_codel in 2.4
-
I actually kind of wish I hadn't stopped work on "bobbie", a better policer. fq_codel is far too gentle and has the wrong goal for inbound shaping. Yes - it works better than anything we've ever tried, but a better policer would have zero delay for all packets at a similar cost in bandwidth and far less cpu. I think. Haven't got around to it. (basically you substitute achieving a rate in a codel like algorithm instead of a target delay). Tis research for someone else to do, I'm pretty broke after helping get sch_cake out the door.
-
Over here ( https://github.com/dtaht/fq_codel_fast ) I'm trying to speed up fq_codel a bit, and add an integrated multi-cpu inbound shaper.
-
@dtaht - thanks for the response - that makes a lot of sense.
Event though I used a cable modem in the example in my previous post, the question was really about GPON. I suppose what ends up happening is that there are buffers at at the GPON card (hardware) for both upload and download. If there is too much data being pushed into the link from upstream servers (i.e. a bunch of people downloading), the buffers start to fill up and packet delay (bufferbloat) occurs for downstream users. Since GPON bandwidth is generally shared among several users, I suppose the severity can vary depending on the amount of users, their usage patterns, and level of congestion. Furthermore, I suppose it's likely easier to experience bufferbloat on the uplink direction since GPON is asymmetric (2.4Gbit down / 1.2Gbit up).
In my personal experience with a gigabit GPON link I have been fortunate: I can set inbound/outbound shaping at >95% of max bandwidth and still not experience any significant delay/bloat.
-
Yes, the GPON folk did not think hard about buffering, and it won't be much of a problem in their early deployments until they start oversubscribing more links. This is a flaw repeated time after time in this industry - 3g grew to suck, 4g was "better", 5g is going to fix 4g... (and 2g, 'cause so many have exited the band now, can be surprisingly good nowadays).
Recently I gave a presentation to broadcom ( http://flent-fremont.bufferbloat.net/~d/broadcom_aug9.pdf ) Another one of my hopes in the bufferbloat project was that someone would solve the over-subscription problem up front for a change - and I thought we'd have a chance with GPON and gfiber, but the team I was on got dissolved about 9 months from being able to deploy.
Sigh:
http://www.dslreports.com/speedtest/results/isp/r3910-google-fiber
-
I'm going to have some serious issues attempting to measure the reduction of bufferbloat from fq_Codel
This is with shaping disabled. Dear lord, what is my ISP doing? I love fiddling and they're taking that away from me. I just realized I forgot to disable BitTorrent. Explains my low upload speed.
-
I just updated from 2.4.3 to the 2.4.4 release candidate and am keen to try out fq_codel. I haven't read all 434 (!) posts of this thread, but can anyone kindly point me to a basic beginner's guide or instructions to setting it up for a simple home connection?
-
@occamsrazor Jimp did a quick bit on codel during the last Hangout, with a config example.
https://www.youtube.com/watch?v=o8nL81DzTlU&t=380
-
@kom said in Playing with fq_codel in 2.4:
@occamsrazor Jimp did a quick bit on codel during the last Hangout, with a config example.
https://www.youtube.com/watch?v=o8nL81DzTlU&t=380
Thanks. That's a nice and easy guide. I followed all the steps but wondering if I've done something wrong as the results are inferior to without. I'm on a 50MB dsl connection and before this got this result:
<a href="http://www.dslreports.com/speedtest/39052963"><img src="http://www.dslreports.com/speedtest/39052963.png"></a>
I had an A and A+ score so maybe the traffic shaper is not even needed. After enabling the shaper as per the video with WANDown bandwidth set to 50MB and WanUp set to 40MB, I get this:
<a href="http://www.dslreports.com/speedtest/39053743"><img src="http://www.dslreports.com/speedtest/39053743.png"></a>
My bufferbloat score is way worse as is my bandwidth. One thing I notice, when I go to PFSense > Status > Queues it says "Traffic shaping is not configured." - is that normal for this traffic shaping method?
-
go to command prompt and type "ipfw sched show" and print the output please
typo :( sorry
-
@zwck said in Playing with fq_codel in 2.4:
ipfw shed show
Thanks, output is:
ipfw: bad command `shed'I set up the two limiters as in the video:
And the rule in the Firewall Floating tab:
Is there supposed to be something showing in the queues tab?
-
ipfw sched show -
@gertjan said in Playing with fq_codel in 2.4:
ipfw sched showThanks, output is:
00001: 50.000 Mbit/s 0 ms burst 0 q65537 50 sl. 0 flows (1 buckets) sched 1 weight 0 lmax 0 pri 0 droptail sched 1 type FQ_CODEL flags 0x0 0 buckets 1 active FQ_CODEL target 5ms interval 100ms quantum 1514 limit 10240 flows 1024 ECN Children flowsets: 1 BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes Pkt/Byte Drp 0 ip 0.0.0.0/0 0.0.0.0/0 2 738 0 0 0 00002: 40.000 Mbit/s 0 ms burst 0 q65538 50 sl. 0 flows (1 buckets) sched 2 weight 0 lmax 0 pri 0 droptail sched 2 type FQ_CODEL flags 0x0 0 buckets 0 active FQ_CODEL target 5ms interval 100ms quantum 1514 limit 10240 flows 1024 ECN Children flowsets: 2 -
@occamsrazor said in Playing with fq_codel in 2.4:
FQ_CODEL
sched shed shit :D
sorry for the typo, when you run the dslspeedtest you should see in this output if fq_codel is active, which from here it seems like it is.
-
@mattund I watched the video linked before and end up with more question then before. Do we use masks or not. ? Do we use ECN or not, i remember on the upload side we should not use ECN.
-
Not sure what changed but now my setup seems to be working OK...
https://www.dslreports.com/speedtest/39073625
This while doing speedtest:
Shell Output - ipfw sched show 00001: 50.000 Mbit/s 0 ms burst 0 q65537 50 sl. 0 flows (1 buckets) sched 1 weight 0 lmax 0 pri 0 droptail sched 1 type FQ_CODEL flags 0x0 0 buckets 1 active FQ_CODEL target 5ms interval 100ms quantum 1514 limit 10240 flows 1024 ECN Children flowsets: 1 BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes Pkt/Byte Drp 0 ip 0.0.0.0/0 0.0.0.0/0 653 974276 0 0 0 00002: 40.000 Mbit/s 0 ms burst 0 q65538 50 sl. 0 flows (1 buckets) sched 2 weight 0 lmax 0 pri 0 droptail sched 2 type FQ_CODEL flags 0x0 0 buckets 1 active FQ_CODEL target 5ms interval 100ms quantum 1514 limit 10240 flows 1024 ECN Children flowsets: 2 0 ip 0.0.0.0/0 0.0.0.0/0 59 3149 0 0 0Is fq_codel adaptive to bandwidth in some way? I'm wondering what to put for the WAN bandwidth values in the limiter, as my speeds can sometimes vary during the day.
Also, I am running pfBlocker developer version. Should the fq_codel floating rule be above or below all the pfb rules?
pfSense CE on Qotom Q355G4 8GB RAM/60GB SSD
Ubiquiti Unifi wired and wireless network, APC UPSs
Mac OSX and IOS devices, QNAP NAS -
@occamsrazor said in Playing with fq_codel in 2.4:
Not sure what changed but now my setup seems to be working OK...
https://www.dslreports.com/speedtest/39073625
This while doing speedtest:
Shell Output - ipfw sched show 00001: 50.000 Mbit/s 0 ms burst 0 q65537 50 sl. 0 flows (1 buckets) sched 1 weight 0 lmax 0 pri 0 droptail sched 1 type FQ_CODEL flags 0x0 0 buckets 1 active FQ_CODEL target 5ms interval 100ms quantum 1514 limit 10240 flows 1024 ECN Children flowsets: 1 BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes Pkt/Byte Drp 0 ip 0.0.0.0/0 0.0.0.0/0 653 974276 0 0 0 00002: 40.000 Mbit/s 0 ms burst 0 q65538 50 sl. 0 flows (1 buckets) sched 2 weight 0 lmax 0 pri 0 droptail sched 2 type FQ_CODEL flags 0x0 0 buckets 1 active FQ_CODEL target 5ms interval 100ms quantum 1514 limit 10240 flows 1024 ECN Children flowsets: 2 0 ip 0.0.0.0/0 0.0.0.0/0 59 3149 0 0 0Is fq_codel adaptive to bandwidth in some way? I'm wondering what to put for the WAN bandwidth values in the limiter, as my speeds can sometimes vary during the day.
Also, I am running pfBlocker developer version. Should the fq_codel floating rule be above or below all the pfb rules?
The pfb rules are block/reject so it should not matter. This should only apply on pass/match rules depending on what You are trying to achieve.
ps. - I don't thin fq_codel is adaptive bandwidth. Normally I set it to 90% link speed a peak hours(like 8-9PM) in the evening.
-
@occamsrazor Conventional wisdom says to measure your WAN link speed several times throughout the week, at various times of the day, and then use 90-95% of your LOWEST value.
-
@kom Thanks. I was doing some testing last night with the fq_codel download bandwidth set to 50Mbit, and when I was running a popular torrent I hit a sustained download rate of over 100Mbit. So it would seem to me that this "limit" isn't actually a hard limit. Can that be right? Or does fq_codel only affect TCP traffic (my torrent client is set to use uTP)
-
@occamsrazor I have no idea. I'm fairly dumb when it comes to QoS, and know just enough to be dangerous. I haven't played with codel at all.
-
you should not be able to achieve a higher rate than htb (hfsc?) specifies. There are a lot of ways to configure htb wrong (like not classifying all packets correctly), there have also been bugs in some versions.
Your initial result (2mbits!?) implied to me you'd got the kbit or mbit setting wrong.
The oddity of your current result implies you are putting utp traffic into another bin entirely. try to build the simplest possible thing that feeds all packets into a single fq_codel instance. Post your code.
The fq part of fq codel works on all packets, the codel part works well on anything that is tcp-friendly, which certainly includes utp and quic.
I don't know enough about bsd to help, we found trying to configure this stuff in the linux sqm-scripts so painful that we ended up replacing it all with the one-liner that is sch_cake.
