Playing with fq_codel in 2.4

dtaht

What you folk are calling a limiter, I call a shaper. The two most common algorithms are token bucket and hfsc. google 'em. I have no idea which or an alternative are being used in the "limiter" concept here (?), nor do I have any idea how this gui is configured, I'm hoping others that are more familiar with bsd and pfsense can chime in. I'm just the co-founder of the bufferbloat project and care deeply seeing about seeing rfc8290 get deployed properly everywhere.

I do think you need to enable things on the wan and queue sections but all I can point to at the moment is how to set up "sqm" properly on things like openwrt. Perhaps one of those tutorials would help in creating an analogy for this OS?

https://openwrt.org/docs/guide-user/network/traffic-shaping/sqm

gsmornot

I'm trying this with a gigabit service and so far no matter what instructions I follow codelq performs better for me. With codelq I see an A each time on the dslreports test. With fq_codel I see a C.

00001: 900.000 Mbit/s 0 ms burst 0
q65537 50 sl. 0 flows (1 buckets) sched 1 weight 0 lmax 0 pri 0 droptail
sched 1 type FQ_CODEL flags 0x0 0 buckets 0 active
FQ_CODEL target 5ms interval 100ms quantum 1514 limit 10240 flows 1024 ECN
Children flowsets: 1
00002: 940.000 Mbit/s 0 ms burst 0
q00002 50 sl. 0 flows (1 buckets) sched 2 weight 1 lmax 0 pri 0 AQM CoDel target 5ms interval 100ms ECN
sched 2 type FQ_CODEL flags 0x0 0 buckets 0 active
FQ_CODEL target 5ms interval 100ms quantum 1514 limit 10240 flows 1024 ECN
Children flowsets: 2

Hugovsky

Remember to flush firewall states between each test.

EDIT: what is your hardware and what packages do you use?

gsmornot

Hardware is SG-3100 and packages are Avahi, nut, openvpn client export, and pfblockerng. So, not a lot. Overall the network is great, the only time I see the score is testing. I'm just saying my score is better with codelq than fq-codel after following the video guide. I reset states between testing.

dtaht

@gsmornot your post was fq_codel both times... Does your bandwidth change vs a vs codelq? If you are running out of cpu...

sensemann

Hello!

I want to priorize two things:

video-livestream (one server in my DMZ)
VoIP
So thats guaranteed that these to applications always have enough bandwidth.
-> If the videostream need all, give them all (expect some 100kbit for the telephones).

How can I reach this goal with fq_codel in 2.4.4?

I already did the setup which was shown in the youtube-video/hangout with the effect, that the download is now around 175Mbit and the upload only 170KBit! (my cable line is around 220 down, 20Mbit up)

00001: 215.000 Mbit/s    0 ms burst 0
q65537  50 sl. 0 flows (1 buckets) sched 1 weight 0 lmax 0 pri 0 droptail
 sched 1 type FQ_CODEL flags 0x0 0 buckets 0 active
 FQ_CODEL target 5ms interval 100ms quantum 1514 limit 10240 flows 1024 ECN
   Children flowsets: 1
00002:  20.000 Mbit/s    0 ms burst 0
q65538  50 sl. 0 flows (1 buckets) sched 2 weight 0 lmax 0 pri 0 droptail
 sched 2 type FQ_CODEL flags 0x0 0 buckets 0 active
 FQ_CODEL target 5ms interval 100ms quantum 1514 limit 10240 flows 1024 ECN
   Children flowsets: 2

Thanks for help in advance!
sensemann

gsmornot

@dtaht said in Playing with fq_codel in 2.4:

@gsmornot your post was fq_codel both times... Does your bandwidth change vs a vs codelq? If you are running out of cpu...

Not sure I follow the reply. In each case for testing I use pretty much all of the CPU. The SG3100 is a fanless ARM processor. Enough for line rate gigabit but maybe pushing it for fq_codel but that is just a guess. CodelQ, setup via shaper (vs limiter for fq_codel) seems to perform just a bit better for my setup. Do you think its CPU related?

adx442

So, I was using the System Patches package to apply the fq_codel GUI elements to 2.4.3.

When I move to 2.4.4, I assume if I don't apply that patch, everything will remain as it is now? Just checking before I upgrade.

goodthings

Hello!
My connection without any limiters is 105/16, overall A, bufferbloat B, quality A.
When I activate fq_codel as per https://youtu.be/o8nL81DzTlU?t=518 with 100/15, the upload bufferbloat improves to only ~22ms but download bufferbloat skyrockets (1000ms avg, 3000ms max) in dslreports. Overall C, bufferbloat F, quality A+.

I tried a lot of things but the result is the same, even when I drop the bandwidths to 75% of actual, download buffer bloat always ends up way worse than without any limiters. Tried a lot of things, ECN on/off, firewall rules floating or LAN, etc, all the same. I did exactly what the youtube video suggested, why would I get such a result?

q65539  50 sl. 0 flows (1 buckets) sched 3 weight 0 lmax 0 pri 0 droptail
 sched 3 type FQ_CODEL flags 0x0 0 buckets 1 active
 FQ_CODEL target 5ms interval 100ms quantum 1514 limit 5120 flows 1024 ECN
   Children flowsets: 1
BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes Pkt/Byte Drp
  0 ip           0.0.0.0/0             0.0.0.0/0     63870 95546322 76 114000 363
00004:  15.000 Mbit/s    0 ms burst 0
q65540  50 sl. 0 flows (1 buckets) sched 4 weight 0 lmax 0 pri 0 droptail
 sched 4 type FQ_CODEL flags 0x0 0 buckets 1 active
 FQ_CODEL target 5ms interval 100ms quantum 1514 limit 10240 flows 1024 NoECN
   Children flowsets: 2
  0 ip           0.0.0.0/0             0.0.0.0/0      263    14216  0    0   0

any ideas/suggestions would be welcome.

@goodthings What hardware do you have? It might be that fq_codel being active is causing your CPU to spike too hard.

If you've already got great bufferbloat figures you can rest happy anyway I'd think.

@sensemann FQ_CODEL doesn't give you knobs to tweak flows, if you want that, you want to use ALTQ.
The idea of FQ_CODEL is that it doesn't let any single flow starve out the others, so your live videostream and your VoIP should just keep working even if a user triggers off a large torrent.

Given the terrible upload speed, maybe you have a CPU limitation, what hardware are you doing this on?

goodthings

@muppet My hardware is a Netgate SG-2220. There are no other packages running, so the CPU (in top via ssh) seems to peak at 25% load at 100 Mbit.

I experimented some more and find that fq_codel on upload and no queue on download seems to give me the best results. As soon as I enable fq_codel for downloads the bufferbloat jumps from 100ms to 2000ms under load and 15ms to 750ms idle according to dslreports. This is even with 30 Mbit set in the limiter. Similar results with PIE. Curious what could cause this. Tried from different clients, same results.

@goodthings It's a good question - I'm unsure how much testing FQ_CODEL has had on the ARM chipsets, maybe it's just not compatible?

EDIT: This is bollocks, the SG-2200 is not ARM based.

w0w

@muppet

What ARM chipset are you talking about?

@w0w Sorry, I totally farked up. I was conflating the SG-3100 post above which is ARM based, but @goodthings has an Intel Atom.

I think I will stop posting about stuff I don't fully understand, my apologies all.

w0w

@goodthings
Do you have any other shaper enabled simultaneously with limiters?
What packages are you using on firewall?
Post result of "ipfw sched show" and "pfctl -sr | grep dnqueue" commands and full result of dslreports test when you have those huge jumps, also

bafonso

I have a 120/20 connection and I just tried the FQ_CODEL approach but I get a strange result. When I use 20Mbit for upload limiter I get 2mbit effective upload on speedtest (ECN or not)... if I use 40Mbit, then I get 4Mbit. I have found if I simply use CODELQ I have better buffer bloat scores on dsl reports and actually 20mbit upload.

goodthings

This post is deleted!

goodthings

I collected all the info but the forum software / akismet detects my postings as spam when i try to attach a dslreport. This all seems more trouble than it's worth so I'm giving up for now, thank you for trying to help.

occamsrazor

A few newbie questions..... What is the best way to monitor what is going on with the fq_codel limiter: "ipfw sched show" from the command prompt, or Diagnostics > Limiter Info.... or it's the same? The latter seems to provide an additional line of info:

Queues:
q00001  50 sl. 0 flows (1 buckets) sched 1 weight 0 lmax 0 pri 0  AQM CoDel target 5ms interval 100ms ECN
q00002  50 sl. 0 flows (1 buckets) sched 2 weight 0 lmax 0 pri 0  AQM CoDel target 5ms interval 100ms ECN

though I guess it is same information.

Can anyone give some information (or point me to a resource elsewhere) on how to interpret the output of "ipfw sched show"? Here is a sample I posted above, but I'm confused about what each part means.....

Shell Output - ipfw sched show

00001:  50.000 Mbit/s    0 ms burst 0
q65537  50 sl. 0 flows (1 buckets) sched 1 weight 0 lmax 0 pri 0 droptail
 sched 1 type FQ_CODEL flags 0x0 0 buckets 1 active
 FQ_CODEL target 5ms interval 100ms quantum 1514 limit 10240 flows 1024 ECN
   Children flowsets: 1
BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes Pkt/Byte Drp
  0 ip           0.0.0.0/0             0.0.0.0/0        2     2932  0    0   0
00002:  40.000 Mbit/s    0 ms burst 0
q65538  50 sl. 0 flows (1 buckets) sched 2 weight 0 lmax 0 pri 0 droptail
 sched 2 type FQ_CODEL flags 0x0 0 buckets 1 active
 FQ_CODEL target 5ms interval 100ms quantum 1514 limit 10240 flows 1024 ECN
   Children flowsets: 2
  0 ip           0.0.0.0/0             0.0.0.0/0        2      112  0    0   0

Why 0 buckets 1 active?
Why 0 flows?
Tot_pkt/bytes Pkt/Byte Drp? What do each of these refer to?
If you don't see any Drp (drops?) does that mean fq_codel was not having any effect?
What does target 5ms interval 100ms refer to?

PS - It would be really cool if there were some kind of dashboard widget to show what's going on the traffic shaper/limit in realtime.

Edit: Also I still don't understand the relationship between the bandwidth limit set and what is subsequently possible. Here is a well-seeded torrent running with bandwidth set to 50mbit download 40mbit upload.....

00001:  50.000 Mbit/s    0 ms burst 0
q65537  50 sl. 0 flows (1 buckets) sched 1 weight 0 lmax 0 pri 0 droptail
 sched 1 type FQ_CODEL flags 0x0 0 buckets 1 active
 FQ_CODEL target 5ms interval 100ms quantum 1514 limit 10240 flows 1024 ECN
   Children flowsets: 1
BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes Pkt/Byte Drp
  0 ip           0.0.0.0/0             0.0.0.0/0      164   236431 13 19396   0
00002:  40.000 Mbit/s    0 ms burst 0
q65538  50 sl. 0 flows (1 buckets) sched 2 weight 0 lmax 0 pri 0 droptail
 sched 2 type FQ_CODEL flags 0x0 0 buckets 1 active
 FQ_CODEL target 5ms interval 100ms quantum 1514 limit 10240 flows 1024 ECN
   Children flowsets: 2
  0 ip           0.0.0.0/0             0.0.0.0/0       43     2132  0    0   0

....and yet download bandwidth was able to far exceed the limit set.