CoDel - How to use



  • @mcwtim:

    The test button within pfsense>system>patches.

    So this is not a directly applicable patch to an installed pfsense which is what I needed to know.

    Thanks.

    Yeah, the patch applies to the source-code. See https://forum.pfsense.org/index.php?topic=76132.0 to get accessto the source-code.

    Please be careful. That patch was posted by me and it is completely untested. There are a few other incorrect values I later found, but this bug is not vital so I neglected to create a new (and very possibly wrong) patch. If I remember correctly, all the values needing correction were in ~/patches/stable/10/altq_codel.diff and can found rather simply with grep.

    I will leave this to the professionals to patch, until I have some experience building pfSense. I wish pfSense's source-code was more accessible. Already I have had to resort to using doktornotor's "illegitmate" pfsense-tools repository on github to refer to code that the unwashed masses can see.



  • Thanks for the clarification and warning. I tried using Codel with a very functional PRIQ with limiter setup this past weekend at a 50+ person LAN party and Codel seemed to inflict a lot of latency (erratic 200+ ms). Disabled it and we went back to our normal 20-50 ms pings to external game servers.

    At home the results seemed to be erratic with a single user and Codel only (different pfSense box). Ended up switching to Barrier Breaker and their SQM fq_codel setup which does actually work very well. Took buffer bloat rating from continuous B-C to a repeatable A+.



  • I wonder if FairQueue in PFSense could be an option for child disciplines also. fq_CoDel, why are you so close yet so far?!



  • the docsis 3.1 standard is a "16ms" target for pie. Pie interpret's target a bit differently than codel. pie is target 20ms in the linux code and pretty different from the docsis-pie version.

    the "target" for codel is what it aims for for local queue delay, while trying to keep the pipe filled. The important part is "keeping the pipe filled". It is certainly feasible to achieve a given queue delay while having terrible throughput (tcp's timing out, and so on).

    we hit the issue (needing to increase target) with sub 2.5Mbit uplinks after publication of the codel paper.

    Codel by itself is very gentle, it gradually reduces queue length to more reasonable amounts and certainly you can briefly see latencies go to 100s, of ms. even thousands of ms on unresponsive traffic.

    fair queuing + codel takes most of the latencies for gamer-ish traffic out of the equation, while still moderating (typically tcp) queue length.

    I am sorry you are having such difficulty with the fairq + codel emulation in pfsense. testing for codel's correct behavior is somewhat straightforward using things like the tcp_upload test in the flent suite, or merely saturating the link and looking for time of first drop, 2nd drop, etc to follow the 1/sqrt(100) rule. To make your life more difficult, there are several mildly different versions of codel out there and I have not reviewed the one in pfsense.

    We tried all sorts of fairq+aqm combining techniques for several years, before incorporating them into one qdisc. two separate modules have flaws - including being difficult to debug.

    Another poster here mentioned they had ping packet loss when using codel. Yes, that will happen. Gotta drop packets, some will be pings. hfsc does some clever priorization so pings more rarely drop - but which would you rather drop, pings? or data?



  • I would rather see what my queue is seeing. So HFSC does mess with the order in which packets are sent within a queue? I was under the impression that HFSC is only a scheduler and only messes with inter-queue ordering, not intra-queue ordering. So if you select a dumb FIFO child queue, will HFSC change the order in which packets from that queue leave the interface? I know fq_codel and any fair queuing AQM will do that.



  • @Harvy66:

    I would rather see what my queue is seeing. So HFSC does mess with the order in which packets are sent within a queue? I was under the impression that HFSC is only a scheduler and only messes with inter-queue ordering, not intra-queue ordering. So if you select a dumb FIFO child queue, will HFSC change the order in which packets from that queue leave the interface? I know fq_codel and any fair queuing AQM will do that.

    You are seeing what your flow's queue is seeing, which is all that really matters (End-to-End principle).

    If you want to know the details of HFSC, read the paper. It is super-fun and the ladies love an HFSC-man.



  • Forgive me if some of this has been repeated already. I just wanted throw in some input on this. I was playing around with CODELQ last night.
    Some observations:
    I was using the speedtest.net test which tests for bufferbloat.
    My link typically achieves 86x5.6Mbps pretty consistently on off peak hours.
    I actually used the bandwidth window to get some settings tweak. Initially I had set the bandwidths to 10% below what my actual speeds were but it cutoff WAY too much of my top end bandwidth. When I did that I was getting an A rating on buffer bloat in both directions but speeds were cut to 75x4.2Mbps
    I finally settled by testing on going higher. It was a balancing act because if I went too high then the bufferbloat was bad again as originally seen. If I went too low, it would knock off too much off the limits of the speeds.

    So for the download (LAN) queue, I went for 87.5Mbps and was able to download 83-84Mbps.
    For the upload (WAN) queue, I went for 6.1Mbps and was able to achieve 5.2 Mbps.
    Both these settings resulted in a "B" rating with approximately 60ms of delay.
    Considering the upload delay had been 512ms at one point, I say not bad to giving up ~7% of my top end upload bandwidth, ~3% download bandwidth.
    I didn't have too much buffer bloat on my download (LAN) because I guess the ISP managed it very well. I believe it was originally at about 100ms

    So this whole thing was trial and error but I consistently get the same results which leads me to believe I have achieved the intended goal.

    I'm using 2.2.2 4G embedded on a Firebox.

    Thanks.



  • Yeah, the whole point of managing bufferbloat is your firewall needs to be the chokepoint, and if your internet connection's rate varies too much, you can get bloat spikes.

    My limited understanding of the bufferbloat problem and some technologies is DOCSIS and DSL don't use native Ethernet which cause them to do some strange batching or fragmenting of your packets. This reduces the efficiently and makes for strange bursts in speeds. The bufferbloat forumns are constantly discussing how to properly emulate how DSL and DOCSIS work in an attempt to artificially create the same bottlenecks in the firewall, which allows better management of bufferbloat.

    All of that being said, you need to set your bandwidth to be less than your peak, and losing some bandwidth is part of the cost of getting rid of bufferbloat when your ISP doesn't manage it for you.

    Personally, I have a 100Mb connection and have my rates set to 99Mb and I get 1ms of bloat on average. But I seem to be a special case.



  • Gazoo - I noticed this too. If I put my full speed in pfsense it is as-if it lowers this amount so it is somewhat less (95%?) of the connection speed you enter for traffic shaping limit.



  • 95% is pretty decent. I see closer to about 97%.



  • @Harvy66:

    95% is pretty decent. I see closer to about 97%.

    Might be 97%, haven't looked in a while. In any event looks like the pfsense coders deliberately lower the speed you enter. Can anyone confirm?



  • I have turned to the dark-side and chose full bandwidth over best latency on my incoming traffic. The average latency increases to ~70ms (~10ms idle) during a link saturating download, but even with artificial throughput limiting I was reaching ~40ms. Addiction to throughput probably stems from years of dial-up… the experience still haunts me. :)

    The difference on uploads is huge though; ~600ms vs ~50ms.

    My ISP is the not-so-great-but-improving Windstream. I have wondered what other people experience on other ISPs and CPE.



  • Since I use HFSC even for download traffic shaping, I can make sure Netflix/Youtube buffering is kept to a minimum. I pretty much can't tell the difference, even when using P2P and saturating the downstream.

    I have 100Mb with a 2:1 ratio between Normal:Low. P2P will be sitting around 97Mb, then I start of Netflix and I pretty much immediately see P2P drop down to ~32Mb/s and Netflix pushes through ~64Mb/s. Even with DSLReports, I still see an A/A+ during the download test. But if P2P is using 0 and DSLReports is using max, and suddenly P2P jumps up to 33, the bufferbloat can get bad, like C/D, but that's because available bandwidth is decreasing instead of increasing and I'm still stuck with regular CoDel and not fq_CoDel.



  • CoDel's default "target/interval" values should be fixed in 2.2.3. Dave mentioned earlier that the "target" is pretty good at dynamically adjusting, so the fix probably has little effect in that area.

    The "interval" is more vital and is noted in the IETF draft as being the only parameter that is required for CoDel to to function, so perhaps changing from  a value of 5ms to a more optimal value of 100ms will improve our CoDel experience. :)

    Disappointingly, I tried loading up a CODELQ queue in 2.2.3, and I still got the old values… Either the install needs to be fresh or my patch is shit and needs more work. See if it is fixed on any of your 2.2.3 setups.



  • I wonder how it'll affect my bufferbloat. Right now I have about 1ms, but it can burst quite high given enough TCP streams growing at the same time.



  • @Nullity:

    CoDel's default "target/interval" values should be fixed in 2.2.3. Dave mentioned earlier that the "target" is pretty good at dynamically adjusting, so the fix probably has little effect in that area.

    The "interval" is more vital and is noted in the IETF draft as being the only parameter that is required for CoDel to to function, so perhaps changing from  a value of 5ms to a more optimal value of 100ms will improve our CoDel experience. :)

    Disappointingly, I tried loading up a CODELQ queue in 2.2.3, and I still got the old values… Either the install needs to be fresh or my patch is shit and needs more work. See if it is fixed on any of your 2.2.3 setups.

    I did a fresh install of 2.2.3 and the target/interval are still inverted.

    [2.2.3-RELEASE][admin@pfSense.localdomain]/root: pfctl -vs queue | grep -i codel
    altq on em0 codel( target 50 interval 5) bandwidth 600Kb tbrsize 1500 
    altq on em1 codel( target 50 interval 5) bandwidth 6Mb tbrsize 6000 
    
    


  • @kieranc:

    @Nullity:

    CoDel's default "target/interval" values should be fixed in 2.2.3. Dave mentioned earlier that the "target" is pretty good at dynamically adjusting, so the fix probably has little effect in that area.

    The "interval" is more vital and is noted in the IETF draft as being the only parameter that is required for CoDel to to function, so perhaps changing from  a value of 5ms to a more optimal value of 100ms will improve our CoDel experience. :)

    Disappointingly, I tried loading up a CODELQ queue in 2.2.3, and I still got the old values… Either the install needs to be fresh or my patch is shit and needs more work. See if it is fixed on any of your 2.2.3 setups.

    I did a fresh install of 2.2.3 and the target/interval are still inverted.

    [2.2.3-RELEASE][admin@pfSense.localdomain]/root: pfctl -vs queue | grep -i codel
    altq on em0 codel( target 50 interval 5) bandwidth 600Kb tbrsize 1500 
    altq on em1 codel( target 50 interval 5) bandwidth 6Mb tbrsize 6000 
    
    

    Fudge…

    Thank you for letting me know. I will get them to revert my commits.

    How can a little fix be so illusive?

    Why must you be so confusing, pfSense-tools!?



  • @Nullity:

    @kieranc:

    @Nullity:

    CoDel's default "target/interval" values should be fixed in 2.2.3. Dave mentioned earlier that the "target" is pretty good at dynamically adjusting, so the fix probably has little effect in that area.

    The "interval" is more vital and is noted in the IETF draft as being the only parameter that is required for CoDel to to function, so perhaps changing from  a value of 5ms to a more optimal value of 100ms will improve our CoDel experience. :)

    Disappointingly, I tried loading up a CODELQ queue in 2.2.3, and I still got the old values… Either the install needs to be fresh or my patch is shit and needs more work. See if it is fixed on any of your 2.2.3 setups.

    I did a fresh install of 2.2.3 and the target/interval are still inverted.

    [2.2.3-RELEASE][admin@pfSense.localdomain]/root: pfctl -vs queue | grep -i codel
    altq on em0 codel( target 50 interval 5) bandwidth 600Kb tbrsize 1500 
    altq on em1 codel( target 50 interval 5) bandwidth 6Mb tbrsize 6000 
    
    

    Fudge…

    Thank you for letting me know. I will get them to revert my commits.

    How can a little fix be so illusive?

    Why must you be so confusing, pfSense-tools!?

    It looks like you're doing it right but the patch hasn't been merged. I've created a pull request on github to do the same thing, let's see what happens

    Do you know why the current codel_alloc(100, 5, 0) results in values of target 50 interval 5? target 5 interval 50 would be better but I dunno where the value of 50 is coming from.


  • Banned

    Was never fixed AFAICT? https://redmine.pfsense.org/issues/4692



  • @doktornotor:

    Was never fixed AFAICT? https://redmine.pfsense.org/issues/4692

    Indeed, has anyone tried the patch? I don't really want to have to build pfsense in order to test it, that's a lot of repos to clone…

    edit: PR has been merged, new values should be applied in 2.2.4 or anything built from here on...



  • I wish  this could be set via config instead of compile time. One problem at a time though, ehh?

    50/5 seems to be working well for me right now. I guess I'll see how my bufferbloat is affected once this change finally makes it. I'm getting 0ms of bufferbloat and full throughput already. According to DSLReports, my bloat can spike, but rarely. More of an issue when doing 32 upload streams.



  • @doktornotor:

    Was never fixed AFAICT? https://redmine.pfsense.org/issues/4692

    https://github.com/pfsense/pfsense-tools/commit/3108a902bd816036a3abffd3ec669767140891a7

    I dunno. I am unsure of many things. :(

    I probably should have updated the redmine submission. The redmine patch was a initial code to show what I had found, hooefulky to help a dev pinpoint the problem.

    The github patches were the best I could do, but I should probably stop trying to patch pfSense considering that I cannot build pfSense to test my code. :(



  • So this is from the latest nightly:

    [2.2.4-DEVELOPMENT][admin@pfSense.localdomain]/root: pfctl -vs queue
    altq on em0 codel( target 50 interval 100) bandwidth 600Kb tbrsize 1500 
    
    

    Interval successfully changed, now we just have to figure out where the target of 50 is coming from….

    Edit: I just set the 'queue limit' to 25 in the GUI and my target is now 25.... Victory?

    Edit2: From 2.2.4 19/07/2015 nightly, with queue limit set to 5:

    [2.2.4-DEVELOPMENT][admin@pfSense.localdomain]/root: pfctl -vs queue
    altq on em1 codel( target 5 interval 100) bandwidth 6Mb tbrsize 6000 
      [ pkts:         85  bytes:       9938  dropped pkts:      0 bytes:      0 ]
      [ qlength:   0/ 50 ]
    
    

    So it wasn't anything I did yesterday that fixed it, but it does seem to be fixed/workable in 2.2.4



  • If qlimit is 0, it defaults to 50, and codel gets the (initial?) target value from qlimit.



  • @Nullity:

    If qlimit is 0, it defaults to 50, and codel gets the (initial?) target value from qlimit.

    is qlimit the queue length, or something else entirely?



  • @kieranc:

    @Nullity:

    If qlimit is 0, it defaults to 50, and codel gets the (initial?) target value from qlimit.

    is qlimit the queue length, or something else entirely?

    qlimit is the queue length which becomes useless when codel is axtive, since codel dynamically controls queue length (AQM).



  • @Nullity:

    @kieranc:

    @Nullity:

    If qlimit is 0, it defaults to 50, and codel gets the (initial?) target value from qlimit.

    is qlimit the queue length, or something else entirely?

    qlimit is the queue length which becomes useless when codel is axtive, since codel dynamically controls queue length (AQM).

    So when using codel the 'queue limit' setting seems to change the target instead… handy, but not very obvious..
    Thanks!



  • Yeah, it is pretty confusing but I'll take CoDel however I can get it. :)
    Ermal ported himself, iirc. Ahead of the curve, that guy! :)

    I still dunno how to view or set codel's parameters when it is a sub-discipline though. Default or gtfo, I suppose…



  • The whole target qlimit thing applies to CoDel for both the scheulder and the child discipline?

    Do you know if the interval changes? The interval is supposed to be 20x the target.



  • @Nullity:

    Yeah, it is pretty confusing but I'll take CoDel however I can get it. :)
    Ermal ported himself, iirc. Ahead of the curve, that guy! :)

    I still dunno how to view or set codel's parameters when it is a sub-discipline though. Default or gtfo, I suppose…

    I've just had a tinker and I can't find anything, but that certainly doesn't mean it's not there.
    I've rarely used BSD, is there some /proc type interface where the information comes from that can be queried directly?



  • @Harvy66:

    The whole target qlimit thing applies to CoDel for both the scheulder and the child discipline?

    Do you know if the interval changes? The interval is supposed to be 20x the target.

    iirc, the sub-discipline setup is purely configured by hard-coded defaults and has no user configurable/viewable params that I am aware of. Hopefully, there is a simple way for a user to view/set the params in that situation. ermal? ;)

    interval is the only value required by codel, so I do not think it changes. Technically, the target should be set based on the interval value, not vice versa.
    afaik, current codel implementations do not automagically set interval to live RTT.

    The CoDel building blocks are able to adapt to different or time-
      varying link rates, to be easily used with multiple queues, to have
      excellent utilization with low delay and to have a simple and
      efficient implementation.  The only setting CoDel requires is its
      interval value, and as 100ms satisfies that definition for normal
      internet usage, CoDel can be parameter-free for consumer use.

    See: https://tools.ietf.org/id/draft-nichols-tsvwg-codel-02.txt

    I have tried to run a thought-experiment concerning how a 5ms interval should negatively affect codel's performance, but I cannot fully comprehend it. I need to setup a bufferbloat lab…



  • @kieranc:

    @Nullity:

    Yeah, it is pretty confusing but I'll take CoDel however I can get it. :)
    Ermal ported himself, iirc. Ahead of the curve, that guy! :)

    I still dunno how to view or set codel's parameters when it is a sub-discipline though. Default or gtfo, I suppose…

    I've just had a tinker and I can't find anything, but that certainly doesn't mean it's not there.
    I've rarely used BSD, is there some /proc type interface where the information comes from that can be queried directly?

    iirc, the values could be gotten through some dev/proc interface, but it required an ioctl system call and could not be done via shell commands.

    Though, I was confused then and now I've forgotten stuff, so I might be sense-making not-so-much.



  • Well, this is fun. It seems to actually perform worse with the 'correct' values in place.
    With 50/5 I was seeing mostly <200ms response time with upstream saturated and a 'B' on dslreports bufferbloat test
    With 5/100 I'm seeing mostly <300ms response time, with more between 200 and 300ms than before, and a 'C' on dslreports bufferbloat test



  • That might explain why the CoDel people were saying they typically saw bufferbloat is low as 30ms, but I was seeing 0ms. PFSense may be more aggressive with the 5ms interval.

    The interval is how often a single packet will be dropped until the packet's time in queue is below the target. If the target is 100ms with a 5ms interval, once you get 100ms of packets, CoDel will start dropping packets every 5ms and slowly increase the rate. It's not exactly how I say it, but close. They have some specific math that makes things everything not as simple as described, but very similar.

    the interval is supposed to be set your your "normal" RTT, and the target should be 1/20th that value. Most services I hit have sub 30ms pings. My interval should be say 45ms and my target 2.25ms.

    If the interval is too high, CoDel will too passive and have increasing bufferbloat, but if it's too low, it will be too aggressive and reduce throughput.

    Maybe this is why PFSense's CoDel gives bad packetloss and throughput on slow connections. If the interval is 5ms, many packets will be dropped in a row.



  • @kieranc:

    Well, this is fun. It seems to actually perform worse with the 'correct' values in place.
    With 50/5 I was seeing mostly <200ms response time with upstream saturated and a 'B' on dslreports bufferbloat test
    With 5/100 I'm seeing mostly <300ms response time, with more between 200 and 300ms than before, and a 'C' on dslreports bufferbloat test

    I think you may have another problem/misconfiguration. You should be seeing MUUUUCH better than 200ms. My ADSL connection goes from 600ms without any traffic-shaping, to 50ms with CoDel on upstream during a fully-saturating, single-stream upload test. My idle ping to first hop is ~10ms.

    but lol…. I have been laughing that the fixed parameter values would actually cause a performance decrease...



  • @Nullity:

    @kieranc:

    Well, this is fun. It seems to actually perform worse with the 'correct' values in place.
    With 50/5 I was seeing mostly <200ms response time with upstream saturated and a 'B' on dslreports bufferbloat test
    With 5/100 I'm seeing mostly <300ms response time, with more between 200 and 300ms than before, and a 'C' on dslreports bufferbloat test

    I think you may have another problem/misconfiguration. You should be seeing MUUUUCH better than 200ms. My ADSL connection goes from 600ms without any traffic-shaping, to 50ms with CoDel on upstream during a fully-saturating, single-stream upload test. My idle ping to first hop is ~10ms.

    but lol…. I have been laughing that the fixed parameter values would actually cause a performance decrease...

    You're absolutely right, my problem is my ISP and their crappy excuse for a router, which I can't easily replace because it also handles the phones.
    My connection will easily hit 2000ms+ if someone is uploading, <200ms is a massive improvement.

    I'm also laughing a little at the results, based on your previous tests it's not a huge surprise but an explaination would be nice!



  • @kieranc:

    @Nullity:

    @kieranc:

    Well, this is fun. It seems to actually perform worse with the 'correct' values in place.
    With 50/5 I was seeing mostly <200ms response time with upstream saturated and a 'B' on dslreports bufferbloat test
    With 5/100 I'm seeing mostly <300ms response time, with more between 200 and 300ms than before, and a 'C' on dslreports bufferbloat test

    I think you may have another problem/misconfiguration. You should be seeing MUUUUCH better than 200ms. My ADSL connection goes from 600ms without any traffic-shaping, to 50ms with CoDel on upstream during a fully-saturating, single-stream upload test. My idle ping to first hop is ~10ms.

    but lol…. I have been laughing that the fixed parameter values would actually cause a performance decrease...

    You're absolutely right, my problem is my ISP and their crappy excuse for a router, which I can't easily replace because it also handles the phones.
    My connection will easily hit 2000ms+ if someone is uploading, <200ms is a massive improvement.

    I'm also laughing a little at the results, based on your previous tests it's not a huge surprise but an explaination would be nice!

    You might test enabling net.inet.tcp.inflight.enable=1 in the System->Advanced->System Tunables tab.

    TCP bandwidth delay product limiting can be enabled by setting the net.inet.tcp.inflight.enable sysctl(8) variable to 1. This instructs the system to attempt to calculate the bandwidth delay product for each connection and limit the amount of data queued to the network to just the amount required to maintain optimum throughput.

    This feature is useful when serving data over modems, Gigabit Ethernet, high speed WAN links, or any other link with a high bandwidth delay product, especially when also using window scaling or when a large send window has been configured. When enabling this option, also set net.inet.tcp.inflight.debug to 0 to disable debugging. For production use, setting net.inet.tcp.inflight.min to at least 6144 may be beneficial. Setting high minimums may effectively disable bandwidth limiting, depending on the link. The limiting feature reduces the amount of data built up in intermediate route and switch packet queues and reduces the amount of data built up in the local host's interface queue. With fewer queued packets, interactive connections, especially over slow modems, will operate with lower Round Trip Times. This feature only effects server side data transmission such as uploading. It has no effect on data reception or downloading.

    Adjusting net.inet.tcp.inflight.stab is not recommended. This parameter defaults to 20, representing 2 maximal packets added to the bandwidth delay product window calculation. The additional window is required to stabilize the algorithm and improve responsiveness to changing conditions, but it can also result in higher ping(8) times over slow links, though still much lower than without the inflight algorithm. In such cases, try reducing this parameter to 15, 10, or 5 and reducing net.inet.tcp.inflight.min to a value such as 3500 to get the desired effect. Reducing these parameters should be done as a last resort only.

    https://www.freebsd.org/doc/handbook/configtuning-kernel-limits.html

    Seems like exactly the type of thing we would be interested in.



  • @Nullity:

    @kieranc:

    @Nullity:

    @kieranc:

    Well, this is fun. It seems to actually perform worse with the 'correct' values in place.
    With 50/5 I was seeing mostly <200ms response time with upstream saturated and a 'B' on dslreports bufferbloat test
    With 5/100 I'm seeing mostly <300ms response time, with more between 200 and 300ms than before, and a 'C' on dslreports bufferbloat test

    I think you may have another problem/misconfiguration. You should be seeing MUUUUCH better than 200ms. My ADSL connection goes from 600ms without any traffic-shaping, to 50ms with CoDel on upstream during a fully-saturating, single-stream upload test. My idle ping to first hop is ~10ms.

    but lol…. I have been laughing that the fixed parameter values would actually cause a performance decrease...

    You're absolutely right, my problem is my ISP and their crappy excuse for a router, which I can't easily replace because it also handles the phones.
    My connection will easily hit 2000ms+ if someone is uploading, <200ms is a massive improvement.

    I'm also laughing a little at the results, based on your previous tests it's not a huge surprise but an explaination would be nice!

    You might test enabling net.inet.tcp.inflight.enable=1 in the System->Advanced->System Tunables tab.

    TCP bandwidth delay product limiting can be enabled by setting the net.inet.tcp.inflight.enable sysctl(8) variable to 1. This instructs the system to attempt to calculate the bandwidth delay product for each connection and limit the amount of data queued to the network to just the amount required to maintain optimum throughput.

    This feature is useful when serving data over modems, Gigabit Ethernet, high speed WAN links, or any other link with a high bandwidth delay product, especially when also using window scaling or when a large send window has been configured. When enabling this option, also set net.inet.tcp.inflight.debug to 0 to disable debugging. For production use, setting net.inet.tcp.inflight.min to at least 6144 may be beneficial. Setting high minimums may effectively disable bandwidth limiting, depending on the link. The limiting feature reduces the amount of data built up in intermediate route and switch packet queues and reduces the amount of data built up in the local host's interface queue. With fewer queued packets, interactive connections, especially over slow modems, will operate with lower Round Trip Times. This feature only effects server side data transmission such as uploading. It has no effect on data reception or downloading.

    Adjusting net.inet.tcp.inflight.stab is not recommended. This parameter defaults to 20, representing 2 maximal packets added to the bandwidth delay product window calculation. The additional window is required to stabilize the algorithm and improve responsiveness to changing conditions, but it can also result in higher ping(8) times over slow links, though still much lower than without the inflight algorithm. In such cases, try reducing this parameter to 15, 10, or 5 and reducing net.inet.tcp.inflight.min to a value such as 3500 to get the desired effect. Reducing these parameters should be done as a last resort only.

    https://www.freebsd.org/doc/handbook/configtuning-kernel-limits.html

    Seems like exactly the type of thing we would be interested in.

    Just for fun, I enabled it and disabled the traffic shaper, during the upload portion of a dslreports test, my ping hit 2700ms :)
    With inflight and codel enabled it seems to be behaving fine, possibly slightly better than without but i'll have to do more testing.



  • Given that the 2.2.4 'correct' settings seem to give worse results than the 'incorrect' 2.2.3 ones (for me at least), it seems that we need the ability to tune both interval and target in order to make codel useful for everyone.
    I'm guessing it would be complicated to add an extra field to the traffic shaper setup page, but since the queue limit field is currently being reused to set the target, could we add some logic to use it to set both target and interval? if the field contains a single integer, use it as a target, if it contains something else (t5i100? 5:100?) then use it as target and interval?
    It's a bit beyond my skill level but it seems like it should be possible in theory, or is there a better way to do it?

    edit: can we just set the interval and derive the target from that, if it's easier?



  • @kieranc:

    Given that the 2.2.4 'correct' settings seem to give worse results than the 'incorrect' 2.2.3 ones (for me at least), it seems that we need the ability to tune both interval and target in order to make codel useful for everyone.
    I'm guessing it would be complicated to add an extra field to the traffic shaper setup page, but since the queue limit field is currently being reused to set the target, could we add some logic to use it to set both target and interval? if the field contains a single integer, use it as a target, if it contains something else (t5i100? 5:100?) then use it as target and interval?
    It's a bit beyond my skill level but it seems like it should be possible in theory, or is there a better way to do it?

    edit: can we just set the interval and derive the target from that, if it's easier?

    Those who created the codel algorithm are the one's who dictate that, and I presume they chose wisely. Target is dynamic anyway, I think.

    Though, if you choose CoDel as the primary/parent scheduler, then you can choose whatever interval/target you want, via command-line, at least for temporary testing purposes.
    Our problem is that we cannot customize CoDel's parameters when it is a sub-discipline aka "Codel Active Queue" checkbox.

    Whether to expose the CoDel params in the GUI or not… if it is anything like the HFSC params, people will needlessly tweak them with unforeseen consequences simply because they are there. I dunno... maybe we can use the System Tunables tab and add custom params and values that way?


Log in to reply