Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    CoDel - How to use

    Scheduled Pinned Locked Moved Traffic Shaping
    206 Posts 30 Posters 123.5k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • H
      Harvy66
      last edited by

      I would rather see what my queue is seeing. So HFSC does mess with the order in which packets are sent within a queue? I was under the impression that HFSC is only a scheduler and only messes with inter-queue ordering, not intra-queue ordering. So if you select a dumb FIFO child queue, will HFSC change the order in which packets from that queue leave the interface? I know fq_codel and any fair queuing AQM will do that.

      1 Reply Last reply Reply Quote 0
      • N
        Nullity
        last edited by

        @Harvy66:

        I would rather see what my queue is seeing. So HFSC does mess with the order in which packets are sent within a queue? I was under the impression that HFSC is only a scheduler and only messes with inter-queue ordering, not intra-queue ordering. So if you select a dumb FIFO child queue, will HFSC change the order in which packets from that queue leave the interface? I know fq_codel and any fair queuing AQM will do that.

        You are seeing what your flow's queue is seeing, which is all that really matters (End-to-End principle).

        If you want to know the details of HFSC, read the paper. It is super-fun and the ladies love an HFSC-man.

        Please correct any obvious misinformation in my posts.
        -Not a professional; an arrogant ignoramous.

        1 Reply Last reply Reply Quote 0
        • G
          gazoo
          last edited by

          Forgive me if some of this has been repeated already. I just wanted throw in some input on this. I was playing around with CODELQ last night.
          Some observations:
          I was using the speedtest.net test which tests for bufferbloat.
          My link typically achieves 86x5.6Mbps pretty consistently on off peak hours.
          I actually used the bandwidth window to get some settings tweak. Initially I had set the bandwidths to 10% below what my actual speeds were but it cutoff WAY too much of my top end bandwidth. When I did that I was getting an A rating on buffer bloat in both directions but speeds were cut to 75x4.2Mbps
          I finally settled by testing on going higher. It was a balancing act because if I went too high then the bufferbloat was bad again as originally seen. If I went too low, it would knock off too much off the limits of the speeds.

          So for the download (LAN) queue, I went for 87.5Mbps and was able to download 83-84Mbps.
          For the upload (WAN) queue, I went for 6.1Mbps and was able to achieve 5.2 Mbps.
          Both these settings resulted in a "B" rating with approximately 60ms of delay.
          Considering the upload delay had been 512ms at one point, I say not bad to giving up ~7% of my top end upload bandwidth, ~3% download bandwidth.
          I didn't have too much buffer bloat on my download (LAN) because I guess the ISP managed it very well. I believe it was originally at about 100ms

          So this whole thing was trial and error but I consistently get the same results which leads me to believe I have achieved the intended goal.

          I'm using 2.2.2 4G embedded on a Firebox.

          Thanks.

          1 Reply Last reply Reply Quote 0
          • H
            Harvy66
            last edited by

            Yeah, the whole point of managing bufferbloat is your firewall needs to be the chokepoint, and if your internet connection's rate varies too much, you can get bloat spikes.

            My limited understanding of the bufferbloat problem and some technologies is DOCSIS and DSL don't use native Ethernet which cause them to do some strange batching or fragmenting of your packets. This reduces the efficiently and makes for strange bursts in speeds. The bufferbloat forumns are constantly discussing how to properly emulate how DSL and DOCSIS work in an attempt to artificially create the same bottlenecks in the firewall, which allows better management of bufferbloat.

            All of that being said, you need to set your bandwidth to be less than your peak, and losing some bandwidth is part of the cost of getting rid of bufferbloat when your ISP doesn't manage it for you.

            Personally, I have a 100Mb connection and have my rates set to 99Mb and I get 1ms of bloat on average. But I seem to be a special case.

            1 Reply Last reply Reply Quote 0
            • T
              tuffcalc
              last edited by

              Gazoo - I noticed this too. If I put my full speed in pfsense it is as-if it lowers this amount so it is somewhat less (95%?) of the connection speed you enter for traffic shaping limit.

              1 Reply Last reply Reply Quote 0
              • H
                Harvy66
                last edited by

                95% is pretty decent. I see closer to about 97%.

                1 Reply Last reply Reply Quote 0
                • T
                  tuffcalc
                  last edited by

                  @Harvy66:

                  95% is pretty decent. I see closer to about 97%.

                  Might be 97%, haven't looked in a while. In any event looks like the pfsense coders deliberately lower the speed you enter. Can anyone confirm?

                  1 Reply Last reply Reply Quote 0
                  • N
                    Nullity
                    last edited by

                    I have turned to the dark-side and chose full bandwidth over best latency on my incoming traffic. The average latency increases to ~70ms (~10ms idle) during a link saturating download, but even with artificial throughput limiting I was reaching ~40ms. Addiction to throughput probably stems from years of dial-up… the experience still haunts me. :)

                    The difference on uploads is huge though; ~600ms vs ~50ms.

                    My ISP is the not-so-great-but-improving Windstream. I have wondered what other people experience on other ISPs and CPE.

                    Please correct any obvious misinformation in my posts.
                    -Not a professional; an arrogant ignoramous.

                    1 Reply Last reply Reply Quote 0
                    • H
                      Harvy66
                      last edited by

                      Since I use HFSC even for download traffic shaping, I can make sure Netflix/Youtube buffering is kept to a minimum. I pretty much can't tell the difference, even when using P2P and saturating the downstream.

                      I have 100Mb with a 2:1 ratio between Normal:Low. P2P will be sitting around 97Mb, then I start of Netflix and I pretty much immediately see P2P drop down to ~32Mb/s and Netflix pushes through ~64Mb/s. Even with DSLReports, I still see an A/A+ during the download test. But if P2P is using 0 and DSLReports is using max, and suddenly P2P jumps up to 33, the bufferbloat can get bad, like C/D, but that's because available bandwidth is decreasing instead of increasing and I'm still stuck with regular CoDel and not fq_CoDel.

                      1 Reply Last reply Reply Quote 0
                      • N
                        Nullity
                        last edited by

                        CoDel's default "target/interval" values should be fixed in 2.2.3. Dave mentioned earlier that the "target" is pretty good at dynamically adjusting, so the fix probably has little effect in that area.

                        The "interval" is more vital and is noted in the IETF draft as being the only parameter that is required for CoDel to to function, so perhaps changing from  a value of 5ms to a more optimal value of 100ms will improve our CoDel experience. :)

                        Disappointingly, I tried loading up a CODELQ queue in 2.2.3, and I still got the old values… Either the install needs to be fresh or my patch is shit and needs more work. See if it is fixed on any of your 2.2.3 setups.

                        Please correct any obvious misinformation in my posts.
                        -Not a professional; an arrogant ignoramous.

                        1 Reply Last reply Reply Quote 0
                        • H
                          Harvy66
                          last edited by

                          I wonder how it'll affect my bufferbloat. Right now I have about 1ms, but it can burst quite high given enough TCP streams growing at the same time.

                          1 Reply Last reply Reply Quote 0
                          • K
                            kieranc
                            last edited by

                            @Nullity:

                            CoDel's default "target/interval" values should be fixed in 2.2.3. Dave mentioned earlier that the "target" is pretty good at dynamically adjusting, so the fix probably has little effect in that area.

                            The "interval" is more vital and is noted in the IETF draft as being the only parameter that is required for CoDel to to function, so perhaps changing from  a value of 5ms to a more optimal value of 100ms will improve our CoDel experience. :)

                            Disappointingly, I tried loading up a CODELQ queue in 2.2.3, and I still got the old values… Either the install needs to be fresh or my patch is shit and needs more work. See if it is fixed on any of your 2.2.3 setups.

                            I did a fresh install of 2.2.3 and the target/interval are still inverted.

                            [2.2.3-RELEASE][admin@pfSense.localdomain]/root: pfctl -vs queue | grep -i codel
altq on em0 codel( target 50 interval 5) bandwidth 600Kb tbrsize 1500 
altq on em1 codel( target 50 interval 5) bandwidth 6Mb tbrsize 6000
                            1 Reply Last reply Reply Quote 0
                            • N
                              Nullity
                              last edited by

                              @kieranc:

                              @Nullity:

                              CoDel's default "target/interval" values should be fixed in 2.2.3. Dave mentioned earlier that the "target" is pretty good at dynamically adjusting, so the fix probably has little effect in that area.

                              The "interval" is more vital and is noted in the IETF draft as being the only parameter that is required for CoDel to to function, so perhaps changing from  a value of 5ms to a more optimal value of 100ms will improve our CoDel experience. :)

                              Disappointingly, I tried loading up a CODELQ queue in 2.2.3, and I still got the old values… Either the install needs to be fresh or my patch is shit and needs more work. See if it is fixed on any of your 2.2.3 setups.

                              I did a fresh install of 2.2.3 and the target/interval are still inverted.

                              [2.2.3-RELEASE][admin@pfSense.localdomain]/root: pfctl -vs queue | grep -i codel
altq on em0 codel( target 50 interval 5) bandwidth 600Kb tbrsize 1500 
altq on em1 codel( target 50 interval 5) bandwidth 6Mb tbrsize 6000

                              Fudge…

                              Thank you for letting me know. I will get them to revert my commits.

                              How can a little fix be so illusive?

                              Why must you be so confusing, pfSense-tools!?

                              Please correct any obvious misinformation in my posts.
                              -Not a professional; an arrogant ignoramous.

                              1 Reply Last reply Reply Quote 0
                              • K
                                kieranc
                                last edited by

                                @Nullity:

                                @kieranc:

                                @Nullity:

                                CoDel's default "target/interval" values should be fixed in 2.2.3. Dave mentioned earlier that the "target" is pretty good at dynamically adjusting, so the fix probably has little effect in that area.

                                The "interval" is more vital and is noted in the IETF draft as being the only parameter that is required for CoDel to to function, so perhaps changing from  a value of 5ms to a more optimal value of 100ms will improve our CoDel experience. :)

                                Disappointingly, I tried loading up a CODELQ queue in 2.2.3, and I still got the old values… Either the install needs to be fresh or my patch is shit and needs more work. See if it is fixed on any of your 2.2.3 setups.

                                I did a fresh install of 2.2.3 and the target/interval are still inverted.

                                [2.2.3-RELEASE][admin@pfSense.localdomain]/root: pfctl -vs queue | grep -i codel
altq on em0 codel( target 50 interval 5) bandwidth 600Kb tbrsize 1500 
altq on em1 codel( target 50 interval 5) bandwidth 6Mb tbrsize 6000

                                Fudge…

                                Thank you for letting me know. I will get them to revert my commits.

                                How can a little fix be so illusive?

                                Why must you be so confusing, pfSense-tools!?

                                It looks like you're doing it right but the patch hasn't been merged. I've created a pull request on github to do the same thing, let's see what happens

                                Do you know why the current codel_alloc(100, 5, 0) results in values of target 50 interval 5? target 5 interval 50 would be better but I dunno where the value of 50 is coming from.

                                1 Reply Last reply Reply Quote 0
                                • D
                                  doktornotor Banned
                                  last edited by

                                  Was never fixed AFAICT? https://redmine.pfsense.org/issues/4692

                                  1 Reply Last reply Reply Quote 0
                                  • K
                                    kieranc
                                    last edited by

                                    @doktornotor:

                                    Was never fixed AFAICT? https://redmine.pfsense.org/issues/4692

                                    Indeed, has anyone tried the patch? I don't really want to have to build pfsense in order to test it, that's a lot of repos to clone…

                                    edit: PR has been merged, new values should be applied in 2.2.4 or anything built from here on...

                                    1 Reply Last reply Reply Quote 0
                                    • H
                                      Harvy66
                                      last edited by

                                      I wish  this could be set via config instead of compile time. One problem at a time though, ehh?

                                      50/5 seems to be working well for me right now. I guess I'll see how my bufferbloat is affected once this change finally makes it. I'm getting 0ms of bufferbloat and full throughput already. According to DSLReports, my bloat can spike, but rarely. More of an issue when doing 32 upload streams.

                                      1 Reply Last reply Reply Quote 0
                                      • N
                                        Nullity
                                        last edited by

                                        @doktornotor:

                                        Was never fixed AFAICT? https://redmine.pfsense.org/issues/4692

                                        https://github.com/pfsense/pfsense-tools/commit/3108a902bd816036a3abffd3ec669767140891a7

                                        I dunno. I am unsure of many things. :(

                                        I probably should have updated the redmine submission. The redmine patch was a initial code to show what I had found, hooefulky to help a dev pinpoint the problem.

                                        The github patches were the best I could do, but I should probably stop trying to patch pfSense considering that I cannot build pfSense to test my code. :(

                                        Please correct any obvious misinformation in my posts.
                                        -Not a professional; an arrogant ignoramous.

                                        1 Reply Last reply Reply Quote 0
                                        • K
                                          kieranc
                                          last edited by

                                          So this is from the latest nightly:

                                          [2.2.4-DEVELOPMENT][admin@pfSense.localdomain]/root: pfctl -vs queue
altq on em0 codel( target 50 interval 100) bandwidth 600Kb tbrsize 1500

                                          Interval successfully changed, now we just have to figure out where the target of 50 is coming from….

                                          Edit: I just set the 'queue limit' to 25 in the GUI and my target is now 25.... Victory?

                                          Edit2: From 2.2.4 19/07/2015 nightly, with queue limit set to 5:

                                          [2.2.4-DEVELOPMENT][admin@pfSense.localdomain]/root: pfctl -vs queue
altq on em1 codel( target 5 interval 100) bandwidth 6Mb tbrsize 6000 
  [ pkts:         85  bytes:       9938  dropped pkts:      0 bytes:      0 ]
  [ qlength:   0/ 50 ]

                                          So it wasn't anything I did yesterday that fixed it, but it does seem to be fixed/workable in 2.2.4

                                          1 Reply Last reply Reply Quote 0
                                          • N
                                            Nullity
                                            last edited by

                                            If qlimit is 0, it defaults to 50, and codel gets the (initial?) target value from qlimit.

                                            Please correct any obvious misinformation in my posts.
                                            -Not a professional; an arrogant ignoramous.

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.