Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    CoDel - How to use

    Traffic Shaping
    30
    206
    96.9k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • H
      Harvy66
      last edited by

      I was wondering if it could be possible for someone to create two new versions of codel in PFSense, one for low bandwidth connections and one for high bandwidth.

      Codel has two parameters. something like a target latency and an interval, which for normal codel is something like 5ms and 100ms. They recommend 0.5ms and 50ms for 40Gb NICs. We may want a less aggressive one for low bandwidth connections, like 1Mb, so maybe 30ms and 150ms. I have no idea what number, just something less aggressive than 5ms on a 1Mb link trying to push a 1500byte packet.

      1 Reply Last reply Reply Quote 0
      • H
        Harvy66
        last edited by

        In theory, the only thing that needs to get changes are those two numbers.

        1 Reply Last reply Reply Quote 0
        • N
          Nonsense
          last edited by

          "Do the same thing for your LAN interface, except your LAN bandwidth will be whatever your download speed is."

          Done, thanks, now my score is a "B."

          1 Reply Last reply Reply Quote 0
          • H
            Harvy66
            last edited by

            You could try switching to fairq instead of codel and see how your milage goes.

            1 Reply Last reply Reply Quote 0
            • H
              Harvy66
              last edited by

              looks like 20ms target may be better for cable modems with a 150 interval, because of how buffering and scheduling works with DOCSIS. The modem needs to see a certain amount of backlog in the buffer in order to schedule more time, so a small target latency can artificially reduce buffering too much for DOCSIS. This may or may not be an issue for the router but the modem. But it does bring up possibilities where different target latencies for codel would be nice.

              1 Reply Last reply Reply Quote 0
              • N
                Nullity
                last edited by

                @Harvy66:

                I was wondering if it could be possible for someone to create two new versions of codel in PFSense, one for low bandwidth connections and one for high bandwidth.

                Codel has two parameters. something like a target latency and an interval, which for normal codel is something like 5ms and 100ms. They recommend 0.5ms and 50ms for 40Gb NICs. We may want a less aggressive one for low bandwidth connections, like 1Mb, so maybe 30ms and 150ms. I have no idea what number, just something less aggressive than 5ms on a 1Mb link trying to push a 1500byte packet.

                If I read the source code properly, pfSense's CoDel implementation supports both "target" ("qlimit" in pfSense) and "interval" but not the GUI. I have no idea what the standard practice is for manually editing the pfSense pf.conf/rules.debug. Here is the pfSense source: https://github.com/doktornotor/pfsense-tools/blob/master/patches/RELENG_10_0/altq_codel.diff#L126
                (Is there no better way to show pfSense source-code than this repo?)

                I will test a 20ms target and see if it changes anything. I thought the problems with CoDel and <768Kbit  connections was caused by more than an undersized target value… cannot hurt to try and see though! :)

                Please correct any obvious misinformation in my posts.
                -Not a professional; an arrogant ignoramous.

                1 Reply Last reply Reply Quote 0
                • N
                  Nullity
                  last edited by

                  When I was looking at the CoDel source, I noticed that the default target and interval might be incorrect. It should be target=5ms and interval=100ms, but it is not on my end.

                  Please SSH to your pfSense device and run "pfctl -vs queue | grep -i codel".
                  I get "altq on em0 codel( target 50 interval 5) bandwidth 300Mb tbrsize 36000". Target should be "5" and interval "100".
                  Here is the problem, or one of them; https://github.com/doktornotor/pfsense-tools/blob/master/patches/RELENG_10_0/altq_codel.diff#L1330
                  "100" and "5" should be switched. I am unsure why my pfSense is showing values of "50" and "5", since I only see "100" and "5" in the source, but then again pfSense source-code is very confusing to me.

                  I wonder if this bug has any affect…

                  Anyone who is familiar with submitting bug-fixes, please do so. I will try to figure it out this weekend, if nobody else does.

                  Please correct any obvious misinformation in my posts.
                  -Not a professional; an arrogant ignoramous.

                  1 Reply Last reply Reply Quote 0
                  • N
                    Nullity
                    last edited by

                    @Nullity:

                    When I was looking at the CoDel source, I noticed that the default target and interval might be incorrect. It should be target=5ms and interval=100ms, but it is not on my end.

                    Please SSH to your pfSense device and run "pfctl -vs queue | grep -i codel".
                    I get "altq on em0 codel( target 50 interval 5) bandwidth 300Mb tbrsize 36000". Target should be "5" and interval "100".
                    Here is the problem, or one of them; https://github.com/doktornotor/pfsense-tools/blob/master/patches/RELENG_10_0/altq_codel.diff#L1330
                    "100" and "5" should be switched. I am unsure why my pfSense is showing values of "50" and "5", since I only see "100" and "5" in the source, but then again pfSense source-code is very confusing to me.

                    I wonder if this bug has any affect…

                    Anyone who is familiar with submitting bug-fixes, please do so. I will try to figure it out this weekend, if nobody else does.

                    Also, I have only figured out how to display the target/interval options for the stand-alone CODELQ configuration. (pfctl -vsq)
                    I do not know how to display the values of target/interval when Codel is being used together with another scheduler like HFSC or PRIQ. (The "Codel Active Queue" check-box.)

                    Any help would be dandy.

                    Please correct any obvious misinformation in my posts.
                    -Not a professional; an arrogant ignoramous.

                    1 Reply Last reply Reply Quote 0
                    • H
                      Harvy66
                      last edited by

                      $ pfctl -vs queue | grep -i codel
                      queue  qDefault on igb0 bandwidth 14.70Mb qlimit 4096 hfsc( codel default )
                      queue  qHigh on igb0 bandwidth 29.40Mb qlimit 4096 hfsc( codel linkshare(48.02Mb 5 29.40Mb) )
                      queue  qNormal on igb0 bandwidth 29.40Mb qlimit 4096 hfsc( codel )
                      queue  qDefault on igb1 bandwidth 14.25Mb qlimit 4096 hfsc( codel default )
                      queue  qHigh on igb1 bandwidth 28.50Mb qlimit 4096 hfsc( codel linkshare(46.55Mb 5 28.50Mb) )
                      queue  qNormal on igb1 bandwidth 28.50Mb qlimit 4096 hfsc( codel )

                      It doesn't say when I use it as a child queue instead of the scheduler.

                      1 Reply Last reply Reply Quote 0
                      • N
                        Nullity
                        last edited by

                        @Harvy66:

                        $ pfctl -vs queue | grep -i codel
                        queue  qDefault on igb0 bandwidth 14.70Mb qlimit 4096 hfsc( codel default )
                        queue  qHigh on igb0 bandwidth 29.40Mb qlimit 4096 hfsc( codel linkshare(48.02Mb 5 29.40Mb) )
                        queue  qNormal on igb0 bandwidth 29.40Mb qlimit 4096 hfsc( codel )
                        queue  qDefault on igb1 bandwidth 14.25Mb qlimit 4096 hfsc( codel default )
                        queue  qHigh on igb1 bandwidth 28.50Mb qlimit 4096 hfsc( codel linkshare(46.55Mb 5 28.50Mb) )
                        queue  qNormal on igb1 bandwidth 28.50Mb qlimit 4096 hfsc( codel )

                        It doesn't say when I use it as a child queue instead of the scheduler.

                        Yeah, I had to create the stand-alone CODELQ queue to test.I looked through pfctl's man page and some source code to track down where details were accessable to userland, but I only found the stand-alone queue.

                        According to my grepping of the pfSense-tools repo, all Codel options are configured backwards. I will continue to verify and then submit a ticket on the bugtracker… I guess...  :o

                        I would like to know how to configure and/or display the values of CODELQ when it is used as a sub-discipline.

                        Please correct any obvious misinformation in my posts.
                        -Not a professional; an arrogant ignoramous.

                        1 Reply Last reply Reply Quote 0
                        • H
                          Harvy66
                          last edited by

                          Thanks for your work! More dedicated than I am. /salute

                          1 Reply Last reply Reply Quote 0
                          • N
                            Nullity
                            last edited by

                            Bug tracker ticket has been submitted. Hopefully I interpreted all this crap properly.

                            If you want to test some proper target/interval values and you are using stand-alone CODELQ queueing, you could use pfctl to see what your current pf.conf/rules.debug queue line is then switch "target" and "interval", either by a rules.debug modification or directly through pfctl. The change will be temporary, since the GUI will eventually over-write rules.debug.

                            As I understand it, pfctl uses system calls (ioctl?) to get super special info from /dev/pf, and that is where CoDel's target/interval values are, but pfctl has no pre-programmed method of extracting the values when CoDel is acting as a secondary algorithm ("Codel Active Queue" checkbox) on top of the primary algorithm.

                            Adding code to pfctl would be rather easy (I naively assume), so I may try to do that, just so we can have the ability to view/change CoDel regardless of it being the primary or secondary scheduling algorithm. Then again, most people probably do not care. CoDel is meant to be a self-configuring scheduler where you do not need to be an expert knob twiddler.

                            The particulars of how a target of 50 and an interval of 5 were functioning seemingly well is interesting… lol, hopefully the "proper" values are better and not worse.  :P

                            Please correct any obvious misinformation in my posts.
                            -Not a professional; an arrogant ignoramous.

                            1 Reply Last reply Reply Quote 0
                            • N
                              Nullity
                              last edited by

                              I tested (ping tests while upload is fully saturated) CoDel in pfSense with proper target(20 for my slow cxn) and interval(100) values, but… it made no difference. I expected otherwise. I tested 5, 20, 30 and 50 as values for target, and for interval I tested 5 and 100. The (incorrect) default in pfSense is target=(50 or 100) and interval=5. If you guys would like to test the stand-alone CoDel with proper target/interval values, in /tmp/rules.debug, replace your current WAN queue lines (they start with "altq" or "queue") with the following line into /tmp/rules.debug and reload with "pfctl -vf /tmp/rules.debug". It is only a temporary change (config.xml, your standard GUI config, creates /tmp/rules.debug on the fly).
                              Modify the "em0" parts to match your interface, and set the bandwidth to your usual.

                              altq on em0 codelq( qlimit 5, interval 105 ) bandwidth 630Kb queue { root_em0 }
                              Edit: To clarify, qlimit = target.

                              Another strange thing I encountered during testing was that with CoDel, I had 5-12% of pings unreplied (tcpdump registered them as sent). I tested 3 times with HFSC and CoDelQ… every time, CoDel has ping loss while HFSC never had one dropped packet. By my understanding of CoDel's specifications, it should not have been dropping packets, especially ICMP.

                              I am being premature, but I am thinking CoDel is not working properly. Here are some tcpdump and ping pastes that tell me the ping request was sent, but never replied, but how can CoDel cause someone to not reply? Any ideas? Am I interpreting tcpdump improperly?
                              I just noticed that when I switch to CoDel, all my firewall rules are still assigned to non-existing HFSC queues. I wonder if that is causing probs...

                              20:09:03.422774 AF IPv4 (2), length 88: (tos 0x0, ttl 63, id 17061, offset 0, flags [none], proto ICMP (1), length 84)
                                  173.187.x.x > 74.125.22.102: ICMP echo request, id 37395, seq 11, length 64
                              20:09:03.451752 AF IPv4 (2), length 88: (tos 0x0, ttl 45, id 0, offset 0, flags [none], proto ICMP (1), length 84)
                                  74.125.22.102 > 173.187.x.x: ICMP echo reply, id 37395, seq 11, length 64
                              20:09:04.424272 AF IPv4 (2), length 88: (tos 0x0, ttl 63, id 32553, offset 0, flags [none], proto ICMP (1), length 84)
                                  173.187.x.x > 74.125.22.102: ICMP echo request, id 37395, seq 12, length 64
                              20:09:05.424160 AF IPv4 (2), length 88: (tos 0x0, ttl 63, id 45955, offset 0, flags [none], proto ICMP (1), length 84)
                                  173.187.x.x > 74.125.22.102: ICMP echo request, id 37395, seq 13, length 64
                              20:09:05.480163 AF IPv4 (2), length 88: (tos 0x0, ttl 45, id 0, offset 0, flags [none], proto ICMP (1), length 84)
                                  74.125.22.102 > 173.187.x.x: ICMP echo reply, id 37395, seq 13, length 64

                              PING google.com (74.125.22.102) 56(84) bytes of data.
                              64 bytes from qh-in-f102.1e100.net (74.125.22.102): icmp_seq=1 ttl=44 time=45.8 ms
                              64 bytes from qh-in-f102.1e100.net (74.125.22.102): icmp_seq=2 ttl=44 time=46.1 ms
                              64 bytes from qh-in-f102.1e100.net (74.125.22.102): icmp_seq=3 ttl=44 time=61.8 ms
                              64 bytes from qh-in-f102.1e100.net (74.125.22.102): icmp_seq=4 ttl=44 time=57.5 ms
                              64 bytes from qh-in-f102.1e100.net (74.125.22.102): icmp_seq=5 ttl=44 time=55.7 ms
                              64 bytes from qh-in-f102.1e100.net (74.125.22.102): icmp_seq=6 ttl=44 time=87.6 ms
                              64 bytes from qh-in-f102.1e100.net (74.125.22.102): icmp_seq=7 ttl=44 time=33.0 ms
                              64 bytes from qh-in-f102.1e100.net (74.125.22.102): icmp_seq=8 ttl=44 time=28.9 ms
                              64 bytes from qh-in-f102.1e100.net (74.125.22.102): icmp_seq=9 ttl=44 time=73.3 ms
                              64 bytes from qh-in-f102.1e100.net (74.125.22.102): icmp_seq=10 ttl=44 time=34.9 ms
                              64 bytes from qh-in-f102.1e100.net (74.125.22.102): icmp_seq=11 ttl=44 time=29.7 ms
                              64 bytes from qh-in-f102.1e100.net (74.125.22.102): icmp_seq=13 ttl=44 time=57.0 ms

                              Please correct any obvious misinformation in my posts.
                              -Not a professional; an arrogant ignoramous.

                              1 Reply Last reply Reply Quote 0
                              • E
                                eri--
                                last edited by

                                Your tcpdump can see the packet but it does not mean that it made it out :)
                                It certainly means that it was sent to the queue which in this case is Codel.

                                You should have statistics of the codel queue tell you what it dropped and what it passed.

                                1 Reply Last reply Reply Quote 0
                                • N
                                  Nullity
                                  last edited by

                                  @ermal:

                                  Your tcpdump can see the packet but it does not mean that it made it out :)
                                  It certainly means that it was sent to the queue which in this case is Codel.

                                  You should have statistics of the codel queue tell you what it dropped and what it passed.

                                  Thanks ermal. Your insight is always welcomed. :)

                                  Quick question or two about CoDel when it is a sub-discipline;
                                  1. are target/interval configurable by the user?
                                  2. are CoDel's stats exposed to the user?

                                  Please correct any obvious misinformation in my posts.
                                  -Not a professional; an arrogant ignoramous.

                                  1 Reply Last reply Reply Quote 0
                                  • E
                                    eri--
                                    last edited by

                                    I have to check the code but normally even when is a sub-discipline all the stats are exported.
                                    When you have Codel as part of HFSC queue the queue stats you see are the ones of Codel book keeping ones.

                                    1 Reply Last reply Reply Quote 0
                                    • M
                                      mcwtim
                                      last edited by

                                      Anyone figure out how to apply the patch?

                                      https://redmine.pfsense.org/issues/4692

                                      Keep getting "no file to patch" when testing, so clearly some steps missing on how to apply it.

                                      1 Reply Last reply Reply Quote 0
                                      • D
                                        doktornotor Banned
                                        last edited by

                                        Testing how? You'd need the full pfsense-tools sources from github plus recompile the thing

                                        1 Reply Last reply Reply Quote 0
                                        • M
                                          mcwtim
                                          last edited by

                                          The test button within pfsense>system>patches.

                                          So this is not a directly applicable patch to an installed pfsense which is what I needed to know.

                                          Thanks.

                                          1 Reply Last reply Reply Quote 0
                                          • N
                                            Nullity
                                            last edited by

                                            @mcwtim:

                                            The test button within pfsense>system>patches.

                                            So this is not a directly applicable patch to an installed pfsense which is what I needed to know.

                                            Thanks.

                                            Yeah, the patch applies to the source-code. See https://forum.pfsense.org/index.php?topic=76132.0 to get accessto the source-code.

                                            Please be careful. That patch was posted by me and it is completely untested. There are a few other incorrect values I later found, but this bug is not vital so I neglected to create a new (and very possibly wrong) patch. If I remember correctly, all the values needing correction were in ~/patches/stable/10/altq_codel.diff and can found rather simply with grep.

                                            I will leave this to the professionals to patch, until I have some experience building pfSense. I wish pfSense's source-code was more accessible. Already I have had to resort to using doktornotor's "illegitmate" pfsense-tools repository on github to refer to code that the unwashed masses can see.

                                            Please correct any obvious misinformation in my posts.
                                            -Not a professional; an arrogant ignoramous.

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post