Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Playing with fq_codel in 2.4

    Scheduled Pinned Locked Moved Traffic Shaping
    1.1k Posts 123 Posters 1.5m Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • L
      Larrikin @q54e3w
      last edited by

      @q54e3w said in Playing with fq_codel in 2.4:

      What was the switch you were using on loan that gave the 360mbps?

      Ubiquity.

      1 Reply Last reply Reply Quote 0
      • L
        Larrikin
        last edited by

        Here is some additional information from the NBN in terms of how they deal with their policy enforcer on uploads (ignore downloads as that is fine).

        *******The PBS defines the length of a burst of Layer 2 traffic (either in bytes or milliseconds as set out below) that may be received at ingress to the NBN Co Network for a burst of traffic that pushes the average Information Rate above the configured bandwidth profile for a PIR traffic class. Traffic in excess of the PBS will be discarded by the NBN Co Network. The PBS is set by NBN Co for each PIR specification, and cannot be modified.

        The PBS is used by the policing functions of the NBN Co Network at ingress to the NBN Co Network to determine whether a stream of ingress data complies with the subscribed PIR. Customer is responsible for ensuring that all ingress traffic is shaped to comply with the PIR/PBS as specified for the required traffic class and interface, before presentation to the UNI-D or NNI as relevant.

        It goes on to define the PBS as:

        Downstream at the NNI: 10ms
        Specific PBS setting in Bytes is dependent on the TC-4 PIR (bandwidth profile) selected
        Upstream at the UNI-D: 40,000 bytes
        The AVC TC-4 PBS is set by NBN Co and cannot be modified by Customer*******

        1 Reply Last reply Reply Quote 0
        • T
          TheNarc
          last edited by

          Apologies in advance if you've already provided this information - I read back through but may have missed it - but are you saying that with the pfSense machine you can never get above ~300Mbps regardless of whether you use limiters at all? Or are you saying that you've tried different limiter settings but never get above that speed? In other words, through your testing have you isolated this apparent speed ceiling specifically to when you use limiters?

          Another interesting test (maybe, I'm not sure whether it would really provide useful information) might be to set up a download limiter. I know that you don't need one, and it would only be temporary. But may be interesting to know whether setting up a download limit with the same parameters as your upload limiter would "over-limit" your download in the same manner.

          I can say that I haven't seen anything like this and I use a roughly identical shaping configuration, but also with dramatically different bandwidth limits. I only have 10Mbps upstream, so it's nowhere near the same throughput.

          One other thought, and this is really off the wall and should make no difference, but it's also trivial to try: maybe specify the bandwidth in different units. For example, 380,000 Kbps.

          L 1 Reply Last reply Reply Quote 0
          • L
            Larrikin @TheNarc
            last edited by

            @TheNarc said in Playing with fq_codel in 2.4:

            Apologies in advance if you've already provided this information - I read back through but may have missed it - but are you saying that with the pfSense machine you can never get above ~300Mbps regardless of whether you use limiters at all? Or are you saying that you've tried different limiter settings but never get above that speed? In other words, through your testing have you isolated this apparent speed ceiling specifically to when you use limiters?

            Yes, I am able to get over 360+mbps when I put a switch between pfsense and the NBN connection (where the switch is doing the shaping). I don't own that switch and reluctant to buy one to fix this when hopefully we can resolve this within pfsense.

            Another interesting test (maybe, I'm not sure whether it would really provide useful information) might be to set up a download limiter. I know that you don't need one, and it would only be temporary. But may be interesting to know whether setting up a download limit with the same parameters as your upload limiter would "over-limit" your download in the same manner.

            I've tried this - no difference in results.

            One other thought, and this is really off the wall and should make no difference, but it's also trivial to try: maybe specify the bandwidth in different units. For example, 380,000 Kbps.

            I'll give this a shot and revert.

            T 1 Reply Last reply Reply Quote 0
            • T
              TheNarc @Larrikin
              last edited by TheNarc

              @Larrikin Sorry about that, somehow I missed that the pfSense box was still in the loop when you were using the Ubiquity switch. Only other thought I have offhand is whether you've tried ALTQ shaping instead of limiters? Again, I see nothing wrong with what you're doing now, but if it simply won't work you can try ALTQ as a point of comparison.

              I can also say that on my upload limiter, I have a much lower limit value (1024 as opposed to 10240) based on information from the following sources:

              https://community.ui.com/questions/Best-Practices-for-Smart-Que-tuning-FQ-CoDel-on-and-ER-X/845b3bd4-676c-4b3e-be0e-2fb9abe97415

              https://www.bufferbloat.net/projects/codel/wiki/Best_practices_for_benchmarking_Codel_and_FQ_Codel/

              Also based on those sources I have my upload limiter quantum set at 300, but then you have a much higher upload than the 100Mbps for which the setting of 300 is recommended.

              L 2 Replies Last reply Reply Quote 0
              • L
                Larrikin @TheNarc
                last edited by

                @TheNarc said in Playing with fq_codel in 2.4:

                @Larrikin Sorry about that, somehow I missed that the pfSense box was still in the loop when you were using the Ubiquity switch. Only other thought I have offhand is whether you've tried ALTQ shaping instead of limiters? Again, I see nothing wrong with what you're doing now, but if it simply won't work you can try ALTQ as a point of comparison.

                I'm unfamiliar with ALTQ. How do I go about setting that up? I can't find anything in the GUI.

                T 1 Reply Last reply Reply Quote 0
                • T
                  TheNarc @Larrikin
                  last edited by

                  @Larrikin Sorry, it's not referred to as such in the GUI. It's set up on the "By Interface" tab from the "Firewall > Traffic Shaper" page. But typically it's easiest to set it up using the wizard (far right tab, Wizards, on the same page). It's been a while since I've set it up myself, but there's good information in the pfSense book:

                  https://docs.netgate.com/pfsense/en/latest/book/trafficshaper/configuring-the-altq-traffic-shaper-with-the-wizard.html

                  Must head to bed here but good luck! Will check back in.

                  1 Reply Last reply Reply Quote 0
                  • L
                    Larrikin @TheNarc
                    last edited by

                    @TheNarc said in Playing with fq_codel in 2.4:

                    @Larrikin Sorry about that, somehow I missed that the pfSense box was still in the loop when you were using the Ubiquity switch. Only other thought I have offhand is whether you've tried ALTQ shaping instead of limiters? Again, I see nothing wrong with what you're doing now, but if it simply won't work you can try ALTQ as a point of comparison.

                    Boom - you are right. Using ALTQ and switching off Explicit Network Congestion on qLink has fixed the problem. I've disabled the limiters and use this shaping and it works! Thank you good sir.

                    1 Reply Last reply Reply Quote 0
                    • L
                      Larrikin
                      last edited by Larrikin

                      For those who want to know how exactly I got it working, you can find the instructions here:

                      https://whirlpool.net.au/wiki/pfsense_traffic_shaping

                      A big thanks to @TheNarc for pointing me in this direction.

                      Also a big thanks to all the others who contributed to helping me troubleshoot. I am most grateful you took your own personal time to help me.

                      Z 1 Reply Last reply Reply Quote 1
                      • Q
                        qwerty123
                        last edited by

                        I have a bit of a problem with my setup, and was hoping to gain some insight into why this may be happening. I will say that I'm no traffic shaping expert, but I have played around with it a bit with varying success. The biggest issue for me is trying to solve bufferbloat and have a better zoom experience.

                        I followed the instructions to get it up and running based on the google hangout post and slideshare instructions here: https://www.slideshare.net/NetgateUSA/pfsense-244-short-topic-miscellany-pfsense-hangout-august-2018

                        In my particular example, when running a speedtest from dslreports, I'm seeing 350Mbps down and about 22Mbps up. I have since used the speeds when configuring pfsense to 340/20. I have also set a 1700 Queue Length (5 * 340 [speed]). My internal network consists of about 11 different networks; 5 of them using VLAN's, the other are using dedicated NIC's in my pfsense box.

                        This does improve my bufferbloat score tremendously from a "D" to an "A". The problem I have are icmp packets dropping. When running a speedtest and pinging something (google.com, 8.8.8.8, Comcast's DNS Servers), I never get a response back while running the speedtest. After the test is over, I start seeing responses again. When somebody is on a Zoom conference, I'm seeing about 2% packet loss on the WAN connection.

                        I came across this bug:

                        https://redmine.pfsense.org/issues/9024

                        This sounds similar to what I've been seeing on my end. I was wondering if anybody else has had a similar experience, and if there's a better way to control this than setting a rule up on every interface.

                        F 1 Reply Last reply Reply Quote 0
                        • F
                          fabrizior @qwerty123
                          last edited by

                          @qwerty123

                          While reviewing that bug, there was a comment made: “ If a separate floating match rule is created for ICMP, then packets will not be dropped.”

                          Have you tried that?

                          Q 1 Reply Last reply Reply Quote 0
                          • Q
                            qwerty123 @fabrizior
                            last edited by

                            @fabrizior said in Playing with fq_codel in 2.4:

                            @qwerty123

                            While reviewing that bug, there was a comment made: “ If a separate floating match rule is created for ICMP, then packets will not be dropped.”

                            Have you tried that?

                            Yup. I first tried to create a regular rule on every interface but was still having problems. I then thought hey, ping is ICMP so let's change this floating rule from "all protocols" to just TCP/UDP. I did that and it fixed my packet loss.

                            Will this reported bug affect anything else or is this fq_codel setup good to use in production?

                            Thank You!

                            F superweaselS 2 Replies Last reply Reply Quote 1
                            • F
                              fabrizior @qwerty123
                              last edited by

                              @qwerty123

                              Define Production...
                              :)

                              Been working here just fine. entire family’s online experiences are vastly improved.

                              Though, as Karma would have it, these errors started at 3am last night:

                              There were error(s) loading the rules: /tmp/rules.debug:218: no routing address with matching address family found. - The line in question reads [218]: pass out quick on { em0 } $GWWAN_DHCP6 inet6 from any to any tracker 1595024432 keep state dnqueue( 2,1) label "USER_RULE: CoDel Limiters IPv6"
                              @ 2020-09-01 03:18:18

                              There were error(s) loading the rules: /tmp/rules.debug:218: no routing address with matching address family found. - The line in question reads [218]: pass out quick on { em0 } $GWWAN_DHCP6 inet6 from any to any tracker 1595024432 keep state dnqueue( 2,1) label "USER_RULE: CoDel Limiters IPv6"
                              @ 2020-09-01 03:18:40

                              Q GertjanG 2 Replies Last reply Reply Quote 0
                              • Q
                                qwerty123 @fabrizior
                                last edited by

                                @fabrizior said in Playing with fq_codel in 2.4:

                                @qwerty123

                                Define Production...
                                :)

                                Been working here just fine. entire family’s online experiences are vastly improved.

                                Good to hear. I haven't had any major issues per say, but have had some things happen in the past couple of months that have made me look more at quality vs quantity. Figuring out my Comcast inactive wifi connection was actually active and the same channel as my home AP. I've also had some quality issues with Zoom and Teams that may or may not be related to Comcast. I'm hoping that traffic shaping improves this. I honestly don't think it will because I'm only using about 5/2 and the connection is a 340/20. But hey, maybe it will make things better.

                                Though, as Karma would have it, these errors started at 3am last night:

                                Sorry to jinx you. ;)

                                F 2 Replies Last reply Reply Quote 0
                                • F
                                  fabrizior @qwerty123
                                  last edited by

                                  @qwerty123
                                  ha! thanks.

                                  Was having a lot of jitter and semi-frequent high percentage of dropped packets earlier this summer.
                                  Turned out that the squirrels were munching on the 75-ft cable from the house to the pole and it was full of water! When the Comcast tech unscrewed it at the service demarcation box on the wall, it ran like a garden hose for over a minute! He laughed and replaced it. service has been solid ever since.

                                  1 Reply Last reply Reply Quote 0
                                  • F
                                    fabrizior @qwerty123
                                    last edited by

                                    @qwerty123

                                    I’d suggest running tests from a hardwired host when you experience issues. wifi adds too many variables to account for if you suspect router issues.

                                    How’d your xfinitywifi re-activate? thats odd.
                                    will go check my XB4 out of curiosity. Didn’t think we had access to those settings when in bridge mode.

                                    Q 1 Reply Last reply Reply Quote 0
                                    • Q
                                      qwerty123 @fabrizior
                                      last edited by

                                      @fabrizior said in Playing with fq_codel in 2.4:

                                      @qwerty123

                                      I’d suggest running tests from a hardwired host when you experience issues. wifi adds too many variables to account for if you suspect router issues.

                                      I agree. When I run these tests, I do it from my hardwired workstation. Too many variables over wifi.

                                      How’d your xfinitywifi re-activate? thats odd.
                                      will go check my XB4 out of curiosity. Didn’t think we had access to those settings when in bridge mode.

                                      This is interesting and extremely frustrating. I have an XB6. When I set it up, I configured it to make wifi 'inactive' through the web UI, then I placed it in bridge mode to use it with pfsense. I discovered a wifi issue a few months ago and decided to change wifi channels. Instead of using the Ubiquiti scan tool, this time I used inSSIDer. When I scanned, I discovered a really hot channel without a SSID and using about 5 or 6 bssid's. I was able to match the bssid's close enough to the mac address of the wifi card in the xb6. When I brought my laptop down to the modem, I discovered an extremely high signal coming off of the xb6.

                                      I asked around and it turns out that even if you have your modem in bridged mode with wifi being 'inactive', they still activate wifi for their personal hotspot. If you disable that, it's STILL active because of their security system. Rumor has it that the security system Xfinity sells uses wifi. So the Xfinity modems have wifi utilizing a channel that can't be disabled because of this.

                                      I don't have the security system and just want to use the modem in bridge mode with wifi totally disabled. Looks like that's not an option unless you use your own modem. I've been down that road before and whenever there's a problem, Comcast always blames your user-owned modem. Plus, I wanted to get rid of my data cap so I pay for the modem rental plus no cap.

                                      1 Reply Last reply Reply Quote 0
                                      • GertjanG
                                        Gertjan @fabrizior
                                        last edited by

                                        @fabrizior said in Playing with fq_codel in 2.4:

                                        There were error(s) loading the rules: /tmp/rules.debug:218: no routing address with matching address family found. - The line in question reads [218]: pass out quick on { em0 } $GWWAN_DHCP6 inet6 from any to any tracker 1595024432 keep state dnqueue( 2,1) label "USER_RULE: CoDel Limiters IPv6"
                                        @ 2020-09-01 03:18:18

                                        This :

                                        /tmp/rules.debug
                                        

                                        is just a file : the one that gets build after 'something' was changed in the firewall rule set. It could be an alias that points to a new IP or network, pfBlockerNG that retrieved new info, or some sheduled firewall rule.

                                        What is on, around this line "218" ?

                                        Check the frequency of this line in the main log file :

                                        66c49821-7b5d-49ea-8614-bf257bb89e41-image.png

                                        Typically, one, two three times a day is normal.
                                        Many more ? Could be ok, just find out the 'why' part.

                                        To make a long story short : disable this :
                                        a9ce0fc2-bb3a-4572-92f8-cfaea270895b-image.png
                                        ( DNS Resolver settings)

                                        No "help me" PM's please. Use the forum, the community will thank you.
                                        Edit : and where are the logs ??

                                        1 Reply Last reply Reply Quote 0
                                        • superweaselS
                                          superweasel @qwerty123
                                          last edited by

                                          @qwerty123 Where did you place the floating rule, above or below the WAN-Out FQ-ColDel rule?

                                          pfSense rig: pfSense SG-4860/120GB SSD
                                          WAN: CenturyLink Gigabit Fiber

                                          Q 1 Reply Last reply Reply Quote 0
                                          • Q
                                            qwerty123 @superweasel
                                            last edited by

                                            @superweasel said in Playing with fq_codel in 2.4:

                                            @qwerty123 Where did you place the floating rule, above or below the WAN-Out FQ-ColDel rule?

                                            I ended up making the rule just for tcp/udp. All the guides say to use "ALL" for the protocols. In most situations, the ISP's mostly going to route tcp/udp and icmp. Since ICMP was the issue, I just changed the protocol "TCP/UDP" instead of "ALL".

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.