Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Playing with fq_codel in 2.4

    Scheduled Pinned Locked Moved Traffic Shaping
    1.1k Posts 123 Posters 1.6m Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • VeldkornetV
      Veldkornet @xciter327
      last edited by

      @xciter327 Even at 95% of 400/40 which is 380/38, the upload speed is nowhere near that. The download at least comes in the vicinity.

      C X 2 Replies Last reply Reply Quote 0
      • C
        csutcliff @Veldkornet
        last edited by

        @Veldkornet I don't suppose it's a PPPoE connection? I had a similar problem.

        VeldkornetV 1 Reply Last reply Reply Quote 0
        • M
          maverick_slo
          last edited by

          Hi how did you resolve pppoe problem?
          Thanks!

          C 1 Reply Last reply Reply Quote 0
          • C
            csutcliff @maverick_slo
            last edited by

            @maverick_slo I didn't. I am only able to use the Download shaper for my PPPoE WAN. It works perfectly on my other WAN which is DHCP.

            1 Reply Last reply Reply Quote 0
            • VeldkornetV
              Veldkornet @csutcliff
              last edited by

              @csutcliff Nope, mine is just a DHCPv4 connection

              1 Reply Last reply Reply Quote 0
              • X
                xciter327 @Veldkornet
                last edited by xciter327

                @Veldkornet said in Playing with fq_codel in 2.4:

                @xciter327 Even at 95% of 400/40 which is 380/38, the upload speed is nowhere near that. The download at least comes in the vicinity.

                Hm. That should not be. Did You follow the guide I linked? Also what hardware is the pfsense running on? Limiters add extra CPU usage. Take a look with "htop -d 1" (need to install it first "pkg install htop") and see if You are peaking the CPU while doing the test.

                VeldkornetV 1 Reply Last reply Reply Quote 0
                • VeldkornetV
                  Veldkornet @xciter327
                  last edited by Veldkornet

                  @xciter327 ah! I didn't see the guide. I've now re-created everything as per the guide... although it didn't change the results...

                  I have a PCEngines APU2. I just had a look in "top" while doing the tests. Didn't really see anything climb very high at all. Even then, if it can handle the download, it should be able to manage the upload which is 10% of the download.

                  If I watch the test, the upload starts strong and climbs to around 20Mbit quickly, for 2 seconds or so, but then drops down to around 4/5 for the remainder of the test where it eventually finishes.

                  1 Reply Last reply Reply Quote 0
                  • X
                    xciter327
                    last edited by

                    Could You perhaps post some picture of your firewall rules and limiter config? I am shaping my Ziggo connection on a Zotac NUC, which theoretically should be less powerful than the APU2. Also make sure You clear the states/reset the firewall when applying the limiters. Keep an eye in the system log for any log messages when You apply the limiters as well.

                    1 Reply Last reply Reply Quote 0
                    • VeldkornetV
                      Veldkornet
                      last edited by Veldkornet

                      I can post screenshots instead of the below.. but it will take lots of space. I just checked everything though, and except for the speed limits, to me it looks the same as in the post. Might as well just be a copy paste. Looking at the floating rule screenshot though, I see that the WAN-In FQ-CoDel queue is pretty small considering all the tests I was doing... normal?

                      Also, see how the upload just dies off:
                      b13103a2-63f8-4d95-8ac4-fc2a13a21e21-image.png

                      8668731a-351d-40d0-8f8d-fd51f481cfc4-image.png

                      FQ_CODEL_OUT

                      Name: FQ_CODEL_OUT
                      Bandwidth: 38 Mbit/s
                      Mask: None
                      Queue Management Algorithm: Tail Drop
                      Scheduler: FQ_CODEL
                      target: 5
                      interval: 100
                      quantum: 300
                      limit: 10240
                      flows: 20480
                      

                      fq_codel_out_q

                      Name: fq_codel_out_q
                      Mask: None
                      Queue Management Algorithm: Tail Drop
                      

                      FQ_CODEL_IN

                      Name: FQ_CODEL_IN
                      Bandwidth: 380 Mbit/s
                      Mask: None
                      Queue Management Algorithm: Tail Drop
                      Scheduler: FQ_CODEL
                      target: 5
                      interval: 100
                      quantum: 300
                      limit: 10240
                      flows: 20480
                      

                      fq_codel_in_q

                      Name: fq_codel_in_q
                      Mask: None
                      Queue Management Algorithm: Tail Drop
                      

                      Firewall Rules - Floating:
                      0cbe9dcf-3a46-4a2b-b529-90c1a1442677-image.png

                      policy routing traceroute workaround

                      Action: Pass
                      Quick: Tick Apply the action immediately on match.
                      Interface: WAN
                      Direction: out
                      Address Family: IPv4
                      Protocol: ICMP
                      ICMP subtypes: Traceroute
                      Source: any
                      Destination: any
                      Description: policy routing traceroute workaround
                      

                      limiter drop echo-reply under load workaround

                      Action: Pass
                      Quick: Tick Apply the action immediately on match.
                      Interface: WAN
                      Direction: any
                      Address Family: IPv4
                      Protocol: ICMP
                      ICMP subtypes: Echo reply, Echo Request
                      Source: any
                      Destination: any
                      Description: limiter drop echo-reply under load workaround
                      

                      WAN-In FQ-CoDel queue

                      Action: Match
                      Interface: WAN
                      Direction: in
                      Address Family: IPv4
                      Protocol: Any
                      Source: any
                      Destination: any
                      Description: WAN-In FQ-CoDel queue
                      Gateway: Default
                      In / Out pipe: fq_codel_in_q / fq_codel_out_q
                      

                      WAN-Out FQ-CoDel queue

                      Action: Match
                      Interface: WAN
                      Direction: out
                      Address Family: IPv4
                      Protocol: Any
                      Source: any
                      Destination: any
                      Description: WAN-Out FQ-CoDel queue
                      Gateway: WAN_DHCP
                      In / Out pipe: fq_codel_out_q / fq_codel_in_q
                      
                      1 Reply Last reply Reply Quote 0
                      • X
                        xciter327
                        last edited by

                        Looks good to me. Mine at home is the same, with lower speeds tough. When You do the dslreports test, You can open up htop(prefer it because it's easier to deal with multiple cores) in one window and "ipfw sched show" in another to see if the limiters are actually matching traffic. Anything else on this box(like squid or snort)?

                        1 Reply Last reply Reply Quote 0
                        • VeldkornetV
                          Veldkornet
                          last edited by Veldkornet

                          @xciter327 said in Playing with fq_codel in 2.4:

                          Looks good to me. Mine at home is the same, with lower speeds tough. When You do the dslreports test, You can open up htop(prefer it because it's easier to deal with multiple cores) in one window and "ipfw sched show" in another to see if the limiters are actually matching traffic. Anything else on this box(like squid or snort)?

                          Well I have Suricata, no squid. Although I turned Suricata off and it made no difference.

                          Download maxes the CPU, but upload doesn't seem to do much...
                          Download:

                          00001:  38.000 Mbit/s    0 ms burst 0
                          q65537  50 sl. 0 flows (1 buckets) sched 1 weight 0 lmax 0 pri 0 droptail
                           sched 1 type FQ_CODEL flags 0x0 0 buckets 1 active
                           FQ_CODEL target 5ms interval 100ms quantum 300 limit 10240 flows 20480 NoECN
                             Children flowsets: 1
                          BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes Pkt/Byte Drp
                            0 ip           0.0.0.0/0             0.0.0.0/0     2478   112953 488 26508   0
                          00002: 380.000 Mbit/s    0 ms burst 0
                          q65538  50 sl. 0 flows (1 buckets) sched 2 weight 0 lmax 0 pri 0 droptail
                           sched 2 type FQ_CODEL flags 0x0 0 buckets 1 active
                           FQ_CODEL target 5ms interval 100ms quantum 300 limit 10240 flows 20480 NoECN
                             Children flowsets: 2
                            0 ip           0.0.0.0/0             0.0.0.0/0     33141 49232048 193 287300  12
                          

                          Capture.PNG
                          Upload:

                          00001:  38.000 Mbit/s    0 ms burst 0
                          q65537  50 sl. 0 flows (1 buckets) sched 1 weight 0 lmax 0 pri 0 droptail
                           sched 1 type FQ_CODEL flags 0x0 0 buckets 1 active
                           FQ_CODEL target 5ms interval 100ms quantum 300 limit 10240 flows 20480 NoECN
                             Children flowsets: 1
                          BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes Pkt/Byte Drp
                            0 ip           0.0.0.0/0             0.0.0.0/0       31    45860  0    0   0
                          00002: 380.000 Mbit/s    0 ms burst 0
                          q65538  50 sl. 0 flows (1 buckets) sched 2 weight 0 lmax 0 pri 0 droptail
                           sched 2 type FQ_CODEL flags 0x0 0 buckets 1 active
                           FQ_CODEL target 5ms interval 100ms quantum 300 limit 10240 flows 20480 NoECN
                             Children flowsets: 2
                            0 ip           0.0.0.0/0             0.0.0.0/0       16      664  0    0   0
                          

                          Capture2.PNG

                          1 Reply Last reply Reply Quote 0
                          • X
                            xciter327
                            last edited by

                            Hm, I don't think You will have a great experience with all those thing's You've loaded on that little box. I read somewhere the APU2 is good for ~400Mpbs without Suricata/other heavy software.

                            I would try disabling all add-on(haProxy, Openvpn, pfblocker, Suricata, snmp, fancy unbound settings, tftp-server etc.) and try vanilla pfsense with just the limiters(via floating rules) and a simple "Allow all" on the LAN side. If your CPU is peaked(like You have on the download test), then better run without limiters.

                            1 Reply Last reply Reply Quote 0
                            • VeldkornetV
                              Veldkornet
                              last edited by

                              Oh? The CPU load on it is almost non-existant usually. This is the first time I've seen it go so high now with the traffic shaping. Even so, I can understand that the load may be an issue for download and I'm okay with that.

                              It's the upload speed that's annoying me...... which is not doing much on the CPU side.

                              1 Reply Last reply Reply Quote 0
                              • X
                                xciter327
                                last edited by xciter327

                                Well If I look at the picture, the upload is mostly pegged by Surricata. So that would be the first thing I disable. Also note that not all packages work normally with limiters. There used to be issues with Suricata/Squid and limiters, I don't know if it was ever fixed.

                                1 Reply Last reply Reply Quote 0
                                • VeldkornetV
                                  Veldkornet
                                  last edited by Veldkornet

                                  Okay, I disabled all of the packages, ran the test, upload good.
                                  Then I enabled each one and tested until the upload speed decreased.

                                  So it seems I have discovered the culprit... I have an OpenVPN Client setup to PIA, this is what's causing the bad upload speeds. I have an alias with an IP range defined, and basically everything within that alias should go over the VPN. And then I have an interface & gateway which is bound to the OpenVPN client; with a Firewall rule which says anything to that alias should use the VPN gateway.

                                  Is there anything special that I would need to do with my rules in this situation? The traffic itself isn’t going over the VPN.

                                  1 Reply Last reply Reply Quote 0
                                  • X
                                    xciter327
                                    last edited by

                                    My guess would be to place it at the very top of the floating rule set. I believe pf does "match most specific" unless "quick match" is selected.

                                    uptownVagrantU 1 Reply Last reply Reply Quote 0
                                    • uptownVagrantU
                                      uptownVagrant @xciter327
                                      last edited by

                                      @xciter327 said in Playing with fq_codel in 2.4:

                                      My guess would be to place it at the very top of the floating rule set. I believe pf does "match most specific" unless "quick match" is selected.

                                      pf is last match wins unless quick is enabled. Quick is enabled by default on interface rules but has to be selected on floating rules

                                      1 Reply Last reply Reply Quote 0
                                      • X
                                        xciter327
                                        last edited by

                                        Good to hear. I did not know about the default quick on interface rules. That explains some things.

                                        1 Reply Last reply Reply Quote 0
                                        • w0wW
                                          w0w
                                          last edited by

                                          I've rebuilt my network a bit, but I am not sure when this was happened — one PC behind pfSense always get the A or A+ rating and the other get the D or even lower grade rating on dslreports, both are on the same LAN. FQ_CODEL configuration is similar to the others in this thread except only that it is applied on the LAN side (LAN to any). When I disable limiters, then both PCs get A or A+ rating. Does anyone have an idea, what is going wrong?

                                          F 1 Reply Last reply Reply Quote 0
                                          • O
                                            ohbobva
                                            last edited by ohbobva

                                            What about Dual WAN?

                                            I've used the previous post (https://forum.netgate.com/post/807490), with my best guess modifications for two WANs (duplicated rules for each WAN, and duplicated queues), and received good grades on the DSLReports test. However, I use a dual WAN fallover setup. I noticed, today, when WAN1 went down and PFSense switched to WAN2, all traffic stopped. Removing the floating rules fixed the issue.

                                            The two WANs have different speeds.

                                            Does anyone have this working with dual WAN setup (different WAN speeds) in a fallover configuration? If so, what do your floating rules look like?

                                            @uptownVagrant Do you have any suggestions to modify your steps for dual WAN w/ fallover?

                                            Thanks, in advance, for any suggestions.

                                            1 Reply Last reply Reply Quote 1
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.