Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Limiters and floating rules

    Scheduled Pinned Locked Moved Traffic Shaping
    10 Posts 5 Posters 2.1k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • S
      stevenb
      last edited by

      I have a multi-wan setup and would like to enable FQ_CODEL on the WAN interfaces.

      To achieve this I have created limiters with child queues for each upload/download limit for each WAN. I have then created a number of MATCH floating rules on each WAN interface to assign the upload/download queues to traffic on those WAN interfaces.

      My problem is that the queuing of inbound (download) packets on the WAN does not seem to function correctly for packets matched by floating rules. This shows up as dropped ping packets when the download speed reaches the limiter speed. I know it is the limiter dropping the packets as I can see the ping responses come in on the WAN interface using packet capture, but they are not forwarded to the LAN interface. Upload or outbound traffic on the WAN seems to work closer to what is expected. Ping times are more in line with what I would expect for FQ_CODEL, however there are still one or two dropped ping packets during the upload speedtests.

      I have tried different variations of the above setup, such as using PASS floating rules with QUICK flag set, etc. Any floating rule variation I have tried has displayed the same characteristics.

      However, if I assign the traffic shaping queues to the Allow all from LAN rule in the LAN zone, the traffic shaping works much more like I would expect for FQ_CODEL. There are no dropped ping packets when the download speed reaches the limit, the download speed is capped at the limiter speed, and there are no dropped pings during upload either.

      My problem is due to the mulit-wan aspect of the setup, I cannot have the traffic shaping assigned on the Allow from LAN rule as that rule will automatically fail between two gateways with different link properties.

      Is anyone seeing similar issues with limiters?

      The best I think I seem to be able to achieve using limiters is only having the primary WAN link working correctly with shaping, or not being able to shape download traffic and flaky shaping of upload traffic. Both present their own issues with our desired setup.

      1 Reply Last reply Reply Quote 0
      • S
        sideout
        last edited by

        Your best bet is to establish a gateway group. Put the WAN gateways in the group and and give them the correct Tier. You can now set a gateway group as a default gateway in 2.4.4. Then under your LAN rule you can change the gateway to the group and let it do the failover for you that way.

        So if you have say 3 WAN's - WAN1 WAN2 and WAN3 . Make the following groups:
        WAN_GW_ALL - WAN1-3 Tier1 on all with Packet Loss / High Latency

        PFSense will round robin traffic now across all 3 WAN's unless packet loss / high latency or a down event removes a GW.
        Make sure under General Setup you assign a DNS server to each WAN. Typically I do this:
        WAN 1 - 8.8.8.8 / 4.2.2.2
        WAN 2 - 8.8.4.4 / 4.2.2.3
        WAN 3 - 4.2.2.4 / OpenDNS server or ISP

        This is similar to how I set it up for my LAN parties but I create a few different groups and rules to send traffic to a specific WAN / modem.

        1 Reply Last reply Reply Quote 0
        • S
          stevenb
          last edited by

          I think you've misunderstood the problem. I don't have any problem with the gateway failover setup. This is working without issue.

          My issue is I can only seem to get the traffic shaping limiters to work as expected if I apply it to the LAN interface allow rule that uses the failover gateway. The issue with this is my traffic shaping limiters are specific for each WAN upload/download speed, but I can only apply a single upload/download queue pair on this rule. When the gateway switches, the traffic shaping limits are no longer correct for the other WAN gateways as the gateways all run at different speeds.

          To apply the correct traffic shapers to each WAN I created floating rules tied to each specific WAN interface. As mentioned in my post though, this doesn't seem to work correctly as it will drop pings completely if the download limit is reached and intermittently drop pings if the upload limit is reached, which is not how I would expect FQ_CODEL to react and not how it reacts if the queues are applied to the LAN interface rule.

          mattundM 1 Reply Last reply Reply Quote 0
          • mattundM
            mattund @stevenb
            last edited by

            @stevenb

            That's not how it's supposed to react; some discussion is going on in the main/gigantic fq_codel thread in this sub-forum, and a bug report was put in as well. FYI, I have seen this in 2.4.3 testing this patch as well. For now, I've made my rules not classify ICMP traffic, and reset states prior to testing again.

            R 1 Reply Last reply Reply Quote 0
            • R
              revengineer @mattund
              last edited by revengineer

              @mattund said in Limiters and floating rules:

              For now, I've made my rules not classify ICMP traffic,....

              Can you explain (or show gui snapshot) how you did that? I have a web-based power strip that pings connected equipment and power cycles them if they don't respond. With FQ_CODEL pings don't come through when pfsense is under heavy load and the equipment is power cycled unnecessarily. It sounds like you have worked out the solution to my problem, but I am not sure how to implement your suggestion.

              Thank you in advance!

              T 1 Reply Last reply Reply Quote 0
              • T
                tibere86 @revengineer
                last edited by

                @revengineer said in Limiters and floating rules:

                @mattund said in Limiters and floating rules:

                For now, I've made my rules not classify ICMP traffic,....

                Can you explain (or show gui snapshot) how you did that? I have a web-based power strip that pings connected equipment and power cycles them if they don't respond. With FQ_CODEL pings don't come through when pfsense is under heavy load and the equipment is power cycled unnecessarily. It sounds like you have worked out the solution to my problem, but I am not sure how to implement your suggestion.

                Thank you in advance!

                +1

                1 Reply Last reply Reply Quote 0
                • mattundM
                  mattund
                  last edited by mattund

                  I've made my floating rule classify only TCP/UDP traffic, and not "All" protocols

                  0_1539699243800_2eb3d109-9e91-4144-ba6f-bed6d29f3e9d-image.png

                  This excludes things like ICMP from being touched by FQ_CODEL. If traffic other than TCP/UDP, but not ICMP is a concern to you, you may want other floating rules for those protocols as well.

                  R 1 Reply Last reply Reply Quote 0
                  • R
                    revengineer @mattund
                    last edited by revengineer

                    @mattund said in Limiters and floating rules:

                    I've made my floating rule classify only TCP/UDP traffic, and not "All" protocols

                    This excludes things like ICMP from being touched by FQ_CODEL. If traffic other than TCP/UDP, but not ICMP is a concern to you, you may want other floating rules for those protocols as well.

                    Got it, that makes sense. I thought there was a way to specify all minus ICMP. Follow up question: Did you reduced the bandwidth for the CoDel traffic to reserve bandwidth for the non TCP/UDP traffic? Or is this typically in the noise and does not matter?

                    mattundM 1 Reply Last reply Reply Quote 0
                    • mattundM
                      mattund @revengineer
                      last edited by

                      @revengineer

                      I count it as "in the noise" per-say, and let it pass my mind. I find ICMP traverses the network leisurely anyway, and besides, I haven't found a way to NOT drop the traffic -- if ICMP is getting dropped a lot of other problems can seep in...

                      R 1 Reply Last reply Reply Quote 0
                      • R
                        revengineer @mattund
                        last edited by

                        @mattund said in Limiters and floating rules:

                        @revengineer

                        I count it as "in the noise" per-say, and let it pass my mind. I find ICMP traverses the network leisurely anyway, and besides, I haven't found a way to NOT drop the traffic -- if ICMP is getting dropped a lot of other problems can seep in...

                        Great, thanks. I made the change and confirm that I can ping my cable modem connected to WAN even under full bandwidth load.

                        1 Reply Last reply Reply Quote 1
                        • First post
                          Last post
                        Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.