Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Limiter with Burst or similar solution needed

    Scheduled Pinned Locked Moved Traffic Shaping
    31 Posts 11 Posters 20.2k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • C
      caust
      last edited by

      Greetings,

      I'd like to configure traffic shaping where, on a given LAN interface, each Source IP's new connections get a burst of up to 100% of a WAN connections bandwidth for 20 seconds. Then the connection's bandwidth is reduced to a hard limit of 100Kbit/s for the remainder of the connection.

      Can I achieve this affect from configs in the pfSense GUI, or shell on a pfSense 2.0.2 appliance? I can upgrade to 2.0.3 if needed. If not, does anyone have ideas on how closest I can come to satisfying the following use case?

      The use case is that we have a Satellite Broadband ISP with the infamous "Fair Access Policy" (FAP). The WAN connection supports resort customers. My goal is to reduce the frequency that the WAN connection is throttled under penalty of breaking FAP thresholds. I hope to achieve this while providing the fastest experience for users who simply browse web pages.

      On an unrelated note: Thank you all for the great info in this forum. I've been able to push the limits of pfsense thanks to all the technical info provided within this forum.

      caust

      1 Reply Last reply Reply Quote 0
      • C
        caust
        last edited by

        I guess I'm heading down the same path as many before me.

        I attempted to use the```
        ipfw pipe config

        
        1) For some reason the burst settings are not being honored, or there are other configurations that are negating the burst setting. A number of posts throughout the web indicate the same finding. I was really bummed because this seems to be an ideal solution. We could give each client IP a bw volume quota. Once used, they would have significantly degraded access. I need burst to make that work though.
        
        2) I can see why pfSense only allows one mask type (either src-ip or dst-ip) because as soon as you start pairing masks, dynamic pipes generate like crazy. There is surely a performance cost at scale.
        
        Does anyone know how to get the burst option working with the Limiters? Are there options for generating dynamic queues based on IP when going the "By Interface" and "By Queue" routes for traffic shaper? I noticed the "Queue" options under packet shaper don't appear to make use of the same ipfw queues but are configured elsewhere.
        
        Any suggestions would be appreciated.
        1 Reply Last reply Reply Quote 0
        • S
          SeventhSon
          last edited by

          I think you can actually do this with the HFSC scheduler, you can do things like burst 100% for 20 seconds, after on 10% or something similar.

          with these values:

          m1:  The amount of bandwidth to allocate to the class for "d" number of milliseconds.
          d:  Millisecond time setting described above.
          m2:  The max limit for this curve for this class.

          1 Reply Last reply Reply Quote 0
          • C
            caust
            last edited by

            Thanks for replying.

            I like that feature of altq. I'm not exactly sure how it behaves in practice then. If many hosts are going through the queue, is it only the first to the queue that experience the burst and then the queue remains at the m2 slope until the queue is momentarily emptied and the m1 slope begins again?

            I'd really like the queues to generate dynamically for each source ip. I don't want to penalize normal internet browsing from an ip at all but only penalize a source ip when they begin large downloads… etc.

            1 Reply Last reply Reply Quote 0
            • S
              SeventhSon
              last edited by

              you're right, this will be for everything, not per source IP, sorry.

              The limiter option doesn't have a burst option in the GUI, I haven't looked into that further.

              1 Reply Last reply Reply Quote 0
              • jimpJ
                jimp Rebel Alliance Developer Netgate
                last edited by

                We don't have a GUI knob for it, but Limiters are based on ipfw/dummynet pipes, and those do support a burst parameter:

                From ipfw(8):

                burst size
                            If the data to be sent exceeds the pipe's bandwidth limit (and
                            the pipe was previously idle), up to size bytes of data are
                            allowed to bypass the dummynet scheduler, and will be sent as
                            fast as the physical link allows.  Any additional data will be
                            transmitted at the rate specified by the pipe bandwidth.  The
                            burst size depends on how long the pipe has been idle; the effec-
                            tive burst size is calculated as follows: MAX( size , bw *
                            pipe_idle_time).

                If someone wanted to hack together a patch, it may be possible to leverage that.

                Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

                Need help fast? Netgate Global Support!

                Do not Chat/PM for help!

                1 Reply Last reply Reply Quote 0
                • S
                  SeventhSon
                  last edited by

                  So you could do this, just not in GUI, I don't know enough about doing this is CLI (well, mostly, how to do it properly to make it stick across reboots/reloads of the firewall). Maybe someone else can help with that.

                  1 Reply Last reply Reply Quote 0
                  • jimpJ
                    jimp Rebel Alliance Developer Netgate
                    last edited by

                    It might actually be easier for someone to add a field to the limiter config to do that than it would to hack it in manually.

                    Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

                    Need help fast? Netgate Global Support!

                    Do not Chat/PM for help!

                    1 Reply Last reply Reply Quote 0
                    • F
                      foxale08
                      last edited by

                      Got tired of waiting so I did it myself. I make no promises it works completely right. It can be cleaned up a bit and put into 2.1 if desired.

                      shaper_burst.patch.txt

                      1 Reply Last reply Reply Quote 0
                      • E
                        eri--
                        last edited by

                        Seems correct implementation so i committed in snapshots.
                        Just test it with new coming snapshots or gitsync

                        1 Reply Last reply Reply Quote 0
                        • X
                          xbipin
                          last edited by

                          problem with the patch is it doesnt upgrade the config, meaning if the old snapshot didnt have a burst value and u upgraded then u get errors in the system log untill u goto the limiter page and feed in a burst value and hit save

                          1 Reply Last reply Reply Quote 0
                          • jimpJ
                            jimp Rebel Alliance Developer Netgate
                            last edited by

                            xbipin - I just committed a fix for that: https://github.com/pfsense/pfsense/commit/f1a17b1a085ca65fafbe28e7c36cf9fc0018f2b0
                            https://github.com/pfsense/pfsense/commit/e43fa2ac995158553a47a0169bfd4946fe44a81b
                            https://github.com/pfsense/pfsense/commit/c32e058108193d17da7085623775c94d21a8bd96

                            Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

                            Need help fast? Netgate Global Support!

                            Do not Chat/PM for help!

                            1 Reply Last reply Reply Quote 0
                            • X
                              xbipin
                              last edited by

                              i quiet dont understand how this burst thing works, firstly, after the patch can we set a burst as zero or blank and secondly if burst is set to 1mb and the pipe also to 1mb and the link supports 2mb then how would it actually work?

                              1 Reply Last reply Reply Quote 0
                              • jimpJ
                                jimp Rebel Alliance Developer Netgate
                                last edited by

                                Burst is an amount of data, it is not a rate

                                Setting a burst of blank/0 is OK and what most people will do to not allow bursting.

                                If you have a 1Mbit/s limit and a 1MB burst, then the user will get 1MB of data at full speed, then be limited to 1Mbit/s.

                                In this example, with no burst set, the user will be limited to 1Mbit/s at all times.

                                Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

                                Need help fast? Netgate Global Support!

                                Do not Chat/PM for help!

                                1 Reply Last reply Reply Quote 0
                                • X
                                  xbipin
                                  last edited by

                                  so wouldnt it be better to put some description on that page saying burst is actual data and not rate and secondly the rules.limiter file shows me this

                                  
                                  pipe 1 config  bw 480Kb burst 480Kb
                                  
                                  pipe 3 config  bw 400Kb burst 400Kb
                                  
                                  

                                  isnt the burst supposed to be KB and not Kb and also the interface doesnt allow to specify the unit separately for rate and burst

                                  1 Reply Last reply Reply Quote 0
                                  • jimpJ
                                    jimp Rebel Alliance Developer Netgate
                                    last edited by

                                    @xbipin:

                                    so wouldnt it be better to put some description on that page saying burst is actual data and not rate and secondly the rules.limiter file shows me this

                                    
                                    pipe 1 config  bw 480Kb burst 480Kb
                                     
                                    
                                    pipe 3 config  bw 400Kb burst 400Kb
                                    
                                    

                                    isnt the burst supposed to be KB and not Kb and also the interface doesnt allow to specify the unit separately for rate and burst

                                    The description could be better, yes. I don't know about the ipfw syntax.

                                    Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

                                    Need help fast? Netgate Global Support!

                                    Do not Chat/PM for help!

                                    1 Reply Last reply Reply Quote 0
                                    • F
                                      foxale08
                                      last edited by

                                      Has anyone been able to make burst work as expected? Speed tests have not shown evidence of the bursting parameter on the child queues in my config. My current assumptions are that either the "pipe_idle_time" is impossibly low as burst values many orders of magnitude higher produce no results and/or the burst only applies to the first packet sent through an idle link. (I took a quick glance at the current dummynet source, I have limited understanding of C/C++ syntax at present.) Some online have suggested changing the kern.hz parameter in /boot/loader.conf (/boot/loader.conf.local). Additionally, would an expire of 1 cause a link to never be considered idle as it is removed so quickly (net.inet.ip.dummynet.expire=1). What do you guys think?

                                      pipe 1 config  bw 14Mb burst 80Mb
                                      queue 1 config pipe 1 mask dst-ip6 /128 dst-ip 0xffffffff
                                      queue 2 config pipe 1 mask dst-ip6 /128 dst-ip 0xffffffff
                                      queue 3 config pipe 1 mask dst-ip6 /128 dst-ip 0xffffffff
                                      
                                      pipe 2 config  bw 2.5Mb burst 40Mb
                                      queue 4 config pipe 2 mask src-ip6 /128 src-ip 0xffffffff
                                      queue 5 config pipe 2 mask src-ip6 /128 src-ip 0xffffffff
                                      queue 6 config pipe 2 mask src-ip6 /128 src-ip 0xffffffff
                                      

                                      *bw values are half of ISP provided bandwidth.

                                      Edit: Burst does work, but I can only prove it at a 'bw' of 25Kb using ping…

                                      1 Reply Last reply Reply Quote 0
                                      • S
                                        stephenminta
                                        last edited by

                                        foxale08 did you ever get any further with this, I am seeing exactly the same results as you with regards to the burst setting.

                                        Thanks

                                        1 Reply Last reply Reply Quote 0
                                        • E
                                          eri--
                                          last edited by

                                          The burst setting will not be applied unless the queue is congested.

                                          So unless you fill the pipe you will not see the effect of bursting.

                                          1 Reply Last reply Reply Quote 0
                                          • S
                                            stephenminta
                                            last edited by

                                            Thanks Ermal. Am I correct in saying then that if I use the settings in the attached screen shot each ip address using more than 2mb of bandwidth (i.e. congesting the queue) will be allowed to burst to the max bandwidth available on the interface until it has passed 10mb of data and then it will drop back to the 2mb rate. In real world use should this not mean that if I run a speedtest from say speedtest.net I should see it burst above 2mb then back equally if I run jperf there should be an initial burst followed by a consistent 2mb.

                                            Many thanks

                                            traffic_limit.jpg
                                            traffic_limit.jpg_thumb

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.