"Hard" limit for the queue



  • I wonder if there is a way to set up a queue not to take more than specified amount of bandwidth, ex.: I'd like my P2P upload queue always stay below 500kbit/s.
    Right now I modified the settings created by Shaper wizard - changed Bandwidth from 1 %  to 500kbit/s, but it doesn't work as I planed, P2P upload is still abowe the limit.
    Please advise - thank you in advance.



  • Firewall>Traffic Shaper>Queues.  To put a 500 Kb limit on the bandwidth, you need to put "500Kb" into the Service Curve Section in the m2 column box next to "The maximum allowed bandwidth for the queue."  Make sure your in the correct queue for P2P.  It takes a few moments to kick in after "Applying" the changes.

    Why do you want to limit it anyway?  The traffic shaper will limit it when neccessary and let it go as fast as it can when nothing else is going on.



  • Thank you for your help :)

    Every think I needed was written on the queue setup screen … I should have read it more carefully  :-[

    I need this rule because I have FTP server using the same connection and it was impossible to download anything from it when P2P was taking most of upload.



  • I have P2P running a lot maxing out both up/down speeds (1.5Mb/1Mb), and when a voice call comes in or I make one using my VoIP service, it sounds fine.  I can even surf fine.  I used the traffic shaper to start with, and then tweaked it.  For example, instead of using ports to prioritize my VoIP, I use my phone adapters IP address since it uses a different and dynamic port (seems to usually use one in the 16000's) for the SIP protocol and uses 5060 to register with my provider.  I have put 300Kb in the minimum bandwidth boxes for the queue so that the FW automatically reserves that amount of bandwidth for VoIP when it detects traffic to and from my phone adapter.  When there's no VoIP calls, everything else on the network can use the full bandwidth again.  Before I changed it to use the IP, all my VoIP traffic would end up falling into the default queue.  However, because the default has a higher priority than P2P, all my calls still came through without a hiccup.

    If the shaper rules for P2P aren't set up right, then it will stomp all over other traffic.  Take bittorent for example.  The port used by most clients is 6881 (I personally use one much, much higher).  Other ports in that range are used also.  I have found that by making a total of 4 rules for it in the traffic shaper, puts virtually all of the P2P traffic into the P2P queue.  I have 2 rules for all outgoing traffic (as opposed to using the "any" setting for traffic direction) with the filtering being done at the LAN interface.  One rule has port 6881 or the appropriate range as the source and "any" as the destination.  The other rule has just the opposite.  The second set of 2 rules is used for incoming traffic with the filtering done at the WAN interface and they mirror the first 2 rules.  Some may argue that this approach is redundant, but I look at it as throwing a "bigger net" out there that will catch ALL of a certain type of traffic.  If the shaping rules are too specific, then traffic that should go to a higher priority queue might go to a lower one and vice-versa.  I'll use 4 variables of a packet (Source IP, Dest IP, Source Port, Dest Port).  If a shaping rule is put in place that specifically says all 4 variables have to match up before it gets put in a specific queue, then not all of that traffic is going to make it to that queue.  But if four differant rules are used that match a packet to only one of those variables, then it is likely that all of a specific type of traffic will be put into the appropriate queue.  Before I tried this approach, the traffic shaping only mostly worked.  With this approach, not only does it work but it is very responsive.

    Other FW's that I've used like Monowall and Coyote Linux w/ Wondershaping scripts (now BrazilFW), both of which I like a lot for their ease of use and tons of features which IMO blow away other do-it-yourself FW boxes and retail routers, always seemed to give me a lag before the traffic shaping would kick in, especially with VoIP calls.  pfSense seems to kick in with its traffic shaping almost instantaneously.  That must be the benefit of ALTQ.

    What you might want to do is, rather than limiting the bandwidth for the P2P queue, do the following.  1) Make sure your P2P shaping rules are like those I described above.  2)Create an FTP Up & Down queue, or use one the wizard creates (possibly the qOthersUpL/qOthersDownL), and give them a priority just one number higher than the default queues.  3)Make sure only FTP traffic is being put into the FTP queues by using shaping rules that include all traffic on port 21 or range or whatever your FTP server uses.  4)Give the FTP queues the minimum amount of bandwidth you want them to have.  (You could give them all of your bandwidth if you wanted.)  When your done, FTP traffic will only use bandwidth when it's needed and it will be guaranteed a minimum amount.  P2P traffic will be kept at bay when the bandwidth is needed by higher priority traffic and then let to go full speed again when nothing else is going on.

    Please let me know how it goes.  I know how frustrating traffic-shaping is especially when what is written about it is explained in a way that is only easily understood by those who code it.



  • Wow, nice work.  Would you mind emailing me the export of your shaper config?  Go to Diagnostics->Backup/Restore and backup the Shaper config there.  I'd love to see it to find room for improvement in the existing wizard (I'm always looking for ways to improve it to keep people from having to deal with the normal UI for modifying stuff).

    –Bill

    @charincol:

    I have P2P running a lot maxing out both up/down speeds (1.5Mb/1Mb), and when a voice call comes in or I make one using my VoIP service, it sounds fine.  I can even surf fine.  I used the traffic shaper to start with, and then tweaked it.  For example, instead of using ports to prioritize my VoIP, I use my phone adapters IP address since it uses a different and dynamic port (seems to usually use one in the 16000's) for the SIP protocol and uses 5060 to register with my provider.  I have put 300Kb in the minimum bandwidth boxes for the queue so that the FW automatically reserves that amount of bandwidth for VoIP when it detects traffic to and from my phone adapter.  When there's no VoIP calls, everything else on the network can use the full bandwidth again.  Before I changed it to use the IP, all my VoIP traffic would end up falling into the default queue.  However, because the default has a higher priority than P2P, all my calls still came through without a hiccup.

    If the shaper rules for P2P aren't set up right, then it will stomp all over other traffic.  Take bittorent for example.  The port used by most clients is 6881 (I personally use one much, much higher).  Other ports in that range are used also.  I have found that by making a total of 4 rules for it in the traffic shaper, puts virtually all of the P2P traffic into the P2P queue.  I have 2 rules for all outgoing traffic (as opposed to using the "any" setting for traffic direction) with the filtering being done at the LAN interface.  One rule has port 6881 or the appropriate range as the source and "any" as the destination.  The other rule has just the opposite.  The second set of 2 rules is used for incoming traffic with the filtering done at the WAN interface and they mirror the first 2 rules.  Some may argue that this approach is redundant, but I look at it as throwing a "bigger net" out there that will catch ALL of a certain type of traffic.  If the shaping rules are too specific, then traffic that should go to a higher priority queue might go to a lower one and vice-versa.  I'll use 4 variables of a packet (Source IP, Dest IP, Source Port, Dest Port).  If a shaping rule is put in place that specifically says all 4 variables have to match up before it gets put in a specific queue, then not all of that traffic is going to make it to that queue.  But if four differant rules are used that match a packet to only one of those variables, then it is likely that all of a specific type of traffic will be put into the appropriate queue.  Before I tried this approach, the traffic shaping only mostly worked.  With this approach, not only does it work but it is very responsive.

    Other FW's that I've used like Monowall and Coyote Linux w/ Wondershaping scripts (now BrazilFW), both of which I like a lot for their ease of use and tons of features which IMO blow away other do-it-yourself FW boxes and retail routers, always seemed to give me a lag before the traffic shaping would kick in, especially with VoIP calls.  pfSense seems to kick in with its traffic shaping almost instantaneously.  That must be the benefit of ALTQ.

    What you might want to do is, rather than limiting the bandwidth for the P2P queue, do the following.  1) Make sure your P2P shaping rules are like those I described above.  2)Create an FTP Up & Down queue, or use one the wizard creates (possibly the qOthersUpL/qOthersDownL), and give them a priority just one number higher than the default queues.  3)Make sure only FTP traffic is being put into the FTP queues by using shaping rules that include all traffic on port 21 or range or whatever your FTP server uses.  4)Give the FTP queues the minimum amount of bandwidth you want them to have.  (You could give them all of your bandwidth if you wanted.)  When your done, FTP traffic will only use bandwidth when it's needed and it will be guaranteed a minimum amount.  P2P traffic will be kept at bay when the bandwidth is needed by higher priority traffic and then let to go full speed again when nothing else is going on.

    Please let me know how it goes.  I know how frustrating traffic-shaping is especially when what is written about it is explained in a way that is only easily understood by those who code it.



  • Billm

    I am going to post my shaper config file at the top of a new post.



  • There are still uses for hard limits on P2P speeds. For instance if I let DC++ download at my max speed ( 100Mbit ) then everything else just bogs down to a slow crawl. Not due to no available bandwidth but cuz the comp is stressed to hanlde so high multiple downloads.

    But i suppose its not really a firewall issue – i use netlimiter for that now and it works... Now what I would like is if Netlimiter could max out the speeds when the screensaver kicks in - but otherwise limit P2P progs. That would be a nice feature.

    /DaK/



  • @charincol:

    Firewall>Traffic Shaper>Queues.  To put a 500 Kb limit on the bandwidth, you need to put "500Kb" into the Service Curve Section in the m2 column box next to "The maximum allowed bandwidth for the queue."  Make sure your in the correct queue for P2P.  It takes a few moments to kick in after "Applying" the changes.

    How about "hard limit" for total upload?

    Solution described abowe doesn't work for qWANRoot.

    I also find out that real upload is higher then value defined in Bandwidth field for qWANRoot queue.



  • Are you running the latest image?  http://www.pfsense.com/~sullrich/1.0-BETA1-TESTING-SNAPSHOT-1-24-06/

    If not then upgrade and rerun the traffic shaper wizard.



  • @sullrich:

    Are you running the latest image?   http://www.pfsense.com/~sullrich/1.0-BETA1-TESTING-SNAPSHOT-1-24-06/

    If not then upgrade and rerun the traffic shaper wizard.

    Right now I have Beta 1 first release.
    I'll try to upgrade soon



  • Just for clarity here, I'm going to have a very complicated installation coming up soon, and I want to make sure this is going to work.

    I need to be able to shape/limit throughput on several levels.  One would be based on either the source IP or mac address (I'm presuming it would have to be by IP), so that if someone is paying for 1Mbit of throughput, they only get 1Mbit total, and the other based on protocol, ie, SIP, IAX2, and SSH get high priorities, web, ftp, get regular, and p2p gets low priority.

    If I'm reading correctly, pfSense can do all of the above, right?  I need to hard limit total throughput available to individual peers connected on an interface as well as shape the overall traffic.  If it's 2am they're welcome to p2p all they want, but I can't have p2p futzing up voice calls, nor can I have a single user dominating the overall available bandwidth.



  • Shaper is being worked on still.  I would not deploy it yet.



  • This wouldn't be a production environment for a couple more months.  Right now I'm just trying to get my technology lined up.  Presuming it were working as expected, is what I said correct?



  • I would wait for beta2 atleast.



  • @sullrich:

    I would wait for beta2 atleast.

    And if you have sis cards I would forget about using the shaper.  We've found an OS level bug that affects those cards (at a minimum).  Intel FXP cards at this time are the only known good cards (I'll be posting a survey to get more info later).

    –Bill



  • Not really trying to be a pest (honest!) but you didn't answer the question. :(

    I know the code is in beta.  I'm not looking for production deployment just yet.  What I'm asking is that bugs aside, is what I'm talking about feasible?  Would traffic shaper, when behaving as expect, deliver what I was referring to?  Could I arbitrarily say, "traffic to and from ip x.x.x.x(.x.x) is not to exceed, 2mbit/sec at any given time" and also be able to prioritize traffic and still say, IAX2 and SIP get high priority, then ssh, vnc, x11, or rdp, then web traffic, etc?  I've gotten the feeling the answer is yes, but I wasn't sure.  I really haven't had the opportunity to dig into pipes and queues, and I wan to be able to contribute more to this project, but got severely sidetracked by a business emergency of sorts that is going to keep me tied up for probably 3 more weeks. :(  OpenVPN has had to take a back seat because of this too, but the feature is so important to my business that I simply MUST find time to get back to it.

    Sorry, wasn't trying to wail in your thread guys….

    Anyhoo, am I on the right track?



  • Possibly.  Try it in beta2

    –Bill


Locked