Interesting Traffinc Shaping needs
-
Hello all!
I have used Pfsense for a while at home but until now have never needed to use shaping. Here is some background on my issue. I am deployed to Iraq and my unit wanted internet. So we pooled our money and purchased a satellite dish and a plan for 2084K down and 512K up. The hardware wasnt cheap and neither is the service and it works fairly well but with some large limitations. We have about 45 people sharing this connection (or should i say trying to share!).
- The ISP has a monthly "on peak" traffic limit of 15 GB. This means that you can download all you want (voip calls to home, email, skype, internet etc) without any penalty with the exception of point 2.
2) The ISP has a fair usage policy where we can only download so much so fast. If we download alot (example 300 mb) in a very short period of time, our "bucket" fills up and the ISP throttles our bandwidth to 1/3. This "bucket" drains out slowly and then we get out bandwidth back (it only drains if you stop using the bandwidth).
I have no idea how to implement this and would really appreciate anyones guidance - we have a little money left ($100 - its not much but if you know what your doing it could be a quick $100!).
What I want to do is this:
1) I want to set up a dedicated amount of bandwidth to be used for voip/skype. I want this bandwidth to be able to be used by other traffic when no calls are being placed, but to always be available if you need to make a call home.
2) I want to be able to throttle certain people that abuse the bandwidth - 1 guy downloading the next episode of lost (not that we dont all love that show!) screws everyone else. If it could do that automatically with a fair usage policy that would be awesome.
3) It is important to cut the use of our bandwidth during peak hours (so as not to go over our 15 GB limit) - i want to cut the amount of bandwidth available during that time so that people dont go crazy during the peak hours. 10 am to 10 pm is our peak hours.
4) To create some sort of throttling when downloading large files - ie, if you are downloading a large file, the traffic shaper will throttle the bandwidth to slow down the download thus allowing the file to download more slowly. This would help us to not hit that "bucket" limit and be throttled by the ISP. Even if no one else is on, if 1 person downloads too much too fast, we get throttled.I know this is a big wish list. I have done enough research to know what i want, but im not sure if it can be done with pfsense. Thanks for your inputs. I really appreciate your time. :)
Nathan Dennen, 1Lt, USAF
- The ISP has a monthly "on peak" traffic limit of 15 GB. This means that you can download all you want (voip calls to home, email, skype, internet etc) without any penalty with the exception of point 2.
-
1) I want to set up a dedicated amount of bandwidth to be used for voip/skype. I want this bandwidth to be able to be used by other traffic when no calls are being placed, but to always be available if you need to make a call home.
After running the wizard open the VoIP queue. Make it realtime parameters look m1=0b d=10 m2=256Kb or 512Kb if you prefer for download. For upload set the m2=128Kb or 256Kb. This makes your guaranteed VoIP bandwidth enough to satisfy the poeple needs with that bandwidth. It will go to other uses if VoIP is not used or part of it will be used and the other go tho the other streams when you are not stressing it. Be sure to make the bandwidth parameter 32Kb and remove any linkshare parameter.
Remove the realtime paramteres from all other queues. And for the p2p queue set its m1=m2 paramter and d=300 plus set upperlimit parameters of this queue to m2=50% of your bandwidth so they do not hog you bucket.2) I want to be able to throttle certain people that abuse the bandwidth - 1 guy downloading the next episode of lost (not that we dont all love that show!) screws everyone else. If it could do that automatically with a fair usage policy that would be awesome.
3) It is important to cut the use of our bandwidth during peak hours (so as not to go over our 15 GB limit) - i want to cut the amount of bandwidth available during that time so that people dont go crazy during the peak hours. 10 am to 10 pm is our peak hours.
4) To create some sort of throttling when downloading large files - ie, if you are downloading a large file, the traffic shaper will throttle the bandwidth to slow down the download thus allowing the file to download more slowly. This would help us to not hit that "bucket" limit and be throttled by the ISP. Even if no one else is on, if 1 person downloads too much too fast, we get throttled.For this install squid proxy from packages and setup to throttle binary or files from extensions. Just click and go for these on the squid configuration. Now you should be set to do all you wanted.
-
Thanks for the fast response! ;D I will look into it tonight if i have a chance and get back to you with any questions.
-Nate
-
Ermal, are you talking about the shaper that's currently available in 1.2 final or do you mean the features you implemented in the shaper bounty/future releases?
-
Both of them either on the newer one some more tweaking can be done ;).
Though i do not remember if m1 < m2 is allowed by the one on 1.2!
-
I currently have 1.2rc4 - is this the one i need to have or should i DL 1.2 final?
-Nate
-
Yeah should be the same.
-
OK - i have my pfsense box up and running. Traffic shaper is configured as mentioned above. I have tried to install squid but it fails.
As a result i thought i might be my version, so i downloaded 1.2 final. Its installed but im still having issues installing squid. Any ideas on thats?My voip calls are going through now, but everyone is downloading as max speed the rest of the time. Squid was my answer to throttling that bandwidth… right? Or is there another way to reduce that traffic in the shaper?
-
Well try setting the queue where http traffic goes with an upperlimit set like:
m1= 32Kb d = 80 m2=10Kb and see.On the other queues set the upperlimit m2=50%. Not on VoIP one ;)
-
Interestingly… skype does not seem to be using toe Voip que. Most everything is working well with the exception of skype. I was under the impression that it would be voip but i was wrong. is there a way to create a que JUST for skype?
-
No, skype is not using static ports, it even tries to hide in port 80 (http) sometimes. That's why it's nearly impossible to shape unless you have the possibility to inspect packages on layer7 (not yet supported).
-
Skype is using the p2p que - i had made it the catch all, but it seems to be the only way to ensure that skype makes it into THAT que! So what i did was set the p2pUp bandwidth to 64K, changed the priority to 7 ( i think thats the highest priority), realtime set m1 = 0Kb, d = 10, m2 = 128Kb and the p2pDown bandwidth to bandwidth to 64K, changed the priority to 7, set m1 = 0Kb, d = 10, m2 = 128Kb.
Then for the qOthersDownH i set bandwidth to 50% (this means 50% of the total bandwidth - ie qRoot right), changed the priority to 2, realtime set m1 = 0Kb, d = 600, m2 = 128Kb.
Then for qOthersUpH i set bandwidth to 25%, changed the priority to 2, realtime set m1 = 0Kb, d = 600, m2 = 64Kb.
What i think im doing is creating a tunnel that will use up to 128kb down and 64Kb up, with a long delay (600) so that it slows http traffic down, and that, when instatiating multiple ques will use up to 50% of the total bandwidth for downloading and 25% total for uploading.
Is that correct or am i way off?
-
Skype is using the p2p que - i had made it the catch all, but it seems to be the only way to ensure that skype makes it into THAT que! So what i did was set the p2pUp bandwidth to 64K, changed the priority to 7 ( i think thats the highest priority), realtime set m1 = 0Kb, d = 10, m2 = 128Kb and the p2pDown bandwidth to bandwidth to 64K, changed the priority to 7, set m1 = 0Kb, d = 10, m2 = 128Kb.
Then for the qOthersDownH i set bandwidth to 50% (this means 50% of the total bandwidth - ie qRoot right), changed the priority to 2, realtime set m1 = 0Kb, d = 600, m2 = 128Kb.
Then for qOthersUpH i set bandwidth to 25%, changed the priority to 2, realtime set m1 = 0Kb, d = 600, m2 = 64Kb.
What i think im doing is creating a tunnel that will use up to 128kb down and 64Kb up, with a long delay (600) so that it slows http traffic down, and that, when instatiating multiple ques will use up to 50% of the total bandwidth for downloading and 25% total for uploading.
Is that correct or am i way off?
Do not use realtime for qOthersUpH or any other queue that is not realtime traffic ie VoIP or Video(trust me on this).
Skype is a pitta, but you have found yourself that you can configure a policy which can catch that too.
So leave VoIP queue as i sugested previously make qDefault with the same parameters as qVoIP and configure the other queues as i said above. The only change you need to make is to be sure that p2p traffic or any uncategorizaed traffic does not go to this qDefault queue.
This should set you up and ready. Though be aware that if skype goes to HTTP you will not be able to catch that, though it happens while you are blocking it or some strange routing/natting is happening at the ISP. -
I really appreciate the help… i am just not getting it.
I recreated the traffic shaper from scratch today. And now for the first time, skype isnt using p2p ques. it used th` voip que once, and then used the queOthersH... i dont understand why its not using the p2p que.
Also, please explain to me since im a noob what the bandwidth entry is, what m1, d and m2 mean. I think that they mean, bandwidth (percentage or fixed) is a function of the TOTAL bandwidth of your root que. Thus if you have 4 ques, at 25%, they will each only be able to use 25% of that que.
m1 is the initial amount of bandwidth assigned, d is a delay to slow down the packets, and m2 is the amount of bandwidth that you actually want to have (either for a minimum or an upper limit).
Now if you have a 1 MB connection, you have 4 ques, you set one que to minimum of 128Kb and then set the bandwidth to, say, 50%, then that que can use up to 50% of the total (or 512Kb) right? if you set 128Kb as a maximum, then setting bandwidth to 50% would be pointless since it would always max out at 12.5% - am i correct?second senario.
if you had 2 ques, 1 mb total bandwidth, que1 was set to 128Kb minimum and 75% bandwidth with a priority of 7, and que2 was set to 256Kb minimum and 50% bandwidth and a priority of 6, then if que 1 wanted 768Kb of bandwidth, it would get it - right - and que2 would be limited to 256Kb? Please tell me if my understanding is incorrect. I dont mind being told that im wrong!!!
-
It is complicated actually :).
A quick explanation:
upperlimit
(per each queue)
m1 - bandwidth in percentage that would be satisfied during a scheduler run/turn.
d - when the shaper is doing its magic delay before this queue gets served again if it has passed m1
m2 - bandwidth hard limit, cumulative. Will delay packet if hit.
linkshare
(cumulative, the sum of all queues should not be more than 100% or the link bandwidth)
m1 - bandwidth in percentage that would be satisfied during a scheduler run/turn.
d - when the shaper is doing its magic delay before this queue gets served again if it has passed m1
m2 - bandwidth that the shaper will try to satisfy durin long run but not a hard limit. If available more would be given.
realtime
(each queue can have up to 80% of total link bandwidth)
m1/d - the same as above
m2- hard limit in the long run which states that it will not get realtime scheduler services if this limit is hit.Now the other explanation is complex since you need to understand the internals of the operating system and why an upperlimit of m1=35Kb d=300 m2=10Kb does not limit you to 10Kb max.
Simple explanation is that this limit is evaluated each d and serves m1 during d and this applies on a per packet bases.
While an upperlimit of m2 = 10Kb is linear and indeed limit all the packets in the queue to 10Kb. -
Hmm that is alot more complicated than i thought! I redid the shaper wizard again and its working much much better. Im not sure how to add some of these rules manually, so i checked the box for the p2p catch all, and then selected all of the individual p2p services. Now skype shows up again in the p2p que! must be associating it with one of the rules… now if i could just figure out which one its using!
As far as bandwith:
p2p up - i assigned 256Kb realtime, 60% bandwidth
p2p down - 256Kb realtime, 25% bandwidth
ack up - 25% (standard)
ack down - 25% (standard)
queOthersUpH - 10%
queOthersDownH - 40%
queOthersUpL - 1% min
queOthersDownL - 1% minIts working MUCH much better now. I am able to skype with my wife and watch others surf the net!!! After reading that last post, i cant claim that i know exactly why it works like it does. I am still trying to learn though. My question is this though - what is the difference between queOthersH and L? most of the http traffic seemed to go through H, so that is why i traffic shaped it the way i did. But sometimes the traffic will go through queOthersL - whats the difference?
Thanks again for all your help and time. I dont take it for granted. ; :D
