PfSense and Shaping Facebook – The Definitive Guide.
-
There are cases where traffic shaping won't help, but I assume it's not an issue because your targeting a lowly 300Kb/s.
You're taking about FB, which tends to use a lot of CDNs, akamai being one of them. I have a 1ms ping to my ISP's akamai CDN. This puts a lower limit on how slow TCP will go.
Current TCP implementations have a minimum window size of two segments. That is 3000 bytes for most cases. With a 1ms RTT, 3000 bytes will roughly be transferred every 1ms. That's 24Mb/s. That means TCP will refuse to transfer data slower than 24Mb/s per TCP connection, assuming the ping stays constant. A traffic policer drops data when it comes in too quickly, which means the data comes in, but the data will be getting dropped a lot.
As long as the limiter/policer has a large enough buffer, it will delay the packets but will cause buffer bloat to do so. If the buffer is too small, it will drop the packets, resulting in high packet loss.
-
@cyber7
thanks for this, I'll implement this one in the near future as I also need this.@Harvy66
honestly ^_^, I only got almost half of it I guess.
if we where to example the 300kb/s one (this tread is doing), then with the one you mentioned with the 23Mb/s (assuming its akamai/facebook which is near the isp), it will result to either "bufferbloat and/or dropped packets" due to it being capped/limited right?sorry for this
-
Correct, but only for connections that have low latency relative to the bandwidth. This applies in my case because I have a 1Gb link, but it's rate limited to much less.
-
I don't see any reason that can't go on LAN with more sanity.
You are masking on destination address in both In and Out. That will mean your users will get a limiter pipe for each facebook destination IP address, not for each LAN host.
You can't match LAN hosts on WAN out floating rules because it's post-NAT (the source address will be the NAT address).
Name: FBupPRI
Bandwidth: 300 Kbit/s
Mask: Source AddressName: FBdownPRI
Bandwidth: 300 Kbit/s
Mask: Destination addressInterface LAN
Action: Pass
Protocol: any
Source: LAN net
Destination Type: Single host or Alias
Destination Address: Facebook
Advanced Features:
In/Out: FBupPRI/FBdownPRIResult: 300kbit/sec up/down for each LAN host for all connections to Facebook addresses.
-
I don't see any reason that can't go on LAN with more sanity.
You are masking on destination address in both In and Out. That will mean your users will get a limiter pipe for each facebook destination IP address, not for each LAN host.
You can't match LAN hosts on WAN out floating rules because it's post-NAT (the source address will be the NAT address).
Name: FBupPRI
Bandwidth: 300 Kbit/s
Mask: Source AddressName: FBdownPRI
Bandwidth: 300 Kbit/s
Mask: Destination addressInterface LAN
Action: Pass
Protocol: any
Source: LAN net
Destination Type: Single host or Alias
Destination Address: Facebook
Advanced Features:
In/Out: FBupPRI/FBdownPRIResult: 300kbit/sec up/down for each LAN host for all connections to Facebook addresses.
Hi Derelict
Definitely going to try this!Thanx
cyber7 -
Result: 300kbit/sec up/down for each LAN host for all connections to Facebook addresses.
some dumb question on this:
each will have 300kbit up/down for every computer on LAN?, lets say I have 3 computers with this implemented and all of them are doing facebook simultaneously, total is 900kb up/down right?how about something 1,000kbit for them to share? like if only 1 user is accessing facebook, then he will have the whole 1,000kbits, but if others joins, then they'll share the 1,000kbits allocation
is this possible perhaps?
-
Result: 300kbit/sec up/down for each LAN host for all connections to Facebook addresses.
some dumb question on this:
each will have 300kbit up/down for every computer on LAN?, lets say I have 3 computers with this implemented and all of them are doing facebook simultaneously, total is 900kb up/down right?how about something 1,000kbit for them to share? like if only 1 user is accessing facebook, then he will have the whole 1,000kbits, but if others joins, then they'll share the 1,000kbits allocation
is this possible perhaps?
New top-level limiter:
Name: FBupPRIPool
Bandwidth: 1000 Kbit/s
Mask: NoneWhile viewing FBupPRIPool click Add new queue
Name: FBupPRIByHost
Mask: Source addressNew top-level limiter:
Name: FBdownPRIPool
Bandwidth: 1000 Kbit/s
Mask: NoneWhile viewing FBdownPRIPool click Add new queue
Name: FBdownPRIByHost
Mask: Destination addressInterface LAN
Action: Pass
Protocol: any
Source: LAN net
Destination Type: Single host or Alias
Destination Address: Facebook
Advanced Features:
In/Out: FBupPRIByHost/FBdownPRIByHostResult: 1000kbit/sec up/down Pool split among all LAN hosts for all connections to Facebook addresses. If only one host, it gets the full 1000kbit.
-
thanks sir, I'll try this one
-
Thanks it works ;D
btw… what if i want to exclude a single pc/ip from the rule ?
thanks again
-
If it doesn't match the rule, or if it matches another rule above it it won't be put through the limiter.
So put a rule above it that matches only that IP address but doesn't set the limiter.
-
If it doesn't match the rule, or if it matches another rule above it it won't be put through the limiter.
So put a rule above it that matches only that IP address but doesn't set the limiter.
You can see my limiter works and works 100% - I did, however make it 1MB/s because the experience at 300kb/s is just not on :)
Here you can see it in working (all the FB ip's and then my one single GW IP)
-
Your point?
-
Your point?
Did you read the entire topic? My point being the original limiting works 100% and does not create multiple 1MB pipes, but a single pipe. ALL FB traffic goes through the pipe and the 1MB pipe gets shared by all the FB ip's.
YOUR point? ;)
cyber7
-
Except it doesn't. If what you're doing works for you, good on you.
It goes through a single pipe because it is post-NAT on WAN out, meaning a single source address, meaning a single pipe.
You are missing the ability for the limiter to try to share the available pipe among LAN users (the users you should care about) by using the child limiters.
But, again, if what you're doing works for you, have at it.
The user I was responding to asked how to exclude a single source IP.
Tell me how you are going to do that post-NAT on WAN out?
-
Hi Derelict
Thanks for the extensive explanation! Please could I pick your brain a bit? (It will also help other users to understand when reading the topic)Are you saying that the big difference between my original writing and yours is that with yours you can manage the LAN IP's you want to limit, but with mine, you do it for the entire LAN?
I suppose if it is true, it is actually ok in my environment where I want to limit ALL FB traffic, not just for some users… BUT, the application of a 'child' limiter (in your example) has such potential for other technologies running away with your bandwidth. For example, Dropbox and any other "clouded" services.
My other headache is YOUTUBE (googlevideo) and limiting that traffic... I found a solution using squid, but that is beyond this subject matter.
kind regards
cyber7 (aka Aubrey Kloppers) -
Hi Derelict
Thanks for the extensive explanation! Please could I pick your brain a bit? (It will also help other users to understand when reading the topic)Are you saying that the big difference between my original writing and yours is that with yours you can manage the LAN IP's you want to limit, but with mine, you do it for the entire LAN?
It all depends on what your goals are. Post-NAT WAN out rules cannot see what the source IP is. That is quite a limiting factor in most cases.
I suppose if it is true, it is actually ok in my environment where I want to limit ALL FB traffic, not just for some users… BUT, the application of a 'child' limiter (in your example) has such potential for other technologies running away with your bandwidth. For example, Dropbox and any other "clouded" services.
Your stated goal is to limit facebook. The hardest part about that is identifying facebook traffic. Your rules won't do anything to limit dropbox either, since it's all on destination Facebook.
Limiters and child limiters work. The outlier is usually bittorrent. And that is usually because people put a WAN pass rule for their torrent port and don't set the limiter there too.
My other headache is YOUTUBE (googlevideo) and limiting that traffic… I found a solution using squid, but that is beyond this subject matter.
The hard part is identifying the traffic. Limiting identified traffic is pretty easy. I think most people who go down this rabbit hole are overthinking things. (Facebook bad, google, ok, googlevideo bad, cnn ok). Fuck it. Just limit/shape them all and make the internet work.
-
The hard part is identifying the traffic. Limiting identified traffic is pretty easy. I think most people who go down this rabbit hole are overthinking things. (Facebook bad, google, ok, googlevideo bad, cnn ok). Fuck it. Just limit/shape them all and make the internet work.
HAHAHA! I like your attitude! I am starting to really think in this direction as well! I have set up limiters (1/2/3Mb/s). It works, but after I implemented your solution, I am looking at making this more "smove" :)
cyber7
-
The hard part is identifying the traffic. Limiting identified traffic is pretty easy. I think most people who go down this rabbit hole are overthinking things. (Facebook bad, google, ok, googlevideo bad, cnn ok). Fuck it. Just limit/shape them all and make the internet work.
HAHAHA! I like your attitude! I am starting to really think in this direction as well! I have set up limiters (1/2/3Mb/s). It works, but after I implemented your solution, I am looking at making this more "smove" :)
cyber7
And you, Derelict, my dear sir ARE A GENIUS! Re-Wrote all my Limiters with your specs and WOW, soooo smove!
cyber7-out
