PfSense and Shaping Facebook – The Definitive Guide.
-
Hi Guys
After struggling for a HUGE amount of time on the problem of limiting the speed of facebook traffic, I finally got my head around this and wrote a guide to do exactly this, LIMIT FACEBOOK TRAFFIC.Things you have to remember, this can not go into your LAN , it HAS to go into your WAN/FLOATING rules. If you try to put it onto your LAN rules, you will not have a good time… :P
You can read the full instruction here:
https://aubreykloppers.wordpress.com/2015/07/22/pfsense-and-shaping-facebook-the-definitive-guide/Enjoy and if you get to Cape Town, please buy me a beer :)
as a last note: This is going to p >:( ss more than a couple of people off, so make sure you can take the punch ;D
ps - I will be writing a proper guide to limiting groups of LAN users over the next day or so…
-
I think your alias list might be a little light:
http://bgp.he.net/search?search%5Bsearch%5D=facebook&commit=Search
-
Hi KOM
The alias-list might be light, but it works :)
The full alias-list that one can use is:
ad4game-a.akamaihd.net adaptv-a.akamaihd.net adaptvcdn-a.akamaihd.net amazonadsi-a.akamaihd.net amznadsi-a.akamaihd.net apiconstasurfinf-a.akamaihd.net appnext-a.akamaihd.net atdmt-a.akamaihd.net av00vimeo-i.akamaihd.net avvimeo-a.akamaihd.net azmtcdn-a.akamaihd.net bcsecure01-a.akamaihd.net bws2midas-a.akamaihd.net bzmtcdn-a.akamaihd.net canvasdp-a.akamaihd.net canvaspl-a.akamaihd.net cdn2sitescout-a.akamaihd.net cdncache-a.akamaihd.net cdncache1-a.akamaihd.net cdnrockyou-a.akamaihd.net cdnstats-a.akamaihd.net cloudinary-a.akamaihd.net combowhosay-a.akamaihd.net comedysec-a.akamaihd.net content-a.akamaihd.net contents-a.akamaihd.net contextual-a.akamaihd.net czmtcdn-a.akamaihd.net distilleryimage0-a.akamaihd.net distilleryimage1-a.akamaihd.net distilleryimage10-a.akamaihd.net distilleryimage11-a.akamaihd.net distilleryimage2-a.akamaihd.net distilleryimage3-a.akamaihd.net distilleryimage4-a.akamaihd.net distilleryimage5-a.akamaihd.net distilleryimage6-a.akamaihd.net distilleryimage7-a.akamaihd.net distilleryimage8-a.akamaihd.net distilleryimage9-a.akamaihd.net ds-aksb-a.akamaihd.net dzmtcdn-a.akamaihd.net evernote-a.akamaihd.net fashiononesec-a.akamaihd.net fbcdn-creative-a.akamaihd.net fbcdn-dragon-a.akamaihd.net fbcdn-gtvideo-a-a.akamaihd.net fbcdn-gtvideo-c-a.akamaihd.net fbcdn-gtvideo-d-a.akamaihd.net fbcdn-gtvideo-e-a.akamaihd.net fbcdn-gtvideo-f-a.akamaihd.net fbcdn-gtvideo-g-a.akamaihd.net fbcdn-gtvideo-h-a.akamaihd.net fbcdn-gtvideo-i-a.akamaihd.net fbcdn-gtvideo-j-a.akamaihd.net fbcdn-gtvideo-k-a.akamaihd.net fbcdn-gtvideo-m-a.akamaihd.net fbcdn-gtvideo-n-a.akamaihd.net fbcdn-gtvideo-o-a.akamaihd.net fbcdn-gtvideo-p-a.akamaihd.net fbcdn-photos-a-a.akamaihd.net fbcdn-photos-a.akamaihd.net fbcdn-photos-b-a.akamaihd.net fbcdn-photos-c-a.akamaihd.net fbcdn-photos-d-a.akamaihd.net fbcdn-photos-e-a.akamaihd.net fbcdn-photos-f-a.akamaihd.net fbcdn-photos-g-a.akamaihd.net fbcdn-photos-h-a.akamaihd.net fbcdn-profile-a.akamaihd.net fbcdn-sphotos-a-a.akamaihd.net fbcdn-sphotos-a.akamaihd.net fbcdn-sphotos-b-a.akamaihd.net fbcdn-sphotos-c-a.akamaihd.net fbcdn-sphotos-d-a.akamaihd.net fbcdn-sphotos-e-a.akamaihd.net fbcdn-sphotos-f-a.akamaihd.net fbcdn-sphotos-g-a.akamaihd.net fbcdn-sphotos-h-a.akamaihd.net fbcdn-static-b-a.akamaihd.net fbcdn-video-a-a.akamaihd.net fbcdn-video-a.akamaihd.net fbcdn-video-b-a.akamaihd.net fbcdn-video-c-a.akamaihd.net fbcdn-video-d-a.akamaihd.net fbcdn-video-e-a.akamaihd.net fbcdn-video-f-a.akamaihd.net fbcdn-video-g-a.akamaihd.net fbcdn-video-h-a.akamaihd.net fbcdn-video-i-a.akamaihd.net fbcdn-video-j-a.akamaihd.net fbcdn-video-k-a.akamaihd.net fbcdn-video-l-a.akamaihd.net fbcdn-video-m-a.akamaihd.net fbcdn-video-n-a.akamaihd.net fbcdn-video-o-a.akamaihd.net fbcdn-video-p-a.akamaihd.net fbcdn-vthumb-a.akamaihd.net fbexternal-a.akamaihd.net fbstatic-a.akamaihd.net foxnewsplayer-a.akamaihd.net fxdepo-a.akamaihd.net gamegos-a.akamaihd.net golfchannel-a.akamaihd.net grvaol-a.akamaihd.net hdapp1004-a.akamaihd.net hdapp1006-a.akamaihd.net hdapp1008-a.akamaihd.net hdliveextra-a.akamaihd.net hdsrc-a.akamaihd.net hfys5200-a.akamaihd.net hof-a.akamaihd.net hrsecsynd-a.akamaihd.net humblebundle-a.akamaihd.net ic41c1c00-ds-aksb-a.akamaihd.net igcdn-photos-a-a.akamaihd.net igcdn-photos-b-a.akamaihd.net igcdn-photos-c-a.akamaihd.net igcdn-photos-d-a.akamaihd.net igcdn-photos-e-a.akamaihd.net igcdn-photos-f-a.akamaihd.net igcdn-photos-g-a.akamaihd.net igcdn-photos-h-a.akamaihd.net igcdn-videos-b-0-a.akamaihd.net igcdn-videos-b-10-a.akamaihd.net igcdn-videos-d-9-a.akamaihd.net igcdn-videos-g-7-a.akamaihd.net igcdn-videos-h-12-a.akamaihd.net inmagazinesec-a.akamaihd.net inmobisdk-a.akamaihd.net instagramimages-a.akamaihd.net instagramstatic-a.akamaihd.net kbdownload1-a.akamaihd.net kbimages1-a.akamaihd.net kbmerch1-a.akamaihd.net kbstatic1-a.akamaihd.net lfavatar-a.akamaihd.net lfzor-a.akamaihd.net mindjolt-a.akamaihd.net mycbslocal-a.akamaihd.net mycdn-a.akamaihd.net myvegas-a.akamaihd.net networkten-a.akamaihd.net pdlvimeocdn-a.akamaihd.net photorankmedia-a.akamaihd.net photorankstatics-a.akamaihd.net prezi-a.akamaihd.net qsearch-a.akamaihd.net rdio-a.akamaihd.net rdio0-a.akamaihd.net rdio1-a.akamaihd.net rdio2-a.akamaihd.net redge-a.akamaihd.net rounds-a.akamaihd.net sharecarepmd-a.akamaihd.net shinezone-a.akamaihd.net snappytv-a.akamaihd.net splitsec-a.akamaihd.net static6-a.akamaihd.net sugarinc-a.akamaihd.net tapjoycdn-a.akamaihd.net tedcdnpa-a.akamaihd.net tedcdnpi-a.akamaihd.net tos-a.akamaihd.net uppercutsec-a.akamaihd.net ustvstaticcdn1-a.akamaihd.net ustvstaticcdn2-a.akamaihd.net vindicoasset-a.akamaihd.net wwwigame-a.akamaihd.net z1photorankmedia-a.akamaihd.net z2photorankmedia-a.akamaihd.net z3photorankmedia-a.akamaihd.net zchan0-a.akamaihd.net zephyrzoosk-a.akamaihd.net zynga1-a.akamaihd.net 0-channel-proxy-04-frc3.facebook.com 0-channel-proxy-06-ash2.facebook.com 0-channel-proxy-06-frc1.facebook.com 0-channel-proxy-07-ash2.facebook.com 0-channel-proxy-13-prn1.facebook.com 0-edge-chat.facebook.com 0-p-04-frc3.channel.facebook.com 0-p-06-ash2.channel.facebook.com 0-p-06-frc1.channel.facebook.com 0-p-07-ash2.channel.facebook.com 0-p-13-prn1.channel.facebook.com 0-undefined.facebook.com 1-channel-proxy-04-frc3.facebook.com 1-channel-proxy-06-ash2.facebook.com 1-channel-proxy-06-frc1.facebook.com 1-channel-proxy-07-ash2.facebook.com 1-channel-proxy-13-prn1.facebook.com 1-edge-chat.facebook.com 1-p-04-frc3.channel.facebook.com 1-p-06-ash2.channel.facebook.com 1-p-06-frc1.channel.facebook.com 1-p-07-ash2.channel.facebook.com 1-p-13-prn1.channel.facebook.com 1-undefined.facebook.com 2-channel-proxy-04-frc3.facebook.com 2-channel-proxy-06-ash2.facebook.com 2-channel-proxy-06-frc1.facebook.com 2-channel-proxy-07-ash2.facebook.com 2-channel-proxy-13-prn1.facebook.com 2-edge-chat.facebook.com 2-p-04-frc3.channel.facebook.com 2-p-06-ash2.channel.facebook.com 2-p-06-frc1.channel.facebook.com 2-p-07-ash2.channel.facebook.com 2-p-13-prn1.channel.facebook.com 2-undefined.facebook.com 3-channel-proxy-04-frc3.facebook.com 3-channel-proxy-06-ash2.facebook.com 3-channel-proxy-06-frc1.facebook.com 3-channel-proxy-07-ash2.facebook.com 3-channel-proxy-13-prn1.facebook.com 3-edge-chat.facebook.com 3-p-04-frc3.channel.facebook.com 3-p-06-ash2.channel.facebook.com 3-p-06-frc1.channel.facebook.com 3-p-07-ash2.channel.facebook.com 3-p-13-prn1.channel.facebook.com 3-undefined.facebook.com 4-channel-proxy-04-frc3.facebook.com 4-channel-proxy-06-ash2.facebook.com 4-channel-proxy-06-frc1.facebook.com 4-channel-proxy-07-ash2.facebook.com 4-channel-proxy-13-prn1.facebook.com 4-edge-chat.facebook.com 4-p-04-frc3.channel.facebook.com 4-p-06-ash2.channel.facebook.com 4-p-06-frc1.channel.facebook.com 4-p-07-ash2.channel.facebook.com 4-p-13-prn1.channel.facebook.com 4-undefined.facebook.com 5-channel-proxy-04-frc3.facebook.com 5-channel-proxy-06-ash2.facebook.com 5-channel-proxy-06-frc1.facebook.com 5-channel-proxy-07-ash2.facebook.com 5-channel-proxy-13-prn1.facebook.com 5-edge-chat.facebook.com 5-p-04-frc3.channel.facebook.com 5-p-06-ash2.channel.facebook.com 5-p-06-frc1.channel.facebook.com 5-p-07-ash2.channel.facebook.com 5-p-13-prn1.channel.facebook.com 5-undefined.facebook.com 6-channel-proxy-04-frc3.facebook.com 6-channel-proxy-06-ash2.facebook.com 6-channel-proxy-06-frc1.facebook.com 6-channel-proxy-07-ash2.facebook.com 6-channel-proxy-13-prn1.facebook.com 6-edge-chat.facebook.com 6-p-04-frc3.channel.facebook.com 6-p-06-ash2.channel.facebook.com 6-p-06-frc1.channel.facebook.com 6-p-07-ash2.channel.facebook.com 6-p-13-prn1.channel.facebook.com 6-undefined.facebook.com af-za.facebook.com api-read.facebook.com api.facebook.com apps.facebook.com b-api.facebook.com b-graph.facebook.com b-www.facebook.com badge.facebook.com channel-proxy-04-frc3.facebook.com channel-proxy-06-ash2.facebook.com channel-proxy-06-frc1.facebook.com channel-proxy-07-ash2.facebook.com channel-proxy-13-prn1.facebook.com connect.facebook.com da-dk.facebook.com de-de.connect.facebook.com developers.facebook.com edge-chat.facebook.com en-gb.facebook.com error.facebook.com es-la.facebook.com et-ee.facebook.com facebook.com fi-fi.facebook.com fr-fr.facebook.com graph.facebook.com hr-hr.facebook.com l.facebook.com lt-lt.facebook.com m.facebook.com m2.facebook.com mbasic.facebook.com mtouch.facebook.com nl-nl.facebook.com p-04-frc3.channel.facebook.com p-06-ash2.channel.facebook.com p-06-frc1.channel.facebook.com p-07-ash2.channel.facebook.com p-13-prn1.channel.facebook.com pixel.facebook.com pt-br.facebook.com s-static.ak.facebook.com secure.facebook.com ssl.connect.facebook.com static.ak.connect.facebook.com static.ak.facebook.com static.facebook.com upload.facebook.com vupload-edge.facebook.com webdav.facebook.com www.facebook.comps - I have had some people over the last couple of days (on my LAN) trying to BS me saying internet is slow, but busting them that it is only FB in front of their managers and seeing the expressions on their faces is WAY BETTER :)
ps2 - Love the Sisters of Mercy avatar!
-
but busting them that it is only FB in front of their managers and seeing the expressions on their faces is WAY BETTER
Perhaps. I find my life is much easier if I work with my users and have them not hate me while still accomplishing my goals
ps2 - Love the Sisters of Mercy avatar!
??? You're talking about your own? Mine is Rush's 'Starman'.
-
@KOM:
but busting them that it is only FB in front of their managers and seeing the expressions on their faces is WAY BETTER
Perhaps. I find my life is much easier if I work with my users and have them not hate me while still accomplishing my goals
ps2 - Love the Sisters of Mercy avatar!
??? You're talking about your own? Mine is Rush's 'Starman'.
I totally agree, but having someone watching FB videos whilst trying to work on an off-site database can only be pleasurable up to a point. When the 4Mbit/s (You can see it is very limited) runs out, drastic steps has to be taken to curb and pull the bandwidth back to business use. If people then become sour with their experience, I point out it is due to their own habits and to detriment of NPO business.
Better to have a couple of haters than have a company on it's knees …
-
Thanks for sharing your configuration. I appreciate having tested configurations to refer to as traffic shaping can be tricky.
There is a small typo in the article: the opening sentence-
'The first ting you have to understand'
And it looks like something might have been truncated at the end-
'If you now look on your Diagnostics/Limiter Info you will see 2 limiters with the 300.000 Kbit/s limiters and anyone using Facebook (even your firewall) will be left using only 300 Kbit/s and your entire' -
I point out it is due to their own habits and to detriment of NPO business.
I prefer to have a stated policy so users know what's expected of them, and measures in place to enforce them. My users are told that social media is off-limits, and I enforce that with URL filtering that only allows non-business stuff at lunchtime. Everyone know the rules, they follow the rules and I don't have to be a BOFH.
-
if I'm understanding it correctly, and implement this on my pfsense box. that any individual using facebook (per device) will only get the speed provided in the limiter field?
side question: if they click a vid in facebook, will it still retain the 300k limit? (assuming limit is 300k from limiter field?)
-
You are 100% correct in your understanding. Although 300Kbit/s is a bit slow, all traffic through this limiter will be affected :)
As a side-note: I have set this to 1500Kbit/s and it works like a charm!
cyber7-out
-
There are cases where traffic shaping won't help, but I assume it's not an issue because your targeting a lowly 300Kb/s.
You're taking about FB, which tends to use a lot of CDNs, akamai being one of them. I have a 1ms ping to my ISP's akamai CDN. This puts a lower limit on how slow TCP will go.
Current TCP implementations have a minimum window size of two segments. That is 3000 bytes for most cases. With a 1ms RTT, 3000 bytes will roughly be transferred every 1ms. That's 24Mb/s. That means TCP will refuse to transfer data slower than 24Mb/s per TCP connection, assuming the ping stays constant. A traffic policer drops data when it comes in too quickly, which means the data comes in, but the data will be getting dropped a lot.
As long as the limiter/policer has a large enough buffer, it will delay the packets but will cause buffer bloat to do so. If the buffer is too small, it will drop the packets, resulting in high packet loss.
-
@cyber7
thanks for this, I'll implement this one in the near future as I also need this.@Harvy66
honestly ^_^, I only got almost half of it I guess.
if we where to example the 300kb/s one (this tread is doing), then with the one you mentioned with the 23Mb/s (assuming its akamai/facebook which is near the isp), it will result to either "bufferbloat and/or dropped packets" due to it being capped/limited right?sorry for this
-
Correct, but only for connections that have low latency relative to the bandwidth. This applies in my case because I have a 1Gb link, but it's rate limited to much less.
-
I don't see any reason that can't go on LAN with more sanity.
You are masking on destination address in both In and Out. That will mean your users will get a limiter pipe for each facebook destination IP address, not for each LAN host.
You can't match LAN hosts on WAN out floating rules because it's post-NAT (the source address will be the NAT address).
Name: FBupPRI
Bandwidth: 300 Kbit/s
Mask: Source AddressName: FBdownPRI
Bandwidth: 300 Kbit/s
Mask: Destination addressInterface LAN
Action: Pass
Protocol: any
Source: LAN net
Destination Type: Single host or Alias
Destination Address: Facebook
Advanced Features:
In/Out: FBupPRI/FBdownPRIResult: 300kbit/sec up/down for each LAN host for all connections to Facebook addresses.
-
I don't see any reason that can't go on LAN with more sanity.
You are masking on destination address in both In and Out. That will mean your users will get a limiter pipe for each facebook destination IP address, not for each LAN host.
You can't match LAN hosts on WAN out floating rules because it's post-NAT (the source address will be the NAT address).
Name: FBupPRI
Bandwidth: 300 Kbit/s
Mask: Source AddressName: FBdownPRI
Bandwidth: 300 Kbit/s
Mask: Destination addressInterface LAN
Action: Pass
Protocol: any
Source: LAN net
Destination Type: Single host or Alias
Destination Address: Facebook
Advanced Features:
In/Out: FBupPRI/FBdownPRIResult: 300kbit/sec up/down for each LAN host for all connections to Facebook addresses.
Hi Derelict
Definitely going to try this!Thanx
cyber7 -
Result: 300kbit/sec up/down for each LAN host for all connections to Facebook addresses.
some dumb question on this:
each will have 300kbit up/down for every computer on LAN?, lets say I have 3 computers with this implemented and all of them are doing facebook simultaneously, total is 900kb up/down right?how about something 1,000kbit for them to share? like if only 1 user is accessing facebook, then he will have the whole 1,000kbits, but if others joins, then they'll share the 1,000kbits allocation
is this possible perhaps?
-
Result: 300kbit/sec up/down for each LAN host for all connections to Facebook addresses.
some dumb question on this:
each will have 300kbit up/down for every computer on LAN?, lets say I have 3 computers with this implemented and all of them are doing facebook simultaneously, total is 900kb up/down right?how about something 1,000kbit for them to share? like if only 1 user is accessing facebook, then he will have the whole 1,000kbits, but if others joins, then they'll share the 1,000kbits allocation
is this possible perhaps?
New top-level limiter:
Name: FBupPRIPool
Bandwidth: 1000 Kbit/s
Mask: NoneWhile viewing FBupPRIPool click Add new queue
Name: FBupPRIByHost
Mask: Source addressNew top-level limiter:
Name: FBdownPRIPool
Bandwidth: 1000 Kbit/s
Mask: NoneWhile viewing FBdownPRIPool click Add new queue
Name: FBdownPRIByHost
Mask: Destination addressInterface LAN
Action: Pass
Protocol: any
Source: LAN net
Destination Type: Single host or Alias
Destination Address: Facebook
Advanced Features:
In/Out: FBupPRIByHost/FBdownPRIByHostResult: 1000kbit/sec up/down Pool split among all LAN hosts for all connections to Facebook addresses. If only one host, it gets the full 1000kbit.
-
thanks sir, I'll try this one
-
Thanks it works ;D
btw… what if i want to exclude a single pc/ip from the rule ?
thanks again
-
If it doesn't match the rule, or if it matches another rule above it it won't be put through the limiter.
So put a rule above it that matches only that IP address but doesn't set the limiter.
-
If it doesn't match the rule, or if it matches another rule above it it won't be put through the limiter.
So put a rule above it that matches only that IP address but doesn't set the limiter.
You can see my limiter works and works 100% - I did, however make it 1MB/s because the experience at 300kb/s is just not on :)
Here you can see it in working (all the FB ip's and then my one single GW IP)
