PfSense and Shaping Facebook – The Definitive Guide.

[cyber7]

Hi Guys
After struggling for a HUGE amount of time on the problem of limiting the speed of facebook traffic, I finally got my head around this and wrote a guide to do exactly this, LIMIT FACEBOOK TRAFFIC.

Things you have to remember, this can not go into your LAN , it HAS to go into your WAN/FLOATING rules.  If you try to put it onto your LAN rules, you will not have a good time…  :P

You can read the full instruction here:
https://aubreykloppers.wordpress.com/2015/07/22/pfsense-and-shaping-facebook-the-definitive-guide/

Enjoy and if you get to Cape Town, please buy me a beer :)

as a last note: This is going to p >:( ss more than a couple of people off, so make sure you can take the punch  ;D

ps - I will be writing a proper guide to limiting groups of LAN users over the next day or so…

[KOM]

I think your alias list might be a little light:

http://bgp.he.net/search?search%5Bsearch%5D=facebook&commit=Search

[cyber7]

Hi KOM

The alias-list might be light, but it works :)

The full alias-list that one can use is:

ad4game-a.akamaihd.net
adaptv-a.akamaihd.net
adaptvcdn-a.akamaihd.net
amazonadsi-a.akamaihd.net
amznadsi-a.akamaihd.net
apiconstasurfinf-a.akamaihd.net
appnext-a.akamaihd.net
atdmt-a.akamaihd.net
av00vimeo-i.akamaihd.net
avvimeo-a.akamaihd.net
azmtcdn-a.akamaihd.net
bcsecure01-a.akamaihd.net
bws2midas-a.akamaihd.net
bzmtcdn-a.akamaihd.net
canvasdp-a.akamaihd.net
canvaspl-a.akamaihd.net
cdn2sitescout-a.akamaihd.net
cdncache-a.akamaihd.net
cdncache1-a.akamaihd.net
cdnrockyou-a.akamaihd.net
cdnstats-a.akamaihd.net
cloudinary-a.akamaihd.net
combowhosay-a.akamaihd.net
comedysec-a.akamaihd.net
content-a.akamaihd.net
contents-a.akamaihd.net
contextual-a.akamaihd.net
czmtcdn-a.akamaihd.net
distilleryimage0-a.akamaihd.net
distilleryimage1-a.akamaihd.net
distilleryimage10-a.akamaihd.net
distilleryimage11-a.akamaihd.net
distilleryimage2-a.akamaihd.net
distilleryimage3-a.akamaihd.net
distilleryimage4-a.akamaihd.net
distilleryimage5-a.akamaihd.net
distilleryimage6-a.akamaihd.net
distilleryimage7-a.akamaihd.net
distilleryimage8-a.akamaihd.net
distilleryimage9-a.akamaihd.net
ds-aksb-a.akamaihd.net
dzmtcdn-a.akamaihd.net
evernote-a.akamaihd.net
fashiononesec-a.akamaihd.net
fbcdn-creative-a.akamaihd.net
fbcdn-dragon-a.akamaihd.net
fbcdn-gtvideo-a-a.akamaihd.net
fbcdn-gtvideo-c-a.akamaihd.net
fbcdn-gtvideo-d-a.akamaihd.net
fbcdn-gtvideo-e-a.akamaihd.net
fbcdn-gtvideo-f-a.akamaihd.net
fbcdn-gtvideo-g-a.akamaihd.net
fbcdn-gtvideo-h-a.akamaihd.net
fbcdn-gtvideo-i-a.akamaihd.net
fbcdn-gtvideo-j-a.akamaihd.net
fbcdn-gtvideo-k-a.akamaihd.net
fbcdn-gtvideo-m-a.akamaihd.net
fbcdn-gtvideo-n-a.akamaihd.net
fbcdn-gtvideo-o-a.akamaihd.net
fbcdn-gtvideo-p-a.akamaihd.net
fbcdn-photos-a-a.akamaihd.net
fbcdn-photos-a.akamaihd.net
fbcdn-photos-b-a.akamaihd.net
fbcdn-photos-c-a.akamaihd.net
fbcdn-photos-d-a.akamaihd.net
fbcdn-photos-e-a.akamaihd.net
fbcdn-photos-f-a.akamaihd.net
fbcdn-photos-g-a.akamaihd.net
fbcdn-photos-h-a.akamaihd.net
fbcdn-profile-a.akamaihd.net
fbcdn-sphotos-a-a.akamaihd.net
fbcdn-sphotos-a.akamaihd.net
fbcdn-sphotos-b-a.akamaihd.net
fbcdn-sphotos-c-a.akamaihd.net
fbcdn-sphotos-d-a.akamaihd.net
fbcdn-sphotos-e-a.akamaihd.net
fbcdn-sphotos-f-a.akamaihd.net
fbcdn-sphotos-g-a.akamaihd.net
fbcdn-sphotos-h-a.akamaihd.net
fbcdn-static-b-a.akamaihd.net
fbcdn-video-a-a.akamaihd.net
fbcdn-video-a.akamaihd.net
fbcdn-video-b-a.akamaihd.net
fbcdn-video-c-a.akamaihd.net
fbcdn-video-d-a.akamaihd.net
fbcdn-video-e-a.akamaihd.net
fbcdn-video-f-a.akamaihd.net
fbcdn-video-g-a.akamaihd.net
fbcdn-video-h-a.akamaihd.net
fbcdn-video-i-a.akamaihd.net
fbcdn-video-j-a.akamaihd.net
fbcdn-video-k-a.akamaihd.net
fbcdn-video-l-a.akamaihd.net
fbcdn-video-m-a.akamaihd.net
fbcdn-video-n-a.akamaihd.net
fbcdn-video-o-a.akamaihd.net
fbcdn-video-p-a.akamaihd.net
fbcdn-vthumb-a.akamaihd.net
fbexternal-a.akamaihd.net
fbstatic-a.akamaihd.net
foxnewsplayer-a.akamaihd.net
fxdepo-a.akamaihd.net
gamegos-a.akamaihd.net
golfchannel-a.akamaihd.net
grvaol-a.akamaihd.net
hdapp1004-a.akamaihd.net
hdapp1006-a.akamaihd.net
hdapp1008-a.akamaihd.net
hdliveextra-a.akamaihd.net
hdsrc-a.akamaihd.net
hfys5200-a.akamaihd.net
hof-a.akamaihd.net
hrsecsynd-a.akamaihd.net
humblebundle-a.akamaihd.net
ic41c1c00-ds-aksb-a.akamaihd.net
igcdn-photos-a-a.akamaihd.net
igcdn-photos-b-a.akamaihd.net
igcdn-photos-c-a.akamaihd.net
igcdn-photos-d-a.akamaihd.net
igcdn-photos-e-a.akamaihd.net
igcdn-photos-f-a.akamaihd.net
igcdn-photos-g-a.akamaihd.net
igcdn-photos-h-a.akamaihd.net
igcdn-videos-b-0-a.akamaihd.net
igcdn-videos-b-10-a.akamaihd.net
igcdn-videos-d-9-a.akamaihd.net
igcdn-videos-g-7-a.akamaihd.net
igcdn-videos-h-12-a.akamaihd.net
inmagazinesec-a.akamaihd.net
inmobisdk-a.akamaihd.net
instagramimages-a.akamaihd.net
instagramstatic-a.akamaihd.net
kbdownload1-a.akamaihd.net
kbimages1-a.akamaihd.net
kbmerch1-a.akamaihd.net
kbstatic1-a.akamaihd.net
lfavatar-a.akamaihd.net
lfzor-a.akamaihd.net
mindjolt-a.akamaihd.net
mycbslocal-a.akamaihd.net
mycdn-a.akamaihd.net
myvegas-a.akamaihd.net
networkten-a.akamaihd.net
pdlvimeocdn-a.akamaihd.net
photorankmedia-a.akamaihd.net
photorankstatics-a.akamaihd.net
prezi-a.akamaihd.net
qsearch-a.akamaihd.net
rdio-a.akamaihd.net
rdio0-a.akamaihd.net
rdio1-a.akamaihd.net
rdio2-a.akamaihd.net
redge-a.akamaihd.net
rounds-a.akamaihd.net
sharecarepmd-a.akamaihd.net
shinezone-a.akamaihd.net
snappytv-a.akamaihd.net
splitsec-a.akamaihd.net
static6-a.akamaihd.net
sugarinc-a.akamaihd.net
tapjoycdn-a.akamaihd.net
tedcdnpa-a.akamaihd.net
tedcdnpi-a.akamaihd.net
tos-a.akamaihd.net
uppercutsec-a.akamaihd.net
ustvstaticcdn1-a.akamaihd.net
ustvstaticcdn2-a.akamaihd.net
vindicoasset-a.akamaihd.net
wwwigame-a.akamaihd.net
z1photorankmedia-a.akamaihd.net
z2photorankmedia-a.akamaihd.net
z3photorankmedia-a.akamaihd.net
zchan0-a.akamaihd.net
zephyrzoosk-a.akamaihd.net
zynga1-a.akamaihd.net
0-channel-proxy-04-frc3.facebook.com
0-channel-proxy-06-ash2.facebook.com
0-channel-proxy-06-frc1.facebook.com
0-channel-proxy-07-ash2.facebook.com
0-channel-proxy-13-prn1.facebook.com
0-edge-chat.facebook.com
0-p-04-frc3.channel.facebook.com
0-p-06-ash2.channel.facebook.com
0-p-06-frc1.channel.facebook.com
0-p-07-ash2.channel.facebook.com
0-p-13-prn1.channel.facebook.com
0-undefined.facebook.com
1-channel-proxy-04-frc3.facebook.com
1-channel-proxy-06-ash2.facebook.com
1-channel-proxy-06-frc1.facebook.com
1-channel-proxy-07-ash2.facebook.com
1-channel-proxy-13-prn1.facebook.com
1-edge-chat.facebook.com
1-p-04-frc3.channel.facebook.com
1-p-06-ash2.channel.facebook.com
1-p-06-frc1.channel.facebook.com
1-p-07-ash2.channel.facebook.com
1-p-13-prn1.channel.facebook.com
1-undefined.facebook.com
2-channel-proxy-04-frc3.facebook.com
2-channel-proxy-06-ash2.facebook.com
2-channel-proxy-06-frc1.facebook.com
2-channel-proxy-07-ash2.facebook.com
2-channel-proxy-13-prn1.facebook.com
2-edge-chat.facebook.com
2-p-04-frc3.channel.facebook.com
2-p-06-ash2.channel.facebook.com
2-p-06-frc1.channel.facebook.com
2-p-07-ash2.channel.facebook.com
2-p-13-prn1.channel.facebook.com
2-undefined.facebook.com
3-channel-proxy-04-frc3.facebook.com
3-channel-proxy-06-ash2.facebook.com
3-channel-proxy-06-frc1.facebook.com
3-channel-proxy-07-ash2.facebook.com
3-channel-proxy-13-prn1.facebook.com
3-edge-chat.facebook.com
3-p-04-frc3.channel.facebook.com
3-p-06-ash2.channel.facebook.com
3-p-06-frc1.channel.facebook.com
3-p-07-ash2.channel.facebook.com
3-p-13-prn1.channel.facebook.com
3-undefined.facebook.com
4-channel-proxy-04-frc3.facebook.com
4-channel-proxy-06-ash2.facebook.com
4-channel-proxy-06-frc1.facebook.com
4-channel-proxy-07-ash2.facebook.com
4-channel-proxy-13-prn1.facebook.com
4-edge-chat.facebook.com
4-p-04-frc3.channel.facebook.com
4-p-06-ash2.channel.facebook.com
4-p-06-frc1.channel.facebook.com
4-p-07-ash2.channel.facebook.com
4-p-13-prn1.channel.facebook.com
4-undefined.facebook.com
5-channel-proxy-04-frc3.facebook.com
5-channel-proxy-06-ash2.facebook.com
5-channel-proxy-06-frc1.facebook.com
5-channel-proxy-07-ash2.facebook.com
5-channel-proxy-13-prn1.facebook.com
5-edge-chat.facebook.com
5-p-04-frc3.channel.facebook.com
5-p-06-ash2.channel.facebook.com
5-p-06-frc1.channel.facebook.com
5-p-07-ash2.channel.facebook.com
5-p-13-prn1.channel.facebook.com
5-undefined.facebook.com
6-channel-proxy-04-frc3.facebook.com
6-channel-proxy-06-ash2.facebook.com
6-channel-proxy-06-frc1.facebook.com
6-channel-proxy-07-ash2.facebook.com
6-channel-proxy-13-prn1.facebook.com
6-edge-chat.facebook.com
6-p-04-frc3.channel.facebook.com
6-p-06-ash2.channel.facebook.com
6-p-06-frc1.channel.facebook.com
6-p-07-ash2.channel.facebook.com
6-p-13-prn1.channel.facebook.com
6-undefined.facebook.com
af-za.facebook.com
api-read.facebook.com
api.facebook.com
apps.facebook.com
b-api.facebook.com
b-graph.facebook.com
b-www.facebook.com
badge.facebook.com
channel-proxy-04-frc3.facebook.com
channel-proxy-06-ash2.facebook.com
channel-proxy-06-frc1.facebook.com
channel-proxy-07-ash2.facebook.com
channel-proxy-13-prn1.facebook.com
connect.facebook.com
da-dk.facebook.com
de-de.connect.facebook.com
developers.facebook.com
edge-chat.facebook.com
en-gb.facebook.com
error.facebook.com
es-la.facebook.com
et-ee.facebook.com
facebook.com
fi-fi.facebook.com
fr-fr.facebook.com
graph.facebook.com
hr-hr.facebook.com
l.facebook.com
lt-lt.facebook.com
m.facebook.com
m2.facebook.com
mbasic.facebook.com
mtouch.facebook.com
nl-nl.facebook.com
p-04-frc3.channel.facebook.com
p-06-ash2.channel.facebook.com
p-06-frc1.channel.facebook.com
p-07-ash2.channel.facebook.com
p-13-prn1.channel.facebook.com
pixel.facebook.com
pt-br.facebook.com
s-static.ak.facebook.com
secure.facebook.com
ssl.connect.facebook.com
static.ak.connect.facebook.com
static.ak.facebook.com
static.facebook.com
upload.facebook.com
vupload-edge.facebook.com
webdav.facebook.com
www.facebook.com

ps - I have had some people over the last couple of days (on my LAN) trying to BS me saying internet is slow, but busting them that it is only FB in front of their managers and seeing the expressions on their faces is WAY BETTER :)

ps2 - Love the Sisters of Mercy avatar!

[KOM]

but busting them that it is only FB in front of their managers and seeing the expressions on their faces is WAY BETTER

Perhaps.  I find my life is much easier if I work with my users and have them not hate me while still accomplishing my goals

ps2 - Love the Sisters of Mercy avatar!

???  You're talking about your own?  Mine is Rush's 'Starman'.

[cyber7]

@KOM:

but busting them that it is only FB in front of their managers and seeing the expressions on their faces is WAY BETTER

Perhaps.  I find my life is much easier if I work with my users and have them not hate me while still accomplishing my goals

ps2 - Love the Sisters of Mercy avatar!

???  You're talking about your own?  Mine is Rush's 'Starman'.

I totally agree, but having someone watching FB videos whilst trying to work on an off-site database can only be pleasurable up to a point.  When the 4Mbit/s (You can see it is very limited) runs out, drastic steps has to be taken to curb and pull the bandwidth back to business use.  If people then become sour with their experience, I point out it is due to their own habits and to detriment of NPO business.

Better to have a couple of haters than have a company on it's knees …

[dotdash]

Thanks for sharing your configuration. I appreciate having tested configurations to refer to as traffic shaping can be tricky.
There is a small typo in the article: the opening sentence-
'The first ting you have to understand'
And it looks like something might have been truncated at the end-
'If you now look on your Diagnostics/Limiter Info you will see 2 limiters with the 300.000 Kbit/s limiters and anyone using Facebook (even your firewall) will be left using only 300 Kbit/s and your entire'

[KOM]

I point out it is due to their own habits and to detriment of NPO business.

I prefer to have a stated policy so users know what's expected of them, and measures in place to enforce them.  My users are told that social media is off-limits, and I enforce that with URL filtering that only allows non-business stuff at lunchtime.  Everyone know the rules, they follow the rules and I don't have to be a BOFH.

[gratis.obake]

if I'm understanding it correctly, and implement this on my pfsense box. that any individual using facebook (per device) will only get the speed provided in the limiter field?

side question: if they click a vid in facebook, will it still retain the 300k limit? (assuming limit is 300k from limiter field?)

[cyber7]

You are 100% correct in your understanding.  Although 300Kbit/s is a bit slow, all traffic through this limiter will be affected :)

As a side-note: I have set this to 1500Kbit/s and it works like a charm!

cyber7-out

[Harvy66]

There are cases where traffic shaping won't help, but I assume it's not an issue because your targeting a lowly 300Kb/s.

You're taking about FB, which tends to use a lot of CDNs, akamai being one of them. I have a 1ms ping to my ISP's akamai CDN. This puts a lower limit on how slow TCP will go.

Current TCP implementations have a minimum window size of two segments. That is 3000 bytes for most cases. With a 1ms RTT, 3000 bytes will roughly be transferred every 1ms. That's 24Mb/s. That means TCP will refuse to transfer data slower than 24Mb/s per TCP connection, assuming the ping stays constant. A traffic policer drops data when it comes in too quickly, which means the data comes in, but the data will be getting dropped a lot.

As long as the limiter/policer has a large enough buffer, it will delay the packets but will cause buffer bloat to do so. If the buffer is too small, it will drop the packets, resulting in high packet loss.

[gratis.obake]

@cyber7
thanks for this, I'll implement this one in the near future as I also need this.

@Harvy66
honestly ^_^, I only got almost half of it I guess.
if we where to example the 300kb/s one (this tread is doing), then with the one you mentioned with the 23Mb/s (assuming its akamai/facebook which is near the isp), it will result to either "bufferbloat and/or dropped packets" due to it being capped/limited right?

sorry for this

[Harvy66]

Correct, but only for connections that have low latency relative to the bandwidth. This applies in my case because I have a 1Gb link, but it's rate limited to much less.

[Derelict]

I don't see any reason that can't go on LAN with more sanity.

You are masking on destination address in both In and Out.  That will mean your users will get a limiter pipe for each facebook destination IP address, not for each LAN host.

You can't match LAN hosts on WAN out floating rules because it's post-NAT (the source address will be the NAT address).

Name: FBupPRI
Bandwidth: 300 Kbit/s
Mask: Source Address

Name: FBdownPRI
Bandwidth: 300 Kbit/s
Mask: Destination address

Interface LAN
Action: Pass
Protocol: any
Source: LAN net
Destination Type: Single host or Alias
Destination Address: Facebook
Advanced Features:
In/Out: FBupPRI/FBdownPRI

Result: 300kbit/sec up/down for each LAN host for all connections to Facebook addresses.

[cyber7]

@Derelict:

I don't see any reason that can't go on LAN with more sanity.

You are masking on destination address in both In and Out.  That will mean your users will get a limiter pipe for each facebook destination IP address, not for each LAN host.

You can't match LAN hosts on WAN out floating rules because it's post-NAT (the source address will be the NAT address).

Name: FBupPRI
Bandwidth: 300 Kbit/s
Mask: Source Address

Name: FBdownPRI
Bandwidth: 300 Kbit/s
Mask: Destination address

Interface LAN
Action: Pass
Protocol: any
Source: LAN net
Destination Type: Single host or Alias
Destination Address: Facebook
Advanced Features:
In/Out: FBupPRI/FBdownPRI

Result: 300kbit/sec up/down for each LAN host for all connections to Facebook addresses.

Hi Derelict
Definitely going to try this!

Thanx
cyber7

[gratis.obake]

@Derelict:

Result: 300kbit/sec up/down for each LAN host for all connections to Facebook addresses.

some dumb question on this:
each will have 300kbit up/down for every computer on LAN?, lets say I have 3 computers with this implemented and all of them are doing facebook simultaneously, total is 900kb up/down right?

how about something 1,000kbit for them to share? like if only 1 user is accessing facebook, then he will have the whole 1,000kbits, but if others joins, then they'll share the 1,000kbits allocation

is this possible perhaps?

[Derelict]

@gratis.obake:

@Derelict:

Result: 300kbit/sec up/down for each LAN host for all connections to Facebook addresses.

some dumb question on this:
each will have 300kbit up/down for every computer on LAN?, lets say I have 3 computers with this implemented and all of them are doing facebook simultaneously, total is 900kb up/down right?

how about something 1,000kbit for them to share? like if only 1 user is accessing facebook, then he will have the whole 1,000kbits, but if others joins, then they'll share the 1,000kbits allocation

is this possible perhaps?

New top-level limiter:

Name: FBupPRIPool
Bandwidth: 1000 Kbit/s
Mask: None

While viewing FBupPRIPool click Add new queue

Name: FBupPRIByHost
Mask: Source address

New top-level limiter:

Name: FBdownPRIPool
Bandwidth: 1000 Kbit/s
Mask: None

While viewing FBdownPRIPool click Add new queue

Name: FBdownPRIByHost
Mask: Destination address

Interface LAN
Action: Pass
Protocol: any
Source: LAN net
Destination Type: Single host or Alias
Destination Address: Facebook
Advanced Features:
In/Out: FBupPRIByHost/FBdownPRIByHost

Result: 1000kbit/sec up/down Pool split among all LAN hosts for all connections to Facebook addresses. If only one host, it gets the full 1000kbit.

[gratis.obake]

thanks sir, I'll try this one

[strike101]

Thanks it works  ;D

btw… what if i want to exclude a single pc/ip from the rule ?

thanks again

[Derelict]

If it doesn't match the rule, or if it matches another rule above it it won't be put through the limiter.

So put a rule above it that matches only that IP address but doesn't set the limiter.

[cyber7]

@Derelict:

If it doesn't match the rule, or if it matches another rule above it it won't be put through the limiter.

So put a rule above it that matches only that IP address but doesn't set the limiter.

You can see my limiter works and works 100%  - I did, however make it 1MB/s because the experience at 300kb/s is just not on :)

Here you can see it in working (all the FB ip's and then my one single GW IP)