Netflix bypassing traffic limiters?
-
Hi. So I'm running pfSense 2.4.2p1 on an old desktop (circa 2007, Intel Q35 platform), and aside from a major issue with a previous install, things have been smooth sailing for the most part. My WAN link is, unfortunately only an ADSL2+ one (~4.6 Mbps down, ~0.9 Mbps up), which means I have to come up with what I feel are hacky solutions to minimal-lag gaming/VOIP while the link is under heavy contention.
So what I settled for in the end was firewall rules and limiters. I created high, normal, and low priority limiters, each with different bandwidth allocations as needed. High priority being stuff like gaming and VoIP, normal being usual HTTP/Email, and low priority being cloud syncing (like iCloud).
In most situations, this works extremely well. There is a global limiter that limits things at a hard 4 Mbps down and 0.8 Mbps up. Then after that I have the other limiters than run in scheduled firewall rules from 9 AM to 23:59 PM. The high priority limiter has 1 Mbps reserved for downstream, with 0.4 Mbps for upstream. Normal has 3 Mbps reserved for downstream, with another 0.4 Mbps reserved for upstream. As for shaping cloud stuff, I went full savage and shaped that to 0.05 Mbps during the peak schedule.
Unfortunately, there is still one issue I can't work my head around. For whatever reason, Netflix traffic seems to completely bypass the limiters in place. I get hit by the full 4.6 Mbps and so my gaming and VoIP experience suffers still. From what I've seen in the traffic graphs, Netflix is multi-threaded in its streams (there are multiple IPs sending the traffic to clients on my LAN). Still, I'm not sure why other multi-threaded downloads are shaped correctly, while Netflix isn't. It would be great if someone could give me insight on this. I've searched on Google but haven't had much luck in doing so.
-
You need to understand how traffic shaping/limiting works:
There is no real way for pfSense to tell a remote server at what speed it should sent data. Instead once the transfer hits the limit pfSense will start dropping packets, in the hope that the remote server will notice it and slow down. As this is normal behaviour it works most of the time. But this entirely depends on the cooperation of the remote server, if that doesn't care for dropped packets it will simply continue to sent as much data as fast as possible.
So check that the data from Netflix goes through the right limiter. If it is and is and it's still pushing over the limits there is not much else you can do with pfSense. In that case the traffic would have to be limited on the other side of the DSL connection to prevent it from choking, and that means your provider has to configure a limiter on their side.
-
Ok. So I did verify that Netflix traffic was groung through the right limiter. And for reference I have the firewall rules setup to "queue" traffic by destination port or IP. I'm not sure of a better way than that, as I'm aware pfSense can't function as a layer 7 firewall. But yeah I initially thought that all servers would respect the dropped packets and tone down on their transfers until minimal packet drops.
Would I be better off with something like fq_codel, or is my WAN link just too slow to see any benefit?
-
I'm not saying this is the reason why you're having issue, just saying it's a common reason.
Depends on your latency. Some people are connected to Netflix via an ISP level CDN. While ADSL may have low bandwidth, like 4Mb, your ping may be ultra low relative to your bandwidth. TCP does respond to loss by reducing the send window, but this window has a minimum size of 2 segments. TCP does not manage bandwidth, it manages segments in flight. How this related to bandwidth is Bandwidth=(Window Size)/(RTT delay). TCP can manipulate the bandwidth indirectly by manipulating the window size, but the RTT is defined by your route. If you have a really low RTT, this makes TCP have a floor on bandwidth that may be too large for your pipe.
-
I suffer for years on slow DSL but ISPs have up the game and are offering more speed for same, even less$ so stay abreast of what they are currently offering. Comcast even allowing Internet-only, they used to charge big time for this and forcing you to buy double-play, triple-play but no longer.
