Pfsense plus hurricane electric breaks netflix IPV6 - proxy error

[Gertjan]

Hi,

The problem boils down to:
@kejianshi:

… Just the people using tunnel brokers.

Netflix made an international statement about that.

The entire issue is : who is on the list (that Netflix manages) that they consider as a proxy ?
You proved, as many did, that he.net is … (the free IPv6 tunnel service).

[kejianshi]

Yeah - I think that netflix is going too far out of its way to attempt to enforce vpn and proxy bans.

Its like a net that is meant to catch tuna but scoops up dolphins, turtles and whatever else happens to be swimming at the time.

I for sure don't use IPV6 as a way to skirt geofiltering and I'd bet the same is true of most people.

Netflix needs to give a pass on those HE IP blocks.

Another way for Netflix to do it might be to fail over to IPV4 on their end as a check when it detects HE IPs rather than forcing the customers to figure out how to do it.

[dennypage]

https://forum.pfsense.org/index.php?topic=112378.msg627503#msg627503

[kejianshi]

Yep - Some people are definitely using it for that purpose.

Still, I don't think its a good reason to slam everyone.

I will be glad when native IPV6 is running on verizon fios so I don't need HE as much.

Although, I sort of like having static IPs for IPV6.

Seems like the internet providers are again going out of there way to make up reasons for you IP to change all the time.

With IPV6, there is no good reason.

[kejianshi]

You know - It would be nice to be able to enter a domain in a field in DNS resolver and tell it on a case by case basis to either remove all IPV6 or IPV4 references to a site.

Might be abit less brute force than blocking entire IPV6 ranges.

[reinderien]

@kejianshi:

You know - It would be nice to be able to enter a domain in a field in DNS resolver and tell it on a case by case basis to either remove all IPV6 or IPV4 references to a site.

Might be a bit less brute force than blocking entire IPV6 ranges.

That's exactly what I got working here:

https://www.reddit.com/r/PFSENSE/comments/6weauh/ipv6_and_netflix_another_option/

[awebster]

I didn't want it to break all IPv6 name resolution, so I used this method instead…works great!

https://forum.pfsense.org/index.php?topic=133172.msg732233#msg732233

[reinderien]

@awebster:

I didn't want it to break all IPv6 name resolution

The single-purpose bind solution does not break all IPv6 resolution. It targets specific domain names as registered through the unbound domain override mechanism. Comparatively speaking, the solution you linked breaks a whole lot more: many millions of IPv6 addresses are all blocked.

[Gertjan]

Exact.
This : https://forum.pfsense.org/index.php?topic=133172.msg732233#msg732233 is very ugly with a many (future) side effects.
This : https://www.reddit.com/r/PFSENSE/comments/6weauh/ipv6_and_netflix_another_option/ is beautiful because it block only listed domain names.

[kejianshi]

I like it, of course.  I just prefer to keep pfsense using as much of its default features as possible.

I'm not sure that switching to bind wouldn't break something (or lots of somethings) after an update/upgrade.

That is my only worry with that solution.

[zskwrel]

See: https://forums.he.net/index.php?topic=3564.msg21004#msg21004

Just add these parameters to the 'Custom options' box under: Services>DNS Resolver>General Settings.

local-zone: "netflix.com" typetransparent
local-data: "netflix.com IN AAAA ::"

local-zone: "netflix.net" typetransparent
local-data: "netflix.net IN AAAA ::"

local-zone: "nflxext.com" typetransparent
local-data: "nflxext.com IN AAAA ::"

local-zone: "nflximg.net" typetransparent
local-data: "nflximg.net IN AAAA ::"

local-zone: "nflxvideo.net" typetransparent
local-data: "nflxvideo.net IN AAAA ::"

local-zone: "www.netflix.com" typetransparent
local-data: "www.netflix.com IN AAAA ::"

local-zone: "customerevents.netflix.com" typetransparent
local-data: "customerevents.netflix.com IN AAAA ::"

local-zone: "secure.netflix.com" typetransparent
local-data: "secure.netflix.com IN AAAA ::"

local-zone: "adtech.nflximg.net" typetransparent
local-data: "adtech.nflximg.net IN AAAA ::"

local-zone: "assets.nflxext.com" typetransparent
local-data: "assets.nflxext.com IN AAAA ::"

local-zone: "codex.nflxext.com" typetransparent
local-data: "codex.nflxext.com IN AAAA ::"

local-zone: "dockhand.netflix.com" typetransparent
local-data: "dockhand.netflix.com IN AAAA ::"

local-zone: "ichnaea.netflix.com" typetransparent
local-data: "ichnaea.netflix.com IN AAAA ::"

local-zone: "art-s.nflximg.net" typetransparent
local-data: "art-s.nflximg.net IN AAAA ::"

local-zone: "tp-s.nflximg.net" typetransparent
local-data: "tp-s.nflximg.net IN AAAA ::"

Caveat: While I found Netflix to play on my PC using several different web browsers after adding the above parameters to unbound, Netflix still refuses to play through the Netflix PC app, Apple TV, or my iPhone Netflix app. If anybody can help explain why only web browsers work I'd be most grateful.


[Derelict]

Re: Pfsense plus hurricane electric breaks netflix IPV6 - proxy error

Netflix intentionally breaks access when using Tunnel Broker IPv6

There. Fixed it for you.

[kejianshi]

Yep.  Someone over there at netflix must be really bored…

Thing is I have plenty of IPV4 addresses that are private and in no vpn related IP blocks.

I really just use HE for IPV6 generically.  It is actually far more useful than the IPV6 that most ISPs provision since it is static.

I also don't believe HE intends their service to be a VPN like service.  What a huge mess.

[zskwrel]

In order to block the Hurricane Electric tunnel from trying to access Netflix I resorted to setting up an alias with the Netflix IPv6 address blocks then set up a reject rule in the LAN firewall rules. This seems to solve the problem of my Netflix apps being blocked.


[kejianshi]

Yep - I've heard somewhere that might work…

[kejianshi]

I could be wrong about this, but I bet there is more than just Netflix out there blocking access to their site if someone is using a IPV6 tunnel.

Wonder if one of the people who make updates for surricata might compile such a list of IPV6 offenders so they can be blocked at the wan by a regularly updated list?
Then everyone who uses a tunnel won't have to update their custom resolver configs every time Netflix does something new.

They could just call the rule set "IPV6 Tunnel Idiots" or something.

[tartan]

I finally put the time in today to fix this.

I really needed to leave ipv6 running here, and I rely on HE for this, since Charter Spectrum Business STILL hasn't rolled out ipv6 in my part of Georgia, USA.

Sure, I could kick Netflix out of my house, and then my wife and kids would kick me out.  I could put them all on an ipv4-only vlan with a dedicated wifi SSID, but that, to me, is a step backward.

After reading through the various Reddit posts and such, I decided to take the brute-force firewalling approach.  Special thanks go to diyftw for his/her contribution around the specific whois lookup, within this thread:
https://www.reddit.com/r/PFSENSE/comments/5ei2t6/netflix_over_hurricane_electric_tunnel_broker

Essentially, I'm rejecting all outbound ipv6 packets to all Netflix ASNs _, as well as to ALL of Amazon AWS, since Netflix uses AWS heavily, and I'm in no mood to keep up with which IP ranges Netflix is using from AWS.

The Netflix ranges update somewhat automatically.  I have a list of ASNs from HE at https://bgp.he.net/search?search%5Bsearch%5D=netflix&commit=Search.  (Thanks again to diyftw for that URL.)  But I don't yet have a way to discover new ASNs should Netflix add them.  I do a whois query nightly against these known ASNs, to at least keep them updated.

The AWS IP ranges update automatically.  Thanks to this lovely blog post, http://docs.aws.amazon.com/general/latest/gr/aws-ip-ranges.html, which references this json file https://ip-ranges.amazonaws.com/ip-ranges.json, I can daily look up what Amazon claims it's using.

So here's how I did it..

1. From the shell of pfsense, add a script somewhere, such as /root/get_netflix_v6, and make it executable.  Notice that the script writes the IP networks into the web directory of pfsense, so they can be used as alias URLs.

#!/bin/sh

# get_netflix_v6

# Netflix ASNs from https://bgp.he.net/search?search%5Bsearch%5D=netflix&commit=Search
AS="AS55095 AS40027 AS394406 AS2906 AS136292"

# Backup previous list
cp /usr/local/www/netflix/netflix_ipv6 /usr/local/www/netflix/netflix_ipv6.`date "+%G%m%d_%H%M"`

# Query for ipv6 networks within Netflix ASNs, and create new list of networks
for i in $AS
do
        whois -h riswhois.ripe.net -- -F -K -i $i | grep "^route6" | awk '{print $2}' | sort
done > /usr/local/www/netflix/netflix_ipv6

# Amazon AWS ipv6 range

# Backup previous list
cp /usr/local/www/netflix/aws_ipv6 /usr/local/www/netflix/aws_ipv6.`date "+%G%m%d_%H%M"`

# Query for ipv6 networks from AWS listing, and create new list of networks
# I didn't feel like installing the json package into pfsense when some simple grep and awk do the trick.
curl -s "https://ip-ranges.amazonaws.com/ip-ranges.json" | \
        grep "ipv6_prefix" | \
        grep -v "ipv6_prefixes" | \
        awk '{print $2}' | \
        awk -F'"' '{print $2}' | \
        sort | \
        uniq \
        > /usr/local/www/netflix/aws_ipv6

2. Setup the web directory, and touch initial, empty files:

mkdir /usr/local/www/netflix
touch /usr/local/www/netflix/{netflix_ipv6,aws_ipv6}

3. Run the script once, then confirm

/root/get_netflix_v6
tail -10 /usr/local/www/netflix/{netflix_ipv6,aws_ipv6}

for example:

[2.4.2-RELEASE][root@xxxx.xxxxxxx.xxx]/root: tail -10 /usr/local/www/netflix/{netflix_ipv6,aws_ipv6}
==> /usr/local/www/netflix/netflix_ipv6 <==
2a00:86c0:39bc::/48
2a00:86c0:39bd::/48
2a00:86c0:4::/48
2a00:86c0:5::/48
2a00:86c0:600::/48
2a00:86c0:601::/48
2a00:86c0:98::/48
2a00:86c0:99::/48
2a00:86c0::/32
2a00:86c0:ff0a::/48

==> /usr/local/www/netflix/aws_ipv6 <==
2a05:d07c:8000::/40
2a05:d07c:c000::/40
2a05:d07e:2000::/40
2a05:d07e:4000::/40
2a05:d07e:8000::/40
2a05:d07e:c000::/40
2a05:d07f:2000::/40
2a05:d07f:4000::/40
2a05:d07f:8000::/40
2a05:d07f:c000::/40

4. Add the cron package from the pfsense package manager.  Go to Services | Cron, and run the update script daily.  I run mine at 3am

0 	3 	* 	* 	* 	root 	/root/get_netflix_v6 2>&1

5. Create two aliases at Firewall | Aliases | URLs

Name = awsv6
Description = Amazon AWS ipv6 networks
Type = URL Table (IPs)
URL Table (IPs) = https://127.0.0.1:443/netflix/aws_ipv6         / 1      <--- the / 1 tells pfsense to re-read the file daily

Name = netflix6
Description = Netflix ipv6 networks
Type = URL Table (IPs)
URL Table (IPs) = https://127.0.0.1:443/netflix/netflix_ipv6         / 1      <--- the / 1 tells pfsense to re-read the file daily

6. Add two rules at Firewall | Rules | Floating – I put mine near the top of the list:

Action = Reject                 <----- You really do want to Reject here, not just Drop, so that the client immediately is informed of such
Quick = yes, Apply the action immediately on match
Interface = (select all the internal network interfaces where you have ipv6 users who want to use Netflix)
Direction = any
Address Family = IPv6
Protocol = Any
Source = any
Destination = Single host or alias = netflix6

And then create a second, identical entry for:
Destination = Single host or alias = awsv6

Click Apply

7. Test!_

[kejianshi]

Yep - Fixed this ages ago also basically by not allowing ipv6 netflix anything. 
Still - I think it is crazy stupid of netflix to so zealously block anything remotely related to a tunnel.

It is stupidity with a capital S.

[tartan]

@kejianshi:

Yep - Fixed this ages ago also basically by not allowing ipv6 netflix anything. 
Still - I think it is crazy stupid of netflix to so zealously block anything remotely related to a tunnel.

It is stupidity with a capital S.

Agreed.  It's too bad they didn't at least talk with the HE team to come up with a better plan.  Surely Netflix realizes tons of ISPs have not yet implemented ipv6, and that tunnels will be used for legit purposes.

Oh well.

Anyway, I wanted to provide that write-up for others who are struggling, and for myself for future reference.

[zskwrel]

This seems to be working for me. Thanks so much!