Pfsense plus hurricane electric breaks netflix IPV6 - proxy error

kejianshi

I could be wrong about this, but I bet there is more than just Netflix out there blocking access to their site if someone is using a IPV6 tunnel.

Wonder if one of the people who make updates for surricata might compile such a list of IPV6 offenders so they can be blocked at the wan by a regularly updated list?
Then everyone who uses a tunnel won't have to update their custom resolver configs every time Netflix does something new.

They could just call the rule set "IPV6 Tunnel Idiots" or something.

tartan

I finally put the time in today to fix this.

I really needed to leave ipv6 running here, and I rely on HE for this, since Charter Spectrum Business STILL hasn't rolled out ipv6 in my part of Georgia, USA.

Sure, I could kick Netflix out of my house, and then my wife and kids would kick me out. I could put them all on an ipv4-only vlan with a dedicated wifi SSID, but that, to me, is a step backward.

After reading through the various Reddit posts and such, I decided to take the brute-force firewalling approach. Special thanks go to diyftw for his/her contribution around the specific whois lookup, within this thread:
https://www.reddit.com/r/PFSENSE/comments/5ei2t6/netflix_over_hurricane_electric_tunnel_broker

Essentially, I'm rejecting all outbound ipv6 packets to all Netflix ASNs _, as well as to ALL of Amazon AWS, since Netflix uses AWS heavily, and I'm in no mood to keep up with which IP ranges Netflix is using from AWS.

The Netflix ranges update somewhat automatically. I have a list of ASNs from HE at https://bgp.he.net/search?search%5Bsearch%5D=netflix&commit=Search. (Thanks again to diyftw for that URL.) But I don't yet have a way to discover new ASNs should Netflix add them. I do a whois query nightly against these known ASNs, to at least keep them updated.

The AWS IP ranges update automatically. Thanks to this lovely blog post, http://docs.aws.amazon.com/general/latest/gr/aws-ip-ranges.html, which references this json file https://ip-ranges.amazonaws.com/ip-ranges.json, I can daily look up what Amazon claims it's using.

So here's how I did it..

From the shell of pfsense, add a script somewhere, such as /root/get_netflix_v6, and make it executable. Notice that the script writes the IP networks into the web directory of pfsense, so they can be used as alias URLs.


#!/bin/sh

# get_netflix_v6

# Netflix ASNs from https://bgp.he.net/search?search%5Bsearch%5D=netflix&commit=Search
AS="AS55095 AS40027 AS394406 AS2906 AS136292"

# Backup previous list
cp /usr/local/www/netflix/netflix_ipv6 /usr/local/www/netflix/netflix_ipv6.`date "+%G%m%d_%H%M"`

# Query for ipv6 networks within Netflix ASNs, and create new list of networks
for i in $AS
do
        whois -h riswhois.ripe.net -- -F -K -i $i | grep "^route6" | awk '{print $2}' | sort
done > /usr/local/www/netflix/netflix_ipv6

# Amazon AWS ipv6 range

# Backup previous list
cp /usr/local/www/netflix/aws_ipv6 /usr/local/www/netflix/aws_ipv6.`date "+%G%m%d_%H%M"`

# Query for ipv6 networks from AWS listing, and create new list of networks
# I didn't feel like installing the json package into pfsense when some simple grep and awk do the trick.
curl -s "https://ip-ranges.amazonaws.com/ip-ranges.json" | \
        grep "ipv6_prefix" | \
        grep -v "ipv6_prefixes" | \
        awk '{print $2}' | \
        awk -F'"' '{print $2}' | \
        sort | \
        uniq \
        > /usr/local/www/netflix/aws_ipv6

Setup the web directory, and touch initial, empty files:


mkdir /usr/local/www/netflix
touch /usr/local/www/netflix/{netflix_ipv6,aws_ipv6}

Run the script once, then confirm


/root/get_netflix_v6
tail -10 /usr/local/www/netflix/{netflix_ipv6,aws_ipv6}

for example:

[2.4.2-RELEASE][root@xxxx.xxxxxxx.xxx]/root: tail -10 /usr/local/www/netflix/{netflix_ipv6,aws_ipv6}
==> /usr/local/www/netflix/netflix_ipv6 <==
2a00:86c0:39bc::/48
2a00:86c0:39bd::/48
2a00:86c0:4::/48
2a00:86c0:5::/48
2a00:86c0:600::/48
2a00:86c0:601::/48
2a00:86c0:98::/48
2a00:86c0:99::/48
2a00:86c0::/32
2a00:86c0:ff0a::/48

==> /usr/local/www/netflix/aws_ipv6 <==
2a05:d07c:8000::/40
2a05:d07c:c000::/40
2a05:d07e:2000::/40
2a05:d07e:4000::/40
2a05:d07e:8000::/40
2a05:d07e:c000::/40
2a05:d07f:2000::/40
2a05:d07f:4000::/40
2a05:d07f:8000::/40
2a05:d07f:c000::/40

Add the cron package from the pfsense package manager. Go to Services | Cron, and run the update script daily. I run mine at 3am


0 	3 	* 	* 	* 	root 	/root/get_netflix_v6 2>&1

Create two aliases at Firewall | Aliases | URLs


Name = awsv6
Description = Amazon AWS ipv6 networks
Type = URL Table (IPs)
URL Table (IPs) = https://127.0.0.1:443/netflix/aws_ipv6         / 1      <--- the / 1 tells pfsense to re-read the file daily

Name = netflix6
Description = Netflix ipv6 networks
Type = URL Table (IPs)
URL Table (IPs) = https://127.0.0.1:443/netflix/netflix_ipv6         / 1      <--- the / 1 tells pfsense to re-read the file daily

Add two rules at Firewall | Rules | Floating – I put mine near the top of the list:


Action = Reject                 <----- You really do want to Reject here, not just Drop, so that the client immediately is informed of such
Quick = yes, Apply the action immediately on match
Interface = (select all the internal network interfaces where you have ipv6 users who want to use Netflix)
Direction = any
Address Family = IPv6
Protocol = Any
Source = any
Destination = Single host or alias = netflix6

And then create a second, identical entry for:
Destination = Single host or alias = awsv6

Click Apply

Test!_

kejianshi

Yep - Fixed this ages ago also basically by not allowing ipv6 netflix anything.
Still - I think it is crazy stupid of netflix to so zealously block anything remotely related to a tunnel.

It is stupidity with a capital S.

tartan

@kejianshi:

Yep - Fixed this ages ago also basically by not allowing ipv6 netflix anything.
Still - I think it is crazy stupid of netflix to so zealously block anything remotely related to a tunnel.

It is stupidity with a capital S.

Agreed. It's too bad they didn't at least talk with the HE team to come up with a better plan. Surely Netflix realizes tons of ISPs have not yet implemented ipv6, and that tunnels will be used for legit purposes.

Oh well.

Anyway, I wanted to provide that write-up for others who are struggling, and for myself for future reference.

zskwrel

This seems to be working for me. Thanks so much!

sporkme

I use HE.net as well, and just noticed this ATV issue today. I actually have a handful of streaming devices and I'd really prefer all of them to use IPv4 - mainly because I'm OTT-only with all my viewing and don't want to screw around with things by possibly pushing traffic over a tunnel, it's just another possible point of failure I don't want to think about.

Given that I know all the MACs of these devices (a few rokus, an ATV and a fire TV), is there some way on the pfsense side to just prevent them from even picking up IPv6 addresses? I know I could VLAN but I really don't want to invest in more gear or cable runs to accomplish this.

Derelict

Put them on a VLAN that doesn't have IPv6 enabled.

Grimson

This is also a good solution that works fine here: https://forum.pfsense.org/index.php?topic=134352.msg737158#msg737158

transistor1

Late to the party, but posting anyway, as I just encountered the Netflix tunnel issue today.

In pfSense 2.4.3-RELEASE-p1, I went to System->Routing, then clicked on the Static Routes tab.

Under Destination Network, I added 2406:da00:ff00::, and selected 48 as the netmask. For Gateway, I selected Null6 ::1

0_1537801517181_35395898-a1da-4f51-a8c9-0c73d1c91990-image.png

0_1537801475241_538e7082-653c-4368-b0f7-50f649cffe00-image.png

This forced Netflix to go through IPv4 instead of my HE tunnel. It was quite easy, and seems to do the trick for me.

bouke

@kejianshi Thanks. This quick fix works fine for me.

msf2000

I'm running into a situation where the Netflix app is not failing over to IPv4 with the reject rule in place for IPv6 addresses. Only solution is to disable IPv6 on the client.

Has anyone had trouble with their setup since implementing the reject rules described (above)?

Derelict

If the client thinks it has IPv6 and gets a response - even though it is a connection refused - why should it try IPv4? In general, if you have a dual stack client you don't want to selectively break IPv6. You want to turn it off. Behavior there will be application and operating system dependent.

Gertjan

Another solution is : inform the local resolver ( unbound) that it should remove any IPv6 when the URL is "netflix" like.

https://forum.netgate.com/topic/118566/netflix-and-he-net-tunnel-fixed-using-unbound-python-module

As soon as I view a film from netflix with my IPv6 enabled PC via pfSense (using he.net for IPv6 addresses) I see this in the DNS log :

Aug 6 17:35:51 	unbound 	52268:1 	info: no-aaaa: blocking AAAA request for anycast.ftl.netflix.com.
Aug 6 17:35:51 	unbound 	52268:1 	info: no-aaaa: blocking AAAA request for anycast.ftl.netflix.com.
Aug 6 17:35:51 	unbound 	52268:1 	info: no-aaaa: blocking AAAA request for oca-api.us-west-2.prodaa.netflix.com.
Aug 6 17:35:51 	unbound 	52268:1 	info: no-aaaa: blocking AAAA request for oca-api.us-west-2.prodaa.netflix.com.
Aug 6 17:35:51 	unbound 	52268:1 	info: no-aaaa: blocking AAAA request for ifo5usjqtzhvl6xjlsanq-euw1.r.nflxso.net.
Aug 6 17:35:51 	unbound 	52268:1 	info: no-aaaa: blocking AAAA request for oca-api.netflix.com.
Aug 6 17:35:48 	unbound 	52268:1 	info: no-aaaa: blocking AAAA request for push.prod.us-west-2.prodaa.netflix.com.
Aug 6 17:35:48 	unbound 	52268:1 	info: no-aaaa: blocking AAAA request for push.prod.netflix.com.
Aug 6 17:35:47 	unbound 	52268:1 	info: no-aaaa: blocking AAAA request for occ-0-56-55.1.nflxso.net.
Aug 6 17:35:47 	unbound 	52268:1 	info: no-aaaa: blocking AAAA request for occ-0-56-55.1.nflxso.net.
Aug 6 17:35:41 	unbound 	52268:1 	info: no-aaaa: blocking AAAA request for ichnaea-web.us-west-2.prodaa.netflix.com.
Aug 6 17:35:41 	unbound 	52268:1 	info: no-aaaa: blocking AAAA request for ichnaea-web.netflix.com.
Aug 6 17:35:39 	unbound 	52268:1 	info: no-aaaa: blocking AAAA request for assets.nflxext.com.
Aug 6 17:35:39 	unbound 	52268:1 	info: no-aaaa: blocking AAAA request for assets.nflxext.com.
Aug 6 17:35:39 	unbound 	52268:1 	info: no-aaaa: blocking AAAA request for codex.nflxext.com.
Aug 6 17:35:39 	unbound 	52268:1 	info: no-aaaa: blocking AAAA request for codex.nflxext.com.
Aug 6 17:35:38 	unbound 	52268:1 	info: no-aaaa: blocking AAAA request for www.eu-west-1.prodaa.netflix.com.
Aug 6 17:35:38 	unbound 	52268:1 	info: no-aaaa: blocking AAAA request for www.netflix.com.

Thus : any Netflix related URL that has an Pv6 is removed from the DNS request reply - only IPv4 persists. So the application use pure IPv4 to contact Netflix. This means : not using IPv6 so not using he.net.
Works !

Napsterbater

@msf2000

@msf2000 said in Pfsense plus hurricane electric breaks netflix IPV6 - proxy error:

I'm running into a situation where the Netflix app is not failing over to IPv4 with the reject rule in place for IPv6 addresses. Only solution is to disable IPv6 on the client.

Has anyone had trouble with their setup since implementing the reject rules described (above)?

Switch Reject for Block and see if Happy Eyeballs kicks in?

kejianshi

So its years later. Jan 2022 and I set this up again on a new pfsense and this time it seems so far that Netflix, Amazon and others are not blocking Hurricane Electric IPV6 anymore. So thats nice! (So far).

Napsterbater

@kejianshi You will notice ALOT of stuff is missing. They no longer outright block you, but they only show you a "Global" view, i.e. stuff available everywhere. Stuff allowed in your location but not allowed everywhere is not shown.

This is my currently block list that seems to catch everything netflix.

2406:da00:ff00::/48
2607:f8b0:4001::/48
2620:108:700f::/48
2a01:578:3::/48
2600:1407:19::/48
2a05:d018:76c::/48
2600:1f18:631e::/48
2607:fb10::/32

Gertjan

@napsterbater said in Pfsense plus hurricane electric breaks netflix IPV6 - proxy error:

but they only show you a "Global" view

You have to login first, right ?
What you can view is probably based upon your IP (4 or 6).
And probably the country from which you subscribed.

kejianshi

@napsterbater Thanks! I tested your "Theory" and turns out it was a fact. I have loaded my alias with those IPs and made the firewall block rule. Now the omitted content is present again. I tested it with "Stargate SG1" which is apparently US-Only content and your are correct. Thanks for the heads up.

Napsterbater

@gertjan said in Pfsense plus hurricane electric breaks netflix IPV6 - proxy error:

@napsterbater said in Pfsense plus hurricane electric breaks netflix IPV6 - proxy error:

but they only show you a "Global" view

You have to login first, right ?
What you can view is probably based upon your IP (4 or 6).
And probably the country from which you subscribed.

Yes.
An HE IPv6 = VPN as far as Netflix is concerned.
No, only Globally available items, your login or "country subscribed in" has no bearing..

Gertjan

For the record : these (are placed in an alias) and then placed on the LAN interface, right ? ::

@napsterbater said in Pfsense plus hurricane electric breaks netflix IPV6 - proxy error:

2406:da00:ff00::/48
2607:f8b0:4001::/48
2620:108:700f::/48
2a01:578:3::/48
2600:1407:19::/48
2a05:d018:76c::/48
2600:1f18:631e::/48
2607:fb10::/32