Netflix and HE.net tunnel fixed using Unbound python module

[twitched]

Most of the solutions for getting around Netflix's block of Hurricane Electric's tunnels involve either creating rules blocking certain IPv6 ranges (which may change from time to time) or setting up a second DNS server just to handle Netflix queries.  Until now I've been using the first, but always thought there should be a better way.

The other day Fillipo Valsorda tweeted out an Unbound python script he uses to solve the problem: https://gist.github.com/FiloSottile/e2cffde2bae1ea0c14eada229543aebd/

Since python was recently enabled in pfSense 2.4 beta's Unbound (https://redmine.pfsense.org/issues/7549), I thought I'd try it. It is working, but it's not yet straight forward.  Here's what I did:

1. Enable the python module in unbound.conf. This was the most difficult part since it requires changing the automatically generated line:

module-config: "iterator"

must be changed to

module-config: "python interator"

Or if you are using DNSSEC, it must be changed to

module-config: "validator python iterator"

Since this is automatically generated, I created a patch that will make the change: https://github.com/twitched/pfsense/commit/1ff1605e8d2e2c9f87aac489fd7af7a407b3440c

Note: If the python module is enabled, there must be a python script specified (Step 3 below)

2.  Create a script to put the python script in the chroot. Here's the script:

#!/bin/sh

#make sure the directory for the python libraries is in the chroot
mkdir -p /var/unbound/usr/local/lib/python2.7

#link the actual python library directory to the chroot's directory
mount -t nullfs /usr/local/lib/python2.7 /var/unbound/usr/local/lib/python2.7

#copy the python script to the /var/unbound directory so 
#unbound-checkconf can find it
cp /root/netflix-no-aaaa.py /var/unbound/

#create a /var/unbound directory in the /var/unbound directory so that 
#unbound can find the script
mkdir -p /var/unbound/var/unbound

#copy the python module into the /var/unbound/var/unbound directory under the chroot 
#directory
cp /root/netflix-no-aaaa.py /var/unbound/var/unbound

#make sure unbound can read it
chown unbound:unbound /var/unbound/var/unbound/netflix-no-aaaa.py

As you can see, it took a little more than copying a file.  unbound-checkconf doesn't seem to recognize that the file is in a chroot while unbound does.  In the configuration (Step 3), you specify the path of the script, and the two interpret it differently meaning that you have to have the file in both places.  I also tried relative paths, which didn't work.

I haven't tested it yet, but you'll probably want to execute this command at startup: https://doc.pfsense.org/index.php/Executing_commands_at_boot_time

3. Add the python script to the unbound config. Here's what it looks like:

The unbound documentation for this directive is here under Python Module Options: https://www.unbound.net/documentation/unbound.conf.html

After that it worked.  Here's what I get, which is exactly what I want.  With no answer for AAAA, it will fall back to an A query.

; <<>> DiG 9.8.3-P1 <<>> AAAA netflix.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 22275
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;netflix.com.			IN	AAAA

;; Query time: 29 msec
;; SERVER: 192.168.X.X#53(192.168.X.X)
;; WHEN: Fri Jul 28 10:37:48 2017
;; MSG SIZE  rcvd: 29

[jimp]

That's nice, I may have to try that out. Usually we do Netflix from the TiVo, Chromecast, or Tablets and none of those use IPv6 so we don't have issues  (just DHCPv6 here, no SLAAC), but the rare times I try to access it on my laptop it doesn't work.

With a little extra work, that could be made more generic so it isn't just Netflix. I can think of a couple other times I've wanted to remove an AAAA response for a host to force it to use IPv4.

[johnpoz]

couldn't you just add the module-config to the custom options box?

hmm…

When I try to enable python module I get

pythonmod: cannot initialize core module: unboundmodule.py

[twitched]

'module-config: "iterator"' is already in the generated file.  I don't think it can be in there twice.

[johnpoz]

Nevermind - helps if you finish the steps before you try and load the module..

Currently working.. Same boat as jimp, don't have a need for this for netflix - but it would be slick to add this into the gui like the domain/host overrides where you do not want certain domains/fqdn to resolve AAAA.. Could be quite handy!!

[Gertjan]

@johnpoz:

…. but it would be slick to add this into the gui like the domain/host overrides where you do not want certain domains/fqdn to resolve AAAA.. Could be quite handy!!

Or the other way around : enforce a pure outbound IPv6 traffic, bringing IPv4 to a halt.
I'm very curious to see how much of the Internet still "works" (i guess I better shut down my "help desk" before trying this).

Btw : I'm using an ISP that just discovered that IPv6 exists, so I'm using he.net for years now. My IPv6 from them, surfacing in Paris has been geo-locked "USA" by Netflix, so they hang up on me right away. But wait … => Their streaming servers are just 3 hops away from my WAN-IPv6 (he.net in Paris). Well ....  ;D

[Nan0tEch]

thx for this solution. Now netflix works again. Just have a question in how does this works for the netflix app by not getting the IPv6 adresses as i block all IPv6 in my network? What could the app be using by getting these IPv6 blocks and blocking the streaming service. I do use a VPN provider and now it's no problem.

[Gertjan]

@Nan0tEch said in Netflix and HE.net tunnel fixed using Unbound python module:

how does this works for the netflix app by not getting the IPv6 adresses as i block all IPv6 in my network?

The issue is / was : devices that run browsers to visit Netflix, or apps on phones or SmartTV could contact Netflix by IPv4 and - if you have it enabled - IPv6.
Some ISP's don't know what IPv6 is, so ipv6.he.net can be used as an IPv6-ISP, in parallel with your classic ISP, doing IPv4 only.

The issue is : like many VPN's, the gateway he.net is using (an Ipv6) is considered as and "VPN endpoint" and listed as such by NetFlix : they don't accepts that I use ipv6.he.net to stream their content.

So, blocking AAAA requests when some device on my LAN want to resolve a Netflix server help : my device steps back to IPv4 only, thus using my classic ISP, Netflix doesn't complain now.

Nite : If you do not use IPv6 your don''t need the unbound-python trick described here.

@Nan0tEch said in Netflix and HE.net tunnel fixed using Unbound python module:

What could the app be using by getting these IPv6 blocks and blocking the streaming service.

What do you mean ?
How it works ?
See the script. It blocks AAAA requests if the URL is on a list, present in the script. You have to edit the script to implicate other URLs.