Pfsense plus hurricane electric breaks netflix IPV6 - proxy error



  • Netflix has started blocking all VPN associated networks that they can detect or identify as VPN or proxy.

    They are clumping Hurricane Electric tunnel broker IPs in with "proxy".

    So, even if you are in the USA and have a real USA IP, you will probably see this:

    So, just having HE IPV6 tunnel on your pfsense will probably cause netflix to throw an error screen in your face about being naughty and using VPNs and proxy.

    So, I had to go into the firewall and make a quick floating rule REJECT all of the following:

    2a01:578:3::/48
    2406:da00:ff00::/48
    2600:1407:19::/48
    2607:f8b0:4001::/48
    2620:108:700f::/48

    These are all associated with amazon cloud services and netflix streaming.

    Rejecting those causes Netflix to fail over to IPV4 and then everything works again.

    I assume people with native IPV6 don't have this problem.  Just the people using tunnel brokers.

    For me, to make it simple I made an alias containing all those IP ranges.

    Then I selected all interfaces in my floating firewall rule and told it to apply as soon as it matched.

    Not sure if this will be a permanent fix or if Netflix will come up with new IP ranges all the time, but for now it works.



  • Hi,

    The problem boils down to:
    @kejianshi:

    … Just the people using tunnel brokers.

    Netflix made an international statement about that.

    The entire issue is : who is on the list (that Netflix manages) that they consider as a proxy ?
    You proved, as many did, that he.net is … (the free IPv6 tunnel service).



  • Yeah - I think that netflix is going too far out of its way to attempt to enforce vpn and proxy bans.

    Its like a net that is meant to catch tuna but scoops up dolphins, turtles and whatever else happens to be swimming at the time.

    I for sure don't use IPV6 as a way to skirt geofiltering and I'd bet the same is true of most people.

    Netflix needs to give a pass on those HE IP blocks.

    Another way for Netflix to do it might be to fail over to IPV4 on their end as a check when it detects HE IPs rather than forcing the customers to figure out how to do it.





  • Yep - Some people are definitely using it for that purpose.

    Still, I don't think its a good reason to slam everyone.

    I will be glad when native IPV6 is running on verizon fios so I don't need HE as much.

    Although, I sort of like having static IPs for IPV6.

    Seems like the internet providers are again going out of there way to make up reasons for you IP to change all the time.

    With IPV6, there is no good reason.



  • You know - It would be nice to be able to enter a domain in a field in DNS resolver and tell it on a case by case basis to either remove all IPV6 or IPV4 references to a site.

    Might be abit less brute force than blocking entire IPV6 ranges.



  • @kejianshi:

    You know - It would be nice to be able to enter a domain in a field in DNS resolver and tell it on a case by case basis to either remove all IPV6 or IPV4 references to a site.

    Might be a bit less brute force than blocking entire IPV6 ranges.

    That's exactly what I got working here:

    https://www.reddit.com/r/PFSENSE/comments/6weauh/ipv6_and_netflix_another_option/



  • I didn't want it to break all IPv6 name resolution, so I used this method instead…works great!

    https://forum.pfsense.org/index.php?topic=133172.msg732233#msg732233



  • @awebster:

    I didn't want it to break all IPv6 name resolution

    The single-purpose bind solution does not break all IPv6 resolution. It targets specific domain names as registered through the unbound domain override mechanism. Comparatively speaking, the solution you linked breaks a whole lot more: many millions of IPv6 addresses are all blocked.



  • Exact.
    This : https://forum.pfsense.org/index.php?topic=133172.msg732233#msg732233 is very ugly with a many (future) side effects.
    This : https://www.reddit.com/r/PFSENSE/comments/6weauh/ipv6_and_netflix_another_option/ is beautiful because it block only listed domain names.



  • I like it, of course.  I just prefer to keep pfsense using as much of its default features as possible.

    I'm not sure that switching to bind wouldn't break something (or lots of somethings) after an update/upgrade.

    That is my only worry with that solution.



  • See: https://forums.he.net/index.php?topic=3564.msg21004#msg21004

    Just add these parameters to the 'Custom options' box under: Services>DNS Resolver>General Settings.

    local-zone: "netflix.com" typetransparent
    local-data: "netflix.com IN AAAA ::"
    
    local-zone: "netflix.net" typetransparent
    local-data: "netflix.net IN AAAA ::"
    
    local-zone: "nflxext.com" typetransparent
    local-data: "nflxext.com IN AAAA ::"
    
    local-zone: "nflximg.net" typetransparent
    local-data: "nflximg.net IN AAAA ::"
    
    local-zone: "nflxvideo.net" typetransparent
    local-data: "nflxvideo.net IN AAAA ::"
    
    local-zone: "www.netflix.com" typetransparent
    local-data: "www.netflix.com IN AAAA ::"
    
    local-zone: "customerevents.netflix.com" typetransparent
    local-data: "customerevents.netflix.com IN AAAA ::"
    
    local-zone: "secure.netflix.com" typetransparent
    local-data: "secure.netflix.com IN AAAA ::"
    
    local-zone: "adtech.nflximg.net" typetransparent
    local-data: "adtech.nflximg.net IN AAAA ::"
    
    local-zone: "assets.nflxext.com" typetransparent
    local-data: "assets.nflxext.com IN AAAA ::"
    
    local-zone: "codex.nflxext.com" typetransparent
    local-data: "codex.nflxext.com IN AAAA ::"
    
    local-zone: "dockhand.netflix.com" typetransparent
    local-data: "dockhand.netflix.com IN AAAA ::"
    
    local-zone: "ichnaea.netflix.com" typetransparent
    local-data: "ichnaea.netflix.com IN AAAA ::"
    
    local-zone: "art-s.nflximg.net" typetransparent
    local-data: "art-s.nflximg.net IN AAAA ::"
    
    local-zone: "tp-s.nflximg.net" typetransparent
    local-data: "tp-s.nflximg.net IN AAAA ::"
    

    Caveat: While I found Netflix to play on my PC using several different web browsers after adding the above parameters to unbound, Netflix still refuses to play through the Netflix PC app, Apple TV, or my iPhone Netflix app. If anybody can help explain why only web browsers work I'd be most grateful.

    ![Screenshot-2017-10-21 pfsense lan - Services DNS Resolver General Settings.png](/public/imported_attachments/1/Screenshot-2017-10-21 pfsense lan - Services DNS Resolver General Settings.png)
    ![Screenshot-2017-10-21 pfsense lan - Services DNS Resolver General Settings.png_thumb](/public/imported_attachments/1/Screenshot-2017-10-21 pfsense lan - Services DNS Resolver General Settings.png_thumb)


  • Netgate

    Re: Pfsense plus hurricane electric breaks netflix IPV6 - proxy error

    Netflix intentionally breaks access when using Tunnel Broker IPv6

    There. Fixed it for you.



  • Yep.  Someone over there at netflix must be really bored…

    Thing is I have plenty of IPV4 addresses that are private and in no vpn related IP blocks.

    I really just use HE for IPV6 generically.  It is actually far more useful than the IPV6 that most ISPs provision since it is static.

    I also don't believe HE intends their service to be a VPN like service.  What a huge mess.



  • In order to block the Hurricane Electric tunnel from trying to access Netflix I resorted to setting up an alias with the Netflix IPv6 address blocks then set up a reject rule in the LAN firewall rules. This seems to solve the problem of my Netflix apps being blocked.

    ![Screenshot-2017-10-21 pfsense lan - Firewall Aliases Edit.png](/public/imported_attachments/1/Screenshot-2017-10-21 pfsense lan - Firewall Aliases Edit.png)
    ![Screenshot-2017-10-21 pfsense lan - Firewall Aliases Edit.png_thumb](/public/imported_attachments/1/Screenshot-2017-10-21 pfsense lan - Firewall Aliases Edit.png_thumb)
    ![Screenshot-2017-10-21 pfsense lan - Firewall Rules LAN.png](/public/imported_attachments/1/Screenshot-2017-10-21 pfsense lan - Firewall Rules LAN.png)
    ![Screenshot-2017-10-21 pfsense lan - Firewall Rules LAN.png_thumb](/public/imported_attachments/1/Screenshot-2017-10-21 pfsense lan - Firewall Rules LAN.png_thumb)



  • Yep - I've heard somewhere that might work…



  • I could be wrong about this, but I bet there is more than just Netflix out there blocking access to their site if someone is using a IPV6 tunnel.

    Wonder if one of the people who make updates for surricata might compile such a list of IPV6 offenders so they can be blocked at the wan by a regularly updated list?
    Then everyone who uses a tunnel won't have to update their custom resolver configs every time Netflix does something new.

    They could just call the rule set "IPV6 Tunnel Idiots" or something.



  • I finally put the time in today to fix this.

    I really needed to leave ipv6 running here, and I rely on HE for this, since Charter Spectrum Business STILL hasn't rolled out ipv6 in my part of Georgia, USA.

    Sure, I could kick Netflix out of my house, and then my wife and kids would kick me out.  I could put them all on an ipv4-only vlan with a dedicated wifi SSID, but that, to me, is a step backward.

    After reading through the various Reddit posts and such, I decided to take the brute-force firewalling approach.  Special thanks go to diyftw for his/her contribution around the specific whois lookup, within this thread:
    https://www.reddit.com/r/PFSENSE/comments/5ei2t6/netflix_over_hurricane_electric_tunnel_broker

    Essentially, I'm rejecting all outbound ipv6 packets to all Netflix ASNs _, as well as to ALL of Amazon AWS, since Netflix uses AWS heavily, and I'm in no mood to keep up with which IP ranges Netflix is using from AWS.

    The Netflix ranges update somewhat automatically.  I have a list of ASNs from HE at https://bgp.he.net/search?search[search]=netflix&commit=Search.  (Thanks again to diyftw for that URL.)  But I don't yet have a way to discover new ASNs should Netflix add them.  I do a whois query nightly against these known ASNs, to at least keep them updated.

    The AWS IP ranges update automatically.  Thanks to this lovely blog post, http://docs.aws.amazon.com/general/latest/gr/aws-ip-ranges.html, which references this json file https://ip-ranges.amazonaws.com/ip-ranges.json, I can daily look up what Amazon claims it's using.

    So here's how I did it..

    1. From the shell of pfsense, add a script somewhere, such as /root/get_netflix_v6, and make it executable.  Notice that the script writes the IP networks into the web directory of pfsense, so they can be used as alias URLs.
    
    #!/bin/sh
    
    # get_netflix_v6
    
    # Netflix ASNs from https://bgp.he.net/search?search%5Bsearch%5D=netflix&commit=Search
    AS="AS55095 AS40027 AS394406 AS2906 AS136292"
    
    # Backup previous list
    cp /usr/local/www/netflix/netflix_ipv6 /usr/local/www/netflix/netflix_ipv6.`date "+%G%m%d_%H%M"`
    
    # Query for ipv6 networks within Netflix ASNs, and create new list of networks
    for i in $AS
    do
            whois -h riswhois.ripe.net -- -F -K -i $i | grep "^route6" | awk '{print $2}' | sort
    done > /usr/local/www/netflix/netflix_ipv6
    
    # Amazon AWS ipv6 range
    
    # Backup previous list
    cp /usr/local/www/netflix/aws_ipv6 /usr/local/www/netflix/aws_ipv6.`date "+%G%m%d_%H%M"`
    
    # Query for ipv6 networks from AWS listing, and create new list of networks
    # I didn't feel like installing the json package into pfsense when some simple grep and awk do the trick.
    curl -s "https://ip-ranges.amazonaws.com/ip-ranges.json" | \
            grep "ipv6_prefix" | \
            grep -v "ipv6_prefixes" | \
            awk '{print $2}' | \
            awk -F'"' '{print $2}' | \
            sort | \
            uniq \
            > /usr/local/www/netflix/aws_ipv6
    
    
    1. Setup the web directory, and touch initial, empty files:
    
    mkdir /usr/local/www/netflix
    touch /usr/local/www/netflix/{netflix_ipv6,aws_ipv6}
    
    
    1. Run the script once, then confirm
    
    /root/get_netflix_v6
    tail -10 /usr/local/www/netflix/{netflix_ipv6,aws_ipv6}
    
    for example:
    
    [2.4.2-RELEASE][root@xxxx.xxxxxxx.xxx]/root: tail -10 /usr/local/www/netflix/{netflix_ipv6,aws_ipv6}
    ==> /usr/local/www/netflix/netflix_ipv6 <==
    2a00:86c0:39bc::/48
    2a00:86c0:39bd::/48
    2a00:86c0:4::/48
    2a00:86c0:5::/48
    2a00:86c0:600::/48
    2a00:86c0:601::/48
    2a00:86c0:98::/48
    2a00:86c0:99::/48
    2a00:86c0::/32
    2a00:86c0:ff0a::/48
    
    ==> /usr/local/www/netflix/aws_ipv6 <==
    2a05:d07c:8000::/40
    2a05:d07c:c000::/40
    2a05:d07e:2000::/40
    2a05:d07e:4000::/40
    2a05:d07e:8000::/40
    2a05:d07e:c000::/40
    2a05:d07f:2000::/40
    2a05:d07f:4000::/40
    2a05:d07f:8000::/40
    2a05:d07f:c000::/40
    
    
    1. Add the cron package from the pfsense package manager.  Go to Services | Cron, and run the update script daily.  I run mine at 3am
    
    0 	3 	* 	* 	* 	root 	/root/get_netflix_v6 2>&1
    
    
    1. Create two aliases at Firewall | Aliases | URLs
    
    Name = awsv6
    Description = Amazon AWS ipv6 networks
    Type = URL Table (IPs)
    URL Table (IPs) = https://127.0.0.1:443/netflix/aws_ipv6         / 1      <--- the / 1 tells pfsense to re-read the file daily
    
    Name = netflix6
    Description = Netflix ipv6 networks
    Type = URL Table (IPs)
    URL Table (IPs) = https://127.0.0.1:443/netflix/netflix_ipv6         / 1      <--- the / 1 tells pfsense to re-read the file daily
    
    
    1. Add two rules at Firewall | Rules | Floating – I put mine near the top of the list:
    
    Action = Reject                 <----- You really do want to Reject here, not just Drop, so that the client immediately is informed of such
    Quick = yes, Apply the action immediately on match
    Interface = (select all the internal network interfaces where you have ipv6 users who want to use Netflix)
    Direction = any
    Address Family = IPv6
    Protocol = Any
    Source = any
    Destination = Single host or alias = netflix6
    
    And then create a second, identical entry for:
    Destination = Single host or alias = awsv6
    
    Click Apply
    
    
    1. Test!_


  • Yep - Fixed this ages ago also basically by not allowing ipv6 netflix anything. 
    Still - I think it is crazy stupid of netflix to so zealously block anything remotely related to a tunnel.

    It is stupidity with a capital S.



  • @kejianshi:

    Yep - Fixed this ages ago also basically by not allowing ipv6 netflix anything. 
    Still - I think it is crazy stupid of netflix to so zealously block anything remotely related to a tunnel.

    It is stupidity with a capital S.

    Agreed.  It's too bad they didn't at least talk with the HE team to come up with a better plan.  Surely Netflix realizes tons of ISPs have not yet implemented ipv6, and that tunnels will be used for legit purposes.

    Oh well.

    Anyway, I wanted to provide that write-up for others who are struggling, and for myself for future reference.



  • This seems to be working for me. Thanks so much!



  • I use HE.net as well, and just noticed this ATV issue today. I actually have a handful of streaming devices and I'd really prefer all of them to use IPv4 - mainly because I'm OTT-only with all my viewing and don't want to screw around with things by possibly pushing traffic over a tunnel, it's just another possible point of failure I don't want to think about.

    Given that I know all the MACs of these devices (a few rokus, an ATV and a fire TV), is there some way on the pfsense side to just prevent them from even picking up IPv6 addresses?  I know I could VLAN but I really don't want to invest in more gear or cable runs to accomplish this.


  • Netgate

    Put them on a VLAN that doesn't have IPv6 enabled.



  • This is also a good solution that works fine here: https://forum.pfsense.org/index.php?topic=134352.msg737158#msg737158


 

© Copyright 2002 - 2018 Rubicon Communications, LLC | Privacy Policy