Problems with Netflix and Amazon. NOT using a VPN.
-
Hello all,
I have a very frustrating problem. I have been using pfsense (currently 2.5.2) built on a Qotom-Q555G6-S05 with my internet provider for almost a year without any issues. My ISP is Taylor Telecom, and I have a 1GB fiber connection. They use the mac address of the router to attach to your assigned external IP address. I had pfsense setup to spoof the mac of the Comtrend router originally suplied by the ISP. I also have Squid, Snort, and pfBlockerNG. Everything was working great.
Then a few days ago, we started having problems with streaming media. Netflix was blocking most titles. Amazon Prime wouldn't stream anything, and specifically gave a message saying, "You are using a VPN or blocking service...". Please note that I am NOT using a VPN, and never have. I tried loading a title from Netflix on my Android phone while connected to my network, and also got a message about using a VPN or blocking service. When I tried it with wifi off, it worked with no problems. Once again, I am NOT using a VPN.
I disabled Squid, Snort, and pfBlockerNG with no change. So I called my ISP, and eventually got a very helpful tech, who spent a lot of her time working on this with me. She said that they had changed some equipment a few days ago to comply with FCC regulations. She also said that the ISP's device is not required to connect, as long as the detected mac is correctly associated with your assigned IP. So I removed the spoofed IP from pfsense, and she changed the assigned mac in their system to the actual mac of my pfsense router's WAN nic. Bingo! Everything was back to working great again. No problems for the rest of the day.
The next day, we turned the TV on, and everything is back to the way it was. Netflix is blocking titles, and I'm getting the VPN or proxy message again.
I've tried everything that I can think of on my side. I've disabled services, and tried whitelisting Netflix and Amazon in dnsbl. I tried opening ports and creating aliases for firewall rules. Nothing has worked. Clearly, something is flagging me as connecting through a VPN, and I have no idea what it is. My ISP doesn't know, and they are kind of pointing the finger back at me, but I don't really accept that the problem is on my side, since everything was fine until some equipment changed on their end.
Does anyone have any idea of something else that I could try? Or better, something specific that I can say to my ISP that they need to check or do? Without the right instructions or verbiage, I don't think anything is going to get done. I'm kind of at my wits end, because streaming is our only source of media, and my whole family is unhappy.
Any help would be appreciated!
Thanks!
-
The first step in troubleshooting this on your end is to do the following to absolutely eliminate the possibility of the add-on packages causing you problems.
Go to the Snort INTERFACES tab and stop Snort on all interfaces. Then click the edit icon on each enabled Snort interface and uncheck the Enabled checkbox. Save the change to disable Snort so it won't start back up. Now go to the BLOCKS tab in Snort and click the button to remove, or clear, all blocked IP addresses.
Now go and disable Squid so that it will not auto-start.
Finally, do the same for pfBlockerNG. Make sure all three of these packages are set so they will NOT auto-start when the firewall boots.
Now reboot the firewall to be 100% sure everything gets reset to a known state. Test Netflix and Amazon Prime again. If it works, then you know one of those three packages is at fault, so you add them back one at the time to see which one breaks it again.
If it still does not work with all the packages disabled, I would take the extra step of removing all three of them from the firewall. Also reset the DNS Resolver to its default out-of-the-box state. Reboot the firewall again and test. If still not working, then you know it's something on your ISP's end at that point. So you will need to work with them to figure it out.
If it works with all the packages removed and the DNS Resolver restored to its out-of-the-box state, then again, you know one of the three packages and its configuration is likely to blame. So install them again one at the time, testing thoroughly with each package addition, to see which one breaks it.
You have three packages that all can cause things to be blocked. You are getting blocked (albeit with a "you are using a VPN" warning, which is not typical when things are blocked locally). So one logical assumption is one or more of your blocking packages is at fault. But your ISP admitted they have recently made changes as well. So could be their fault.
Did your WAN IP address change recently? Perhaps your ISP's address space is mistakenly marked by some list used by Netflix and Amazon as a "VPN provider" address block ???
-
Okay... I've done all of that except for resetting DNS Resolver. How do I reset it as you described?
-
Set it for "resolving", so under SERVICES > DNS RESOLVER, check the box for "Enable DNS Resolver". Likely that is already set, especially if you were formerly using DNSBL.
Make sure that the Custom Options box is cleared out. You may need to click the Display Custom Options button to see the box.
I only mentioned the DNS Resolver section because sometimes folks monkey with the defaults and wind up screwing things up.
-
Thanks. I tried all of the steps you gave me, including completely removing Squid, Snort, and pfBlockerNG. And cleared DNS Resolver.
No change.
It's simply GOT to be coming from the ISP side. I just emailed the support tech to see if she can give me a new IP address.
-
I agree. It's most likely on their end. But at least by doing all that I suggested you conclusively proved it's not any of the packages. Sorry for giving you all the homework, but sometimes it can be on the user's end.
That message you are getting really makes me think the IP subnet you have on the WAN is somehow on a "bad IPs" list that the streaming providers are using. Might be on there by mistake -- sort of like how someone's mail server can accidentally wind up on some list of email spammers. The fact it worked on your phone after you turned off Wi-Fi is another valuable clue. Without Wi-Fi enabled, you would be accessing the content on an IP address provided by your cell phone provider.
One last suggestion, track your WAN IP as you work with the ISP. See if it changes. Like maybe it changed yesterday when you worked with them to alter the MAC, but then overnight it "renewed" and reverted to the old IP.
You may already know this, but there is no "magic" used by Netflix and others to identify VPNs. They are just looking at the source IP address that is connecting to their service, and if that IP is within a netblock identified as belonging to a VPN provider, they block you and send you the error message. Thus why I'm saying maybe your ISP has some address space that is mistakenly identified as VPN space when it really isn't.
-
Got it. That sounds like the most likely explanation. And no problem about the work, It helps prove that it isn't a screwup on my end.
I asked for a new IP, and she's questioning whether they are seeing my pfsense router as a VPN, since they can't identify it as branded device.
-
@tomz said in Problems with Netflix and Amazon. NOT using a VPN.:
I asked for a new IP, and she's questioning whether they are seeing my pfsense router as a VPN, since they can't identify it as branded device.
Highly unlikely as thousands of pfSense users around the world would be impacted. I've had zero issues with Netflix. I've run pfSense on my own third-party hardware, and currently on a Netgate-branded appliance. Both worked fine with Netflix and Amazon prime.
I sense "grasping at straws" on the part of your ISP with that comeback ...
-
I agree. They gave me a new IP in a completely different pool, and now everything is working again!
Last time, it worked until the next day. I'm not going to touch anything on the router over the weekend, and see if it stays up.
Thank you for all your help. It really helped bolster my argument, and pointed me in the right direction to to negotiate with my ISP.
I'll post back with what happens.
-
@tomz said in Problems with Netflix and Amazon. NOT using a VPN.:
I agree. They gave me a new IP in a completely different pool, and now everything is working again!
Last time, it worked until the next day. I'm not going to touch anything on the router over the weekend, and see if it stays up.
Thank you for all your help. It really helped bolster my argument, and pointed me in the right direction to to negotiate with my ISP.
I'll post back with what happens.
Glad you got it sorted out. I would watch your firewall's WAN IP and see if it changes. If it does, and Netflix breaks again, you know the cause.
Your ISP might want to investigate the original IP netblock you were assigned. Perhaps it is on a VPN list by mistake, and if they use it with some of their other customers, they might have the same issue.
