Squid + Disney/ABC apps on iOS = no worky!

AirCooledTiger

Hi, all -

My kids have been bugging me for some time now that they can't use certain apps like DisneyXD or the ABC video on demand app on their iOS devices. The error message they were getting was "you appear to be using the app from outside the US" and went on to say that it isn't allowed. I finally started looking into it and, long story short, I found out Disney/ABC blocks access to content that is set via a proxy server! Apparently they've had people using proxies to get around their digital rights restrictions so now they just block any and all traffic coming from a proxy.

I have been running pfSense 2.0.3 and squid in transparent proxy mode for at least a year now with no real problems until encountering this. I tried changing some squid parameters to try to hide the fact that I'm using a proxy but it didn't work. Anyone have any ideas on how to fool Disney's proxy check so I can re-enable squid on my network?

Tikimotel

Have you tried to add Disney stuff to a "proxy.pac" file?

My "proxy.pac" file served by pfSense on port 80 (/usr/local/www/proxy.pac)

function FindProxyForURL(url, host) {

  url = url.toLowerCase();
  host = host.toLowerCase();
	hostip=dnsResolve(host);
	isHttp=(url.substring(0,5) == "http:");
	isHttps=(url.substring(0,6) == "https:")

	// Always bypass local
	if(0
		|| isPlainHostName(host)
		|| isInNet(hostip, "10.0.0.0", "255.0.0.0") 
		|| isInNet(hostip, "172.16.0.0", "255.240.0.0") 
		|| isInNet(hostip, "192.168.0.0", "255.255.0.0")
		|| isInNet(hostip, "127.0.0.0", "255.255.255.0")
	) { return "DIRECT"; }

	// Forward non-http(s) and some hosts to forward proxy (or DIRECT)
	if(0
		|| (!isHttp && !isHttps) // Skip all non http(s)
		|| dnsDomainIs(host, "microsoft.com")
		|| dnsDomainIs(host, "windowsupdate.com")
		|| dnsDomainIs(host, "eset.com")
		|| dnsDomainIs(host, "mcafee.com") // McAfee
		|| dnsDomainIs(host, "siteadvisor.com") // McAfee
		|| dnsDomainIs(host, "hackerwatch.com") // McAfee
		|| dnsDomainIs(host, "hackerwatch.org") // McAfee
		|| dnsDomainIs(host, "avg.com")
		|| dnsDomainIs(host, "grisoft.cz")
		|| dnsDomainIs(host, "avgfree.com")
		|| dnsDomainIs(host, "avg.cz")
		|| dnsDomainIs(host, "symantecliveupdate.com")
		|| dnsDomainIs(host, "thawte.com")

	) { return "DIRECT"; }

	if(isHttps) {
		// Skip HTTPS
		return "DIRECT";
	}

	// Otherwise, go through our proxy or if it fails, through bypass
	return "PROXY 192.168.0.1:3128; DIRECT";
}

So maybe you can add the Disney stuff to connect "DIRECT" (and configure the devices to use the automatic proxy file?)

AirCooledTiger

@Tikimotel:

Have you tried to add Disney stuff to a "proxy.pac" file?

My "proxy.pac" file served by pfSense on port 80 (/usr/local/www/proxy.pac)

function FindProxyForURL(url, host) {

  url = url.toLowerCase();
  host = host.toLowerCase();
	hostip=dnsResolve(host);
	isHttp=(url.substring(0,5) == "http:");
	isHttps=(url.substring(0,6) == "https:")

	// Always bypass local
	if(0
		|| isPlainHostName(host)
		|| isInNet(hostip, "10.0.0.0", "255.0.0.0") 
		|| isInNet(hostip, "172.16.0.0", "255.240.0.0") 
		|| isInNet(hostip, "192.168.0.0", "255.255.0.0")
		|| isInNet(hostip, "127.0.0.0", "255.255.255.0")
	) { return "DIRECT"; }

	// Forward non-http(s) and some hosts to forward proxy (or DIRECT)
	if(0
		|| (!isHttp && !isHttps) // Skip all non http(s)
		|| dnsDomainIs(host, "microsoft.com")
		|| dnsDomainIs(host, "windowsupdate.com")
		|| dnsDomainIs(host, "eset.com")
		|| dnsDomainIs(host, "mcafee.com") // McAfee
		|| dnsDomainIs(host, "siteadvisor.com") // McAfee
		|| dnsDomainIs(host, "hackerwatch.com") // McAfee
		|| dnsDomainIs(host, "hackerwatch.org") // McAfee
		|| dnsDomainIs(host, "avg.com")
		|| dnsDomainIs(host, "grisoft.cz")
		|| dnsDomainIs(host, "avgfree.com")
		|| dnsDomainIs(host, "avg.cz")
		|| dnsDomainIs(host, "symantecliveupdate.com")
		|| dnsDomainIs(host, "thawte.com")
		
	) { return "DIRECT"; }

	if(isHttps) {
		// Skip HTTPS
		return "DIRECT";
	}

	// Otherwise, go through our proxy or if it fails, through bypass
	return "PROXY 192.168.0.1:3128; DIRECT";
}

So maybe you can add the Disney stuff to connect "DIRECT" (and configure the devices to use the automatic proxy file?)

Hmmm… The problem is I have no way of knowing what netblocks the content is originating from. It could be hosted by Akamai or some other CDN so IPs could be constantly changing. For the moment I have disabled squid. Two things happened: 1) web performance improved significantly and 2) my wife noticed that content we weren't able to access on our Samsung smart TV now works. I suspect others will experience this same problem.

doktornotor

@AirCooledTiger:

web performance improved significantly

Sounds like good reason to leave it disabled forever. As for the OP:

kejianshi

Try this. Go to a website that checks for proxy's like:
http://www.lagado.com/proxy-test

You should get something that shows if there is a proxy in use on your system.

I suspect that the services you want to use are reading the forwarder info that is default for squid.
Try this:

Services > Proxy server > General settings

Disable X-Forward

Disable VIA

Now go back and try again the proxy check.
http://www.lagado.com/proxy-test

By disabling X-Forward and VIA pfsense will give all websites the impression that not only are you not using a proxy but that also your computer is plugged straight into the internet with no NAT. Your public IP and your basic browser info is all they will see. They will happily serve you up content now. Assuming your public IP is in the USA.
In all cases, I see no advantage for you in someone knowing you are using a proxy, so I just disable those settings.

This is a very long way of saying, I think disabling X-Forward and Via will fix all your problems and allow you to keep using squid.
To verify this, I'm watching "Austin & Alley" on watchdisneychannel.go.com/austin–ally
and http://watchdisneyxd.go.com/kickin-it
on my LAN with transparent squid proxy

And now I will get off this channel before I get sick.

AirCooledTiger

@kejianshi:

Try this. Go to a website that checks for proxy's like:
http://www.lagado.com/proxy-test

You should get something that shows if there is a proxy in use on your system.

I suspect that the services you want to use are reading the forwarder info that is default for squid.
Try this:

Services > Proxy server > General settings

Disable X-Forward

Disable VIA

Now go back and try again the proxy check.
http://www.lagado.com/proxy-test

By disabling X-Forward and VIA pfsense will give all websites the impression that not only are you not using a proxy but that also your computer is plugged straight into the internet with no NAT. Your public IP and your basic browser info is all they will see. They will happily serve you up content now. Assuming your public IP is in the USA.
In all cases, I see no advantage for you in someone knowing you are using a proxy, so I just disable those settings.

This is a very long way of saying, I think disabling X-Forward and Via will fix all your problems and allow you to keep using squid.
To verify this, I'm watching "Austin & Alley" on watchdisneychannel.go.com/austin–ally
and http://watchdisneyxd.go.com/kickin-it
on my LAN with transparent squid proxy

And now I will get off this channel before I get sick.

Thanks! I know I tried disabling one of those but maybe I didn't try both. I will give this a try and report back.

jimp

@doktornotor:

As for the OP:

http://i41.tinypic.com/2ynmbf6.jpg

Perry the Platypus frowns upon such blasphemy.