[SOLVED]Help needed: bypass squid and squidGuard for iTunes, AppleStore, Android

slim2016

squid proxy server / general /[Transparent Proxy Settings] - Bypass Proxy for These Destination IPs

That should do the trick

GL

Hello ,
thanks for your reply but this is not the definitive solution.
apple is using akamai network and their servers ip are continuosly changing…
adding the IPs to squid works for a while, then you are back with the problem.

The IP s are also geolocated and changing country by country.

Is there a definitive set of squid directives to be added to squid.conf and squidGuard.conf to fix the problem forever?

After adding the Windows Update fixing to squidGuard.conf, I am sure this can be done also for Apple, but my knowledge of squid is not allowing me to do that alone.

Pls, help me and the rest of pfsense supporters who are also Apple users.
thans in advance.

Ciao,
GL

sichent

Not a user of transparent proxy but not sure why adding these as SNI bypass should not work on the recent Squid
https://docs.diladele.com/faq/squid/sslbump_exlusions/apple_app_store.html

GL

thanks for the reply, however actually the webgui allows you to add exclusions only with IPs, and because Apple is using Akamai, servers IP are floating and they are potentially hundreds…
we need to add the exclusion in the config files, as per the Windows Update workaround, the problem is that me and most of the not advanced users do not know how.
I tried to replicate that solution using the windows update workaround, but it is not working. I am missing something...
Is there anyone able to help us?
Thanks in advance.
Ciao,
GL

GL

I found the solution: simply there is no solution.
If you use Squid as https transparent proxy, the only info not encrypted that reaches Squid is the IP of the servers, so Squid has no opportunity to read the domain names of the servers.
If you want to use Squid on https, you must use it as explicit proxy.
This is also why in pfsense webgui you can exclude Squid only based on IPs and not domain names.

sichent

Not exactly correct; the version of Squid in pfSense is good enough to use the SNI info from the HTTPS connection being established.
The acl you need to use is ssl::server_name

For example this what we use when filtering all connections except spliced in Web Safety (does not apply to SquidGuard!!):

acl ssl_exclude_domains ssl::server_name "/opt/websafety/etc/squid/ssl/exclude/domains.conf"

ssl_bump peek step1 all
ssl_bump splice ssl_exclude_domains
ssl_bump stare step2 all
ssl_bump bump all

I am not sure where to put that in pfSense UI - but should not be very complex to find.

GL

thanks for your input, I will try this we and will let everybody know the outcome.
Cross our fingers…
I think this will be a very important input for most of the "home" users.
ciao,
GL

GL

I implemented the solution and for now it seems working, I am testing it.
here a step by step guide of what i implemented.

Go to Services->Squid Proxy Server
Enable and configure HTTPS transparent proxy
Go to the bottom of the page, click Show Advanced Options
Cut and past the following text in the box "Custom Options (Before Auth)":

acl ssl_exclude_domains ssl::server_name "/usr/local/etc/squid/exclude_domains.conf"

ssl_bump peek step1 all
ssl_bump splice ssl_exclude_domains
ssl_bump stare step2 all
ssl_bump bump all

wait to save
connect with secure shell to pfsense and login
choose option 8 "Shell"
cd /usr/local/etc/squid
ee exclude_domains.conf
input the following text :

.apple.com
.mzstatic.com
.icloud.com
.dropbox.com
.microsoft.com
.oneDrive.com
.live.com
.messenger.live.com
.skype.com
.trouter.com
.login.live.com
.whatsapp.com
.whatsapp.net

press "esc" then press "a", again press "a"
go back to pfsense webgui and save your squid configuration
restart squid service

The configuration in exclude_domains.conf should let you use your Apple devices with iTunes and Apple store, let you use WhatsApp, sync with iCloud (pls check also allowed ports in your firewall rules), sync with OneDrive, let your skype work with https transparent proxy, allow you to update with Microsoft in Win10.

I am still testing, if someone will implement it, pls let us know the outcomes.

GL

Adding support also for Android devices.

Add following entries to exclude_domains.conf file:

.ggpht.com
.play.googleapis.com
android.clients.google.com
www.googleapis.com
.gvt1.com

It seems working for me, up to now.

GL

Adding following entries to exclude_domains.conf you should bypass Netflix. I am still testing this.

.netflix.com
.llnwd.net
.edgesuite.net
.nflximg.com
.nflxvideo.net

GL

Up to today no main issues found.
Everything seems working well.
Thanks to sichent for his help and suggestion!

tapout72

@gl said in [SOLVED]Help needed: bypass squid and squidGuard for iTunes, AppleStore, Android:

acl ssl_exclude_domains ssl::server_name "/usr/local/etc/squid/exclude_domains.conf"
ssl_bump peek step1 all
ssl_bump splice ssl_exclude_domains
ssl_bump stare step2 all
ssl_bump bump all

GL, I know this is an old thread but I tried your solution to try and bypass netflix. However, I get a Privacy error NET::ERR_CERT_AUTHORITY_INVALID on every site I try to access after applying the Custom Options (Before Auth).

Any ideas on where I went wrong?

rob_kae

@tapout72 did you manage to sort this? I have the same problem.

vlurk

I implemented this solution in pfSense 2.7.0 and you can make this solution resilient to package updates by leveraging the whitelist in the ACL tab. In the default config of the package, you can already splice exceptions and bump everything by default. The alias of the acl it creates for sslbump is simply sslwhitelist. And you can reference it in custom options, so this solution would look like this:

ssl_bump peek step1 all
ssl_bump splice sslwhitelist
ssl_bump stare step2 all
ssl_bump bump all

Note that the ACL accepts regex! Which is what I use exclusively. Here is an example of a regex to match microsoft.com domain and subdomains:

(^|.)microsoft.com$

I now get all the domains I have defined logged as TCP_TUNNEL, so I can tell this is working well too.

azmodeuz

@vlurk Thank you for this guide. I have the same issue but with Viber. How can I use your settings for viber desktop App