[SOLVED]Help needed: bypass squid and squidGuard for iTunes, AppleStore, Android



  • Hello All,
    I have been able to apply the Windows Upgrade bypass without problems in the SquidGuard.conf file, so that all my Win devices can update bypassing Squid (running with https transparent proxy and squidGuard).

    I have problems with my Apple deivces which cannot access itunes, apple store nor upgrade.
    The only solution found on the net is to list Apple ip addresses in the squid webgui.

    Apple, however, is using Akamai network, as Microsoft, and trying to bypass squid via IP addresses is pratically impossible (Akamai network is huge). If you input some Akamai address you find via nslookup in squid bypass, the patch is working for a while before you will be redirected to some other server.

    I am sure there exist a way to let Apple devices bypass squid and squidGuard via squid.conf and squidGuard.conf files, but I have not been able to reach the result. The only success for me has been to stop squid from restarting up to when I did not restore the old squid.conf file…

    Is there anybody who can tell us how, what and where we should add/modify in the squid.conf and SquidGuard.conf files, as per the Wind Update bypass, to get the result ( ie bypassing squid when addressing apple web sites)?

    Thanks in advance to the nice and kind guy who will give us the instructions.
    I am sure that your answer will become one of the most viewed posts on this board....

    Ciao,

    GL



  • just un update: I would like to bypass squid only when reaching itunes and AppleStore for upgrading and/or downloading new apps.
    during normal navigation I would like to be proxied by squid.

    If I remove SSL transparet proxy option and leave squid as http transparent proxy only, everything works fine.



  • squid proxy server /  general /[Transparent Proxy Settings] - Bypass Proxy for These Destination IPs

    That should do the trick



  • Hello ,
    thanks for your reply but this is not the definitive solution.
    apple is using akamai network and their servers ip are continuosly changing…
    adding the IPs to squid works for a while, then you are back with the problem.

    The IP s are also geolocated and changing country by country.

    Is there a definitive set of squid directives to be added to squid.conf and squidGuard.conf to fix the problem forever?

    After adding the Windows Update fixing to squidGuard.conf, I am sure this can be done also for Apple, but my knowledge of squid  is not allowing me to do that alone.

    Pls, help me and the rest of pfsense supporters who are also Apple users.
    thans in advance.

    Ciao,
    GL


  • Banned

    Not a user of transparent proxy but not sure why adding these as SNI bypass should not work on the recent Squid
    https://docs.diladele.com/faq/squid/sslbump_exlusions/apple_app_store.html



  • thanks for the reply, however actually the webgui allows you to add exclusions only with IPs, and because Apple is using Akamai, servers IP are floating and they are potentially hundreds…
    we need to add the exclusion in the config files, as per the Windows Update workaround, the problem is that me and most of the not advanced users do not know how.
    I tried to replicate that solution using the windows update workaround, but it is not working. I am missing something...
    Is there anyone able to help us?
    Thanks in advance.
    Ciao,
    GL



  • I found the solution: simply there is no solution.
    If you use Squid as https transparent proxy, the only info not encrypted that reaches Squid is the IP of the servers, so Squid has no opportunity to read the domain names of the servers.
    If you want to use Squid on https, you must use it as explicit proxy.
    This is also why in pfsense webgui you can exclude Squid only based on IPs and not domain names.


  • Banned

    Not exactly correct; the version of Squid in pfSense is good enough to use the SNI info from the HTTPS connection being established.
    The acl you need to use is ssl::server_name

    For example this what we use when filtering all connections except spliced in Web Safety (does not apply to SquidGuard!!):

    acl ssl_exclude_domains ssl::server_name "/opt/websafety/etc/squid/ssl/exclude/domains.conf"

    ssl_bump peek step1 all
    ssl_bump splice ssl_exclude_domains
    ssl_bump stare step2 all
    ssl_bump bump all

    I am not sure where to put that in pfSense UI - but should not be very complex to find.



  • thanks for your input, I will try this we and will let everybody know the outcome.
    Cross our fingers…
    I think this will be a very important input for most of the "home" users.
    ciao,
    GL



  • I implemented the solution and for now it seems working, I am testing it.
    here a step by step guide of what i implemented.

    1. Go to Services->Squid Proxy Server

    2. Enable and configure HTTPS transparent proxy

    3. Go to the bottom of the page, click Show Advanced Options

    4. Cut and past the following text in the box "Custom Options (Before Auth)":

    acl ssl_exclude_domains ssl::server_name "/usr/local/etc/squid/exclude_domains.conf"

    ssl_bump peek step1 all
    ssl_bump splice ssl_exclude_domains
    ssl_bump stare step2 all
    ssl_bump bump all

    1. wait to save

    2. connect with secure shell to pfsense and login

    3. choose option 8 "Shell"

    4. cd /usr/local/etc/squid

    5. ee exclude_domains.conf

    6. input the following text :

    .apple.com
    .mzstatic.com
    .icloud.com
    .dropbox.com
    .microsoft.com
    .oneDrive.com
    .live.com
    .messenger.live.com
    .skype.com
    .trouter.com
    .login.live.com
    .whatsapp.com
    .whatsapp.net

    1. press "esc" then press "a", again press "a"

    2. go back to pfsense webgui and save your squid configuration

    3. restart squid service

    The configuration in exclude_domains.conf should let you use your Apple devices with iTunes and Apple store, let you use WhatsApp, sync with iCloud (pls check also allowed ports in your firewall rules), sync with OneDrive, let your skype work with https transparent proxy, allow you to update with Microsoft in Win10.

    I am still testing, if someone will implement it, pls let us know the outcomes.



  • Adding support also for Android devices.

    Add following entries to exclude_domains.conf file:

    .ggpht.com
    .play.googleapis.com
    android.clients.google.com
    www.googleapis.com
    .gvt1.com

    It seems working for me, up to now.



  • Adding following entries to exclude_domains.conf you should bypass Netflix. I am still testing this.

    .netflix.com
    .llnwd.net
    .edgesuite.net
    .nflximg.com
    .nflxvideo.net



  • Up to today no main issues found.
    Everything seems working well.
    Thanks to sichent for his help and suggestion!