Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    [SOLVED]Help needed: bypass squid and squidGuard for iTunes, AppleStore, Android

    Scheduled Pinned Locked Moved Cache/Proxy
    17 Posts 7 Posters 9.2k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • G
      GL
      last edited by

      Hello All,
      I have been able to apply the Windows Upgrade bypass without problems in the SquidGuard.conf file, so that all my Win devices can update bypassing Squid (running with https transparent proxy and squidGuard).

      I have problems with my Apple deivces which cannot access itunes, apple store nor upgrade.
      The only solution found on the net is to list Apple ip addresses in the squid webgui.

      Apple, however, is using Akamai network, as Microsoft, and trying to bypass squid via IP addresses is pratically impossible (Akamai network is huge). If you input some Akamai address you find via nslookup in squid bypass, the patch is working for a while before you will be redirected to some other server.

      I am sure there exist a way to let Apple devices bypass squid and squidGuard via squid.conf and squidGuard.conf files, but I have not been able to reach the result. The only success for me has been to stop squid from restarting up to when I did not restore the old squid.conf file…

      Is there anybody who can tell us how, what and where we should add/modify in the squid.conf and SquidGuard.conf files, as per the Wind Update bypass, to get the result ( ie bypassing squid when addressing apple web sites)?

      Thanks in advance to the nice and kind guy who will give us the instructions.
      I am sure that your answer will become one of the most viewed posts on this board....

      Ciao,

      GL

      1 Reply Last reply Reply Quote 0
      • G
        GL
        last edited by

        just un update: I would like to bypass squid only when reaching itunes and AppleStore for upgrading and/or downloading new apps.
        during normal navigation I would like to be proxied by squid.

        If I remove SSL transparet proxy option and leave squid as http transparent proxy only, everything works fine.

        1 Reply Last reply Reply Quote 0
        • S
          slim2016
          last edited by

          squid proxy server /  general /[Transparent Proxy Settings] - Bypass Proxy for These Destination IPs

          That should do the trick

          1 Reply Last reply Reply Quote 0
          • G
            GL
            last edited by

            Hello ,
            thanks for your reply but this is not the definitive solution.
            apple is using akamai network and their servers ip are continuosly changing…
            adding the IPs to squid works for a while, then you are back with the problem.

            The IP s are also geolocated and changing country by country.

            Is there a definitive set of squid directives to be added to squid.conf and squidGuard.conf to fix the problem forever?

            After adding the Windows Update fixing to squidGuard.conf, I am sure this can be done also for Apple, but my knowledge of squid  is not allowing me to do that alone.

            Pls, help me and the rest of pfsense supporters who are also Apple users.
            thans in advance.

            Ciao,
            GL

            1 Reply Last reply Reply Quote 0
            • S
              sichent Banned
              last edited by

              Not a user of transparent proxy but not sure why adding these as SNI bypass should not work on the recent Squid
              https://docs.diladele.com/faq/squid/sslbump_exlusions/apple_app_store.html

              1 Reply Last reply Reply Quote 0
              • G
                GL
                last edited by

                thanks for the reply, however actually the webgui allows you to add exclusions only with IPs, and because Apple is using Akamai, servers IP are floating and they are potentially hundreds…
                we need to add the exclusion in the config files, as per the Windows Update workaround, the problem is that me and most of the not advanced users do not know how.
                I tried to replicate that solution using the windows update workaround, but it is not working. I am missing something...
                Is there anyone able to help us?
                Thanks in advance.
                Ciao,
                GL

                1 Reply Last reply Reply Quote 0
                • G
                  GL
                  last edited by

                  I found the solution: simply there is no solution.
                  If you use Squid as https transparent proxy, the only info not encrypted that reaches Squid is the IP of the servers, so Squid has no opportunity to read the domain names of the servers.
                  If you want to use Squid on https, you must use it as explicit proxy.
                  This is also why in pfsense webgui you can exclude Squid only based on IPs and not domain names.

                  1 Reply Last reply Reply Quote 0
                  • S
                    sichent Banned
                    last edited by

                    Not exactly correct; the version of Squid in pfSense is good enough to use the SNI info from the HTTPS connection being established.
                    The acl you need to use is ssl::server_name

                    For example this what we use when filtering all connections except spliced in Web Safety (does not apply to SquidGuard!!):

                    acl ssl_exclude_domains ssl::server_name "/opt/websafety/etc/squid/ssl/exclude/domains.conf"

                    ssl_bump peek step1 all
                    ssl_bump splice ssl_exclude_domains
                    ssl_bump stare step2 all
                    ssl_bump bump all

                    I am not sure where to put that in pfSense UI - but should not be very complex to find.

                    1 Reply Last reply Reply Quote 0
                    • G
                      GL
                      last edited by

                      thanks for your input, I will try this we and will let everybody know the outcome.
                      Cross our fingers…
                      I think this will be a very important input for most of the "home" users.
                      ciao,
                      GL

                      1 Reply Last reply Reply Quote 0
                      • G
                        GL
                        last edited by

                        I implemented the solution and for now it seems working, I am testing it.
                        here a step by step guide of what i implemented.

                        1. Go to Services->Squid Proxy Server

                        2. Enable and configure HTTPS transparent proxy

                        3. Go to the bottom of the page, click Show Advanced Options

                        4. Cut and past the following text in the box "Custom Options (Before Auth)":

                        acl ssl_exclude_domains ssl::server_name "/usr/local/etc/squid/exclude_domains.conf"

                        ssl_bump peek step1 all
                        ssl_bump splice ssl_exclude_domains
                        ssl_bump stare step2 all
                        ssl_bump bump all

                        1. wait to save

                        2. connect with secure shell to pfsense and login

                        3. choose option 8 "Shell"

                        4. cd /usr/local/etc/squid

                        5. ee exclude_domains.conf

                        6. input the following text :

                        .apple.com
                        .mzstatic.com
                        .icloud.com
                        .dropbox.com
                        .microsoft.com
                        .oneDrive.com
                        .live.com
                        .messenger.live.com
                        .skype.com
                        .trouter.com
                        .login.live.com
                        .whatsapp.com
                        .whatsapp.net

                        1. press "esc" then press "a", again press "a"

                        2. go back to pfsense webgui and save your squid configuration

                        3. restart squid service

                        The configuration in exclude_domains.conf should let you use your Apple devices with iTunes and Apple store, let you use WhatsApp, sync with iCloud (pls check also allowed ports in your firewall rules), sync with OneDrive, let your skype work with https transparent proxy, allow you to update with Microsoft in Win10.

                        I am still testing, if someone will implement it, pls let us know the outcomes.

                        V 1 Reply Last reply Reply Quote 2
                        • G
                          GL
                          last edited by

                          Adding support also for Android devices.

                          Add following entries to exclude_domains.conf file:

                          .ggpht.com
                          .play.googleapis.com
                          android.clients.google.com
                          www.googleapis.com
                          .gvt1.com

                          It seems working for me, up to now.

                          1 Reply Last reply Reply Quote 0
                          • G
                            GL
                            last edited by

                            Adding following entries to exclude_domains.conf you should bypass Netflix. I am still testing this.

                            .netflix.com
                            .llnwd.net
                            .edgesuite.net
                            .nflximg.com
                            .nflxvideo.net

                            1 Reply Last reply Reply Quote 0
                            • G
                              GL
                              last edited by

                              Up to today no main issues found.
                              Everything seems working well.
                              Thanks to sichent for his help and suggestion!

                              1 Reply Last reply Reply Quote 0
                              • T
                                tapout72
                                last edited by

                                @gl said in [SOLVED]Help needed: bypass squid and squidGuard for iTunes, AppleStore, Android:

                                acl ssl_exclude_domains ssl::server_name "/usr/local/etc/squid/exclude_domains.conf"
                                ssl_bump peek step1 all
                                ssl_bump splice ssl_exclude_domains
                                ssl_bump stare step2 all
                                ssl_bump bump all

                                GL, I know this is an old thread but I tried your solution to try and bypass netflix. However, I get a Privacy error NET::ERR_CERT_AUTHORITY_INVALID on every site I try to access after applying the Custom Options (Before Auth).

                                Any ideas on where I went wrong?

                                R 1 Reply Last reply Reply Quote 0
                                • R
                                  rob_kae @tapout72
                                  last edited by

                                  @tapout72 did you manage to sort this? I have the same problem.

                                  1 Reply Last reply Reply Quote 0
                                  • V
                                    vlurk @GL
                                    last edited by

                                    I implemented this solution in pfSense 2.7.0 and you can make this solution resilient to package updates by leveraging the whitelist in the ACL tab. In the default config of the package, you can already splice exceptions and bump everything by default. The alias of the acl it creates for sslbump is simply sslwhitelist. And you can reference it in custom options, so this solution would look like this:

                                    ssl_bump peek step1 all
                                    ssl_bump splice sslwhitelist
                                    ssl_bump stare step2 all
                                    ssl_bump bump all

                                    Note that the ACL accepts regex! Which is what I use exclusively. Here is an example of a regex to match microsoft.com domain and subdomains:

                                    (^|.)microsoft.com$

                                    I now get all the domains I have defined logged as TCP_TUNNEL, so I can tell this is working well too.

                                    A 1 Reply Last reply Reply Quote 1
                                    • A
                                      azmodeuz @vlurk
                                      last edited by

                                      @vlurk Thank you for this guide. I have the same issue but with Viber. How can I use your settings for viber desktop App

                                      1 Reply Last reply Reply Quote 0
                                      • First post
                                        Last post
                                      Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.