• Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login
Netgate Discussion Forum
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login

Problem accessing Spotify via web browser app through Squid

Cache/Proxy
5
12
1.2k
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • M
    michmoor LAYER 8 Rebel Alliance @oldschoolrouterjockey
    last edited by Feb 14, 2024, 9:40 PM

    @oldschoolrouterjockey
    You're most likely MITM yourself. I get it, want to peek inside a TLS session but some sites (majority) wont work for numerous reasons such as certificate pinning.
    Either way you need to whitelist the site. Thats it.

    Firewall: NetGate,Palo Alto-VM,Juniper SRX
    Routing: Juniper, Arista, Cisco
    Switching: Juniper, Arista, Cisco
    Wireless: Unifi, Aruba IAP
    JNCIP,CCNP Enterprise

    O 1 Reply Last reply Feb 14, 2024, 9:57 PM Reply Quote 0
    • O
      oldschoolrouterjockey @michmoor
      last edited by Feb 14, 2024, 9:57 PM

      @michmoor actually nothing as extensive as that; I'm just doing a simple squid with the Transparent HTTP Proxy option set, and not setting the HTTPS/SSL Interception, so not doing any MITM. or at least I think not. I mean non of my banking sites or anything TLS related ever complains about invalid certs.

      🔒 Log in to view

      M 1 Reply Last reply Feb 14, 2024, 10:06 PM Reply Quote 0
      • M
        michmoor LAYER 8 Rebel Alliance @oldschoolrouterjockey
        last edited by Feb 14, 2024, 10:06 PM

        @oldschoolrouterjockey
        Yep , transparent proxy is also an issue as well.

        1. Squid Proxy will be depreciated in future releases on pfSense so its best to start moving away from it. see here

        2. Transparent proxy on pfsense is currently broken. See here

        Your option is to create an alias, call it 'BypassProxy' and put the spotify IP(s) in there. Then go to your Squid configuration and in the 'Bypass Proxy for These' put in the alias you created. You dont want that traffic going through a proxy. Also ensure that you have port 443 open in your firewall rules so that bypassed traffic can go outbound.

        Firewall: NetGate,Palo Alto-VM,Juniper SRX
        Routing: Juniper, Arista, Cisco
        Switching: Juniper, Arista, Cisco
        Wireless: Unifi, Aruba IAP
        JNCIP,CCNP Enterprise

        O J 2 Replies Last reply Feb 15, 2024, 7:18 PM Reply Quote 0
        • O
          oldschoolrouterjockey @michmoor
          last edited by Feb 15, 2024, 7:18 PM

          @michmoor
          OK I did all that, I added a ton of different spotify hosts I found in a HAR trace to the firewall alias I created then added that alias to the bypass. It didnt really help anything though so I then added my local ip address to the bypass for the source IP option. Still didnt help.

          (im going to make a new post as now I have a TON more questions on squid/proxy/transparent in general)

          So I just disabled squid completely and went back to trying to use the local dynamic socks option via ssh to my local pfsense, and found that sometimes I can get logged in. Sometimes. But then after I do get logged then then things only seem to sort of work, sort of. It takes a few minutes after clicking play on something for its audio to actually start. Then if a commercial ever comes on, after its over it takes a few mins to start playing again. Its just really strange and is making me question all I think I know about networking, proxy, etc.

          So even using the dynamic proxy, somehow they know? Or maybe is not so much as "they know and are intentionally trying to cause problems when people use proxies" as much as something is just weird going on?

          M G 2 Replies Last reply Feb 15, 2024, 8:07 PM Reply Quote 0
          • M
            michmoor LAYER 8 Rebel Alliance @oldschoolrouterjockey
            last edited by Feb 15, 2024, 8:07 PM

            @oldschoolrouterjockey

            Forward proxies being notoriously difficult to troubleshoot and are known to be problematic in web browsing do initially seem like the culprit in your setup. As you said you added the IPs to the Bypass List and it still isnt working correctly therefore it isnt Squid causing an issue. My next question would be do you have any browser extensions that could be at play here? What happens when you go into incognito mode in the browser?

            Firewall: NetGate,Palo Alto-VM,Juniper SRX
            Routing: Juniper, Arista, Cisco
            Switching: Juniper, Arista, Cisco
            Wireless: Unifi, Aruba IAP
            JNCIP,CCNP Enterprise

            O 1 Reply Last reply Feb 16, 2024, 4:57 PM Reply Quote 0
            • G
              Gertjan @oldschoolrouterjockey
              last edited by Feb 16, 2024, 6:54 AM

              @oldschoolrouterjockey said in Problem accessing Spotify via web browser app through Squid:

              (im going to make a new post as now I have a TON more questions on squid/proxy/transparent in general)

              With this update upcoming : Deprecation of Squid Add-On Package For pfSense Software ? 😊

              No "help me" PM's please. Use the forum, the community will thank you.
              Edit : and where are the logs ??

              1 Reply Last reply Reply Quote 1
              • O
                oldschoolrouterjockey @michmoor
                last edited by Feb 16, 2024, 4:57 PM

                @michmoor said in Problem accessing Spotify via web browser app through Squid:

                @oldschoolrouterjockey

                Forward proxies being notoriously difficult to troubleshoot and are known to be problematic in web browsing do initially seem like the culprit in your setup. As you said you added the IPs to the Bypass List and it still isnt working correctly therefore it isnt Squid causing an issue. My next question would be do you have any browser extensions that could be at play here? What happens when you go into incognito mode in the browser?

                OK So I loaded a brand new FF profile, zero extensions on it, then set the FF config proxy settings to socks (localhost:3128 to correspond to my local SSH session to my netgate), also set the proxy dnx when using socksv5. I can login to spotify just fine, but still takes nearly 2 minutes after hitting play to actually have a stream of music. I can then pause/play instantaneously, but forwarding to the next song again gets me the 2 minute wait.

                With the corp vpn disconnected and FF proxy settings set to direct, it all works fine as youd expect.

                M 1 Reply Last reply Feb 16, 2024, 6:48 PM Reply Quote 0
                • M
                  michmoor LAYER 8 Rebel Alliance @oldschoolrouterjockey
                  last edited by Feb 16, 2024, 6:48 PM

                  @oldschoolrouterjockey
                  So the problem isnt with pfSense or Squid.
                  Im sure you have a specific use case to go through a socks proxy but that could be whats causing you the pain.
                  Or you are visiting the site while on the corp vpn.
                  In either case a direct connection through pfsense doesnt cause an issue it seems.

                  Firewall: NetGate,Palo Alto-VM,Juniper SRX
                  Routing: Juniper, Arista, Cisco
                  Switching: Juniper, Arista, Cisco
                  Wireless: Unifi, Aruba IAP
                  JNCIP,CCNP Enterprise

                  1 Reply Last reply Reply Quote 0
                  • E
                    enzo5
                    last edited by Dec 23, 2024, 8:17 AM

                    If Spotify isn't working through Squid, ensure its domains are allowed, skip SSL bumping for those domains, and check the Squid logs for any issues. You can also clear the cache and update Squid to the latest version. If the problem persists, consider bypassing Squid temporarily or using the Spotify desktop app instead.

                    1 Reply Last reply Reply Quote 1
                    • J
                      JonathanLee @oldschoolrouterjockey
                      last edited by JonathanLee Dec 23, 2024, 7:44 PM Dec 23, 2024, 7:30 PM

                      @oldschoolrouterjockey

                      I hope this helps,

                      Squid has been updated upstream to fix all the concerns. Again, some sites do certificate pinning.

                      Squid has been updated to address the security concerns that Netgate was concerned about in the past.

                      All you need to do to fix this is add some custom settings, Keep in mind both michmoor and I do not work for Netgate I wish I did but I am in school still. I do know that the depreciated notice was sent long before Squid dev team fixed the issues a couple versions ago. I personally purchased my system because at the time the web cache was advertised.

                      To fix this add the URL requests domain to the do not cache list on squid and also create a custom list, or you can attempt to enable what is called tproxy, you will have to research that I have used that in the past it may not be on the new version of Squid.

                      Try this create a list of domain issues save it as a text file in some location on the firewall.
                      I did /usr/local/pkg/
                      Inside the text file I add the sites that cause issues,

                      acl BrokenButTrustedServers dstdomain "/usr/local/pkg/dstdom.broken"
                      acl DomainMismatch ssl_error SQUID_X509_V_ERR_DOMAIN_MISMATCH
                      sslproxy_cert_error allow BrokenButTrustedServers DomainMismatch
                      sslproxy_cert_error deny all
                      

                      or you can create your pass lists like this

                      acl NoSSLIntercept ssl::server_name_regex -i "/usr/local/pkg/reg.url.nobump"
                      acl NoBumpDNS dstdomain "/usr/local/pkg/dns.nobump"
                      acl SSL_Intercept_Terminate dstdomain "/usr/local/pkg/url.bump"
                      acl block_hours time 00:30-05:00
                      ssl_bump terminate all block_hours
                      http_access deny all block_hours
                      acl getmethod method GET
                      acl to_ipv6 dst ipv6 #my ISP does not provide ipv6 I have a different boot environment for hurricane tunnel broker 
                      acl from_ipv6 src ipv6
                      
                      acl HttpAccess dstdomain "/usr/local/pkg/http.access"
                      acl windowsupdate dstdomain "/usr/local/pkg/windowsupdate"
                      acl https_login url_regex -i ^https.*(login|Login).* #do not save or look at anything with login in it do not even miss cache it
                      cache deny https_login
                      
                      
                      acl splice_only src IP ADDRESS PLACE HERE
                      
                      acl splice_only_mac arp MAC ADDRESS PLACE HERE
                      
                      acl active_use annotate_client active=true
                      
                      acl bump_only src IP ADDRESS PLACE HERE
                      
                      acl bump_only_mac arp MAC ADDRESS PLACE HERE
                      
                      collapsed_forwarding on
                      negative_dns_ttl 5 minutes
                      shutdown_lifetime 1 seconds
                      #pipeline_prefetch 50 Do Not Use this causes risk at times 
                      happy_eyeballs_connect_timeout 10
                      memory_pools on
                      digest_generation off
                      
                      ssl_bump peek step1
                      ssl_bump terminate SSL_Intercept_Terminate
                      miss_access deny no_miss active_use
                      ssl_bump splice https_login active_use
                      ssl_bump splice splice_only_mac splice_only active_use
                      ssl_bump splice NoBumpDNS active_use
                      ssl_bump splice NoSSLIntercept active_use
                      ssl_bump bump bump_only_mac bump_only active_use
                      acl activated note active_use true
                      ssl_bump terminate !activated
                      

                      For this example, configuration you are going to need certificates installed on the devices.

                      Squid works really well, as it does not use DNS requests to inspect traffic and block, it's an amazing set of software. Squid has a support email also. Squid is no joke it needs to be respected. I hope this helps.

                      Also make sure your webcache does not call out to a wpad for security if you are on the proxy do not use wpad anymore. Added security.

                      acl wpad urlpath_regex ^/wpad.dat$
                      acl wpad urlpath_regex ^/proxy.pac$
                      acl wpad urlpath_regex ^/wpad.da$
                      deny_info TCP_RESET wpad
                      reply_header_access Content-Type deny wpad
                      http_access deny wpad
                      http_access deny !safeports
                      http_access deny CONNECT !sslports
                      http_access allow localhost manager
                      http_access deny manager
                      #cachemgr_passwd disable offline_toggle reconfigure shutdown
                      #cachemgr_passwd CLASSIFIED PASSWORD WAS HERE would require php adaption to use squidlite
                      eui_lookup on
                      acl no_miss url_regex -i gateway\.facebook\.com\/ws\/realtime\?
                      acl no_miss url_regex -i web-chat-e2ee\.facebook\.com\/ws\/chat #do not cache or save any chat anything
                      acl CONNECT method CONNECT
                      acl wuCONNECT dstdomain www.update.microsoft.com #http connect
                      acl wuCONNECT dstdomain sls.microsoft.com
                      http_access allow CONNECT wuCONNECT localnet
                      http_access allow CONNECT wuCONNECT localhost
                      http_access allow CONNECT windowsupdate localnet
                      http_access allow CONNECT windowsupdate localhost
                      http_access allow CONNECT HttpAccess localnet
                      http_access allow CONNECT HttpAccess localhost
                      http_access deny manager
                      http_access deny to_ipv6
                      http_access deny from_ipv6
                      

                      It is complex and took me years to get it to work, I will never pretend I know everything about it. I am in school as a full computer science major student. Again, I am a student, so this was the only way to learn about proxy code for me was to start tinkering with this. Most everything for proxy is closed source that works like this, this was used as an educational tool for cybersecurity for me. I am sure there was many times I had messed up items. I have learned a lot and still do. At this point some websites are slow for me and others are lighting fast.

                      It is like a tank, keep in mind the config is pulled from several places. ACLS come before the bump slice stuff always I got some mixed up in this post, some of my devices few are always bump everything and others are splice always and do not inspect anything at all stuff that is not mine is bypassed that way.

                      Make sure to upvote

                      1 Reply Last reply Reply Quote 0
                      • J
                        JonathanLee @michmoor
                        last edited by Dec 23, 2024, 7:46 PM

                        @michmoor You ever configure Squid with http tproxy ? I tested this it was amazing again any reboot or enable reset puts it back to the old way.

                        Make sure to upvote

                        1 Reply Last reply Reply Quote 0
                        • First post
                          Last post
                        Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.