Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Squid StoreID and Facebook plus caching Windows updates

    Scheduled Pinned Locked Moved Cache/Proxy
    13 Posts 1 Posters 1.9k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • JonathanLeeJ
      JonathanLee
      last edited by

      Small addition you must change the owership of the file so that Squid can use it.

      chgrp -Rf proxy /var/squid/storeid
      chown -Rf squid /var/squid/storeid
      chmod -Rf 775 /var/squid/storeid

      Or else it will say helper program exiting to fast and kill the cache

      Make sure to upvote

      1 Reply Last reply Reply Quote 0
      • JonathanLeeJ
        JonathanLee
        last edited by

        does anyone know what the /$6 means at the end of the text files Squid does not cover this information I know it means in reg ex to count over however how can it could over if it is reading a file it would need a variable to store it after so that is where I get confused here.

        Make sure to upvote

        1 Reply Last reply Reply Quote 0
        • JonathanLeeJ
          JonathanLee
          last edited by

          Please if anyone else knows how to improve this please yet me know this configuration project has been going on for a long time for me. It works well now however I feel I could improve on the security side.

          Goal I do not want a container to be able to be downloaded into the cache and escape into the firewall file system.

          Make sure to upvote

          1 Reply Last reply Reply Quote 0
          • JonathanLeeJ
            JonathanLee
            last edited by

            update:
            Dynamic and Update Content
            Custom refresh_patterns

            acl getmethod method GET
acl rewritedoms dstdomain .facebook.com .akamaihd.net .fbcdn.net .google.com .static.com .apple.com .oracle.com .sun.com .java.com .adobe.com .steamstatic.com .steampowered.com .steamcontent.com .google.com

always_direct allow !getmethod
store_id_access deny connect
store_id_access deny !getmethod
store_id_access allow rewritedoms
store_id_program /usr/local/libexec/squid/storeid_file_rewrite /var/squid/storeid/storeid_rewrite.txt
store_id_children 10 startup=5 idle=1 concurrency=0

#APPLE STUFF
refresh_pattern -i apple.com/..(cab|exe|msi|msu|msf|asf|wmv|wma|dat|zip|dist)$ 0 80% 43200  refresh-ims

#apple update
refresh_pattern -i (download|adcdownload).apple.com/.*\.(pkg|dmg) 4320 100% 43200 
refresh_pattern -i appldnld\.apple\.com 129600 100% 129600     
refresh_pattern -i phobos\.apple\.com 129600 100% 129600     
refresh_pattern -i iosapps\.itunes\.apple\.com 129600 100% 129600     

# Updates: Windows
refresh_pattern -i microsoft.com/..(cab|exe|msi|msu|msf|asf|wma|dat|zip)$ 4320 80% 43200  refresh-ims
refresh_pattern -i windowsupdate.com/..(cab|exe|msi|msu|msf|asf|wma|wmv)|dat|zip)$ 4320 80% 43200  refresh-ims
refresh_pattern -i windows.com/..(cab|exe|msi|msu|msf|asf|wmv|wma|dat|zip)$ 4320 80% 43200  refresh-ims
refresh_pattern -i microsoft.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 4320 80% 43200 
refresh_pattern -i windowsupdate.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 4320 80% 43200 
refresh_pattern -i windows.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 4320 80% 43200 
refresh_pattern -i .*windowsupdate.com/.*\.(cab|exe)                     259200 100% 259200   
refresh_pattern -i .*update.microsoft.com/.*\.(cab|exe|dll|msi|psf)                  259200 100% 259200   
refresh_pattern windowsupdate.com/.*\.(cab|exe|dll|msi|psf) 10080 100% 43200 
refresh_pattern download.microsoft.com/.*\.(cab|exe|dll|msi|psf) 10080 100% 43200 
refresh_pattern www.microsoft.com/.*\.(cab|exe|dll|msi|psf) 10080 100% 43200 
refresh_pattern au.download.windowsupdate.com/.*\.(cab|exe|dll|msi|psf) 4320 100% 43200 
refresh_pattern bg.v4.pr.dl.ws.microsoft.com/.*\.(cab|exe|dll|msi|psf) 4320 100% 43200
#windows update NEW UPDATE 0.04
refresh_pattern update.microsoft.com/.*\.(cab|exe)                  43200 100% 129600    
refresh_pattern ([^.]+\.)?(download|(windows)?update)\.(microsoft\.)?com/.*\.(cab|exe|msi|msp|psf) 4320 100% 43200  
refresh_pattern update.microsoft.com/.*\.(cab|exe|dll|msi|psf) 10080 100% 43200 
refresh_pattern -i \.update.microsoft.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 525600 100% 525600       
refresh_pattern -i \.windowsupdate.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 525600 100% 525600       
refresh_pattern -i \.download.microsoft.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 525600 100% 525600       
refresh_pattern -i \.ws.microsoft.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 525600 100% 525600       

refresh_pattern ([^.]+\.)?(cs|content[1-9]|hsar|content-origin|client-download).[steampowered|steamcontent].com/.*\.* 43200 100% 43200 reload-into-ims ignore-reload ignore-no-store override-expire override-lastmod
refresh_pattern ([^.]+\.)?.akamai.steamstatic.com/.*\.* 43200 100% 43200 reload-into-ims ignore-reload ignore-no-store override-expire override-lastmod

refresh_pattern -i ([^.]+\.)?.adobe.com/.*\.(zip|exe) 43200 100% 43200 reload-into-ims ignore-reload ignore-no-store override-expire override-lastmod
refresh_pattern -i ([^.]+\.)?.java.com/.*\.(zip|exe) 43200 100% 43200 reload-into-ims ignore-reload ignore-no-store override-expire override-lastmod
refresh_pattern -i ([^.]+\.)?.sun.com/.*\.(zip|exe) 43200 100% 43200 reload-into-ims ignore-reload ignore-no-store override-expire override-lastmod
refresh_pattern -i ([^.]+\.)?.oracle.com/.*\.(zip|exe|tar.gz) 43200 100% 43200 reload-into-ims ignore-reload ignore-no-store override-expire override-lastmod

refresh_pattern -i appldnld\.apple\.com 43200 100% 43200 ignore-reload ignore-no-store override-expire override-lastmod
refresh_pattern -i ([^.]+\.)?apple.com/.*\.(ipa) 43200 100% 43200 ignore-reload ignore-no-store override-expire override-lastmod
 
refresh_pattern -i ([^.]+\.)?.google.com/.*\.(exe|crx) 10080 80% 43200 override-expire override-lastmod ignore-no-cache ignore-reload reload-into-ims ignore-private
refresh_pattern -i ([^.]+\.)?g.static.com/.*\.(exe|crx) 10080 80% 43200 override-expire override-lastmod ignore-no-cache ignore-reload reload-into-ims ignore-private

#FACEBOOK
refresh_pattern ^http?://*.facebook.com/*  10080 80% 43200 override-expire override-lastmod ignore-no-cache ignore-reload reload-into-ims ignore-private

#FACEBOOK IMAGES  
refresh_pattern -i pixel.facebook.com..(jpg|png|gif|ico|css|js)  10080 80% 43200 override-expire override-lastmod ignore-no-cache ignore-reload reload-into-ims ignore-private
refresh_pattern -i .akamaihd.net..(jpg|png|gif|ico|css|js)  10080 80% 43200 override-expire override-lastmod ignore-no-cache ignore-reload reload-into-ims ignore-private   
refresh_pattern -i (facebook.com).(jpg|png|gif)  10080 80% 43200 store-stale override-expire override-lastmod ignore-no-cache ignore-reload reload-into-ims ignore-private 
refresh_pattern static.(xx|ak).fbcdn.net.(jpg|gif|png) 10080 80% 43200 override-expire override-lastmod ignore-no-cache ignore-reload reload-into-ims ignore-private
refresh_pattern ^https?://profile.ak.fbcdn.net*.(jpg|gif|png)  10080 80% 43200 override-expire override-lastmod ignore-no-cache ignore-reload reload-into-ims ignore-private

#FACEBOOK VIDEO
refresh_pattern -i .video.ak.fbcdn.net.*.(mp4|flv|mp3|amf) 10080 80% 43200 override-expire override-lastmod ignore-no-cache ignore-reload reload-into-ims ignore-private   
refresh_pattern (audio|video)/(webm|mp4) 10080 80% 43200 override-expire override-lastmod ignore-no-cache ignore-reload reload-into-ims ignore-private

refresh_pattern -i squid\.internal 10080 80% 79900 override-lastmod override-expire ignore-reload ignore-no-store ignore-must-revalidate ignore-private ignore-auth

range_offset_limit 200 MB windowsupdate
maximum_object_size 200 MB windowsupdate
range_offset_limit 0
quick_abort_min -1 KB

            update
            Custom Options (SSL/MITM)

            acl manager proto cache_object
acl windowsupdate dstdomain windowsupdate.microsoft.com
acl windowsupdate dstdomain .update.microsoft.com
acl windowsupdate dstdomain download.windowsupdate.com
acl windowsupdate dstdomain redir.metaservices.microsoft.com
acl windowsupdate dstdomain images.metaservices.microsoft.com
acl windowsupdate dstdomain c.microsoft.com
acl windowsupdate dstdomain www.download.windowsupdate.com
acl windowsupdate dstdomain wustat.windows.com
acl windowsupdate dstdomain crl.microsoft.com
acl windowsupdate dstdomain sls.microsoft.com
acl windowsupdate dstdomain productactivation.one.microsoft.com
acl windowsupdate dstdomain ntservicepack.microsoft.com
acl localhost src 192.168.1.1/32

acl CONNECT method CONNECT
acl wuCONNECT dstdomain www.update.microsoft.com
acl wuCONNECT dstdomain sls.microsoft.com

http_access allow CONNECT wuCONNECT localnet
http_access allow CONNECT wuCONNECT localhost
http_access allow windowsupdate localnet
http_access allow windowsupdate localhost
http_access deny manager

acl BrokenButTrustedServers dstdomain "/usr/local/pkg/dstdom.broken"
acl DomainMismatch ssl_error SQUID_X509_V_ERR_DOMAIN_MISMATCH
sslproxy_cert_error allow BrokenButTrustedServers DomainMismatch
sslproxy_cert_error deny all

acl splice_only src 192.168.1.8 #Tasha iPhone
acl splice_only src 192.168.1.10 #Jon iPhone
acl splice_only src 192.168.1.11 #Amazon Fire
acl splice_only src 192.168.1.15 #Tasha HP
acl splice_only src 192.168.1.16 #iPad

acl NoSSLIntercept ssl::server_name_regex -i "/usr/local/pkg/url.nobump"

acl markBumped annotate_client bumped=true
acl bump_only src 192.168.1.3 #webtv
acl bump_only src 192.168.1.4 #toshiba
acl bump_only src 192.168.1.5 #imac
acl bump_only src 192.168.1.9 #macbook
acl bump_only src 192.168.1.13 #dell

ssl_bump peek step1
ssl_bump splice splice_only
ssl_bump splice NoSSLIntercept
ssl_bump bump bump_only markBumped
ssl_bump stare all

acl markedBumped note bumped true
url_rewrite_access deny markedBumped
http_access deny all

#acl SSLIntercept ssl::server_name_regex -i "/usr/local/pkg/url.bump"
#ssl_bump bump SSLIntercept

            Theses changes made it work better.

            Please if anyone else like to research this let me know if you see anything off?

            Make sure to upvote

            1 Reply Last reply Reply Quote 0
            • JonathanLeeJ
              JonathanLee
              last edited by JonathanLee

              Update to custom options new items tried today caused increased performance

              cache deny https_login
              read_ahead_gap 32 KB
              negative_ttl 1 second
              connect_timeout 30 seconds
              request_timeout 60 seconds
              half_closed_clients off
              shutdown_lifetime 10 seconds
              negative_dns_ttl 1 seconds
              ignore_unknown_nameservers on
              pipeline_prefetch 100

              I have been testing the above options and they seem to increase performance drastically.

              acl manager proto cache_object
acl localhost src 192.168.1.1/32
acl https_login url_regex -i ^https.*(login|Login).*
acl CONNECT method CONNECT
acl wuCONNECT dstdomain www.update.microsoft.com
acl wuCONNECT dstdomain sls.microsoft.com
http_access allow CONNECT wuCONNECT localnet
http_access allow CONNECT wuCONNECT localhost
http_access allow windowsupdate localnet
http_access allow windowsupdate localhost
http_access deny manager

acl BrokenButTrustedServers dstdomain "/usr/local/pkg/dstdom.broken"
acl DomainMismatch ssl_error SQUID_X509_V_ERR_DOMAIN_MISMATCH
sslproxy_cert_error allow BrokenButTrustedServers DomainMismatch
sslproxy_cert_error deny all

acl splice_only src 192.168.1.8 #Tasha iPhone
acl splice_only src 192.168.1.10 #Jon iPhone
acl splice_only src 192.168.1.11 #Amazon Fire
acl splice_only src 192.168.1.15 #Tasha HP
acl splice_only src 192.168.1.16 #iPad

acl NoSSLIntercept ssl::server_name_regex -i "/usr/local/pkg/url.nobump"

acl markBumped annotate_client bumped=true
acl bump_only src 192.168.1.3 #webtv
acl bump_only src 192.168.1.4 #toshiba
acl bump_only src 192.168.1.5 #imac
acl bump_only src 192.168.1.9 #macbook
acl bump_only src 192.168.1.13 #dell

cache deny https_login
read_ahead_gap 32 KB
negative_ttl 1 second
connect_timeout 30 seconds
request_timeout 60 seconds
half_closed_clients off
shutdown_lifetime 10 seconds
negative_dns_ttl 1 seconds
ignore_unknown_nameservers on
pipeline_prefetch 100

ssl_bump peek step1
ssl_bump none https_login
ssl_bump splice splice_only
ssl_bump splice NoSSLIntercept
ssl_bump bump bump_only markBumped
ssl_bump stare all

acl markedBumped note bumped true
url_rewrite_access deny markedBumped
http_access deny all

#acl SSLIntercept ssl::server_name_regex -i "/usr/local/pkg/url.bump"
#ssl_bump bump SSLIntercept

              update Custom refresh_patterns this area is in the config first so acl windows update can be used in general configuration also.

              acl getmethod method GET

acl windowsupdate dstdomain windowsupdate.microsoft.com
acl windowsupdate dstdomain .update.microsoft.com
acl windowsupdate dstdomain download.windowsupdate.com
acl windowsupdate dstdomain redir.metaservices.microsoft.com
acl windowsupdate dstdomain images.metaservices.microsoft.com
acl windowsupdate dstdomain c.microsoft.com
acl windowsupdate dstdomain www.download.windowsupdate.com
acl windowsupdate dstdomain wustat.windows.com
acl windowsupdate dstdomain crl.microsoft.com
acl windowsupdate dstdomain sls.microsoft.com
acl windowsupdate dstdomain productactivation.one.microsoft.com
acl windowsupdate dstdomain ntservicepack.microsoft.com
acl windowsupdate dstdomain dc1-st.ksn.kaspersky-labs.com
acl windowsupdate dstdomain dc1-file.ksn.kaspersky-labs.com
acl windowsupdate dstdomain dc1.ksn.kaspersky-labs.com

acl rewritedoms dstdomain .facebook.com .akamaihd.net .fbcdn.net .google.com .static.com .apple.com .oracle.com .sun.com .java.com .adobe.com .steamstatic.com .steampowered.com .steamcontent.com .google.com

always_direct allow !getmethod
store_id_access deny connect
store_id_access deny !getmethod
store_id_access allow rewritedoms
store_id_program /usr/local/libexec/squid/storeid_file_rewrite /var/squid/storeid/storeid_rewrite.txt
store_id_children 10 startup=5 idle=1 concurrency=0

#APPLE STUFF
refresh_pattern -i apple.com/..(cab|exe|msi|msu|msf|asf|wmv|wma|dat|zip|dist)$ 0 80% 43200  refresh-ims

#apple update
refresh_pattern -i (download|adcdownload).apple.com/.*\.(pkg|dmg) 4320 100% 43200 
refresh_pattern -i appldnld\.apple\.com 129600 100% 129600     
refresh_pattern -i phobos\.apple\.com 129600 100% 129600     
refresh_pattern -i iosapps\.itunes\.apple\.com 129600 100% 129600     

# Updates: Windows
refresh_pattern -i microsoft.com/..(cab|exe|msi|msu|msf|asf|wma|dat|zip)$ 4320 80% 43200  refresh-ims
refresh_pattern -i windowsupdate.com/..(cab|exe|msi|msu|msf|asf|wma|wmv)|dat|zip)$ 4320 80% 43200  refresh-ims
refresh_pattern -i windows.com/..(cab|exe|msi|msu|msf|asf|wmv|wma|dat|zip)$ 4320 80% 43200  refresh-ims
refresh_pattern -i microsoft.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 4320 80% 43200 
refresh_pattern -i windowsupdate.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 4320 80% 43200 
refresh_pattern -i windows.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 4320 80% 43200 
refresh_pattern -i .*windowsupdate.com/.*\.(cab|exe) 259200 100% 259200   
refresh_pattern -i .*update.microsoft.com/.*\.(cab|exe|dll|msi|psf) 259200 100% 259200   
refresh_pattern windowsupdate.com/.*\.(cab|exe|dll|msi|psf) 10080 100% 43200 
refresh_pattern download.microsoft.com/.*\.(cab|exe|dll|msi|psf) 10080 100% 43200 
refresh_pattern www.microsoft.com/.*\.(cab|exe|dll|msi|psf) 10080 100% 43200 
refresh_pattern au.download.windowsupdate.com/.*\.(cab|exe|dll|msi|psf) 4320 100% 43200 
refresh_pattern bg.v4.pr.dl.ws.microsoft.com/.*\.(cab|exe|dll|msi|psf) 4320 100% 43200
#windows update NEW UPDATE 0.04
refresh_pattern update.microsoft.com/.*\.(cab|exe) 43200 100% 129600    
refresh_pattern ([^.]+\.)?(download|(windows)?update)\.(microsoft\.)?com/.*\.(cab|exe|msi|msp|psf) 4320 100% 43200  
refresh_pattern update.microsoft.com/.*\.(cab|exe|dll|msi|psf) 10080 100% 43200 
refresh_pattern -i \.update.microsoft.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 525600 100% 525600       
refresh_pattern -i \.windowsupdate.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 525600 100% 525600       
refresh_pattern -i \.download.microsoft.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 525600 100% 525600       
refresh_pattern -i \.ws.microsoft.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 525600 100% 525600       

refresh_pattern ([^.]+\.)?(cs|content[1-9]|hsar|content-origin|client-download).[steampowered|steamcontent].com/.*\.* 43200 100% 43200 reload-into-ims ignore-reload ignore-no-store override-expire override-lastmod
refresh_pattern ([^.]+\.)?.akamai.steamstatic.com/.*\.* 43200 100% 43200 reload-into-ims ignore-reload ignore-no-store override-expire override-lastmod

refresh_pattern -i ([^.]+\.)?.adobe.com/.*\.(zip|exe) 43200 100% 43200 reload-into-ims ignore-reload ignore-no-store override-expire override-lastmod
refresh_pattern -i ([^.]+\.)?.java.com/.*\.(zip|exe) 43200 100% 43200 reload-into-ims ignore-reload ignore-no-store override-expire override-lastmod
refresh_pattern -i ([^.]+\.)?.sun.com/.*\.(zip|exe) 43200 100% 43200 reload-into-ims ignore-reload ignore-no-store override-expire override-lastmod
refresh_pattern -i ([^.]+\.)?.oracle.com/.*\.(zip|exe|tar.gz) 43200 100% 43200 reload-into-ims ignore-reload ignore-no-store override-expire override-lastmod

refresh_pattern -i appldnld\.apple\.com 43200 100% 43200 ignore-reload ignore-no-store override-expire override-lastmod
refresh_pattern -i ([^.]+\.)?apple.com/.*\.(ipa) 43200 100% 43200 ignore-reload ignore-no-store override-expire override-lastmod
 
refresh_pattern -i ([^.]+\.)?.google.com/.*\.(exe|crx) 10080 80% 43200 override-expire override-lastmod ignore-no-cache ignore-reload reload-into-ims ignore-private
refresh_pattern -i ([^.]+\.)?g.static.com/.*\.(exe|crx) 10080 80% 43200 override-expire override-lastmod ignore-no-cache ignore-reload reload-into-ims ignore-private

#FACEBOOK
refresh_pattern ^http?://*.facebook.com/*  10080 80% 43200 override-expire override-lastmod ignore-no-cache ignore-reload reload-into-ims ignore-private

#FACEBOOK IMAGES  
refresh_pattern -i pixel.facebook.com..(jpg|png|gif|ico|css|js)  10080 80% 43200 override-expire override-lastmod ignore-no-cache ignore-reload reload-into-ims ignore-private
refresh_pattern -i .akamaihd.net..(jpg|png|gif|ico|css|js) 10080 80% 43200 override-expire override-lastmod ignore-no-cache ignore-reload reload-into-ims ignore-private   
refresh_pattern -i (facebook.com).(jpg|png|gif) 10080 80% 43200 store-stale override-expire override-lastmod ignore-no-cache ignore-reload reload-into-ims ignore-private 
refresh_pattern static.(xx|ak).fbcdn.net.(jpg|gif|png) 10080 80% 43200 override-expire override-lastmod ignore-no-cache ignore-reload reload-into-ims ignore-private
refresh_pattern ^https?://profile.ak.fbcdn.net*.(jpg|gif|png) 10080 80% 43200 override-expire override-lastmod ignore-no-cache ignore-reload reload-into-ims ignore-private

#FACEBOOK VIDEO
refresh_pattern -i .video.ak.fbcdn.net.*.(mp4|flv|mp3|amf) 10080 80% 43200 override-expire override-lastmod ignore-no-cache ignore-reload reload-into-ims ignore-private   
refresh_pattern (audio|video)/(webm|mp4) 10080 80% 43200 override-expire override-lastmod ignore-no-cache ignore-reload reload-into-ims ignore-private

refresh_pattern -i squid\.internal 10080 80% 79900 override-lastmod override-expire ignore-reload ignore-no-store ignore-must-revalidate ignore-private ignore-auth

range_offset_limit 512 MB windowsupdate
maximum_object_size 512 MB windowsupdate
range_offset_limit 0
quick_abort_min -1 KB

              Update to non bump list

              ^.*gateway.facebook.com/ws/realtime?

              This addition corrects many issues with delays and auto log outs this must be spliced and marked as non bump to function effectively on the web cache accelerator system.

              #Sites to be splice
^.*gateway\.facebook\.com\/ws\/realtime\?
^.*conviva\.com.*
license\.adrise\.tv.*
c2r\.ts\.cdn\.office\.net
^.*cdn\.office\.net
^.*bitdefender\.net
config\.teams\.microsoft\.com
^.*.azure-devices\.net
substrate\.office\.com.*
hulu\.playback\.edge\.bamgrid\.com
assetshuluimcom-a\.akamaihd\.net
hulu\.sc\.omtrdc\.net
infinity-c33\.youboranqs01\.com
beacons\.extremereach\.io
^.*tubi\.video
^.*tubi\.io
a-fds\.youborafds01\.com
youboranqs01\.com
amzpvxrayasset-a\.akamaihd\.net
pv-cdn.net
^.*media-amazon\.com
aiv-delivery\.net
unagi\.amazon\.com
atv-ps\.amazon\.com
pv-cdn\.net
fls-na\.amazon\.com
^.*aiv-cdn\.net
c0a299900000\.local
update\.microsoft\.com
update\.microsoft\.com\.akadns\.net
delivery\.mp\.microsoft\.com
appldnld\.apple\.com
configuration\.apple\.com
gdmf\.apple\.com
mesu\.apple\.com
oscdn\.apple\.com
osrecovery\.apple\.com
skl\.apple\.com
swcdn\.apple\.com
swdist\.apple\.com
swscan\.apple\.com
updates-http\.cdn-apple\.com
updates\.cdn-apple\.com
appldnld\.apple\.com\.edgesuite\.net
entrust\.net
digicert\.com
apple-cloudkit\.com
apple-livephotoskit\.com
gc\.apple\.com
icloud-content\.com
apple\.com
cdn-apple\.com
icloud\.com
api\.apple-cloudkit\.com
^.*appattest\.apple\.com
^.*itunes\.apple\.com
^.*mzstatic\.com
itunes\.com
music\.apple\.com
app-site-association\.cdn-apple\.com
app-site-association\.networking\.apple\.com
xp\.apple\.com
play\.google\.com
android\.com
^((alt[0-9]-mtalk\.)|(mtalk\.)|(mtalk-(staging|dev)\.))google\.com
google-analytics\.com
googleusercontent\.com
^((gvt)([0-9]))\.com
ggpht\.com
dl\.google\.com
dl-ssl\.google\.com
android\.clients\.google\.com
^(((clients)[0-9])|accounts)\.google\.(com|us)
connectivitycheck\.android\.com
android\.clients\.google\.com
device-provisioning\.googleapis\.com
omahaproxy\.appspot\.com
payments\.google\.com
googleapis\.com
notifications\.google\.com
^(pki|(crl|ocsp)\.pki)\.google\.com
ogs\.google\.com
googleapis\.com
androidmanagement\.googleapis\.com
mservice\.bankofamerica\.com
privacyportal-bofa\.my\.onetrust\.com
bankofamerica\.com
mcafee\.com
kaspersky\.com
kaspersky-labs\.com
dc1-st\.ksn\.kaspersky-labs\.com
dc1-file\.ksn\.kaspersky-labs\.com
dc1\.ksn\.kaspersky-labs\.com
olui2m\.fs\.ml\.com
ml\.com
^.*zoom\.us
^.*teams\.microsoft\.com
teams\.events\.data\.microsoft\.com
statics\.teams\.cdn\.office\.net
^.*(outlook\.)(office365|office)\.com
edge-chat\.facebook\.com
internet\.speedpay\.com
^.*hulustream\.com
cws-hulu\.conviva\.com
^.*hulu\.com
hulu\.hb\.omtrdc\.net
^.*dssott\.com
prod-ripcut-delivery\.disney-plus\.net
^(disney\.(content|connections))\.edge\.bamgrid\.com
disney\.api\.edge\.bamgrid\.com
disney\.playback\.edge\.bamgrid\.com
disney\.my\.sentry\.io
^.*amazonvideo\.com
unagi-na\.amazon\.com
events\.data\.microsoft\.com
tubi\.io
production-public\.tubi\.io
tubitv\.com
caauthservice\.state\.gov
studentaid\.gov
mohela\.com
www\.whitehouse\.gov
www\.rcsdk8\.org
rcsdk8\.powerschool\.com
www\.weaveinc\.org
^.*cdn\.nintendo\.net
^.*bitdefender\.net

              Make sure to upvote

              1 Reply Last reply Reply Quote 0
              • JonathanLeeJ
                JonathanLee
                last edited by JonathanLee

                update researching

                added to stop storing misses for real time checks. It is really not needed its a cache item as it is in real time so why store in the cache.

                acl no_miss url_regex -i ^.*gateway\.facebook\.com\/ws\/realtime\?
miss_access deny no_miss

                researching vary expired header issues config seems to have negative effect on performance I have sense removed this test

                vary_ignore_expire on

                acl manager proto cache_object
acl localhost src 192.168.1.1/32
acl https_login url_regex -i ^https.*(login|Login).*
acl no_miss url_regex -i ^.*gateway\.facebook\.com\/ws\/realtime\?
acl CONNECT method CONNECT
acl wuCONNECT dstdomain www.update.microsoft.com
acl wuCONNECT dstdomain sls.microsoft.com
http_access allow CONNECT wuCONNECT localnet
http_access allow CONNECT wuCONNECT localhost
http_access allow windowsupdate localnet
http_access allow windowsupdate localhost
http_access deny manager

acl BrokenButTrustedServers dstdomain "/usr/local/pkg/dstdom.broken"
acl DomainMismatch ssl_error SQUID_X509_V_ERR_DOMAIN_MISMATCH
sslproxy_cert_error allow BrokenButTrustedServers DomainMismatch
sslproxy_cert_error deny all

acl splice_only src 192.168.1.8 #Tasha iPhone
acl splice_only src 192.168.1.10 #Jon iPhone
acl splice_only src 192.168.1.11 #Amazon Fire
acl splice_only src 192.168.1.15 #Tasha HP
acl splice_only src 192.168.1.16 #iPad

acl NoSSLIntercept ssl::server_name_regex -i "/usr/local/pkg/url.nobump"

acl markBumped annotate_client bumped=true
acl bump_only src 192.168.1.3 #webtv
acl bump_only src 192.168.1.4 #toshiba
acl bump_only src 192.168.1.5 #imac
acl bump_only src 192.168.1.9 #macbook
acl bump_only src 192.168.1.13 #dell

cache deny https_login
read_ahead_gap 32 KB
negative_ttl 1 second
connect_timeout 30 seconds
request_timeout 60 seconds
half_closed_clients off
shutdown_lifetime 10 seconds
negative_dns_ttl 1 seconds
ignore_unknown_nameservers on
pipeline_prefetch 100
vary_ignore_expire on


ssl_bump peek step1
miss_access deny no_miss 
ssl_bump splice https_login
ssl_bump splice splice_only
ssl_bump splice NoSSLIntercept
ssl_bump bump bump_only markBumped
ssl_bump stare all

acl markedBumped note bumped true
url_rewrite_access deny markedBumped
http_access deny all

#acl SSLIntercept ssl::server_name_regex -i "/usr/local/pkg/url.bump"
#ssl_bump bump SSLIntercept

                Make sure to upvote

                1 Reply Last reply Reply Quote 0
                • JonathanLeeJ
                  JonathanLee
                  last edited by

                  Adaptions made ordering of placement of
                  refresh_pattern -i squid.internal 10080 80% 79900 override-lastmod override-expire ignore-reload ignore-no-store ignore-must-revalidate ignore-private ignore-auth

                  testing :
                  refresh_all_ims on
                  reload_into_ims on

                  This seems to get a lot more hits this way.

                  #cachemgr_passwd disable offline_toggle reconfigure shutdown
                  #cachemgr_passwd Secret all

                  ability to control what can be accessed inside of cachemgr.cgi if you have this enabled

                  acl getmethod method GET

acl windowsupdate dstdomain windowsupdate.microsoft.com
acl windowsupdate dstdomain .update.microsoft.com
acl windowsupdate dstdomain download.windowsupdate.com
acl windowsupdate dstdomain redir.metaservices.microsoft.com
acl windowsupdate dstdomain images.metaservices.microsoft.com
acl windowsupdate dstdomain c.microsoft.com
acl windowsupdate dstdomain www.download.windowsupdate.com
acl windowsupdate dstdomain wustat.windows.com
acl windowsupdate dstdomain crl.microsoft.com
acl windowsupdate dstdomain sls.microsoft.com
acl windowsupdate dstdomain productactivation.one.microsoft.com
acl windowsupdate dstdomain ntservicepack.microsoft.com
acl windowsupdate dstdomain dc1-st.ksn.kaspersky-labs.com
acl windowsupdate dstdomain dc1-file.ksn.kaspersky-labs.com
acl windowsupdate dstdomain dc1.ksn.kaspersky-labs.com

acl rewritedoms dstdomain .facebook.com .akamaihd.net .fbcdn.net .google.com .static.com .apple.com .oracle.com .sun.com .java.com .adobe.com .steamstatic.com .steampowered.com .steamcontent.com .google.com

store_id_program /usr/local/libexec/squid/storeid_file_rewrite /var/squid/storeid/storeid_rewrite.txt
store_id_children 10 startup=5 idle=1 concurrency=0
always_direct allow !getmethod
store_id_access deny connect
store_id_access deny !getmethod
store_id_access allow rewritedoms
refresh_all_ims on
reload_into_ims on

refresh_pattern -i squid\.internal 10080 80% 79900 override-lastmod override-expire ignore-reload ignore-no-store ignore-must-revalidate ignore-private ignore-auth

#APPLE STUFF
refresh_pattern -i apple.com/..(cab|exe|msi|msu|msf|asf|wmv|wma|dat|zip|dist)$ 0 80% 43200  refresh-ims

#apple update
refresh_pattern -i (download|adcdownload).apple.com/.*\.(pkg|dmg) 4320 100% 43200 
refresh_pattern -i appldnld\.apple\.com 129600 100% 129600     
refresh_pattern -i phobos\.apple\.com 129600 100% 129600     
refresh_pattern -i iosapps\.itunes\.apple\.com 129600 100% 129600     

# Updates: Windows
refresh_pattern -i microsoft.com/..(cab|exe|msi|msu|msf|asf|wma|dat|zip)$ 4320 80% 43200  refresh-ims
refresh_pattern -i windowsupdate.com/..(cab|exe|msi|msu|msf|asf|wma|wmv)|dat|zip)$ 4320 80% 43200  refresh-ims
refresh_pattern -i windows.com/..(cab|exe|msi|msu|msf|asf|wmv|wma|dat|zip)$ 4320 80% 43200  refresh-ims
refresh_pattern -i microsoft.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 4320 80% 43200 
refresh_pattern -i windowsupdate.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 4320 80% 43200 
refresh_pattern -i windows.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 4320 80% 43200 
refresh_pattern -i .*windowsupdate.com/.*\.(cab|exe) 259200 100% 259200   
refresh_pattern -i .*update.microsoft.com/.*\.(cab|exe|dll|msi|psf) 259200 100% 259200   
refresh_pattern windowsupdate.com/.*\.(cab|exe|dll|msi|psf) 10080 100% 43200 
refresh_pattern download.microsoft.com/.*\.(cab|exe|dll|msi|psf) 10080 100% 43200 
refresh_pattern www.microsoft.com/.*\.(cab|exe|dll|msi|psf) 10080 100% 43200 
refresh_pattern au.download.windowsupdate.com/.*\.(cab|exe|dll|msi|psf) 4320 100% 43200 
refresh_pattern bg.v4.pr.dl.ws.microsoft.com/.*\.(cab|exe|dll|msi|psf) 4320 100% 43200
#windows update NEW UPDATE 0.04
refresh_pattern update.microsoft.com/.*\.(cab|exe) 43200 100% 129600    
refresh_pattern ([^.]+\.)?(download|(windows)?update)\.(microsoft\.)?com/.*\.(cab|exe|msi|msp|psf) 4320 100% 43200  
refresh_pattern update.microsoft.com/.*\.(cab|exe|dll|msi|psf) 10080 100% 43200 
refresh_pattern -i \.update.microsoft.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 525600 100% 525600       
refresh_pattern -i \.windowsupdate.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 525600 100% 525600       
refresh_pattern -i \.download.microsoft.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 525600 100% 525600       
refresh_pattern -i \.ws.microsoft.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 525600 100% 525600       

refresh_pattern ([^.]+\.)?(cs|content[1-9]|hsar|content-origin|client-download).[steampowered|steamcontent].com/.*\.* 43200 100% 43200 reload-into-ims ignore-reload ignore-no-store override-expire override-lastmod
refresh_pattern ([^.]+\.)?.akamai.steamstatic.com/.*\.* 43200 100% 43200 reload-into-ims ignore-reload ignore-no-store override-expire override-lastmod

refresh_pattern -i ([^.]+\.)?.adobe.com/.*\.(zip|exe) 43200 100% 43200 reload-into-ims ignore-reload ignore-no-store override-expire override-lastmod
refresh_pattern -i ([^.]+\.)?.java.com/.*\.(zip|exe) 43200 100% 43200 reload-into-ims ignore-reload ignore-no-store override-expire override-lastmod
refresh_pattern -i ([^.]+\.)?.sun.com/.*\.(zip|exe) 43200 100% 43200 reload-into-ims ignore-reload ignore-no-store override-expire override-lastmod
refresh_pattern -i ([^.]+\.)?.oracle.com/.*\.(zip|exe|tar.gz) 43200 100% 43200 reload-into-ims ignore-reload ignore-no-store override-expire override-lastmod

refresh_pattern -i appldnld\.apple\.com 43200 100% 43200 ignore-reload ignore-no-store override-expire override-lastmod
refresh_pattern -i ([^.]+\.)?apple.com/.*\.(ipa) 43200 100% 43200 ignore-reload ignore-no-store override-expire override-lastmod
 
refresh_pattern -i ([^.]+\.)?.google.com/.*\.(exe|crx) 10080 80% 43200 override-expire override-lastmod ignore-no-cache ignore-reload reload-into-ims ignore-private
refresh_pattern -i ([^.]+\.)?g.static.com/.*\.(exe|crx) 10080 80% 43200 override-expire override-lastmod ignore-no-cache ignore-reload reload-into-ims ignore-private

#FACEBOOK
refresh_pattern ^http?://*.facebook.com/*  10080 80% 43200 override-expire override-lastmod ignore-no-cache ignore-reload reload-into-ims ignore-private

#FACEBOOK IMAGES  
refresh_pattern -i pixel.facebook.com..(jpg|png|gif|ico|css|js)  10080 80% 43200 override-expire override-lastmod ignore-no-cache ignore-reload reload-into-ims ignore-private
refresh_pattern -i .akamaihd.net..(jpg|png|gif|ico|css|js) 10080 80% 43200 override-expire override-lastmod ignore-no-cache ignore-reload reload-into-ims ignore-private   
refresh_pattern -i (facebook.com).(jpg|png|gif) 10080 80% 43200 store-stale override-expire override-lastmod ignore-no-cache ignore-reload reload-into-ims ignore-private 
refresh_pattern static.(xx|ak).fbcdn.net.(jpg|gif|png) 10080 80% 43200 override-expire override-lastmod ignore-no-cache ignore-reload reload-into-ims ignore-private
refresh_pattern ^https?://profile.ak.fbcdn.net*.(jpg|gif|png) 10080 80% 43200 override-expire override-lastmod ignore-no-cache ignore-reload reload-into-ims ignore-private

#FACEBOOK VIDEO
refresh_pattern -i .video.ak.fbcdn.net.*.(mp4|flv|mp3|amf) 10080 80% 43200 override-expire override-lastmod ignore-no-cache ignore-reload reload-into-ims ignore-private   
refresh_pattern (audio|video)/(webm|mp4) 10080 80% 43200 override-expire override-lastmod ignore-no-cache ignore-reload reload-into-ims ignore-private


range_offset_limit 512 MB windowsupdate
maximum_object_size 512 MB windowsupdate
range_offset_limit 0
quick_abort_min -1 KB

                  acl manager proto cache_object
acl localhost src 192.168.1.1/32
#cachemgr_passwd disable offline_toggle reconfigure shutdown
#cachemgr_passwd secret all
acl https_login url_regex -i ^https.*(login|Login).*
acl no_miss url_regex -i ^.*gateway\.facebook\.com\/ws\/realtime\?
acl CONNECT method CONNECT
acl wuCONNECT dstdomain www.update.microsoft.com
acl wuCONNECT dstdomain sls.microsoft.com
http_access allow CONNECT wuCONNECT localnet
http_access allow CONNECT wuCONNECT localhost
http_access allow windowsupdate localnet
http_access allow windowsupdate localhost
http_access deny manager

acl BrokenButTrustedServers dstdomain "/usr/local/pkg/dstdom.broken"
acl DomainMismatch ssl_error SQUID_X509_V_ERR_DOMAIN_MISMATCH
sslproxy_cert_error allow BrokenButTrustedServers DomainMismatch
sslproxy_cert_error deny all

acl splice_only src 192.168.1.8 #Tasha iPhone
acl splice_only src 192.168.1.10 #Jon iPhone
acl splice_only src 192.168.1.11 #Amazon Fire
acl splice_only src 192.168.1.15 #Tasha HP
acl splice_only src 192.168.1.16 #iPad

acl NoSSLIntercept ssl::server_name_regex -i "/usr/local/pkg/url.nobump"

acl markBumped annotate_client bumped=true
acl bump_only src 192.168.1.3 #webtv
acl bump_only src 192.168.1.4 #toshiba
acl bump_only src 192.168.1.5 #imac
acl bump_only src 192.168.1.9 #macbook
acl bump_only src 192.168.1.13 #dell

cache deny https_login
read_ahead_gap 32 KB
negative_ttl 1 second
connect_timeout 30 seconds
request_timeout 60 seconds
half_closed_clients off
shutdown_lifetime 10 seconds
negative_dns_ttl 1 seconds
ignore_unknown_nameservers on
pipeline_prefetch 100

ssl_bump peek step1
miss_access deny no_miss 
ssl_bump splice https_login
ssl_bump splice splice_only
ssl_bump splice NoSSLIntercept
ssl_bump bump bump_only markBumped
ssl_bump stare all

acl markedBumped note bumped true
url_rewrite_access deny markedBumped
http_access deny all

#acl SSLIntercept ssl::server_name_regex -i "/usr/local/pkg/url.bump"
#ssl_bump bump SSLIntercept

                  Make sure to upvote

                  JonathanLeeJ 1 Reply Last reply Reply Quote 0
                  • JonathanLeeJ
                    JonathanLee @JonathanLee
                    last edited by

                    @JonathanLee

                    ssl_bump peek step1
miss_access deny no_miss 
ssl_bump splice https_login markBumped
ssl_bump splice splice_only markBumped
ssl_bump splice NoSSLIntercept markBumped
ssl_bump bump bump_only
ssl_bump stare all

acl markedBumped note bumped true
url_rewrite_access deny markedBumped

                    This seems to have more hit 304s

                    Make sure to upvote

                    1 Reply Last reply Reply Quote 0
                    • JonathanLeeJ
                      JonathanLee
                      last edited by 

                      refresh_pattern -i .(video-lax\d\-\d\.xx|video\.ak)\.fbcdn.net.*\.(mp4|flv|mp3|amf) 10080 80% 43200 override-expire override-lastmod ignore-no-cache ignore-reload reload-into-ims ignore-private

                      This works better for cache of videos and acceleration the old pattern no longer worked as of a couple days ago they adapted it to have lax for my area and added numerical info in the URL this radically increased cache and acceleration times for me with use of their new URL scheme

                      Make sure to upvote

                      1 Reply Last reply Reply Quote 0
                      • JonathanLeeJ
                        JonathanLee
                        last edited by JonathanLee

                        Fix for speed issues was to use domain acl for most of the no bump splice items, this drastically speeds up system also.

                        Researching CIPHERs also with this please ignore cipher changes this was my tests with cipher testing site for more use of high ciphers

                        acl localhost src 192.168.1.1/32
#cachemgr_passwd disable offline_toggle reconfigure shutdown
#cachemgr_passwd REDACTED! all
acl no_miss url_regex -i gateway\.facebook\.com\/ws\/realtime\?
acl no_miss url_regex -i web-chat-e2ee\.facebook\.com\/ws\/chat	
acl CONNECT method CONNECT
acl wuCONNECT dstdomain www.update.microsoft.com
acl wuCONNECT dstdomain sls.microsoft.com
http_access allow CONNECT wuCONNECT localnet
http_access allow CONNECT wuCONNECT localhost
http_access allow windowsupdate localnet
http_access allow windowsupdate localhost
http_access deny manager

acl BrokenButTrustedServers dstdomain "/usr/local/pkg/dstdom.broken"
acl DomainMismatch ssl_error SQUID_X509_V_ERR_DOMAIN_MISMATCH
sslproxy_cert_error allow BrokenButTrustedServers DomainMismatch
sslproxy_cert_error deny all

acl splice_only src 192.168.1.8 #Tasha iPhone
acl splice_only src 192.168.1.10 #Jon iPhone
acl splice_only src 192.168.1.11 #Amazon Fire
acl splice_only src 192.168.1.15 #Tasha HP
acl splice_only src 192.168.1.16 #iPad

acl NoSSLIntercept ssl::server_name_regex -i "/usr/local/pkg/reg.url.nobump"
acl NoBumpDNS dstdomain "/usr/local/pkg/dns.nobump"

acl markBumped annotate_client bumped=true
acl bump_only src 192.168.1.3 #webtv
acl bump_only src 192.168.1.4 #toshiba
acl bump_only src 192.168.1.5 #imac
acl bump_only src 192.168.1.9 #macbook
acl bump_only src 192.168.1.13 #dell

ssl_bump peek step1
miss_access deny no_miss 
ssl_bump splice https_login
ssl_bump splice splice_only
ssl_bump splice NoBumpDNS
ssl_bump splice NoSSLIntercept
ssl_bump bump bump_only markBumped
ssl_bump stare all

acl markedBumped note bumped true
url_rewrite_access deny markedBumped

read_ahead_gap 64 KB
negative_ttl 1 second
connect_timeout 30 seconds
request_timeout 60 seconds
half_closed_clients off
shutdown_lifetime 10 seconds
negative_dns_ttl 1 seconds
ignore_unknown_nameservers on
pipeline_prefetch 100

#acl SSLIntercept ssl::server_name_regex -i "/usr/local/pkg/url.bump"
#ssl_bump bump SSLIntercept

                        acl getmethod method GET

tls_outgoing_options cipher=HIGH:MEDIUM:!RC4:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!SRP:!DSS

tls_outgoing_options options=NO_SSLv3,SINGLE_DH_USE,SINGLE_ECDH_USE

acl windowsupdate dstdomain windowsupdate.microsoft.com
acl windowsupdate dstdomain .update.microsoft.com
acl windowsupdate dstdomain download.windowsupdate.com
acl windowsupdate dstdomain redir.metaservices.microsoft.com
acl windowsupdate dstdomain images.metaservices.microsoft.com
acl windowsupdate dstdomain c.microsoft.com
acl windowsupdate dstdomain www.download.windowsupdate.com
acl windowsupdate dstdomain wustat.windows.com
acl windowsupdate dstdomain crl.microsoft.com
acl windowsupdate dstdomain sls.microsoft.com
acl windowsupdate dstdomain productactivation.one.microsoft.com
acl windowsupdate dstdomain ntservicepack.microsoft.com
acl windowsupdate dstdomain dc1-st.ksn.kaspersky-labs.com
acl windowsupdate dstdomain dc1-file.ksn.kaspersky-labs.com
acl windowsupdate dstdomain dc1.ksn.kaspersky-labs.com

acl rewritedoms dstdomain .facebook.com .akamaihd.net .fbcdn.net .google.com .static.com .apple.com .oracle.com .sun.com .java.com .adobe.com .steamstatic.com .steampowered.com .steamcontent.com .google.com

store_id_program /usr/local/libexec/squid/storeid_file_rewrite /var/squid/storeid/storeid_rewrite.txt
store_id_children 10 startup=5 idle=1 concurrency=0
always_direct allow !getmethod
store_id_access deny connect
store_id_access deny !getmethod
store_id_access allow rewritedoms
reload_into_ims on
max_stale 20 years
minimum_expiry_time 0


refresh_pattern -i squid\.internal 10080 80% 79900 override-lastmod override-expire ignore-reload ignore-no-store ignore-must-revalidate ignore-private ignore-auth

#APPLE STUFF
refresh_pattern -i apple.com/..(cab|exe|msi|msu|msf|asf|wmv|wma|dat|zip|dist)$ 0 80% 43200  refresh-ims

#apple update
refresh_pattern -i (download|adcdownload).apple.com/.*\.(pkg|dmg) 4320 100% 43200 
refresh_pattern -i appldnld\.apple\.com 129600 100% 129600     
refresh_pattern -i phobos\.apple\.com 129600 100% 129600     
refresh_pattern -i iosapps\.itunes\.apple\.com 129600 100% 129600     

# Updates: Windows
refresh_pattern -i microsoft.com/..(cab|exe|msi|msu|msf|asf|wma|dat|zip)$ 4320 80% 43200  refresh-ims
refresh_pattern -i windowsupdate.com/..(cab|exe|msi|msu|msf|asf|wma|wmv)|dat|zip)$ 4320 80% 43200  refresh-ims
refresh_pattern -i windows.com/..(cab|exe|msi|msu|msf|asf|wmv|wma|dat|zip)$ 4320 80% 43200  refresh-ims
refresh_pattern -i microsoft.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 4320 80% 43200 
refresh_pattern -i windowsupdate.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 4320 80% 43200 
refresh_pattern -i windows.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 4320 80% 43200 
refresh_pattern -i .*windowsupdate.com/.*\.(cab|exe) 259200 100% 259200   
refresh_pattern -i .*update.microsoft.com/.*\.(cab|exe|dll|msi|psf) 259200 100% 259200   
refresh_pattern windowsupdate.com/.*\.(cab|exe|dll|msi|psf) 10080 100% 43200 
refresh_pattern download.microsoft.com/.*\.(cab|exe|dll|msi|psf) 10080 100% 43200 
refresh_pattern www.microsoft.com/.*\.(cab|exe|dll|msi|psf) 10080 100% 43200 
refresh_pattern au.download.windowsupdate.com/.*\.(cab|exe|dll|msi|psf) 4320 100% 43200 
refresh_pattern bg.v4.pr.dl.ws.microsoft.com/.*\.(cab|exe|dll|msi|psf) 4320 100% 43200
#windows update NEW UPDATE 0.04
refresh_pattern update.microsoft.com/.*\.(cab|exe) 43200 100% 129600    
refresh_pattern ([^.]+\.)?(download|(windows)?update)\.(microsoft\.)?com/.*\.(cab|exe|msi|msp|psf) 4320 100% 43200  
refresh_pattern update.microsoft.com/.*\.(cab|exe|dll|msi|psf) 10080 100% 43200 
refresh_pattern -i \.update.microsoft.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 525600 100% 525600       
refresh_pattern -i \.windowsupdate.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 525600 100% 525600       
refresh_pattern -i \.download.microsoft.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 525600 100% 525600       
refresh_pattern -i \.ws.microsoft.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 525600 100% 525600       

refresh_pattern ([^.]+\.)?(cs|content[1-9]|hsar|content-origin|client-download).[steampowered|steamcontent].com/.*\.* 43200 100% 43200 reload-into-ims ignore-reload ignore-no-store override-expire override-lastmod
refresh_pattern ([^.]+\.)?.akamai.steamstatic.com/.*\.* 43200 100% 43200 reload-into-ims ignore-reload ignore-no-store override-expire override-lastmod

refresh_pattern -i ([^.]+\.)?.adobe.com/.*\.(zip|exe) 43200 100% 43200 reload-into-ims ignore-reload ignore-no-store override-expire override-lastmod
refresh_pattern -i ([^.]+\.)?.java.com/.*\.(zip|exe) 43200 100% 43200 reload-into-ims ignore-reload ignore-no-store override-expire override-lastmod
refresh_pattern -i ([^.]+\.)?.sun.com/.*\.(zip|exe) 43200 100% 43200 reload-into-ims ignore-reload ignore-no-store override-expire override-lastmod
refresh_pattern -i ([^.]+\.)?.oracle.com/.*\.(zip|exe|tar.gz) 43200 100% 43200 reload-into-ims ignore-reload ignore-no-store override-expire override-lastmod

refresh_pattern -i appldnld\.apple\.com 43200 100% 43200 ignore-reload ignore-no-store override-expire override-lastmod
refresh_pattern -i ([^.]+\.)?apple.com/.*\.(ipa) 43200 100% 43200 ignore-reload ignore-no-store override-expire override-lastmod
 
refresh_pattern -i ([^.]+\.)?.google.com/.*\.(exe|crx) 10080 80% 43200 override-expire override-lastmod ignore-no-cache ignore-reload reload-into-ims ignore-private
refresh_pattern -i ([^.]+\.)?g.static.com/.*\.(exe|crx) 10080 80% 43200 override-expire override-lastmod ignore-no-cache ignore-reload reload-into-ims ignore-private

#FACEBOOK
refresh_pattern ^http?://*.facebook.com/*  10080 80% 43200 override-expire override-lastmod ignore-no-cache ignore-reload reload-into-ims ignore-private

#FACEBOOK IMAGES  
refresh_pattern -i pixel.facebook.com..(jpg|png|gif|ico|css|js)  10080 80% 43200 override-expire override-lastmod ignore-no-cache ignore-reload reload-into-ims ignore-private
refresh_pattern -i .akamaihd.net..(jpg|png|gif|ico|css|js) 10080 80% 43200 override-expire override-lastmod ignore-no-cache ignore-reload reload-into-ims ignore-private   
refresh_pattern -i (facebook.com).(jpg|png|gif) 10080 80% 43200 store-stale override-expire override-lastmod ignore-no-cache ignore-reload reload-into-ims ignore-private 
refresh_pattern static.(xx|ak).fbcdn.net.(jpg|gif|png) 10080 80% 43200 override-expire override-lastmod ignore-no-cache ignore-reload reload-into-ims ignore-private
refresh_pattern (scontent\-lax\d\-\d\.xx|.ak)\.fbcdn.net.*(jpg|gif|png) 10080 80% 43200 override-expire override-lastmod ignore-no-cache ignore-reload reload-into-ims ignore-private

refresh_pattern ^https?://profile.ak.fbcdn.net*.(jpg|gif|png) 10080 80% 43200 override-expire override-lastmod ignore-no-cache ignore-reload reload-into-ims ignore-private

#FACEBOOK VIDEO
refresh_pattern -i .(video-lax\d\-\d\.xx|video\.ak)\.fbcdn.net.*\.(mp4|flv|mp3|amf) 10080 80% 43200 override-expire override-lastmod ignore-no-cache ignore-reload reload-into-ims ignore-private
refresh_pattern (audio|video)/(webm|mp4) 10080 80% 43200 override-expire override-lastmod ignore-no-cache ignore-reload reload-into-ims ignore-private
acl https_login url_regex -i ^https.*(login|Login).*
cache deny https_login

range_offset_limit 512 MB windowsupdate
range_offset_limit 4 MB
range_offset_limit 0

                        quick_abort_min -1 KB

                        Files that go with this as an example

                        .dssott.com
.prod-ripcut-delivery.disney-plus.net
.disney.api.edge.bamgrid.com
.disney.playback.edge.bamgrid.com
.disney.my.sentry.io
.hulustream.com
.hulu.com
.hulu.hb.omtrdc.net
.hulu.playback.edge.bamgrid.com
.assetshuluimcom-a.akamaihd.net
.hulu.sc.omtrdc.net
.beacons.extremereach.io
.tubi.video
.tubi.io
.tubitv.com
.a-fds.youborafds01.com
.license.adrise.tv
.amzpvxrayasset-a.akamaihd.net
.pv-cdn.net
.media-amazon.com
.aiv-delivery.net
.unagi.amazon.com
.atv-ps.amazon.com
.pv-cdn.net
.fls-na.amazon.com
.aiv-cdn.net
.c0a299900000.local
.conviva.com
.cdn.office.net
.bitdefender.net
.azure-devices.net
.substrate.office.com
.update.microsoft.com
.update.microsoft.com.akadns.net
.delivery.mp.microsoft.com
.appldnld.apple.com
.configuration.apple.com
.gdmf.apple.com
.mesu.apple.com
.oscdn.apple.com
.osrecovery.apple.com
.skl.apple.com
.swcdn.apple.com
.swdist.apple.com
.swscan.apple.com
.appldnld.apple.com.edgesuite.net
.entrust.net
.digicert.com
.apple-cloudkit.com
.apple-livephotoskit.com
.gc.apple.com
.icloud-content.com
.cdn-apple.com
.icloud.com
.appattest.apple.com
.itunes.apple.com
.mzstatic.com
.itunes.com
.music.apple.com
.app-site-association.networking.apple.com
.xp.apple.com
.play.google.com
.android.com
.google-analytics.com
.googleusercontent.com
.ggpht.com
.dl.google.com
.dl-ssl.google.com
.android.clients.google.com
.android.clients.google.com
.omahaproxy.appspot.com
.payments.google.com
.googleapis.com
.notifications.google.com
.ogs.google.com
.googleapis.com
.privacyportal-bofa.my.onetrust.com
.bankofamerica.com
.mcafee.com
.kaspersky.com
.kaspersky-labs.com
.ml.com
.zoom.us
.teams.microsoft.com
.edge-chat.facebook.com
.internet.speedpay.com
.amazonvideo.com
.unagi-na.amazon.com
.events.data.microsoft.com
.caauthservice.state.gov
.studentaid.gov
.mohela.com
www.whitehouse.gov
www.rcsdk8.org
.rcsdk8.powerschool.com
www.weaveinc.org
.cdn.nintendo.net

                        regular expression file

                        #Sites to be splice
(disney\.(content|connections))\.edge\.bamgrid\.com
web-chat-e2ee\.facebook\.com\/ws\/chat	
gateway\.facebook\.com\/ws\/realtime\?
^((alt[0-9]-mtalk\.)|(mtalk\.)|(mtalk-(staging|dev)\.))google\.com
^((gvt)([0-9]))\.com
^(((clients)[0-9])|accounts)\.google\.(com|us)
^(pki|(crl|ocsp)\.pki)\.google\.com
(outlook\.)(office365|office)\.com
infinity-c[0-9][0-9]\.youboranqs[0-9][0-9]\.com

                        This change has a major improvement

                        Use of command

                        squid -k parse

                        helped direct me to use dstdomain acts over the hundreds of regex items that was causing performance issues.

                        Make sure to upvote

                        1 Reply Last reply Reply Quote 0
                        • JonathanLeeJ JonathanLee referenced this topic on
                        • JonathanLeeJ
                          JonathanLee
                          last edited by

                          Continued Research:

                          Changes to following have a massive increase in hit ratios:

                          Local Cache

                          acl block_hours time 00:30-05:00
ssl_bump terminate all block_hours
http_access deny all block_hours
acl getmethod method GET
acl to_ipv6 dst ipv6
acl from_ipv6 src ipv6

tls_outgoing_options cipher=HIGH:MEDIUM:!RC4:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!SRP:!DSS
tls_outgoing_options options=NO_SSLv3,SINGLE_DH_USE,SINGLE_ECDH_USE

acl HttpAccess dstdomain "/usr/local/pkg/http.access"
acl windowsupdate dstdomain "/usr/local/pkg/windowsupdate"
acl rewritedoms dstdomain "/usr/local/pkg/desdom"

store_id_program /usr/local/libexec/squid/storeid_file_rewrite /var/squid/storeid/storeid_rewrite.txt
store_id_children 10 startup=5 idle=1 concurrency=0


#always_direct allow !getmethod #CHANGE HERE NOT USING SQUID WITH PEERS


#store_id_access deny connect #CHANGE HERE


store_id_access deny !getmethod
store_id_access allow rewritedoms


#store_id_access deny all #CHANGE HERE

refresh_all_ims on
reload_into_ims on
max_stale 20 years
minimum_expiry_time 0

refresh_pattern -i squid\.internal 10080 80% 79900 override-lastmod override-expire ignore-reload ignore-no-store ignore-private

#FACEBOOK
refresh_pattern ^https.*.facebook.com/* 10080 80% 43200

#FACEBOOK IMAGES  
refresh_pattern -i pixel.facebook.com..(jpg|png|gif|ico|css|js|jpg?) 10080 80% 43200
refresh_pattern -i .akamaihd.net..(jpg|png|gif|ico|css|js|jpg?) 10080 80% 43200 
refresh_pattern -i facebook.com.(jpg|png|gif|jpg?) 10080 80% 43200 store-stale
refresh_pattern static.(xx|ak).fbcdn.net.(jpg|gif|png|jpg?) 10080 80% 43200
refresh_pattern ^https.*profile.ak.fbcdn.net.*(jpg|gif|png|jpg?) 10080 80% 43200
refresh_pattern ^https.*fbcdn.net.*(jpg|gif|png|jpg?) 10080 80% 43200

#FACEBOOK VIDEO
refresh_pattern -i .video.ak.fbcdn.net.*.(mp4|flv|mp3|amf) 10080 80% 43200
refresh_pattern (audio|video)/(webm|mp4) 10080 80% 43200

#APPLE STUFF
refresh_pattern -i apple.com/..(cab|exe|msi|msu|msf|asf|wmv|wma|dat|zip|dist)$ 0 80% 43200  refresh-ims

#apple update
refresh_pattern -i (download|adcdownload).apple.com/.*\.(pkg|dmg) 4320 100% 43200
refresh_pattern -i appldnld\.apple\.com 129600 100% 129600
refresh_pattern -i phobos\.apple\.com 129600 100% 129600
refresh_pattern -i iosapps\.itunes\.apple\.com 129600 100% 129600

# Updates: Windows
refresh_pattern -i microsoft.com/..(cab|exe|msi|msu|msf|asf|wma|dat|zip)$ 4320 80% 43200  refresh-ims
refresh_pattern -i windowsupdate.com/..(cab|exe|msi|msu|msf|asf|wma|wmv)|dat|zip)$ 4320 80% 43200  refresh-ims
refresh_pattern -i windows.com/..(cab|exe|msi|msu|msf|asf|wmv|wma|dat|zip)$ 4320 80% 43200  refresh-ims
refresh_pattern -i microsoft.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 4320 80% 43200 
refresh_pattern -i windowsupdate.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 4320 80% 43200 
refresh_pattern -i windows.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 4320 80% 43200 
refresh_pattern -i .*windowsupdate.com/.*\.(cab|exe) 259200 100% 259200   
refresh_pattern -i .*update.microsoft.com/.*\.(cab|exe|dll|msi|psf) 259200 100% 259200   
refresh_pattern windowsupdate.com/.*\.(cab|exe|dll|msi|psf) 10080 100% 43200 
refresh_pattern download.microsoft.com/.*\.(cab|exe|dll|msi|psf) 10080 100% 43200 
refresh_pattern www.microsoft.com/.*\.(cab|exe|dll|msi|psf) 10080 100% 43200 
refresh_pattern au.download.windowsupdate.com/.*\.(cab|exe|dll|msi|psf) 4320 100% 43200 
refresh_pattern bg.v4.pr.dl.ws.microsoft.com/.*\.(cab|exe|dll|msi|psf) 4320 100% 43200
#windows update NEW UPDATE 0.04
refresh_pattern update.microsoft.com/.*\.(cab|exe) 43200 100% 129600    
refresh_pattern ([^.]+\.)?(download|(windows)?update)\.(microsoft\.)?com/.*\.(cab|exe|msi|msp|psf) 4320 100% 43200  
refresh_pattern update.microsoft.com/.*\.(cab|exe|dll|msi|psf) 10080 100% 43200 
refresh_pattern -i \.update.microsoft.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 525600 100% 525600       
refresh_pattern -i \.windowsupdate.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 525600 100% 525600       
refresh_pattern -i \.download.microsoft.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 525600 100% 525600       
refresh_pattern -i \.ws.microsoft.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 525600 100% 525600       
    
refresh_pattern ([^.]+\.)?(cs|content[1-9]|hsar|content-origin|client-download).[steampowered|steamcontent].com/.*\.* 43200 100% 43200     
refresh_pattern ([^.]+\.)?.akamai.steamstatic.com/.*\.* 43200 100% 43200

refresh_pattern -i ([^.]+\.)?.adobe.com/.*\.(zip|exe) 43200 100% 43200
refresh_pattern -i ([^.]+\.)?.java.com/.*\.(zip|exe) 43200 100% 43200
refresh_pattern -i ([^.]+\.)?.sun.com/.*\.(zip|exe) 43200 100% 43200
refresh_pattern -i ([^.]+\.)?.oracle.com/.*\.(zip|exe|tar.gz) 43200 100% 43200

refresh_pattern -i appldnld\.apple\.com 43200 100% 43200
refresh_pattern -i ([^.]+\.)?apple.com/.*\.(ipa) 43200 100% 43200
 
refresh_pattern -i ([^.]+\.)?.google.com/.*\.(exe|crx) 10080 80% 43200
refresh_pattern -i ([^.]+\.)?g.static.com/.*\.(exe|crx) 10080 80% 43200

acl https_login url_regex -i ^https.*(login|Login).*
cache deny https_login

range_offset_limit 512 MB windowsupdate
range_offset_limit 4 MB
range_offset_limit 0
quick_abort_min -1 KB

                          cachemgr_passwd disable offline_toggle reconfigure shutdown
cachemgr_passwd CLASSFIED_REDACTED all
eui_lookup on
acl no_miss url_regex -i gateway\.facebook\.com\/ws\/realtime\?
acl no_miss url_regex -i web-chat-e2ee\.facebook\.com\/ws\/chat
acl CONNECT method CONNECT
acl wuCONNECT dstdomain www.update.microsoft.com
acl wuCONNECT dstdomain sls.microsoft.com
http_access allow CONNECT wuCONNECT localnet
http_access allow CONNECT wuCONNECT localhost
http_access allow windowsupdate localnet
http_access allow windowsupdate localhost
http_access allow HttpAccess localnet
http_access allow HttpAccess localhost
http_access deny manager
http_access deny to_ipv6
http_access deny from_ipv6

acl BrokenButTrustedServers dstdomain "/usr/local/pkg/dstdom.broken"
acl DomainMismatch ssl_error SQUID_X509_V_ERR_DOMAIN_MISMATCH
sslproxy_cert_error allow BrokenButTrustedServers DomainMismatch
sslproxy_cert_error deny all

acl splice_only src 192.168.1.8 #Tasha iPhone
acl splice_only src 192.168.1.10 #Jon iPhone
acl splice_only src 192.168.1.11 #Amazon Fire
acl splice_only src 192.168.1.15 #Tasha HP
acl splice_only src 192.168.1.16 #iPad

acl splice_only_mac arp REDACTED MAC ADDRESS
acl splice_only_mac arp REDACTED MAC ADDRESS
acl splice_only_mac arp REDACTED MAC ADDRESS
acl splice_only_mac arp REDACTED MAC ADDRESS
acl splice_only_mac arp REDACTED MAC ADDRESS

acl NoSSLIntercept ssl::server_name_regex -i "/usr/local/pkg/reg.url.nobump"
acl NoBumpDNS dstdomain "/usr/local/pkg/dns.nobump"

acl markBumped annotate_client bumped=true
acl active_use annotate_client active=true
acl bump_only src 192.168.1.3 #webtv
acl bump_only src 192.168.1.4 #toshiba
acl bump_only src 192.168.1.5 #imac
acl bump_only src 192.168.1.9 #macbook
acl bump_only src 192.168.1.13 #dell

acl bump_only_mac arp REDACTED MAC ADDRESS
acl bump_only_mac arp REDACTED MAC ADDRESS
acl bump_only_mac arp REDACTED MAC ADDRESS
acl bump_only_mac arp REDACTED MAC ADDRESS
acl bump_only_mac arp REDACTED MAC ADDRESS

ssl_bump peek step1
miss_access deny no_miss active_use
ssl_bump splice https_login active_use
ssl_bump splice splice_only_mac splice_only active_use
ssl_bump splice NoBumpDNS active_use
ssl_bump splice NoSSLIntercept active_use
ssl_bump bump bump_only_mac bump_only active_use
acl activated note active_use true
ssl_bump terminate !activated

acl markedBumped note bumped true
url_rewrite_access deny markedBumped

#workers 3
#read_ahead_gap 32 KB

negative_ttl 1 second
connect_timeout 30 seconds
request_timeout 60 seconds

#half_closed_clients off

shutdown_lifetime 10 seconds
negative_dns_ttl 1 seconds

#ignore_unknown_nameservers on
#client_persistent_connections off
#server_persistent_connections off

pipeline_prefetch 100

#acl SSLIntercept ssl::server_name_regex -i "/usr/local/pkg/url.bump"
#ssl_bump bump SSLIntercept

                          Also changes were made to utilize a SWAP partition I created a FREEBSD based swap on an external drive and or you can use a SSD drive.

                          WARNING IF YOU DO NOT KNOW HOW TO CORRECTLY PARTITION A DRVIE DO NOT ATTEMPT THIS AS YOU CAN DESTROY ALL SOFTWARE.

                          I had to use the SWAP on the SSD and or use an external drive as a swap. This was done to help with updates to ClamAV as it will start to swap until update is competed.

                          /etc/fstab

                          # Device		Mountpoint	FStype	Options		Dump	Pass#
/dev/msdosfs/EFISYS	/boot/efi	msdosfs	rw,noatime,noauto	0	0
/dev/msdosfs/DTBFAT0	/boot/msdos	msdosfs	rw,noatime,noauto	0	0
/dev/da0		none	swap	sw		0	0

                          Make sure to upvote

                          1 Reply Last reply Reply Quote 0
                          • JonathanLeeJ
                            JonathanLee
                            last edited by

                            This seems to improve speeds

                            http_upgrade_request_protocols websocket allow all 
accept_filter httpready
accept_filter dataready
collapsed_forwarding on
half_closed_clients off
pipeline_prefetch 6

                            Make sure to upvote

                            1 Reply Last reply Reply Quote 0
                            • First post
                              Last post
                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.