Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Improve Custom refresh pattern

    Scheduled Pinned Locked Moved Cache/Proxy
    111 Posts 27 Posters 50.2k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • JonathanLeeJ
      JonathanLee @High_Voltage
      last edited by

      @high_voltage I got my Raspberry Pi to work with a different mirror I edited sources to one that allowed https. When I run apt-get update it uses a different mirror now I use the constant com's mirror.

      Edit this file
      /etc/apt/sources.list

      Add a https source from the update mirrors for example in Raspberry Pi Linux I changed it to a https source.

      Screenshot_20220116-072528.png

      Screenshot_20220116-072615.png

      Check out other countries some are almost all https like Germany.

      Make sure to upvote

      JonathanLeeJ High_VoltageH 2 Replies Last reply Reply Quote 0
      • JonathanLeeJ
        JonathanLee @JonathanLee
        last edited by

        @jonathanlee it has got to work the same for refreshers for other Linux flavors also.

        Make sure to upvote

        JonathanLeeJ 1 Reply Last reply Reply Quote 0
        • JonathanLeeJ
          JonathanLee @JonathanLee
          last edited by

          @jonathanlee that was the only way to get Linux updates to work with Squid for me, it was doing the same thing as Windows updates, Squid would show a http and when you looked at Squid guard's live connections it would only show 0.

          Make sure to upvote

          JonathanLeeJ 1 Reply Last reply Reply Quote 0
          • JonathanLeeJ
            JonathanLee @JonathanLee
            last edited by

            @jonathanlee

            Made new post with this specific issue.

            https://forum.netgate.com/topic/169166/warning-possible-bypass-attempt-found-multiple-slashes-where-only-one-is-expected-http-dl-delivery-mp-microsoft-com-filestreamingservice-files/3?_=1642466910316

            Make sure to upvote

            1 Reply Last reply Reply Quote 0
            • JonathanLeeJ
              JonathanLee @KOM
              last edited by

              @kom

              Here it is, per your request, a Windows 10 update cached and delivered to another machine. Notice the HIT

              1643495363814-hit.jpeg

              (IMAGE: Windows dynamic refresh patterns to work recently)

              Make sure to upvote

              KOMK 1 Reply Last reply Reply Quote 0
              • High_VoltageH
                High_Voltage @JonathanLee
                last edited by High_Voltage

                @jonathanlee For whatever reason, it's worth noting I literally only just discovered 2 weeks ago that apparently a good chunk of my problems were due to transparent squid and clam AV, having clamAV set to scan all mode was causing random issues I cannot even begin to pinpoint. Setting it to scan Web only fixed everything, but having it set to scan all mode for whatever reason would cause apt packages To fail at trying to receive header information. Even http connections failed due to this.

                JonathanLeeJ 1 Reply Last reply Reply Quote 1
                • JonathanLeeJ
                  JonathanLee @High_Voltage
                  last edited by JonathanLee

                  @high_voltage I think it is the same as if you were to do a ClamAV scan on Kali Linux. So many packages and tools come up as issues when they are in fact only Pen Testing tools. In PFsense Curl, and many other items are included in packages and may scan as false positives also as they are not on a client machine however part of a firewall. It should have a scan Squid Cache option, that is what should be scanned right? Think about the number of items stored in the content accelerator that could be invasive. Why does squid not include scan local cache as an option?

                  Make sure to upvote

                  High_VoltageH 1 Reply Last reply Reply Quote 0
                  • High_VoltageH
                    High_Voltage @JonathanLee
                    last edited by

                    @jonathanlee no, i mean it broke traffic entirely.

                    JonathanLeeJ 1 Reply Last reply Reply Quote 0
                    • JonathanLeeJ
                      JonathanLee @High_Voltage
                      last edited by JonathanLee

                      @high_voltage wow that's different. I had issues where I needed to clear the cache before the traffic would flow again, almost like a container was in the cache.

                      Make sure to upvote

                      1 Reply Last reply Reply Quote 0
                      • KOMK
                        KOM @JonathanLee
                        last edited by

                        @jonathanlee Huh? last time I posted in this thread was 4 years ago.

                        JonathanLeeJ 1 Reply Last reply Reply Quote 0
                        • JonathanLeeJ
                          JonathanLee @KOM
                          last edited by

                          @kom sorry I thought you wanted to see a Windows 10 update run that was cached.

                          Make sure to upvote

                          KOMK 1 Reply Last reply Reply Quote 0
                          • KOMK
                            KOM @JonathanLee
                            last edited by

                            @jonathanlee Perhaps four years ago I did. I don't remember since it's been four years. I don't even use squid anymore. It's completely useless other than as a base for squidguard URL filtering.

                            JonathanLeeJ 1 Reply Last reply Reply Quote 0
                            • JonathanLeeJ
                              JonathanLee @KOM
                              last edited by JonathanLee

                              @kom I respectfully disagree with "useless", I use it for HTTPS cache anti-virus scanning of HTTPS websites and HTTP. Dynamic caching, URL filtering, and blocking. Don't get me wrong it is rather complicated to understand, however the vast abilities that it has to customize a network environment by need is what sets it apart. It can do many things. It is just a challenge to learn. It has also protected my system from many hidden issues that Clam AV stops and reports with HTTPS alongside pup detection as well as generates clear reports. It's Mirrored Analytics down to a granular level.

                              Make sure to upvote

                              1 Reply Last reply Reply Quote 0
                              • D
                                dmalick @kivimart
                                last edited by

                                @kivimart is it working for squid version - 4.45

                                A 1 Reply Last reply Reply Quote 0
                                • A
                                  aGeekhere @dmalick
                                  last edited by

                                  @dmalick You can use the latest here https://github.com/mmd123/squid-cache-dynamic_refresh-list

                                  Yes it works with the latest squid

                                  Never Fear, A Geek is Here!

                                  D 1 Reply Last reply Reply Quote 1
                                  • D
                                    dmalick @aGeekhere
                                    last edited by

                                    @ageekhere it is working thanks

                                    JonathanLeeJ 1 Reply Last reply Reply Quote 1
                                    • JonathanLeeJ
                                      JonathanLee @dmalick
                                      last edited by

                                      @dmalick keep in mind that sometimes if changes in a website are very small it will still load old information if you use ssl intercept and have it set up to cache https. I have had a issue with a time card that would not load a new line because the watermark was to low to require a refresh, however on a different machine it would see the new time card. Just be aware that things are still a work in progress.

                                      Make sure to upvote

                                      D 2 Replies Last reply Reply Quote 1
                                      • D
                                        dmalick @JonathanLee
                                        last edited by

                                        This post is deleted!
                                        1 Reply Last reply Reply Quote 0
                                        • D
                                          dmalick @JonathanLee
                                          last edited by

                                          @jonathanlee my configuration (is it fine)

                                          # This file is automatically generated by pfSense
# Do not edit manually !

http_port 192.168.1.3:3128 ssl-bump generate-host-certificates=on dynamic_cert_mem_cache_size=50MB cert=/usr/local/etc/squid/serverkey.pem cafile=/usr/local/share/certs/ca-root-nss.crt capath=/usr/local/share/certs/ cipher=EECDH+ECDSA+AESGCM:EECDH+aRSA+AESGCM:EECDH+ECDSA+SHA384:EECDH+ECDSA+SHA256:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH+aRSA+RC4:EECDH:EDH+aRSA:HIGH:!RC4:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!SRP:!DSS tls-dh=prime256v1:/etc/dh-parameters.2048 options=NO_SSLv3,SINGLE_DH_USE,SINGLE_ECDH_USE

icp_port 0
digest_generation off
dns_v4_first on
pid_filename /var/run/squid/squid.pid
cache_effective_user squid
cache_effective_group proxy
error_default_language en
icon_directory /usr/local/etc/squid/icons
visible_hostname pfSense.local.dev
cache_mgr admin@localhost
access_log /var/squid/logs/access.log
cache_log /var/squid/logs/cache.log
cache_store_log none
netdb_filename /var/squid/logs/netdb.state
pinger_enable on
pinger_program /usr/local/libexec/squid/pinger
sslcrtd_program /usr/local/libexec/squid/security_file_certgen -s /var/squid/lib/ssl_db -M 4MB -b 2048
tls_outgoing_options cafile=/usr/local/share/certs/ca-root-nss.crt
tls_outgoing_options capath=/usr/local/share/certs/
tls_outgoing_options options=NO_SSLv3,SINGLE_DH_USE,SINGLE_ECDH_USE
tls_outgoing_options cipher=EECDH+ECDSA+AESGCM:EECDH+aRSA+AESGCM:EECDH+ECDSA+SHA384:EECDH+ECDSA+SHA256:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH+aRSA+RC4:EECDH:EDH+aRSA:HIGH:!RC4:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!SRP:!DSS
tls_outgoing_options flags=DONT_VERIFY_PEER
sslcrtd_children 25
sslproxy_cert_error allow all
sslproxy_cert_adapt setValidAfter all
sslproxy_cert_adapt setValidBefore all
sslproxy_cert_adapt setCommonName all

logfile_rotate 7
debug_options rotate=7
shutdown_lifetime 3 seconds
forwarded_for delete
via off
httpd_suppress_version_string on
uri_whitespace strip


cache_mem 3072 MB
maximum_object_size_in_memory 51200 KB
memory_replacement_policy heap GDSF
cache_replacement_policy heap LFUDA
minimum_object_size 0 KB
maximum_object_size 512 MB
cache_dir aufs /var/squid/cache 20000 32 256
offline_mode off
cache_swap_low 90
cache_swap_high 95
acl donotcache dstdomain "/var/squid/acl/donotcache.acl"
cache deny donotcache
cache allow all
# Add any of your own refresh_pattern entries above these.
refresh_pattern ^ftp:    1440  20%  10080
refresh_pattern ^gopher:  1440  0%  1440
refresh_pattern -i (/cgi-bin/|\?) 0  0%  0
refresh_pattern .    0  20%  4320


#Remote proxies


# Setup some default acls
# ACLs all, manager, localhost, and to_localhost are predefined.
acl allsrc src all
acl safeports port 21 70 80 210 280 443 488 563 591 631 777 901 8080 3128 3129 1025-65535 
acl sslports port 443 563 8080 

acl purge method PURGE
acl connect method CONNECT

# Define protocols used for redirects
acl HTTP proto HTTP
acl HTTPS proto HTTPS

# SslBump Peek and Splice
# http://wiki.squid-cache.org/Features/SslPeekAndSplice
# http://wiki.squid-cache.org/ConfigExamples/Intercept/SslBumpExplicit
# Match against the current step during ssl_bump evaluation [fast]
# Never matches and should not be used outside the ssl_bump context.
#
# At each SslBump step, Squid evaluates ssl_bump directives to find
# the next bumping action (e.g., peek or splice). Valid SslBump step
# values and the corresponding ssl_bump evaluation moments are:
#   SslBump1: After getting TCP-level and HTTP CONNECT info.
#   SslBump2: After getting TLS Client Hello info.
#   SslBump3: After getting TLS Server Hello info.
# These ACLs exist even when 'SSL/MITM Mode' is set to 'Custom' so that
# they can be used there for custom configuration.
acl step1 at_step SslBump1
acl step2 at_step SslBump2
acl step3 at_step SslBump3
acl allowed_subnets src 192.168.1.44/32 192.168.1.41/32
acl whitelist dstdom_regex -i "/var/squid/acl/whitelist.acl"
acl sslwhitelist ssl::server_name_regex -i "/var/squid/acl/whitelist.acl"
acl blacklist dstdom_regex -i "/var/squid/acl/blacklist.acl"
http_access allow manager localhost

http_access deny manager
http_access allow purge localhost
http_access deny purge
http_access deny !safeports
http_access deny CONNECT !sslports

# Always allow localhost connections
http_access allow localhost

request_body_max_size 0 KB
delay_pools 1
delay_class 1 2
delay_parameters 1 -1/-1 -1/-1
delay_initial_bucket_level 100
delay_access 1 allow allsrc

# Reverse Proxy settings


# Custom options before auth


# Always allow access to whitelist domains
http_access allow whitelist
# Block access to blacklist domains
http_access deny blacklist
# Set YouTube safesearch restriction
acl youtubedst dstdomain -n www.youtube.com m.youtube.com youtubei.googleapis.com youtube.googleapis.com www.youtube-nocookie.com
request_header_access YouTube-Restrict deny all
request_header_add YouTube-Restrict none youtubedst
ssl_bump peek step1
ssl_bump splice sslwhitelist
ssl_bump bump all
# Setup allowed ACLs
http_access allow allowed_subnets
# Default block all to be sure
http_access deny allsrc

icap_enable on
icap_send_client_ip on
icap_send_client_username on
icap_client_username_encode off
icap_client_username_header X-Authenticated-User
icap_preview_enable on
icap_preview_size 1024

icap_service service_avi_req reqmod_precache icap://127.0.0.1:1344/squid_clamav bypass=on
adaptation_access service_avi_req allow all
icap_service service_avi_resp respmod_precache icap://127.0.0.1:1344/squid_clamav bypass=on
adaptation_access service_avi_resp allow all
                                          JonathanLeeJ 3 Replies Last reply Reply Quote 1
                                          • JonathanLeeJ
                                            JonathanLee @dmalick
                                            last edited by

                                            @dmalick here is mine I run a file that lets specific devices splice and specific websites.

                                            #This file is automatically generated by pfSense
                                            #Do not edit manually !
                                            http_port 192.168.1.1:3128 ssl-bump generate-host-certificates=on dynamic_cert_mem_cache_size=10MB cert=/usr/local/etc/squid/serverkey.pem cafile=/usr/local/share/certs/ca-root-nss.crt capath=/usr/local/share/certs/ cipher=EECDH+ECDSA+AESGCM:EECDH+aRSA+AESGCM:EECDH+ECDSA+SHA384:EECDH+ECDSA+SHA256:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH+aRSA+RC4:EECDH:EDH+aRSA:HIGH:!RC4:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!SRP:!DSS tls-dh=prime256v1:/etc/dh-parameters.2048 options=NO_SSLv3,SINGLE_DH_USE,SINGLE_ECDH_USE

                                            http_port 127.0.0.1:3128 intercept ssl-bump generate-host-certificates=on dynamic_cert_mem_cache_size=10MB cert=/usr/local/etc/squid/serverkey.pem cafile=/usr/local/share/certs/ca-root-nss.crt capath=/usr/local/share/certs/ cipher=EECDH+ECDSA+AESGCM:EECDH+aRSA+AESGCM:EECDH+ECDSA+SHA384:EECDH+ECDSA+SHA256:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH+aRSA+RC4:EECDH:EDH+aRSA:HIGH:!RC4:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!SRP:!DSS tls-dh=prime256v1:/etc/dh-parameters.2048 options=NO_SSLv3,SINGLE_DH_USE,SINGLE_ECDH_USE

                                            https_port 127.0.0.1:3129 intercept ssl-bump generate-host-certificates=on dynamic_cert_mem_cache_size=10MB cert=/usr/local/etc/squid/serverkey.pem cafile=/usr/local/share/certs/ca-root-nss.crt capath=/usr/local/share/certs/ cipher=EECDH+ECDSA+AESGCM:EECDH+aRSA+AESGCM:EECDH+ECDSA+SHA384:EECDH+ECDSA+SHA256:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH+aRSA+RC4:EECDH:EDH+aRSA:HIGH:!RC4:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!SRP:!DSS tls-dh=prime256v1:/etc/dh-parameters.2048 options=NO_SSLv3,SINGLE_DH_USE,SINGLE_ECDH_USE

                                            icp_port 3130
                                            digest_generation off
                                            dns_v4_first on
                                            pid_filename /var/run/squid/squid.pid
                                            cache_effective_user squid
                                            cache_effective_group proxy
                                            error_default_language en
                                            icon_directory /usr/local/etc/squid/icons
                                            visible_hostname proxy.pfsense.secure
                                            cache_mgr jonathanlee571@gmail.com
                                            access_log /var/squid/logs/access.log
                                            cache_log /var/squid/logs/cache.log
                                            cache_store_log none
                                            netdb_filename /var/squid/logs/netdb.state
                                            pinger_enable on
                                            pinger_program /usr/local/libexec/squid/pinger
                                            sslcrtd_program /usr/local/libexec/squid/security_file_certgen -s /var/squid/lib/ssl_db -M 4MB -b 2048
                                            tls_outgoing_options cafile=/usr/local/share/certs/ca-root-nss.crt
                                            tls_outgoing_options capath=/usr/local/share/certs/
                                            tls_outgoing_options options=NO_SSLv3,SINGLE_DH_USE,SINGLE_ECDH_USE
                                            tls_outgoing_options cipher=EECDH+ECDSA+AESGCM:EECDH+aRSA+AESGCM:EECDH+ECDSA+SHA384:EECDH+ECDSA+SHA256:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH+aRSA+RC4:EECDH:EDH+aRSA:HIGH:!RC4:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!SRP:!DSS
                                            sslcrtd_children 5

                                            logfile_rotate 0
                                            debug_options rotate=0
                                            shutdown_lifetime 3 seconds
                                            #Allow local network(s) on interface(s)
                                            acl localnet src 192.168.0.0/16
                                            forwarded_for on
                                            via off
                                            httpd_suppress_version_string on
                                            uri_whitespace strip

                                            #WINDOWS
                                            refresh_pattern -i windowsupdate.com/..(cab|exe|dll|ms[i|u|f|p]|[ap]sf|wm[v|a]|dat|zip|psf) 43200 80% 129600 reload-into-ims
                                            refresh_pattern -i microsoft.com/.                                            .(cab|exe|dll|ms[i|u|f|p]|[ap]sf|wm[v|a]|dat|zip|psf) 43200 80% 129600 reload-into-ims
                                            refresh_pattern -i windows.com/..(cab|exe|dll|ms[i|u|f|p]|[ap]sf|wm[v|a]|dat|zip|psf) 43200 80% 129600 reload-into-ims
                                            refresh_pattern -i microsoft.com.akadns.net/.                                            .(cab|exe|dll|ms[i|u|f|p]|[ap]sf|wm[v|a]|dat|zip|psf) 43200 80% 129600 reload-into-ims
                                            refresh_pattern -i deploy.akamaitechnologies.com/..(cab|exe|dll|ms[i|u|f|p]|[ap]sf|wm[v|a]|dat|zip|psf) 43200 80% 129600 reload-into-ims
                                            refresh_pattern -i msedge.b.tlu.dl.delivery.mp.microsoft.com/.                                            .(cab|exe|dll|ms[i|u|f|p]|[ap]sf|wm[v|a]|dat|zip|psf) 43200 80% 129600 reload-into-ims

                                            #FACEBOOK
                                            refresh_pattern ^http.facebook.com. 720 100% 4320

                                            #FACEBOOK IMAGES
                                            refresh_pattern -i pixel.facebook.com..(jpg|png|gif|ico|css|js) 241920 80% 241920
                                            refresh_pattern -i .akamaihd.net..(jpg|png|gif|ico|css|js) 241920 80% 241920
                                            refresh_pattern -i ((facebook.com)|(85.131.151.39)).(jpg|png|gif) 241920 99% 241920 store-stale
                                            refresh_pattern static.(xx|ak).fbcdn.net.(jpg|gif|png) 241920 99% 241920
                                            refresh_pattern ^http.profile.ak.fbcdn.net.(jpg|gif|png) 241920 99% 241920

                                            #FACEBOOK VIDEO
                                            refresh_pattern -i .video.ak.fbcdn.net.*.(mp4|flv|mp3|amf) 10080 80% 43200
                                            refresh_pattern (audio|video)/(webm|mp4) 129600 99% 129600 store-stale
                                            #refresh_pattern -i ^http.squid.internal. 241920 100% 241920 store-stale

                                            #YAHOO
                                            refresh_pattern ^http.mail.yahoo.com. 720 100% 4320
                                            refresh_pattern ^http.yahoo. 720 100% 4320
                                            refresh_pattern ^http.yimg. 720 100% 4320

                                            #apple update
                                            refresh_pattern -i apple.com/..(cab|exe|msi|msu|msf|asf|wmv|wma|dat|zip|dist)$ 0 80% 43200 refresh-ims
                                            #refresh_pattern -i (download|adcdownload).apple.com/.*.(pkg|dmg) 4320 100% 43200
                                            refresh_pattern -i appldnld.apple.com 129600 100% 129600
                                            refresh_pattern -i phobos.apple.com 129600 100% 129600
                                            refresh_pattern -i iosapps.itunes.apple.com 129600 100% 129600

                                            #GOOGLE STUFF
                                            refresh_pattern ^http.gmail. 720 100% 4320
                                            refresh_pattern ^http.google. 720 100% 4320

                                            #AntiVirus
                                            refresh_pattern ^http.kaspersky. 43200 100% 43200
                                            refresh_pattern ^http.geo.kaspersky.com. 43200 100% 43200
                                            refresh_pattern kaspersky.*.avc$ 43200 100% 43200
                                            refresh_pattern kaspersky 43200 100% 43200

                                            #Office/Windows
                                            refresh_pattern ^http.officecdn.microsoft.com. 43200 100% 43200
                                            refresh_pattern ^http.officecdn.microsoft.com.edgesuite.net. 43200 100% 43200
                                            refresh_pattern ^http.delivery.mp.microsoft.com/filestreamingservice/files. 43200 100% 43200
                                            refresh_pattern ^http.download.windowsupdate.com. 43200 100% 43200
                                            refresh_pattern ^http.download.windowsupdate.com/. 43200 100% 43200
                                            refresh_pattern ^http.au.download.windowsupdate.com/. 43200 100% 43200
                                            refresh_pattern ^http.tlu.dl.delivery.mp.microsoft.com/filestreamingservice/files. 43200 100% 43200

                                            #Website
                                            refresh_pattern -i (.|-)(xml|js|jsp|txt|css)(?.*)?$ 360 40% 1440
                                            #end new refresh patterns

                                            refresh_pattern .(ico|video-stats)$ 129600 100% 129600

                                            #MISC FILE CACHING HERE
                                            refresh_pattern -i .(3gp|7z|ace|asx|avi|bin|cab|dat|deb|rpm|divx|dvr-ms)(?|$) 43800 100% 129600
                                            refresh_pattern -i .(rar|jar|gz|tgz|tar|bz2|iso)(?|$) 43800 100% 129600
                                            refresh_pattern -i .(m1v|M2V|M2P|MOD|MOV|FLV)(?|$) 43800 100% 129600
                                            refresh_pattern -i .(jp(e?g|e|2)|gif|pn[pg]|bm?|tiff?|ico|swf|css|js)(?|$) 43800 100% 129600
                                            refresh_pattern -i .(mp(e?g|a|e|1|2|3|4)|mk(a|v)|ms(i|u|p))(?|$) 43800 100% 129600
                                            refresh_pattern -i .(og(x|v|a|g)|rar|rm|r(a|p)m|snd|vob|wav)(?|$) 43800 100% 129600
                                            refresh_pattern -i .(pp(s|t)|wax|wm(a|v)|wmx|wpl|zip|cb(r|z|t))(?|$) 43800 100% 129600
                                            refresh_pattern -i .(woff|txt|exe|dmg|webm)(?|$) 43800 100% 129600
                                            refresh_pattern -i .(css)(?|$) 10080 60% 43800 # CSS
                                            refresh_pattern -i .(js)(?|$) 10080 60% 10080 # JS
                                            refresh_pattern -i .(doc|pdf)(?|$) 10080 90% 43200 # DOC | PDF
                                            refresh_pattern -i .(html|htm)(?|$) 1440 60% 10080 # HTML | HTM
                                            refresh_pattern -i .(iso|avi|wav|mp3|mp4|mpeg|swf|flv|x-flv)$ 43200 90% 432000
                                            refresh_pattern -i .(deb|rpm|exe|zip|tar|tgz|ram|rar|bin|ppt|doc|docx|tiff)$ 10080 90% 43200
                                            refresh_pattern -i .(html|htm|css|js)$ 1440 40% 40320
                                            refresh_pattern -i .index.(html|htm)$ 0 40% 10080
                                            refresh_pattern -i .(ppt|pptx|doc|docx|docm|docb|dot|pdf|pub|ps)$ 100000 90% 200000 refresh-ims
                                            refresh_pattern -i .(xls|xlsx|xlt|xlm|xlsm|xltm|xlw|csv|txt)$ 100000 90% 200000 refresh-ims
                                            refresh_pattern -i .(app|bin|deb|rpm|drpm|exe|zip|zipx|tar|tgz|tbz2|tlz|iso|arj|cfs|dar|jar)$ 100000 90% 200000 refresh-ims
                                            refresh_pattern -i .(bz|bz2|ipa|ram|rar|uxx|gz|msi|dll|lz|lzma|7z|s7z|Z|z|zz|sz)$ 100000 90% 200000 refresh-ims
                                            refresh_pattern -i .(exe|msi)$ 0 90% 200000 refresh-ims
                                            refresh_pattern -i .(cab|psf|vidt|apk|wtex|hz|ova|ovf)$ 100000 90% 200000 refresh-ims
                                            refresh_pattern -i .(xml|flow|asp|aspx)$ 0 90% 200000 refresh-ims
                                            refresh_pattern -i .(json)$ 0 90% 200000 refresh-ims
                                            refresh_pattern -i .(asx|mp2|mp3|mp4|mp5|wmv|flv|mts|f4v|f4|pls|midi|mid)$ 100000 90% 200000 refresh-ims
                                            refresh_pattern -i .(mpa|m2a|mpe|avi|mov|mpg|mpeg|mpg3|mpg4|mpg5)$ 100000 90% 200000 refresh-ims
                                            refresh_pattern -i .(m1s|mp2v|m2v|m2s|m2ts|mp2t|wmx|rm|rmvb|3pg|3gpp|omg|ogm|asf|war)$ 100000 90% 200000 refresh-ims
                                            refresh_pattern -i .(swf|js|ejs)$ 100000 90% 200000 refresh-ims
                                            refresh_pattern -i .(wav|css|class|dat|zsci|ver|advcs)$ 100000 90% 200000 refresh-ims
                                            refresh_pattern -i .(gif|png|ico|jpg|jpeg|jp2|webp)$ 100000 90% 200000 refresh-ims
                                            refresh_pattern -i .(jpx|j2k|j2c|fpx|bmp|tif|tiff|bif)$ 100000 90% 20000 refresh-ims
                                            refresh_pattern -i .(pcd|pict|rif|exif|hdr|bpg|img|jif|jfif)$ 100000 90% 200000 refresh-ims
                                            refresh_pattern -i .(woff|woff2|eps|ttf|otf|svg|svgi|svgz|ps|ps1|acsm|eot)$ 100000 90% 200000 refresh-ims
                                            refresh_pattern -i (.|-)(mid|midi|mpg|mpeg|ram|cav|acc|alz|apk|at3|bke|arc|ass|ba|big|bik|bkf|bld|c4|cals|clipflair|cpt|daa|dmg|ddz|dpe|egg|egt|ecab|ess|gho|ghs|gz|ipg|jar|lbr|lqr|lha|lz|lzo|lzma|lzx|mbw|mc.meta|mpq|nth|osz|pak|par|par2|paf|pyk|pk3|pk4|rag|sen|sitx|skb|tb|tib|uha|uue|viv|vsa|z|zoo|nrg|adf|adz|dms|dsk|d64|sdi|mds|mdx|cdi|cue|cif|c2d|daa|b6t)(?.*)?$ 43200 100% 432000
                                            refresh_pattern -i (.|-)(mp3|m4a|aa?c3?|wm?av?|og(x|v|a|g)|ape|mka|au|aiff|zip|flac|m4(b|r)|m1v|m2(v|p)|mo(d|v)|arj|appx|lha|lzh|on2) 43200 100% 432000
                                            refresh_pattern -i (.|-)(exe|bin|(n|t)ar|acv|(r|j)ar|t?gz|(g|b)z(ip)?2?|7?z(ip)?|wm[v|a]|patch|diff|mar|vpu|inc|r(a|p)m|kom|iso|sys|[ap]sf|ms[i|u|f]|dat|msi|cab|psf|dvr-ms|ace|asx|qt|xt|esd) 43200 100% 432000
                                            refresh_pattern -i (.|-)(ico(.)?|pn[pg]|css|(g|t)iff?|jpe?g(2|3|4)?|psd|c(d|b)r|cad|bmp|img) 43200 100% 432000
                                            refresh_pattern -i (.|-)(webm|(x-)?swf|mp(eg)?(3|4)|mpe?g(av)?|(x-)?f(l|4)v|divx?|rmvb?|mov|trp|ts|avi|m38u|wmv|wmp|m4v|mkv|asf|dv|vob|3gp?2?) 43200 100% 432000
                                            refresh_pattern -i (.|-)(docx?|xlsx?|pptx?|rtf|xml|pdf|tiff?|txt) 43200 100% 432000

                                            #new refresh patterns 2
                                            refresh_pattern -i (.|-)(ini|def|sig|upt|mid|midi|mpg|mpeg|ram|cav|acc|alz|apk|at3|bke|arc|ass|ba|big|bik|bkf|bld|c4|cals|clipflair|cpt|daa|dmg|ddz|dpe|egg|egt|ecab|ess|esd|gho|ghs|gz|ipg|jar|lbr|lqr|lha|lz|lzo|lzma|lzx|mbw|mc.meta|mpq|nth|osz|pak|par|par2|paf|pyk|pk3|pk4|rag|sen|sitx|skb|tb|tib|uha|uue|viv|vsa|z|zoo|nrg|adf|adz|dms|dsk|d64|sdi|mds|mdx|cdi|cue|cif|c2d|daa|b6t)(?.*)?$ 43200 100% 432000
                                            #end new refresh patterns 2

                                            #new refresh patterns
                                            refresh_pattern -i (.|-)(mp3|m4a|aa?c3?|wm?av?|og(x|v|a|g)|ape|mka|au|aiff|zip|flac|m4(b|r)|m1v|m2(v|p)|mo(d|v)|arj|appx|lha|lzh|on2)(?.)?$ 43200 100% 432000
                                            refresh_pattern -i (.|-)(exe|bin|(n|t)ar|acv|(r|j)ar|t?gz|(g|b)z(ip)?2?|7?z(ip)?|wm[v|a]|patch|diff|mar|vpu|inc|r(a|p)m|kom|iso|sys|[ap]sf|ms[i|u|f]|dat|msi|cab|psf|dvr-ms|ace|asx|qt|xt|esd)(?.                                            )?$ 43200 100% 432000
                                            refresh_pattern -i (.|-)(ico(.)?|pn[pg]|css|(g|t)iff?|jpe?g(2|3|4)?|psd|c(d|b)r|cad|bmp|img)(?.)?$ 43200 100% 432000
                                            refresh_pattern -i (.|-)(webm|(x-)?swf|mp(eg)?(3|4)|mpe?g(av)?|(x-)?f(l|4)v|divx?|rmvb?|mov|trp|ts|avi|m38u|wmv|wmp|m4v|mkv|asf|dv|vob|3gp?2?)(?.)?$ 43200 100% 432000
                                            refresh_pattern -i (.|-)(docx?|xlsx?|pptx?|rtf|xml|pdf|tiff?|txt)(?.                                            )?$ 43200 100% 432000
                                            refresh_pattern -i .(rar|jar|gz|tgz|tar|bz2|iso|m1v|m2(v|p)|mo(d|v)|flv) 129600 100% 129600
                                            refresh_pattern (Release|Packages(.gz)*)$ 0 20% 2880

                                            #GENERIC CACHING BELOW
                                            refresh_pattern -i .(cdn) 10800 100% 43800
                                            refresh_pattern -i (cdn) 10800 100% 43800
                                            refresh_pattern -i (.|-)(xml|js|jsp|txt|css)?$ 360 40% 1440
                                            refresh_pattern (get_video?|videoplayback?|videodownload?|.flv?) 129600 100% 129600
                                            refresh_pattern (get_video?|videoplayback?id|videoplayback.id|videodownload?|.flv?) 129600 100% 129600
                                            refresh_pattern ^.                                            (utm.gif|ads?|rmxads.com|ad.z5x.net|bh.contextweb.com|bstats.adbrite.com|a1.interclick.com|ad.trafficmp.com|ads.cubics.com|ad.xtendmedia.com|.googlesyndication.com|advertising.com|yieldmanager|game-advertising.com|pixel.quantserve.com|adperium.com|doubleclick.net|adserving.cpxinteractive.com|syndication.com|media.fastclick.net).* 129600 20% 129600
                                            refresh_pattern ^.*safebrowsing.google 129600 100% 129600
                                            refresh_pattern ^http://((cbk|mt|khm|mlt)[0-9]?).google.co(m|.uk) 129600 100% 129600
                                            refresh_pattern ytimg.com.                                            .jpg 129600 100% 129600

                                            #YOUTUBE
                                            refresh_pattern .ytimg? 10800 90% 10800
                                            refresh_pattern -i (yimg|twimg).com.* 1440 100% 129600
                                            refresh_pattern -i (ytimg|ggpht).com.* 1440 80% 129600
                                            refresh_pattern -i (get_video?|videoplayback?|videodownload?|.mp4|.webm|.flv|((audio|video)/(webm|mp4))) 241920 100% 241920 store-stale
                                            refresh_pattern -i ^https?://..googlevideo.com/videoplayback. 10080 99% 43200 store-stale
                                            refresh_pattern -i ^https?://..googlevideo.com/videoplayback.$ 241920 100% 241920 store-stale

                                            #XBOX
                                            refresh_pattern -i lancache-microsoft 525600 100% 525600
                                            refresh_pattern -i images-eds.xboxlive.com 525600 100% 525600

                                            #catch all
                                            refresh_pattern -i (/cgi-bin/|?) 0 0% 0
                                            refresh_pattern . 0 30% 43200
                                            range_offset_limit 0

                                            cache_mem 64 MB
                                            maximum_object_size_in_memory 256 KB
                                            memory_replacement_policy heap GDSF
                                            cache_replacement_policy heap LFUDA
                                            minimum_object_size 0 KB
                                            maximum_object_size 5000 MB
                                            cache_dir diskd /var/squid/cache 15000 64 256
                                            offline_mode off
                                            cache_swap_low 90
                                            cache_swap_high 95
                                            acl donotcache dstdomain '/var/squid/acl/donotcache.acl'
                                            cache deny donotcache
                                            cache allow all
                                            #Add any of your own refresh_pattern entries above these.
                                            refresh_pattern ^ftp: 1440 20% 10080
                                            refresh_pattern ^gopher: 1440 0% 1440
                                            refresh_pattern -i (/cgi-bin/|?) 0 0% 0
                                            refresh_pattern . 0 20% 4320

                                            #Remote proxies

                                            #Setup some default acls
                                            #ACLs all, manager, localhost, and to_localhost are predefined.
                                            acl allsrc src all
                                            acl safeports port 21 70 80 210 280 443 488 563 591 631 777 901 8080 3128 3129 1025-65535 3128 3129 1344 3130 5228-5230
                                            acl sslports port 443 563 8080 8854 8810 8806 1344 3128 3129 3130 5228-5230

                                            acl purge method PURGE
                                            acl connect method CONNECT

                                            #Define protocols used for redirects
                                            acl HTTP proto HTTP
                                            acl HTTPS proto HTTPS

                                            #SslBump Peek and Splice
                                            #http://wiki.squid-cache.org/Features/SslPeekAndSplice
                                            #http://wiki.squid-cache.org/ConfigExamples/Intercept/SslBumpExplicit
                                            #Match against the current step during ssl_bump evaluation [fast]
                                            #Never matches and should not be used outside the ssl_bump context.

                                            #At each SslBump step, Squid evaluates ssl_bump directives to find
                                            #the next bumping action (e.g., peek or splice). Valid SslBump step
                                            #values and the corresponding ssl_bump evaluation moments are:
                                            #SslBump1: After getting TCP-level and HTTP CONNECT info.
                                            #SslBump2: After getting TLS Client Hello info.
                                            #SslBump3: After getting TLS Server Hello info.
                                            #These ACLs exist even when 'SSL/MITM Mode' is set to 'Custom' so that
                                            #they can be used there for custom configuration.
                                            acl step1 at_step SslBump1
                                            acl step2 at_step SslBump2
                                            acl step3 at_step SslBump3
                                            acl allowed_subnets src 192.168.1.0/24 192.168.20.0/24 127.0.0.1/32
                                            acl unrestricted_hosts src '/var/squid/acl/unrestricted_hosts.acl'
                                            acl whitelist dstdom_regex -i '/var/squid/acl/whitelist.acl'
                                            http_access allow manager localhost

                                            http_access deny manager
                                            http_access allow purge localhost
                                            http_access deny purge
                                            http_access deny !safeports
                                            http_access deny CONNECT !sslports

                                            #Always allow localhost connections
                                            http_access allow localhost

                                            quick_abort_min 512 KB
                                            quick_abort_max 512 KB
                                            quick_abort_pct 90
                                            request_body_max_size 0 KB
                                            delay_pools 1
                                            delay_class 1 2
                                            delay_parameters 1 -1/-1 -1/-1
                                            delay_initial_bucket_level 100

                                            Do not throttle unrestricted hosts

                                            delay_access 1 deny unrestricted_hosts
                                            delay_access 1 allow allsrc

                                            #Reverse Proxy settings

                                            #Package Integration
                                            url_rewrite_program /usr/local/bin/squidGuard -c /usr/local/etc/squidGuard/squidGuard.conf
                                            url_rewrite_bypass off
                                            url_rewrite_children 16 startup=8 idle=4 concurrency=0

                                            #Custom options before auth

                                            #These hosts do not have any restrictions
                                            http_access allow unrestricted_hosts
                                            #Always allow access to whitelist domains
                                            http_access allow whitelist
                                            #Set YouTube safesearch restriction
                                            acl youtubedst dstdomain -n www.youtube.com m.youtube.com youtubei.googleapis.com youtube.googleapis.com www.youtube-nocookie.com
                                            request_header_access YouTube-Restrict deny all
                                            request_header_add YouTube-Restrict moderate youtubedst
                                            acl sglog url_regex -i sgr=ACCESSDENIED
                                            http_access deny sglog
                                            #Custom SSL/MITM options before auth
                                            acl splice_only src 192.168.20.11 #Xbox
                                            acl splice_only src 192.168.1.11 #Amazon Fire
                                            acl splice_only src 192.168.1.10 #Tasha Work
                                            acl NoSSLIntercept ssl::server_name_regex -i '/usr/local/pkg/url.nobump'
                                            ssl_bump peek step1 all
                                            ssl_bump splice NoSSLIntercept
                                            ssl_bump splice splice_only
                                            ssl_bump stare step2
                                            ssl_bump bump step3

                                            #Setup allowed ACLs
                                            #Allow local network(s) on interface(s)
                                            http_access allow allowed_subnets
                                            http_access allow localnet

                                            Default block all to be sure

                                            http_access deny allsrc

                                            icap_enable on
                                            icap_send_client_ip on
                                            icap_send_client_username off
                                            icap_client_username_encode off
                                            icap_client_username_header X-Authenticated-User
                                            icap_preview_enable on
                                            icap_preview_size 1024

                                            icap_service service_avi_req reqmod_precache icap://127.0.0.1:1344/squid_clamav bypass=off
                                            adaptation_access service_avi_req allow all
                                            icap_service service_avi_resp respmod_precache icap://127.0.0.1:1344/squid_clamav bypass=on
                                            adaptation_access service_avi_resp allow all

                                            Make sure to upvote

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.