Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Improve Custom refresh pattern

    Scheduled Pinned Locked Moved Cache/Proxy
    111 Posts 27 Posters 46.2k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • A
      avdh
      last edited by

      Services -> Squid Proxy Server -> Local Cache Tab - > Look at bottom of page under Custom refresh_patterns

      1 Reply Last reply Reply Quote 0
      • N
        Ninno
        last edited by

        Windows 10 update use also .esd files

        1 Reply Last reply Reply Quote 0
        • C
          CardinS2U
          last edited by

          thanks! good to have a decent pattern to start with.

          1 Reply Last reply Reply Quote 0
          • V
            Valeriy
            last edited by

            FYI, Squid -k parse is complaining that 'ignore-no-cache' is obsolete.. So it has no effect.

            Running 2.3.3-DEVELOPMENT (amd64) built on Sat Feb 04 16:18:22 CST 2017 with Squid 3.5.23

            Processing: refresh_pattern -i (\.|-)(mid|midi|mpg|mpeg|ram|cav|acc|alz|apk|at3|bke|arc|ass|ba|big|bik|bkf|bld|c4|cals|clipflair|cpt|daa|dmg|ddz|dpe|egg|egt|ecab|ess|gho|ghs|gz|ipg|jar|lbr|lqr|lha|lz|lzo|lzma|lzx|mbw|mc.meta|mpq|nth|osz|pak|par|par2|paf|pyk|pk3|pk4|rag|sen|sitx|skb|tb|tib|uha|uue|viv|vsa|z|zoo|nrg|adf|adz|dms|dsk|d64|sdi|mds|mdx|cdi|cue|cif|c2d|daa|b6t)(\?.*)?$ 43200 100% 432000 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-no-store ignore-private ignore-auth
            UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
            Processing: refresh_pattern -i (\.|-)(mp3|m4a|aa?c3?|wm?av?|og(x|v|a|g)|ape|mka|au|aiff|zip|flac|m4(b|r)|m1v|m2(v|p)|mo(d|v)|arj|appx|lha|lzh|on2)(\?.*)?$ 43200 100% 432000 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-no-store ignore-private ignore-auth
            UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
            Processing: refresh_pattern -i (\.|-)(exe|bin|(n|t)ar|acv|(r|j)ar|t?gz|(g|b)z(ip)?2?|7?z(ip)?|wm[v|a]|patch|diff|mar|vpu|inc|r(a|p)m|kom|iso|sys|[ap]sf|ms[i|u|f]|dat|msi|cab|psf|dvr-ms|ace|asx|qt|xt|esd)(\?.*)?$ 43200 100% 432000 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-no-store ignore-private ignore-auth
            UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
            Processing: refresh_pattern -i (\.|-)(ico(.*)?|pn[pg]|css|(g|t)iff?|jpe?g(2|3|4)?|psd|c(d|b)r|cad|bmp|img)(\?.*)?$ 43200 100% 432000 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-no-store ignore-private ignore-auth
            UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
            Processing: refresh_pattern -i (\.|-)(webm|(x-)?swf|mp(eg)?(3|4)|mpe?g(av)?|(x-)?f(l|4)v|divx?|rmvb?|mov|trp|ts|avi|m38u|wmv|wmp|m4v|mkv|asf|dv|vob|3gp?2?)(\?.*)?$ 43200 100% 432000 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-no-store ignore-private ignore-auth
            UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
            Processing: refresh_pattern -i (\.|-)(docx?|xlsx?|pptx?|rtf|xml|pdf|tiff?|txt)(\?.*)?$ 43200 100% 432000 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-no-store ignore-private ignore-auth
            UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
            Processing: refresh_pattern -i (\.|-)(xml|js|jsp|txt|css)(\?.*)?$ 360 40% 1440 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-no-store ignore-private ignore-auth
            UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
            Processing: refresh_pattern -i .index.(html|htm)$ 0 40% 1440
            Processing: refresh_pattern . 30 25% 1440
            Processing: refresh_pattern -i \.(3gp|7z|ace|asx|avi|bin|cab|dat|deb|rpm|divx|dvr-ms)      129600 100% 129600 ignore-no-cache ignore-no-store ignore-private override-expire override-lastmod reload-into-ims ignore-reload
            UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
            Processing: refresh_pattern -i \.(rar|jar|gz|tgz|tar|bz2|iso|m1v|m2(v|p)|mo(d|v)|(x-|)flv) 129600 100% 129600 ignore-no-cache ignore-no-store ignore-private override-expire override-lastmod reload-into-ims ignore-reload
            UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
            /usr/local/etc/squid/squid.conf line 52: refresh_pattern -i \.(rar|jar|gz|tgz|tar|bz2|iso|m1v|m2(v|p)|mo(d|v)|(x-|)flv) 129600 100% 129600 ignore-no-cache ignore-no-store ignore-private override-expire override-lastmod reload-into-ims ignore-reload
            refreshAddToList: Invalid regular expression '\.(rar|jar|gz|tgz|tar|bz2|iso|m1v|m2(v|p)|mo(d|v)|(x-|)flv)': empty (sub)expression
            Processing: refresh_pattern -i \.(jp(e?g|e|2)|gif|pn[pg]|bm?|tiff?|ico|swf|css|js)         129600 100% 129600 ignore-no-cache ignore-no-store ignore-private override-expire override-lastmod reload-into-ims ignore-reload
            UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
            Processing: refresh_pattern -i \.(mp(e?g|a|e|1|2|3|4)|mk(a|v)|ms(i|u|p))                   129600 100% 129600 ignore-no-cache ignore-no-store ignore-private override-expire override-lastmod reload-into-ims ignore-reload
            UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
            Processing: refresh_pattern -i \.(og(x|v|a|g)|rar|rm|r(a|p)m|snd|vob|wav)                  129600 100% 129600 ignore-no-cache ignore-no-store ignore-private override-expire override-lastmod reload-into-ims ignore-reload
            UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
            Processing: refresh_pattern -i \.(pp(s|t)|wax|wm(a|v)|wmx|wpl|zip|cb(r|z|t))               129600 100% 129600 ignore-no-cache ignore-no-store ignore-private override-expire override-lastmod reload-into-ims ignore-reload
            UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
            Processing: refresh_pattern ^gopher:  1440  0%  1440
            Processing: refresh_pattern ^ftp:    10080 95% 43200 override-lastmod reload-into-ims
            Processing: refresh_pattern -i \.(doc|pdf)$           100080 90% 43200 override-expire ignore-no-cache ignore-no-store ignore-private reload-into-ims
            UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
            Processing: refresh_pattern -i \.(html|htm)$          1440   40% 40320 ignore-no-cache ignore-no-store ignore-private override-expire reload-into-ims
            UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
            Processing: refresh_pattern (Release|Packages(.gz)*)$    0   20%  2880
            Processing: refresh_pattern .                          180   95% 43200 override-lastmod reload-into-ims
            Processing: refresh_pattern -i (/cgi-bin/|\?)         0      0%      0
            Processing: refresh_pattern \.(ico|video-stats)$ 129600 100% 129600 override-expire ignore-reload ignore-no-cache ignore-no-store ignore-private ignore-auth override-lastmod ignore-must-revalidate
            UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
            Processing: refresh_pattern imeem.*\.flv$                           0     0%         0 override-lastmod override-expire
            Processing: refresh_pattern \.rapidshare.*\/[0-9]*\/.*\/[^\/]* 161280    90%    161280 ignore-reload
            Processing: refresh_pattern (get_video\?|videoplayback\?|videodownload\?|\.flv?) 129600 100% 129600 ignore-no-cache ignore-no-store ignore-private override-expire override-lastmod reload-into-ims
            UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
            Processing: refresh_pattern (get_video\?|videoplayback\?id|videoplayback.*id|videodownload\?|\.flv?) 129600 100% 129600 ignore-no-cache ignore-no-store ignore-private override-expire override-lastmod reload-into-ims
            UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
            Processing: refresh_pattern ^.*(utm\.gif|ads\?|rmxads\.com|ad\.z5x\.net|bh\.contextweb\.com|bstats\.adbrite\.com|a1\.interclick\.com|ad\.trafficmp\.com|ads\.cubics\.com|ad\.xtendmedia\.com|\.googlesyndication\.com|advertising\.com|yieldmanager|game-advertising\.com|pixel\.quantserve\.com|adperium\.com|doubleclick\.net|adserving\.cpxinteractive\.com|syndication\.com|media.fastclick.net).* 129600 20% 129600 ignore-no-cache ignore-no-store ignore-private override-expire ignore-reload ignore-auth ignore-must-revalidate
            UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
            Processing: refresh_pattern ^.*safebrowsing.*google                                  129600 100% 129600 override-expire ignore-reload ignore-no-cache ignore-private ignore-auth ignore-must-revalidate
            UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
            Processing: refresh_pattern ^http://((cbk|mt|khm|mlt)[0-9]?)\.google\.co(m|\.uk)     129600 100% 129600 override-expire ignore-reload ignore-private
            Processing: refresh_pattern ytimg\.com.*\.jpg                                        129600 100% 129600 override-expire ignore-reload
            Processing: refresh_pattern images\.friendster\.com.*\.(png|gif)                     129600 100% 129600 override-expire ignore-reload
            Processing: refresh_pattern garena\.com                                              129600 100% 129600 override-expire reload-into-ims
            Processing: refresh_pattern photobucket.*\.(jp(e?g|e|2)|tiff?|bmp|gif|png)           129600 100% 129600 override-expire ignore-reload
            Processing: refresh_pattern vid\.akm\.dailymotion\.com.*\.on2\?                      129600 100% 129600 ignore-no-cache override-expire override-lastmod
            UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
            Processing: refresh_pattern mediafire.com\/images.*\.(jp(e?g|e|2)|tiff?|bmp|gif|png) 129600 100% 129600 reload-into-ims override-expire ignore-private
            Processing: refresh_pattern ^http:\/\/images|pics|thumbs[0-9]\.                      129600 100% 129600 reload-into-ims ignore-no-cache ignore-no-store ignore-reload override-expire
            UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
            Processing: refresh_pattern ^http:\/\/www.onemanga.com.*\/                           129600 100% 129600 reload-into-ims ignore-no-cache ignore-no-store ignore-reload override-expire
            UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
            Processing: refresh_pattern guru.avg.com/.*\.(bin)                              43200 100% 43200  ignore-no-cache ignore-no-store ignore-reload reload-into-ims
            UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
            Processing: refresh_pattern (avgate|avira).*(idx|gz)$                           43200 100% 43200  ignore-no-cache ignore-no-store ignore-reload reload-into-ims
            UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
            Processing: refresh_pattern kaspersky.*\.avc$                                   43200 100% 43200  ignore-no-cache ignore-no-store ignore-reload reload-into-ims
            UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
            Processing: refresh_pattern kaspersky                                           43200 100% 43200  ignore-no-cache ignore-no-store ignore-reload reload-into-ims
            UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
            Processing: refresh_pattern update.nai.com/.*\.(gem|zip|mcs)                    43200 100% 43200  ignore-no-cache ignore-no-store ignore-reload reload-into-ims
            UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
            Processing: refresh_pattern ^http:\/\/liveupdate.symantecliveupdate.com.*\(zip) 43200 100% 43200  ignore-no-cache ignore-no-store ignore-reload reload-into-ims
            UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
            Processing: refresh_pattern -i symantecliveupdate.com/.*\.(zip|exe)             43200 100% 43200 reload-into-ims
            Processing: refresh_pattern -i avast.com/.*\.(vpu|vpaa) 4320 100% 43200 reload-into-ims
            Processing: refresh_pattern -i avira-update.com/.*\.* 720 100% 10800 reload-into-ims
            Processing: refresh_pattern windowsupdate.com/.*\.(cab|exe)                     43200 100% 129600 ignore-no-cache ignore-no-store ignore-reload reload-into-ims
            UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
            Processing: refresh_pattern update.microsoft.com/.*\.(cab|exe)                  43200 100% 129600 ignore-no-cache ignore-no-store ignore-reload reload-into-ims
            UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
            Processing: refresh_pattern download.microsoft.com/.*\.(cab|exe)                43200 100% 129600 ignore-no-cache ignore-no-store ignore-reload reload-into-ims
            UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
            Processing: refresh_pattern -i microsoft.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 4320 80% 43200 reload-into-ims
            Processing: refresh_pattern -i windowsupdate.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 4320 80% 43200 reload-into-ims
            Processing: refresh_pattern -i windows.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 4320 80% 43200 reload-into-ims
            Processing: refresh_pattern ([^.]+\.)?(download|(windows)?update)\.(microsoft\.)?com/.*\.(cab|exe|msi|msp|psf) 4320 100% 43200 reload-into-ims
            Processing: refresh_pattern update.microsoft.com/.*\.(cab|exe|dll|msi|psf) 10080 100% 43200 reload-into-ims
            Processing: refresh_pattern windowsupdate.com/.*\.(cab|exe|dll|msi|psf) 10080 100% 43200 reload-into-ims
            Processing: refresh_pattern download.microsoft.com/.*\.(cab|exe|dll|msi|psf) 10080 100% 43200 reload-into-ims
            Processing: refresh_pattern www.microsoft.com/.*\.(cab|exe|dll|msi|psf) 10080 100% 43200 reload-into-ims
            Processing: refresh_pattern au.download.windowsupdate.com/.*\.(cab|exe|dll|msi|psf) 4320 100% 43200 reload-into-ims
            Processing: refresh_pattern bg.v4.pr.dl.ws.microsoft.com/.*\.(cab|exe|dll|msi|psf) 4320 100% 43200 reload-into-ims
            Processing: refresh_pattern -i (download|adcdownload).apple.com/.*\.(pkg|dmg) 4320 100% 43200 reload-into-ims
            Processing: refresh_pattern ((facebook.com)|(85.131.151.39)).*\.(jpg|png|gif) 129600 100% 129600 ignore-reload override-expire ignore-no-cache ignore-no-store
            UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
            Processing: refresh_pattern -i \.fbcdn.net.*\.(jpg|gif|png|swf|mp3)           129600 100% 129600 ignore-reload override-expire ignore-no-cache ignore-no-store
            UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
            Processing: refresh_pattern static\.ak\.fbcdn\.net*\.(jpg|gif|png)            129600 100% 129600 ignore-reload override-expire ignore-no-cache ignore-no-store
            UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
            Processing: refresh_pattern ^http:\/\/profile\.ak\.fbcdn.net*\.(jpg|gif|png)  129600 100% 129600 ignore-reload override-expire ignore-no-cache ignore-no-store
            UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
            Processing: refresh_pattern ^http:\/\/openx.*\.(jp(e?g|e|2)|gif|pn[pg]|swf|ico|css|tiff?) 129600 100% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache ignore-no-store
            UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
            Processing: refresh_pattern ^http:\/\/ads(1|2|3).kompas.com.*\/                           43200  100% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache ignore-no-store
            UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
            Processing: refresh_pattern ^http:\/\/img.ads.kompas.com.*\/                              43200  100% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache ignore-no-store
            UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
            Processing: refresh_pattern .kompasimages.com.*\.(jpg|gif|png|swf)                        43200  100% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache ignore-no-store
            UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
            Processing: refresh_pattern ^http:\/\/openx.kompas.com.*\/                                43200  100% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache ignore-no-store
            UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
            Processing: refresh_pattern kaskus.\us.*\.(jp(e?g|e|2)|gif|png|swf)                       43200  100% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache ignore-no-store
            UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
            Processing: refresh_pattern ^http:\/\/img.kaskus.us.*\.(jpg|gif|png|swf)                  43200  100% 129600 reload-into-ims ignore-reload override-expire ignore-no-cache ignore-no-store
            UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
            Processing: refresh_pattern ^http:\/\/\.www[0-9][0-9]\.indowebster\.com\/(.*)(mp3|rar|zip|flv|wmv|3gp|mp(4|3)|exe|msi|zip) 43200 100% 129600 reload-into-ims  ignore-reload override-expire ignore-no-cache ignore-no-store  ignore-auth
            UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
            Processing: refresh_pattern -i ^http://(khm?)([^/]*?)\.google\.(de|com)     129600 100% 129600 ignore-no-cache ignore-no-store ignore-private override-expire override-lastmod reload-into-ims ignore-reload
            UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
            /usr/local/etc/squid/squid.conf line 136: refresh_pattern -i ^http://(khm?)([^/]*?)\.google\.(de|com)     129600 100% 129600 ignore-no-cache ignore-no-store ignore-private override-expire override-lastmod reload-into-ims ignore-reload
            refreshAddToList: Invalid regular expression '^http://(khm?)([^/]*?)\.google\.(de|com)': repetition-operator operand invalid
            Processing: refresh_pattern -i ^http://ecn\.t\d\.tiles\.virtualearth\.net/tiles/\w*\.jpeg     129600 100% 129600 ignore-no-cache ignore-no-store ignore-private override-expire override-lastmod reload-into-ims ignore-reload
            UPGRADE: refresh_pattern option 'ignore-no-cache' is obsolete. Remove it.
            
            
            1 Reply Last reply Reply Quote 0
            • V
              Valeriy
              last edited by

              Two invalid expressions.
              Somebody can fix them?

              /usr/local/etc/squid/squid.conf line 52: refresh_pattern -i .(rar|jar|gz|tgz|tar|bz2|iso|m1v|m2(v|p)|mo(d|v)|(x-|)flv) 129600 100% 129600 ignore-no-cache ignore-no-store ignore-private override-expire override-lastmod reload-into-ims ignore-reload
              refreshAddToList: Invalid regular expression '.(rar|jar|gz|tgz|tar|bz2|iso|m1v|m2(v|p)|mo(d|v)|(x-|)flv)': empty (sub)expression

              _  kid1| /usr/local/etc/squid/squid.conf line 136: refresh_pattern -i ^http://(khm?)([^/]?).google.(de|com)    129600 100% 129600 ignore-no-cache ignore-no-store ignore-private override-expire override-lastmod reload-into-ims ignore-reload
              refreshAddToList: Invalid regular expression '^http://(khm?)([^/]
              ?).google.(de|com)': repetition-operator operand invalid_

              1 Reply Last reply Reply Quote 0
              • KOMK
                KOM
                last edited by

                For the first one, I think the problem is here:

                (x-|)

                There is nothing on the other side of the OR.  I don't know what file type would be handled by the extension x-.  To fix it, just remove the entire (x-|).

                For the second one, I don't know for sure, but it seems like a lot of regex just to handle .google..

                1 Reply Last reply Reply Quote 0
                • D
                  doktornotor Banned
                  last edited by

                  I would strongly urge everyone to ditch the pointless idea of trying to cache dynamic sites. It doesn't work in like 99% of cases, plus breaks Squid. And please ditch ALL those antivirus patterns. None of them works. The AVs are streaming updates with new sigs to clients every couple of minutes these days. There's nothing to cache.

                  1 Reply Last reply Reply Quote 0
                  • S
                    saluto
                    last edited by

                    So why use squid? Because most sites are changing to https.

                    Various web content are in videos, overloading the network.

                    1 Reply Last reply Reply Quote 0
                    • KOMK
                      KOM
                      last edited by

                      So why use squid?

                      I use it as a filtering platform for squidguard.

                      Various web content are in videos

                      Caching videos is pointless unless everyone is watching the same ones.

                      I have seen commercial products that take squid and heavily modify it with all kinds of custom stuff, and they claim they can cache almost everything.

                      1 Reply Last reply Reply Quote 0
                      • V
                        Valeriy
                        last edited by

                        I use squid for caching, it saves bandwidth for Microsoft Updates and Apple updates + SquidGuard as Ad+Malware+porn filter

                        1 Reply Last reply Reply Quote 0
                        • KOMK
                          KOM
                          last edited by

                          it saves bandwidth for Microsoft Updates

                          I have yet to see a working config with squid and WU.  Most people give up and use WSUS.

                          1 Reply Last reply Reply Quote 0
                          • V
                            Valeriy
                            last edited by

                            I cannt use WSUS, PCs are not under my control.

                            1 Reply Last reply Reply Quote 0
                            • KOMK
                              KOM
                              last edited by

                              If you have a working squid config that caches WU for Win7-10 clients, I'd love to see it.

                              JonathanLeeJ 1 Reply Last reply Reply Quote 0
                              • D
                                danbak365
                                last edited by

                                These refresh pattern made a big difference on my WI-fi network.
                                Since I started using it my clients have been very happy with the speed.
                                Even though, now it seems like they need some update, I still appreciate this well done job.
                                Thanks

                                1 Reply Last reply Reply Quote 0
                                • A
                                  aGeekhere
                                  last edited by

                                  You can use squidlite to check if the refresh pattens are improving the hit rate. I have found that the comodo antivirus updates were caching ( cannot retest at the moment) and so did window updates. The best thing is to test if the hit ratio is improving.

                                  Never Fear, A Geek is Here!

                                  1 Reply Last reply Reply Quote 0
                                  • S
                                    saluto
                                    last edited by

                                    How did he do that?
                                    https://www.youtube.com/watch?v=18QSf6jb1eM

                                    1 Reply Last reply Reply Quote 0
                                    • V
                                      Valeriy
                                      last edited by

                                      @KOM

                                      Here are changes made to squid.conf (or through GUI on pfsense in corresponding custom fields)
                                      that allows caching of Windows Update for me:

                                      
                                      acl Windows_Update dstdomain windowsupdate.microsoft.com
                                      acl Windows_Update dstdomain .update.microsoft.com
                                      acl Windows_Update dstdomain download.windowsupdate.com
                                      acl Windows_Update dstdomain www.download.windowsupdate.com
                                      acl Windows_Update dstdomain au.download.windowsupdate.com
                                      acl Windows_Update dstdomain bg.v4.pr.dl.ws.microsoft.com
                                      range_offset_limit 200 MB Windows_Update
                                      quick_abort_min 1000 KB
                                      quick_abort_max 5000 KB
                                      quick_abort_pct 80
                                      request_body_max_size 0 KB
                                      
                                      refresh_pattern -i .*windowsupdate.com/.*\.(cab|exe)                     259200 100% 259200 ignore-no-store ignore-reload reload-into-ims
                                      refresh_pattern -i .*update.microsoft.com/.*\.(cab|exe|dll|msi|psf)                  259200 100% 259200 ignore-no-store ignore-reload reload-into-ims
                                      refresh_pattern -i windows.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 4320 80% 43200 reload-into-ims
                                      refresh_pattern windowsupdate.com/.*\.(cab|exe|dll|msi|psf) 10080 100% 43200 reload-into-ims
                                      refresh_pattern download.microsoft.com/.*\.(cab|exe|dll|msi|psf) 10080 100% 43200 reload-into-ims
                                      refresh_pattern www.microsoft.com/.*\.(cab|exe|dll|msi|psf) 10080 100% 43200 reload-into-ims
                                      refresh_pattern au.download.windowsupdate.com/.*\.(cab|exe|dll|msi|psf) 4320 100% 43200 reload-into-ims
                                      refresh_pattern bg.v4.pr.dl.ws.microsoft.com/.*\.(cab|exe|dll|msi|psf) 4320 100% 43200 reload-into-ims
                                      
                                      cache_mem 15000 MB 
                                      # Should be less than 50% of avail. RAM
                                      
                                      maximum_object_size_in_memory 5000000 KB 
                                      #should be less than cache_mem 
                                      
                                      memory_replacement_policy heap LFUDA
                                      cache_replacement_policy heap LFUDA
                                      
                                      minimum_object_size 1000 KB
                                      #Do not store very small object on disk
                                      
                                      maximum_object_size 5000 MB 
                                      #should be less or equal to maximum_object_size_in_memory
                                      
                                      

                                      Here is output for today access.log of squid (number of HITS):
                                      cat  /var/squid/logs/access.log | grep HIT | grep windowsupdate -c
                                      2860
                                      cat /var/squid/logs/access.log | grep HIT | grep microsoft.com -c
                                      19128

                                      Here is output for today access.log of squid (number of MISSes):
                                      cat /var/squid/logs/access.log | grep MISS | grep windowsupdate -c
                                      61834
                                      cat /var/squid/logs/access.log | grep MISS | grep microsoft.com -c
                                      55567

                                      I had to rebuild my disk cache 2 days ago, so I assume not everything is cached yet. Or some of my settings needs improvement…

                                      It might not be ideal setup.
                                      If you have some comments, improvements, please let me know.

                                      Regards,

                                      Valeriy

                                      1 Reply Last reply Reply Quote 0
                                      • S
                                        saluto
                                        last edited by

                                        How did your dynamic cache get now?

                                        1 Reply Last reply Reply Quote 0
                                        • V
                                          Valeriy
                                          last edited by

                                          @saluto:

                                          How did your dynamic cache get now?

                                          It is good.
                                          This is result for yesterday:

                                          
                                          [2.3.3-DEVELOPMENT]/var/squid/logs: cat /var/squid/logs/access.log.2 | grep MISS | grep microsoft.com -c
                                          96270
                                          [2.3.3-DEVELOPMENT]/var/squid/logs: cat /var/squid/logs/access.log.2 | grep HIT | grep microsoft.com -c
                                          21177
                                          (18% HITS vs. 82% MISS)
                                          [2.3.3-DEVELOPMENT]/var/squid/logs: cat /var/squid/logs/access.log.2 | grep HIT | grep windowsupdate -c
                                          8352
                                          [2.3.3-DEVELOPMENT]/var/squid/logs: cat /var/squid/logs/access.log.2 | grep MISS | grep windowsupdate -c
                                          34794
                                          (19% HITS vs. 81% MISS)
                                          
                                          
                                          1 Reply Last reply Reply Quote 0
                                          • KOMK
                                            KOM
                                            last edited by

                                            :o

                                            81% miss rate seems awfully high for something that's supposedly working good.

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.