Navigation

    Netgate Discussion Forum
    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search

    Squid Package Tuning

    pfSense Packages
    8
    29
    15255
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • S
      scooterdouglas last edited by

      Hi,

      I've been playing around with Squid Caches, what to keep and what to discard, however my regular expression (regex) is rusty (almost non-existent).

      I've read the Squid Packing Tuning guid from here:  http://doc.pfsense.org/index.php/Squid_Package_Tuning and noticed the following two lines:

      refresh_pattern windowsupdate.com/..(cab|exe) 4320 100% 43200 reload-into-ims;
      refresh_pattern au.download.windowsupdate.com/.      .(cab|exe) 4320 100% 43200 reload-into-ims;

      Could this not be reduced to one line with the following regex command:

      refresh_pattern ([^.]+.|)windowsupdate.com/.*.(cab|exe) 4320 100% 43200 reload-into-ims;

      And while we're at it, how about changing:

      (cab|exe)

      to the pick up Office Updates as well:

      (cab|exe|msi|msp)

      Just a thought (or a brain fart ;D)!

      Scott.

      1 Reply Last reply Reply Quote 0
      • jimp
        jimp Rebel Alliance Developer Netgate last edited by

        If you've tested that and it's known to work, I'll update the docs. There's no reason additional regexes couldn't be used to simplify it, afaik.

        Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

        Need help fast? Netgate Global Support!

        Do not Chat/PM for help!

        1 Reply Last reply Reply Quote 0
        • S
          scooterdouglas last edited by

          Jim,

          Thanks, I've done some limited testing, I only have a couple of PCs with Windows to test with!  I've modified the expression to this:

          refresh_pattern ([^.]+.|)(download|(windows|)update|).(microsoft.|)com/.*.(cab|exe|msi|msp) 4320 100% 43200 reload-into-ims;

          Checking my Squid access.log, I could see plenty of hits (both TCP_MEM_HIT/200 and TCP_REFRESH_HIT/200), but will need to wait till "Patch Tuesday" (14th December) to check further.

          Scott.

          1 Reply Last reply Reply Quote 0
          • jimp
            jimp Rebel Alliance Developer Netgate last edited by

            Sounds good, update the thread then and let us know how it went. I'll update the docs if it worked.

            Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

            Need help fast? Netgate Global Support!

            Do not Chat/PM for help!

            1 Reply Last reply Reply Quote 0
            • N
              Nachtfalke last edited by

              Hi,

              it is not a question pointing exactly to the topic, but could you tell what *.MSP files are ?

              1 Reply Last reply Reply Quote 0
              • S
                scooterdouglas last edited by

                Nachtfalke,

                I would say your question is relevant to the topic.

                Files with the .MSP extension are Windows Installer Patch files, they are mainly used for updating the Microsoft Office suites.

                Scott.

                1 Reply Last reply Reply Quote 0
                • N
                  Nachtfalke last edited by

                  Hi,

                  thanks for your reply. I'm using pfsense2 and squid at the moment only in a testing environment and I used the "old" regex patters with cab and exe, later added msi because of your post above and it works fine.
                  I will try it with the MSP extension, too, but I cannot remeber, that ever downloaded MSP files…I just downloaded service packs for my office suites as .EXE.

                  Nevertheless, thanks for your explanation

                  PS: Isn't there a tool, where I could transform a "normal" expression like
                  http://.microsoft.com/ to a regexp ?

                  Thanks

                  1 Reply Last reply Reply Quote 0
                  • S
                    scooterdouglas last edited by

                    All,

                    It looks as if the regex is working.

                    This is my main XP machine, which has Office 2003 on it, as you can see lots of misses as this is the first box to use Windows update:

                    
1292365669.857    101 192.168.3.199 TCP_MISS/200           405 HEAD http://download.windowsupdate.com/msdownload/update/software/secu/2010/11/windowsxp-kb2423089-x86-enu_94cce8eb153f98affd81951f2123b74f14a59925.exe - DEFAULT_PARENT/havp application/octet-stream
1292365671.711   1850 192.168.3.199 TCP_MISS/200        515349 GET  http://download.windowsupdate.com/msdownload/update/software/secu/2010/11/windowsxp-kb2423089-x86-enu_94cce8eb153f98affd81951f2123b74f14a59925.exe - DEFAULT_PARENT/havp application/octet-stream
1292365672.277     35 192.168.3.199 TCP_MISS/200           406 HEAD http://download.windowsupdate.com/msdownload/update/software/secu/2010/11/windowsxp-kb2436673-x86-express-enu_277a9af7b64b8d5776c3b4398c4e1ce88b7ba0dd.cab - DEFAULT_PARENT/havp application/octet-stream
1292365673.237    955 192.168.3.199 TCP_MISS/200        262026 GET  http://download.windowsupdate.com/msdownload/update/software/secu/2010/11/windowsxp-kb2436673-x86-express-enu_277a9af7b64b8d5776c3b4398c4e1ce88b7ba0dd.cab - DEFAULT_PARENT/havp application/octet-stream
1292365682.846     39 192.168.3.199 TCP_MISS/200           407 HEAD http://download.windowsupdate.com/msdownload/update/software/secu/2010/11/windowsxp-kb2436673-x86-enu_9fc5b54a23314ea57d995f4536edb09a0751db82.psf - DEFAULT_PARENT/havp application/octet-stream
1292365683.663    775 192.168.3.199 TCP_MISS/206        194230 GET  http://download.windowsupdate.com/msdownload/update/software/secu/2010/11/windowsxp-kb2436673-x86-enu_9fc5b54a23314ea57d995f4536edb09a0751db82.psf - DEFAULT_PARENT/havp multipart/byteranges
1292365685.395    110 192.168.3.199 TCP_MISS/200           404 HEAD http://download.windowsupdate.com/msdownload/update/software/uprl/2010/11/windowsxp-kb2443685-x86-enu_d5ce3060d0317ae2ba8b37f43377004ba3cb5a2b.exe - DEFAULT_PARENT/havp application/octet-stream
1292365687.238   1838 192.168.3.199 TCP_MISS/200        513812 GET  http://download.windowsupdate.com/msdownload/update/software/uprl/2010/11/windowsxp-kb2443685-x86-enu_d5ce3060d0317ae2ba8b37f43377004ba3cb5a2b.exe - DEFAULT_PARENT/havp application/octet-stream
1292365687.767     48 192.168.3.199 TCP_MISS/200           405 HEAD http://download.windowsupdate.com/msdownload/update/software/secu/2010/11/windowsxp-kb2440591-x86-enu_4ecb42986b6e12ccc9e83c8bdda536f94f6d8488.exe - DEFAULT_PARENT/havp application/octet-stream
1292365689.619   1847 192.168.3.199 TCP_MISS/200        515861 GET  http://download.windowsupdate.com/msdownload/update/software/secu/2010/11/windowsxp-kb2440591-x86-enu_4ecb42986b6e12ccc9e83c8bdda536f94f6d8488.exe - DEFAULT_PARENT/havp application/octet-stream
1292365690.097     35 192.168.3.199 TCP_MISS/200           405 HEAD http://download.windowsupdate.com/msdownload/update/software/crup/2010/11/windowsxp-kb2467659-x86-enu_921027bf773a08190462844c45ad682a6d0d03d7.exe - DEFAULT_PARENT/havp application/octet-stream
1292365691.882   1779 192.168.3.199 TCP_MISS/200        495893 GET  http://download.windowsupdate.com/msdownload/update/software/crup/2010/11/windowsxp-kb2467659-x86-enu_921027bf773a08190462844c45ad682a6d0d03d7.exe - DEFAULT_PARENT/havp application/octet-stream
1292365692.368     37 192.168.3.199 TCP_MISS/200           407 HEAD http://download.windowsupdate.com/msdownload/update/software/secu/2010/11/ie8-windowsxp-kb2416400-x86-enu_84550c3ba8570c64937a5f154745abf7a80c533d.exe - DEFAULT_PARENT/havp application/octet-stream
1292365732.463  40090 192.168.3.199 TCP_MISS/200      10479895 GET  http://download.windowsupdate.com/msdownload/update/software/secu/2010/11/ie8-windowsxp-kb2416400-x86-enu_84550c3ba8570c64937a5f154745abf7a80c533d.exe - DEFAULT_PARENT/havp application/octet-stream
1292365736.932     34 192.168.3.199 TCP_MISS/200           407 HEAD http://download.windowsupdate.com/msdownload/update/software/uprl/2010/12/windows-kb890830-v3.14-delta_e74875038e701d2e1388e20fb8bca1d43a8f9563.exe - DEFAULT_PARENT/havp application/octet-stream
1292365753.236  16259 192.168.3.199 TCP_MISS/200       4579167 GET  http://download.windowsupdate.com/msdownload/update/software/uprl/2010/12/windows-kb890830-v3.14-delta_e74875038e701d2e1388e20fb8bca1d43a8f9563.exe - DEFAULT_PARENT/havp application/octet-stream
1292365753.506     39 192.168.3.199 TCP_MISS/200           406 HEAD http://download.windowsupdate.com/msdownload/update/software/secu/2010/11/windowsxp-kb2296199-x86-express-enu_31c4c9074a8b5e4ec33ce33f93af3b81d245dc94.cab - DEFAULT_PARENT/havp application/octet-stream
1292365754.487    939 192.168.3.199 TCP_MISS/200        244426 GET  http://download.windowsupdate.com/msdownload/update/software/secu/2010/11/windowsxp-kb2296199-x86-express-enu_31c4c9074a8b5e4ec33ce33f93af3b81d245dc94.cab - DEFAULT_PARENT/havp application/octet-stream
1292365755.948     38 192.168.3.199 TCP_MISS/200           406 HEAD http://download.windowsupdate.com/msdownload/update/software/secu/2010/11/windowsxp-kb2296199-x86-enu_9868146acbe92b31aecec3be7732ca5b2dc8ae77.psf - DEFAULT_PARENT/havp application/octet-stream
1292365756.317    364 192.168.3.199 TCP_MISS/206         72986 GET  http://download.windowsupdate.com/msdownload/update/software/secu/2010/11/windowsxp-kb2296199-x86-enu_9868146acbe92b31aecec3be7732ca5b2dc8ae77.psf - DEFAULT_PARENT/havp multipart/byteranges
1292365804.754     36 192.168.3.199 TCP_MISS/200           406 HEAD http://download.windowsupdate.com/msdownload/update/software/defu/2010/12/am_delta_patch1_fd1798cd440388fc04aacd9d63d23f7298d6a56d.exe - DEFAULT_PARENT/havp application/octet-stream
1292365806.411   1616 192.168.3.199 TCP_MISS/200        457510 GET  http://download.windowsupdate.com/msdownload/update/software/defu/2010/12/am_delta_patch1_fd1798cd440388fc04aacd9d63d23f7298d6a56d.exe - DEFAULT_PARENT/havp application/octet-stream
1292365757.800    102 192.168.3.199 TCP_MISS/200           405 HEAD http://download.windowsupdate.com/msdownload/update/software/secu/2010/12/gpfilt_f86338f746ba4aa86b88a6823f4444caae326e46.cab - DEFAULT_PARENT/havp application/octet-stream
1292365760.378   2572 192.168.3.199 TCP_MISS/200        729535 GET  http://download.windowsupdate.com/msdownload/update/software/secu/2010/12/gpfilt_f86338f746ba4aa86b88a6823f4444caae326e46.cab - DEFAULT_PARENT/havp application/octet-stream
1292365760.502     35 192.168.3.199 TCP_MISS/200           406 HEAD http://download.windowsupdate.com/msdownload/update/software/secu/2010/12/mspub_09d2531c6cd96a26d2a83c1a0ff8c9bc91aeeef2.cab - DEFAULT_PARENT/havp application/octet-stream
1292365772.090  11547 192.168.3.199 TCP_MISS/200       2912623 GET  http://download.windowsupdate.com/msdownload/update/software/secu/2010/12/mspub_09d2531c6cd96a26d2a83c1a0ff8c9bc91aeeef2.cab - DEFAULT_PARENT/havp application/octet-stream
1292365772.270     36 192.168.3.199 TCP_MISS/200           405 HEAD http://download.windowsupdate.com/msdownload/update/software/crup/2010/12/outlook_49ecfe899e24ee3a825a61a1d72b6b6ba081bfc2.cab - DEFAULT_PARENT/havp application/octet-stream
1292365790.045  17770 192.168.3.199 TCP_MISS/200       5084434 GET  http://download.windowsupdate.com/msdownload/update/software/crup/2010/12/outlook_49ecfe899e24ee3a825a61a1d72b6b6ba081bfc2.cab - DEFAULT_PARENT/havp application/octet-stream
1292365790.216     35 192.168.3.199 TCP_MISS/200           406 HEAD http://download.windowsupdate.com/msdownload/update/software/crup/2010/12/outlfltr_e353aad09af692bcd477d82a7e1dedf146333e12.cab - DEFAULT_PARENT/havp application/octet-stream
1292365804.603  14380 192.168.3.199 TCP_MISS/200       3551773 GET  http://download.windowsupdate.com/msdownload/update/software/crup/2010/12/outlfltr_e353aad09af692bcd477d82a7e1dedf146333e12.cab - DEFAULT_PARENT/havp application/octet-stream

                    Now this is my (old) laptop:

                    
1292365986.419     11 192.168.3.197 TCP_HIT/200            413 HEAD http://download.windowsupdate.com/msdownload/update/software/secu/2010/11/windowsxp-kb2423089-x86-enu_94cce8eb153f98affd81951f2123b74f14a59925.exe - NONE/- application/octet-stream
1292365987.846   1350 192.168.3.197 TCP_HIT/200         515357 GET  http://download.windowsupdate.com/msdownload/update/software/secu/2010/11/windowsxp-kb2423089-x86-enu_94cce8eb153f98affd81951f2123b74f14a59925.exe - NONE/- application/octet-stream
1292365991.217     11 192.168.3.197 TCP_HIT/200            414 HEAD http://download.windowsupdate.com/msdownload/update/software/secu/2010/11/windowsxp-kb2436673-x86-express-enu_277a9af7b64b8d5776c3b4398c4e1ce88b7ba0dd.cab - NONE/- application/octet-stream
1292365991.920    676 192.168.3.197 TCP_HIT/200         262034 GET  http://download.windowsupdate.com/msdownload/update/software/secu/2010/11/windowsxp-kb2436673-x86-express-enu_277a9af7b64b8d5776c3b4398c4e1ce88b7ba0dd.cab - NONE/- application/octet-stream
1292366002.587      0 192.168.3.197 TCP_MEM_HIT/200        415 HEAD http://download.windowsupdate.com/msdownload/update/software/secu/2010/11/windowsxp-kb2436673-x86-enu_9fc5b54a23314ea57d995f4536edb09a0751db82.psf - NONE/- application/octet-stream
1292366003.398    748 192.168.3.197 TCP_MISS/206        194230 GET  http://download.windowsupdate.com/msdownload/update/software/secu/2010/11/windowsxp-kb2436673-x86-enu_9fc5b54a23314ea57d995f4536edb09a0751db82.psf - DEFAULT_PARENT/havp multipart/byteranges
1292366009.603     11 192.168.3.197 TCP_HIT/200            412 HEAD http://download.windowsupdate.com/msdownload/update/software/uprl/2010/11/windowsxp-kb2443685-x86-enu_d5ce3060d0317ae2ba8b37f43377004ba3cb5a2b.exe - NONE/- application/octet-stream
1292366010.985   1339 192.168.3.197 TCP_HIT/200         513820 GET  http://download.windowsupdate.com/msdownload/update/software/uprl/2010/11/windowsxp-kb2443685-x86-enu_d5ce3060d0317ae2ba8b37f43377004ba3cb5a2b.exe - NONE/- application/octet-stream
1292366014.465     10 192.168.3.197 TCP_HIT/200            413 HEAD http://download.windowsupdate.com/msdownload/update/software/secu/2010/11/windowsxp-kb2440591-x86-enu_4ecb42986b6e12ccc9e83c8bdda536f94f6d8488.exe - NONE/- application/octet-stream
1292366015.631   1115 192.168.3.197 TCP_HIT/200         515869 GET  http://download.windowsupdate.com/msdownload/update/software/secu/2010/11/windowsxp-kb2440591-x86-enu_4ecb42986b6e12ccc9e83c8bdda536f94f6d8488.exe - NONE/- application/octet-stream
1292366019.214     10 192.168.3.197 TCP_HIT/200            413 HEAD http://download.windowsupdate.com/msdownload/update/software/crup/2010/11/windowsxp-kb2467659-x86-enu_921027bf773a08190462844c45ad682a6d0d03d7.exe - NONE/- application/octet-stream
1292366020.592   1292 192.168.3.197 TCP_HIT/200         495901 GET  http://download.windowsupdate.com/msdownload/update/software/crup/2010/11/windowsxp-kb2467659-x86-enu_921027bf773a08190462844c45ad682a6d0d03d7.exe - NONE/- application/octet-stream
1292366023.936     11 192.168.3.197 TCP_HIT/200            415 HEAD http://download.windowsupdate.com/msdownload/update/software/secu/2010/11/ie8-windowsxp-kb2416400-x86-enu_84550c3ba8570c64937a5f154745abf7a80c533d.exe - NONE/- application/octet-stream
1292366042.305  18320 192.168.3.197 TCP_HIT/200       10479903 GET  http://download.windowsupdate.com/msdownload/update/software/secu/2010/11/ie8-windowsxp-kb2416400-x86-enu_84550c3ba8570c64937a5f154745abf7a80c533d.exe - NONE/- application/octet-stream
1292366055.405     11 192.168.3.197 TCP_HIT/200            415 HEAD http://download.windowsupdate.com/msdownload/update/software/uprl/2010/12/windows-kb890830-v3.14-delta_e74875038e701d2e1388e20fb8bca1d43a8f9563.exe - NONE/- application/octet-stream
1292366065.523   9985 192.168.3.197 TCP_HIT/200        4579175 GET  http://download.windowsupdate.com/msdownload/update/software/uprl/2010/12/windows-kb890830-v3.14-delta_e74875038e701d2e1388e20fb8bca1d43a8f9563.exe - NONE/- application/octet-stream
1292366067.905     11 192.168.3.197 TCP_HIT/200            414 HEAD http://download.windowsupdate.com/msdownload/update/software/secu/2010/11/windowsxp-kb2296199-x86-express-enu_31c4c9074a8b5e4ec33ce33f93af3b81d245dc94.cab - NONE/- application/octet-stream
1292366068.572    610 192.168.3.197 TCP_HIT/200         244434 GET  http://download.windowsupdate.com/msdownload/update/software/secu/2010/11/windowsxp-kb2296199-x86-express-enu_31c4c9074a8b5e4ec33ce33f93af3b81d245dc94.cab - NONE/- application/octet-stream
1292366073.061      0 192.168.3.197 TCP_MEM_HIT/200        414 HEAD http://download.windowsupdate.com/msdownload/update/software/secu/2010/11/windowsxp-kb2296199-x86-enu_9868146acbe92b31aecec3be7732ca5b2dc8ae77.psf - NONE/- application/octet-stream
1292366073.430    342 192.168.3.197 TCP_MISS/206         72986 GET  http://download.windowsupdate.com/msdownload/update/software/secu/2010/11/windowsxp-kb2296199-x86-enu_9868146acbe92b31aecec3be7732ca5b2dc8ae77.psf - DEFAULT_PARENT/havp multipart/byteranges
1292366078.192     98 192.168.3.197 TCP_MISS/200           407 HEAD http://download.windowsupdate.com/msdownload/update/software/defu/2010/12/am_delta_ecfd87fd2d14882a4ce6d5822c2da25b171c50ea.exe - DEFAULT_PARENT/havp application/octet-stream
1292366091.758  13537 192.168.3.197 TCP_MISS/200       3988775 GET  http://download.windowsupdate.com/msdownload/update/software/defu/2010/12/am_delta_ecfd87fd2d14882a4ce6d5822c2da25b171c50ea.exe - DEFAULT_PARENT/havp application/octet-stream

                    And finally my (very old) XP box that I keep updated just in case:

                    
1292366376.917     11 192.168.3.196 TCP_HIT/200            413 HEAD http://download.windowsupdate.com/msdownload/update/software/secu/2010/11/windowsxp-kb2423089-x86-enu_94cce8eb153f98affd81951f2123b74f14a59925.exe - NONE/- application/octet-stream
1292366378.236   1302 192.168.3.196 TCP_HIT/200         515357 GET  http://download.windowsupdate.com/msdownload/update/software/secu/2010/11/windowsxp-kb2423089-x86-enu_94cce8eb153f98affd81951f2123b74f14a59925.exe - NONE/- application/octet-stream
1292366381.193     11 192.168.3.196 TCP_HIT/200            414 HEAD http://download.windowsupdate.com/msdownload/update/software/secu/2010/11/windowsxp-kb2436673-x86-express-enu_277a9af7b64b8d5776c3b4398c4e1ce88b7ba0dd.cab - NONE/- application/octet-stream
1292366381.679    477 192.168.3.196 TCP_HIT/200         262034 GET  http://download.windowsupdate.com/msdownload/update/software/secu/2010/11/windowsxp-kb2436673-x86-express-enu_277a9af7b64b8d5776c3b4398c4e1ce88b7ba0dd.cab - NONE/- application/octet-stream
1292366397.496      0 192.168.3.196 TCP_MEM_HIT/200        415 HEAD http://download.windowsupdate.com/msdownload/update/software/secu/2010/11/windowsxp-kb2436673-x86-enu_9fc5b54a23314ea57d995f4536edb09a0751db82.psf - NONE/- application/octet-stream
1292366398.291    781 192.168.3.196 TCP_MISS/206        194230 GET  http://download.windowsupdate.com/msdownload/update/software/secu/2010/11/windowsxp-kb2436673-x86-enu_9fc5b54a23314ea57d995f4536edb09a0751db82.psf - DEFAULT_PARENT/havp multipart/byteranges
1292366403.734     11 192.168.3.196 TCP_HIT/200            412 HEAD http://download.windowsupdate.com/msdownload/update/software/uprl/2010/11/windowsxp-kb2443685-x86-enu_d5ce3060d0317ae2ba8b37f43377004ba3cb5a2b.exe - NONE/- application/octet-stream
1292366404.598    852 192.168.3.196 TCP_HIT/200         513820 GET  http://download.windowsupdate.com/msdownload/update/software/uprl/2010/11/windowsxp-kb2443685-x86-enu_d5ce3060d0317ae2ba8b37f43377004ba3cb5a2b.exe - NONE/- application/octet-stream
1292366407.613     11 192.168.3.196 TCP_HIT/200            413 HEAD http://download.windowsupdate.com/msdownload/update/software/secu/2010/11/windowsxp-kb2440591-x86-enu_4ecb42986b6e12ccc9e83c8bdda536f94f6d8488.exe - NONE/- application/octet-stream
1292366408.577    949 192.168.3.196 TCP_HIT/200         515869 GET  http://download.windowsupdate.com/msdownload/update/software/secu/2010/11/windowsxp-kb2440591-x86-enu_4ecb42986b6e12ccc9e83c8bdda536f94f6d8488.exe - NONE/- application/octet-stream
1292366411.623     11 192.168.3.196 TCP_HIT/200            413 HEAD http://download.windowsupdate.com/msdownload/update/software/crup/2010/11/windowsxp-kb2467659-x86-enu_921027bf773a08190462844c45ad682a6d0d03d7.exe - NONE/- application/octet-stream
1292366412.904   1261 192.168.3.196 TCP_HIT/200         495901 GET  http://download.windowsupdate.com/msdownload/update/software/crup/2010/11/windowsxp-kb2467659-x86-enu_921027bf773a08190462844c45ad682a6d0d03d7.exe - NONE/- application/octet-stream
1292366373.177    129 192.168.3.196 TCP_REFRESH_MISS/200   405 HEAD http://download.windowsupdate.com/msdownload/update/software/updt/2010/11/ie8-windowsxp-kb2447568-x86-enu_6db7985fc12f63020fc905ffa507d7366fe182c4.exe - DEFAULT_PARENT/havp application/octet-stream
1292366374.075    799 192.168.3.196 TCP_REFRESH_HIT/200 498964 GET  http://download.windowsupdate.com/msdownload/update/software/updt/2010/11/ie8-windowsxp-kb2447568-x86-enu_6db7985fc12f63020fc905ffa507d7366fe182c4.exe - DEFAULT_PARENT/havp application/octet-stream
1292366415.862     10 192.168.3.196 TCP_HIT/200            415 HEAD http://download.windowsupdate.com/msdownload/update/software/secu/2010/11/ie8-windowsxp-kb2416400-x86-enu_84550c3ba8570c64937a5f154745abf7a80c533d.exe - NONE/- application/octet-stream
1292366436.737  20856 192.168.3.196 TCP_HIT/200       10479903 GET  http://download.windowsupdate.com/msdownload/update/software/secu/2010/11/ie8-windowsxp-kb2416400-x86-enu_84550c3ba8570c64937a5f154745abf7a80c533d.exe - NONE/- application/octet-stream
1292366446.862     11 192.168.3.196 TCP_HIT/200            415 HEAD http://download.windowsupdate.com/msdownload/update/software/uprl/2010/12/windows-kb890830-v3.14-delta_e74875038e701d2e1388e20fb8bca1d43a8f9563.exe - NONE/- application/octet-stream
1292366455.849   8967 192.168.3.196 TCP_HIT/200        4579175 GET  http://download.windowsupdate.com/msdownload/update/software/uprl/2010/12/windows-kb890830-v3.14-delta_e74875038e701d2e1388e20fb8bca1d43a8f9563.exe - NONE/- application/octet-stream
1292366457.686     11 192.168.3.196 TCP_HIT/200            414 HEAD http://download.windowsupdate.com/msdownload/update/software/secu/2010/11/windowsxp-kb2296199-x86-express-enu_31c4c9074a8b5e4ec33ce33f93af3b81d245dc94.cab - NONE/- application/octet-stream
1292366458.285    588 192.168.3.196 TCP_HIT/200         244434 GET  http://download.windowsupdate.com/msdownload/update/software/secu/2010/11/windowsxp-kb2296199-x86-express-enu_31c4c9074a8b5e4ec33ce33f93af3b81d245dc94.cab - NONE/- application/octet-stream
1292366463.064      0 192.168.3.196 TCP_MEM_HIT/200        414 HEAD http://download.windowsupdate.com/msdownload/update/software/secu/2010/11/windowsxp-kb2296199-x86-enu_9868146acbe92b31aecec3be7732ca5b2dc8ae77.psf - NONE/- application/octet-stream
1292366463.449    370 192.168.3.196 TCP_MISS/206         72986 GET  http://download.windowsupdate.com/msdownload/update/software/secu/2010/11/windowsxp-kb2296199-x86-enu_9868146acbe92b31aecec3be7732ca5b2dc8ae77.psf - DEFAULT_PARENT/havp multipart/byteranges
1292366467.666     11 192.168.3.196 TCP_HIT/200            415 HEAD http://download.windowsupdate.com/msdownload/update/software/defu/2010/12/am_delta_ecfd87fd2d14882a4ce6d5822c2da25b171c50ea.exe - NONE/- application/octet-stream
1292366475.638   7963 192.168.3.196 TCP_HIT/200        3988783 GET  http://download.windowsupdate.com/msdownload/update/software/defu/2010/12/am_delta_ecfd87fd2d14882a4ce6d5822c2da25b171c50ea.exe - NONE/- application/octet-stream

                    Just need some more people to test it!

                    ;)

                    1 Reply Last reply Reply Quote 0
                    • jimp
                      jimp Rebel Alliance Developer Netgate last edited by

                      Sounds good, I went ahead and updated the doc wiki article.

                      Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

                      Need help fast? Netgate Global Support!

                      Do not Chat/PM for help!

                      1 Reply Last reply Reply Quote 0
                      • S
                        scooterdouglas last edited by

                        Jim,

                        One last thing, near the bottom of the Wiki page, there is an update for AVG, here are a couple more updates that could be applied:

                        refresh_pattern ([^.]+.|)avg.com/..(bin) 4320 100% 43200 reload-into-ims;
                        refresh_pattern ([^.]+.|)spywareblaster.net/.                        .(dtb) 4320 100% 64800 reload-into-ims;
                        refresh_pattern ([^.]+.|)symantecliveupdate.com/..(zip|exe) 43200 100% 43200 reload-into-ims
                        refresh_pattern ([^.]+.|)avast.com/.                        .(vpu|vpaa) 4320 100% 43200 reload-into-ims

                        I put the SpywareBlaster update to 45 days as it is not a regular update as the others

                        Scott.

                        1 Reply Last reply Reply Quote 0
                        • jimp
                          jimp Rebel Alliance Developer Netgate last edited by

                          I added those to the page, thanks!

                          Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

                          Need help fast? Netgate Global Support!

                          Do not Chat/PM for help!

                          1 Reply Last reply Reply Quote 0
                          • _
                            _igor_ last edited by

                            and this one for Apple, including developer-downloads…

                            refresh_pattern ([^.]+.|)(download|adcdownload).(apple.|)com/.*.(pkg|dmg) 4320 100% 43200 reload-into-ims;

                            1 Reply Last reply Reply Quote 0
                            • jimp
                              jimp Rebel Alliance Developer Netgate last edited by

                              Added that one, too. Thanks!

                              Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

                              Need help fast? Netgate Global Support!

                              Do not Chat/PM for help!

                              1 Reply Last reply Reply Quote 0
                              • N
                                Nachtfalke last edited by

                                Hi,

                                thanks for your great job.

                                Perhaps somebody could help me with updates for Kaspersky for Windows Workstations Updates ?
                                If I update on client, it connects to one of this sites:
                                http://dnl-01.geo.kaspersky.com
                                …
                                http://dnl-19.geo.kaspersky.com

                                And McAfee seems to connect to:
                                update.nai.com

                                But I don't know, which extensions will be downloaded :(

                                Thanks for your help!

                                1 Reply Last reply Reply Quote 0
                                • _
                                  _igor_ last edited by

                                  Best thing is looking at your squid-logs. There you have the full path of that update-files. Other way could be a manual download of the respective files via Webbrowser. The rest is copy/paste.

                                  1 Reply Last reply Reply Quote 0
                                  • D
                                    DigitalJer last edited by

                                    SWEET, this is one of my favourite features of pfsense + Squid.

                                    Nice work, thanks for sharing!

                                    –------------------------------------------------
                                    2.4.3-RELEASE (amd64)
                                    built on Mon Mar 26 18:02:04 CDT 2018
                                    FreeBSD 11.1-RELEASE-p7
                                    VM in ESXi 5.5
                                    1 x 1000baseTX (WAN)
                                    1 x 1000baseTX (LAN)

                                    1 Reply Last reply Reply Quote 0
                                    • S
                                      sam0t last edited by

                                      Came across this this informative post while searching for troubleshooting guidance with squid and Windows Updates. The problem being that the Windows Updates are not caching, otherwise the Squid is working as intended.

                                      As suggested in this post, I have copy/pasted the following entry to Squid > Services > Proxy server > General settings > Custom Settings -box

                                      refresh_pattern ([^.]+.|)(download|(windows|)update|).(microsoft.|)com/.*.(cab|exe|msi|msp) 4320 100% 43200 reload-into-ims;

                                      However it does not seem to be working and when I was removing the cache folder and executing the command "squid -k" I get the following error:

                                      refreshAddToList: Invalid regular expression '([^.]+.|)(download|(windows|)update|).(microsoft.|)com/.*.(cab|exe|msi|msp)': empty (sub)expression

                                      Any ideas about this?

                                      1 Reply Last reply Reply Quote 0
                                      • S
                                        serangku last edited by

                                        same here … squid -k, report error with that pattern

                                        some one must validate again with that refresh pattern ...
                                        or ... simple ignore since squid still running ?

                                        thanks ...

                                        1 Reply Last reply Reply Quote 0
                                        • N
                                          Nachtfalke last edited by

                                          Hi,

                                          I am using the following pattern and there is no error in syslog when rebooting or reloading squid:

                                          refresh_pattern ([^.]+.|)avg.com/.*\.(bin) 4320 100% 43200 reload-into-ims;
refresh_pattern ([^.]+.|)spywareblaster.net/.*\.(dtb) 4320 100% 4320 reload-into-ims;
refresh_pattern ([^.]+.|)symantecliveupdate.com/.*\.(zip|exe) 43200 100% 43200 reload-into-ims;
refresh_pattern ([^.]+.|)avast.com/.*\.(vpu|vpaa) 4320 100% 43200 reload-into-ims;
refresh_pattern ([^.]+.|)adobe.com/.*\.(exe|msi) 4320 100% 43200 reload-into-ims;
refresh_pattern ([^.]+.|)(download|adcdownload).(apple.|)com/.*\.(pkg|dmg) 8640 100% 86400 reload-into-ims;
refresh_pattern ([^.]+.|)(download|(windows|)update|).(microsoft.|)com/.*\.(cab|exe|msi|msp) 8640 100% 86400 reload-into-ims;
refresh_pattern ([^.]+.|)ubuntu.com/.*\.(tar|bz|bz2|gpg|gz|zip|deb) 8640 100% 86400 reload-into-ims;
range_offset_limit -1;
                                          1 Reply Last reply Reply Quote 0
                                          • _
                                            _igor_ last edited by

                                            I had this errors only with squid3, since some days. With squid 2.7 and squid3 also before never had such problems.

                                            1 Reply Last reply Reply Quote 0
                                            • N
                                              Nachtfalke last edited by

                                              I am using squid 2.7

                                              1 Reply Last reply Reply Quote 0
                                              • S
                                                serangku last edited by

                                                yes, on sys log gui there is no error with that pattern …
                                                please try on pfsense shell with command squid -k rotate or parse

                                                
[2.0-BETA5][root@server.localan.vvt]/root(11): squid -k parse
2011/02/08 09:15:39| pattern.conf line 1: refresh_pattern ([^.]+.|)avg.com/.*\.(bin) 4320 100% 43200 reload-into-ims
2011/02/08 09:15:39| parse_refreshpattern: Invalid regular expression '([^.]+.|)avg.com/.*\.(bin)': empty (sub)expression
2011/02/08 09:15:39| pattern.conf line 2: refresh_pattern ([^.]+.|)spywareblaster.net/.*\.(dtb) 4320 100% 4320 reload-into-ims
2011/02/08 09:15:39| parse_refreshpattern: Invalid regular expression '([^.]+.|)spywareblaster.net/.*\.(dtb)': empty (sub)expression
2011/02/08 09:15:39| pattern.conf line 3: refresh_pattern ([^.]+.|)symantecliveupdate.com/.*\.(zip|exe) 43200 100% 43200 reload-into-ims
2011/02/08 09:15:39| parse_refreshpattern: Invalid regular expression '([^.]+.|)symantecliveupdate.com/.*\.(zip|exe)': empty (sub)expression
2011/02/08 09:15:39| pattern.conf line 4: refresh_pattern ([^.]+.|)avast.com/.*\.(vpu|vpaa) 4320 100% 43200 reload-into-ims
2011/02/08 09:15:39| parse_refreshpattern: Invalid regular expression '([^.]+.|)avast.com/.*\.(vpu|vpaa)': empty (sub)expression
2011/02/08 09:15:39| pattern.conf line 5: refresh_pattern ([^.]+.|)adobe.com/.*\.(exe|msi) 4320 100% 43200 reload-into-ims
2011/02/08 09:15:39| parse_refreshpattern: Invalid regular expression '([^.]+.|)adobe.com/.*\.(exe|msi)': empty (sub)expression
2011/02/08 09:15:39| pattern.conf line 6: refresh_pattern ([^.]+.|)(download|adcdownload).(apple.|)com/.*\.(pkg|dmg) 8640 100% 86400 reload-into-ims
2011/02/08 09:15:39| parse_refreshpattern: Invalid regular expression '([^.]+.|)(download|adcdownload).(apple.|)com/.*\.(pkg|dmg)': empty (sub)expression
2011/02/08 09:15:39| pattern.conf line 7: refresh_pattern ([^.]+.|)(download|(windows|)update|).(microsoft.|)com/.*\.(cab|exe|msi|msp) 8640 100% 86400 reload-into-ims
2011/02/08 09:15:39| parse_refreshpattern: Invalid regular expression '([^.]+.|)(download|(windows|)update|).(microsoft.|)com/.*\.(cab|exe|msi|msp)': empty (sub)expression
2011/02/08 09:15:39| pattern.conf line 8: refresh_pattern ([^.]+.|)ubuntu.com/.*\.(tar|bz|bz2|gpg|gz|zip|deb) 8640 100% 86400 reload-into-ims
2011/02/08 09:15:39| parse_refreshpattern: Invalid regular expression '([^.]+.|)ubuntu.com/.*\.(tar|bz|bz2|gpg|gz|zip|deb)': empty (sub)expression

                                                or is simple to ignore it …
                                                if ... when it works it works beautifully!

                                                thanks

                                                1 Reply Last reply Reply Quote 0
                                                • N
                                                  Nachtfalke last edited by

                                                  Hi,

                                                  you are right. If I do squid -k rotate than there is the following output

                                                  2011/02/08 07:50:54| squid.conf line 74: refresh_pattern ([^.]+.|)avg.com/.*\.(b                                   in) 4320 100% 43200 reload-into-ims
2011/02/08 07:50:54| parse_refreshpattern: Invalid regular expression '([^.]+.|)                                   avg.com/.*\.(bin)': empty (sub)expression
2011/02/08 07:50:54| squid.conf line 75: refresh_pattern ([^.]+.|)spywareblaster                                   .net/.*\.(dtb) 4320 100% 4320 reload-into-ims
2011/02/08 07:50:54| parse_refreshpattern: Invalid regular expression '([^.]+.|)                                   spywareblaster.net/.*\.(dtb)': empty (sub)expression
2011/02/08 07:50:54| squid.conf line 76: refresh_pattern ([^.]+.|)symantecliveup                                   date.com/.*\.(zip|exe) 43200 100% 43200 reload-into-ims
2011/02/08 07:50:54| parse_refreshpattern: Invalid regular expression '([^.]+.|)                                   symantecliveupdate.com/.*\.(zip|exe)': empty (sub)expression
2011/02/08 07:50:54| squid.conf line 77: refresh_pattern ([^.]+.|)avast.com/.*\.                                   (vpu|vpaa) 4320 100% 43200 reload-into-ims
2011/02/08 07:50:54| parse_refreshpattern: Invalid regular expression '([^.]+.|)                                   avast.com/.*\.(vpu|vpaa)': empty (sub)expression
2011/02/08 07:50:54| squid.conf line 78: refresh_pattern ([^.]+.|)adobe.com/.*\.                                   (exe|msi) 4320 100% 43200 reload-into-ims
2011/02/08 07:50:54| parse_refreshpattern: Invalid regular expression '([^.]+.|)                                   adobe.com/.*\.(exe|msi)': empty (sub)expression
2011/02/08 07:50:54| squid.conf line 79: refresh_pattern ([^.]+.|)(download|adcd                                   ownload).(apple.|)com/.*\.(pkg|dmg) 8640 100% 86400 reload-into-ims
2011/02/08 07:50:54| parse_refreshpattern: Invalid regular expression '([^.]+.|)                                   (download|adcdownload).(apple.|)com/.*\.(pkg|dmg)': empty (sub)expression
2011/02/08 07:50:54| squid.conf line 80: refresh_pattern ([^.]+.|)(download|(win                                   dows|)update|).(microsoft.|)com/.*\.(cab|exe|msi|msp) 8640 100% 86400 reload-int                                   o-ims
2011/02/08 07:50:54| parse_refreshpattern: Invalid regular expression '([^.]+.|)                                   (download|(windows|)update|).(microsoft.|)com/.*\.(cab|exe|msi|msp)': empty (sub                                   )expression
2011/02/08 07:50:54| squid.conf line 81: refresh_pattern ([^.]+.|)ubuntu.com/.*\                                   .(tar|bz|bz2|gpg|gz|zip|deb) 8640 100% 86400 reload-into-ims
2011/02/08 07:50:54| parse_refreshpattern: Invalid regular expression '([^.]+.|)                                   ubuntu.com/.*\.(tar|bz|bz2|gpg|gz|zip|deb)': empty (sub)expression

                                                  But it seems to work and to cache updates from windows and from adobe (this are the two I tested yesterday).

                                                  1 Reply Last reply Reply Quote 0
                                                  • S
                                                    sam0t last edited by

                                                    I got bit older system at hands, Pfsense 1.2-RELEASE with the following squid packages installed:

                                                    squid Network 2.6.21_11
                                                    squidGuard Current: 1.2.0_1 pkg v.1.5 , Installed: 1.2.0_1-2

                                                    Maybe this is the reason for Windows Updates not caching.

                                                    1 Reply Last reply Reply Quote 0
                                                    • S
                                                      serangku last edited by

                                                      @Nachtfalke:

                                                      Hi,

                                                      you are right. If I do squid -k rotate than there is the following output

                                                      But it seems to work and to cache updates from windows and from adobe (this are the two I tested yesterday).

                                                      also when checking on cachemanager …
                                                      that pattern won't show on report .... unless this "([^.]+.|)" deleted, it's invalid regex
                                                      then pattern will show on report
                                                      well … it seem need validate on wiki from squid experience ...

                                                      thanks

                                                      1 Reply Last reply Reply Quote 0
                                                      • K
                                                        khan last edited by

                                                        Dear serangku
                                                        Look at this i m trying to solve this too

                                                        http://forum.pfsense.org/index.php/topic,33158.0.html

                                                        1 Reply Last reply Reply Quote 0
                                                        • S
                                                          serangku last edited by

                                                          i'll try tomorrow …

                                                          thanks for sharing

                                                          1 Reply Last reply Reply Quote 0
                                                          • S
                                                            scooterdouglas last edited by

                                                            @serangku:

                                                            also when checking on cachemanager …
                                                            that pattern won't show on report .... unless this "([^.]+.|)" deleted, it's invalid regex
                                                            then pattern will show on report
                                                            well … it seem need validate on wiki from squid experience ...

                                                            I'm sorry that everyone is having difficulty, I have only tried the regex on a 2-3 machines and everything appeared to be OK.  I running Squid 2.7 with pfSense 1.2.3 and I'm not getting any errors when I run squid -k rotate.  But that is not to say the regex is "faulty".  I had hoped that more people would have tested it before Jim had updated the wiki page, I think it might be time for myself to try a fresh install of version 1.2.3 or maybe try version 2 beta to see what is wrong.

                                                            Thanks.

                                                            1 Reply Last reply Reply Quote 0
                                                            • N
                                                              Nachtfalke last edited by

                                                              Hi,

                                                              in the past we talked about caching Kaspersky updates:
                                                              I tried this:

                                                              
refresh_pattern -i .*kaspersky\.com/.*\.(.*) 1440 100% 1440 reload-into-ims override-expire override-lastmode;
refresh_pattern -i .*kaspersky-labs\.com/.*\.(.*) 1440 100% 1440 reload-into-ims override-expire override-lastmode;

                                                              I think this is working just fine for the "Kaspersky Offline Updater Tool"
                                                              http://support.kaspersky.com/updater?level=2

                                                              Need more testing if it works with the "normal" updater function of the client software.
                                                              Perhaps someone else will find some time for this.

                                                              Further I found out, that if we use

                                                              range_offset_limit -1

                                                              squid is still downloading files even if the user has left a website or canceld some download.
                                                              To avoid this, I am using this:

                                                              
quick_abort_min 0 KB;
quick_abort_max 0 KB;
quick_abort_pct 100;

                                                              For windows updates I am using this without any errors in syslog and with success:

                                                              
refresh_pattern -i .*microsoft\.com/.*\.(cab|exe|msi|msp) 129600 100% 129600 reload-into-ims;
refresh_pattern -i .*windowsupdate\.com/.*\.(cab|exe|msi|msp) 129600 100% 129600 reload-into-ims;

                                                              To avoid, that a user is getting still a "Error 404" if a website wasn't available for a short time and then ist up again because of the high time for negative dns I am using this:

                                                              
negative_ttl 5 second;
negative_dns_ttl 5 second;

                                                              I am using this all in squid "Custom options".
                                                              It would be really nice if someone else could test and verify this or post some other usefull squid options.
                                                              For me it was hard to find these parameters and so I wnat to share them and get some feedback.

                                                              1 Reply Last reply Reply Quote 0
                                                              • First post
                                                                Last post