Navigation

    Netgate Discussion Forum
    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search

    Not caching any thing at all?

    pfSense Packages
    4
    9
    3374
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • S
      saywhaat last edited by

      Hi guys, I've been tasked to create a cache server for Windows Updates. I don't have much experience with Squid but it seems like it works pretty well (from what I have read).

      So I downloaded pfSense VMWare image and booted up the machine, installed the Squid package and adjusted a couple things.

      My test network is as so…(I am double NATing with the pfSense in place but I don't think it should matter..)

      Internet(Office LAN to be accurate)-->pfSenseWAN-->pfSenseLAN(DHCP Server)-->Workstations

      Now i've tried to set it up as a transparent proxy/non transparent proxy..nothing seems to happen. Proxy bind to either WAN/LAN..nothing happens.

      My custom Options:

      refresh_pattern ([^.]+.|)(download|(windows|)update|).(microsoft.|)com/.*\.(cab|exe|msi|msp) 4320 100% 43200 reload-into-ims;range_offset_limit -1; 
      refresh_pattern ([^.]+\.)?(download|(windows)?update)\.(microsoft\.)?com/.*\.(cab|exe|msi|msp|psf) 4320 100% 43200 reload-into-ims; range_offset_limit -1;
      

      I have both examples from the wiki site in there just to make sure I get it right..

      Hard Disk Cache size is 10000MB
      Min Obj Size is 1024KB
      Maximum Obj Size is 512000KB

      So like I said, I don't have much experience with Squid but from the looks of it, it didn't seem too hard to be able to set up? A division of my company has computers coming and going which all need updates after being reloaded, that is the purpose of this. Is getting this thing to work just a matter of adjusting some Squid settings and going? So far, no good =( My test is, I will download an IE install package from here http://windows.microsoft.com/en-US/internet-explorer/downloads/ie-9/worldwide-languages on one machine, then download the same package on another. So far, nothing to indicate I'm pulling down from the Squid Cache itself. Same T1 speed when I re-download it. Any help is much appreciated!

      1 Reply Last reply Reply Quote 0
      • marcelloc
        marcelloc last edited by

        Microsoft wsus isn't a better way to manager Windows updates?

        1 Reply Last reply Reply Quote 0
        • S
          saywhaat last edited by

          @marcelloc:

          Microsoft wsus isn't a better way to manager Windows updates?

          Much better yes, but we service at least 30 new PCs per day (all different/unique) that will not be part of a domain. We don't want to make registry changes to alter the update path. I was hoping to get Squid working as it seems pretty painless and transparent once it works.

          1 Reply Last reply Reply Quote 0
          • P
            phil.davis last edited by

            WSUS has to run on a Windows Server. With client systems coming and going, you would have to point the systems at the Windows Server WSUS (run a reg script with the settings) and then restore the settings at the end (so that traveling systems will get new updates from Microsoft or wherever they are going to be put in the network).

            So, if there is no Windows Server close by, or you don't want to bother changing and restoring Windows Update settings on each machine that comes through, then it can be useful to have some other cache of updates.

            1 Reply Last reply Reply Quote 0
            • marcelloc
              marcelloc last edited by

              Enable squid logs to see if it is working.

              You may need to increase cache total size and object size too.

              1 Reply Last reply Reply Quote 0
              • S
                saywhaat last edited by

                @marcelloc:

                Enable squid logs to see if it is working.

                You may need to increase cache total size and object size too.

                Here is a snippet from cache.log..

                2012/03/08 20:08:14| Reconfiguring Squid Cache (version 2.7.STABLE9)...
                2012/03/08 20:08:14| FD 14 Closing HTTP connection
                2012/03/08 20:08:14| FD 15 Closing HTTP connection
                2012/03/08 20:08:14| FD 17 Closing HTCP socket
                2012/03/08 20:08:14| FD 18 Closing SNMP socket
                2012/03/08 20:08:14| logfileClose: closing log /var/squid/logs/access.log
                2012/03/08 20:08:14| Including Configuration File: /usr/local/etc/squid/squid.conf (depth 0)
                2012/03/08 20:08:14| Cache dir '/var/squid/cache' size remains unchanged at 10240000 KB
                2012/03/08 20:08:14| squid.conf line 67: refresh_pattern ([^.]+.|)(download|(windows|)update|).(microsoft.|)com/.*\.(cab|exe|msi|msp) 4320 100% 43200 reload-into-ims
                2012/03/08 20:08:14| parse_refreshpattern: Invalid regular expression '([^.]+.|)(download|(windows|)update|).(microsoft.|)com/.*\.(cab|exe|msi|msp)': empty (sub)expression
                2012/03/08 20:08:14| Initialising SSL.
                2012/03/08 20:08:14| logfileOpen: opening log /var/squid/logs/access.log
                2012/03/08 20:08:14| Store logging disabled
                2012/03/08 20:08:14| Referer logging is disabled.
                2012/03/08 20:08:14| DNS Socket created at 0.0.0.0, port 32268, FD 12
                2012/03/08 20:08:14| Adding domain endeavor from /etc/resolv.conf
                2012/03/08 20:08:14| Adding nameserver 10.1.1.5 from /etc/resolv.conf
                2012/03/08 20:08:14| Adding nameserver 8.8.8.8 from /etc/resolv.conf
                2012/03/08 20:08:14| Accepting proxy HTTP connections at 10.1.1.15, port 3128, FD 14.
                2012/03/08 20:08:14| Accepting transparently proxied HTTP connections at 127.0.0.1, port 3128, FD 15.
                2012/03/08 20:08:14| Accepting HTCP messages on port 4827, FD 17.
                2012/03/08 20:08:14| Accepting SNMP messages on port 3401, FD 18.
                2012/03/08 20:08:14| WCCP Disabled.
                2012/03/08 20:08:14| Loaded Icons.
                2012/03/08 20:08:14| Ready to serve requests.
                

                I downloaded the same MS file on two different workstations after I made the configuration changes (changed my min size to 2MB and max size to 1GB)..no entries in the log..no indication it's caching. The "Invalid Regular Expression", is that something to take note of?

                1 Reply Last reply Reply Quote 0
                • N
                  Nachtfalke last edited by

                  Choose this pattern:

                  refresh_pattern -i .*apple\.com/.*\.(pkg|dmg) 259200 100% 259200 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private;
                  refresh_pattern -i .*microsoft\.com/.*\.(cab|exe|msi|msp) 259200 100% 259200 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private;
                  refresh_pattern -i .*windowsupdate\.com/.*\.(cab|exe|msi|msp) 259200 100% 259200 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private;
                  refresh_pattern -i .*ubuntu\.com/.*\.(tar|bz|bz2|gpg|gz|zip|deb) 259200 100% 259200 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private;
                  range_offset_limit 100 MB;
                  quick_abort_pct 60;
                  
                  

                  They are working for me with Windows XP updates and Windows 7 updates.

                  1 Reply Last reply Reply Quote 0
                  • S
                    saywhaat last edited by

                    @Nachtfalke:

                    Choose this pattern:

                    refresh_pattern -i .*apple\.com/.*\.(pkg|dmg) 259200 100% 259200 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private;
                    refresh_pattern -i .*microsoft\.com/.*\.(cab|exe|msi|msp) 259200 100% 259200 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private;
                    refresh_pattern -i .*windowsupdate\.com/.*\.(cab|exe|msi|msp) 259200 100% 259200 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private;
                    refresh_pattern -i .*ubuntu\.com/.*\.(tar|bz|bz2|gpg|gz|zip|deb) 259200 100% 259200 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private;
                    range_offset_limit 100 MB;
                    quick_abort_pct 60;
                    
                    

                    They are working for me with Windows XP updates and Windows 7 updates.

                    Thanks, this worked..at least with files I download directly from the Microsoft site. I'm having the techs run automatic updates today, keeping my fingers crossed. I also added in a couple more MS subdomains. So if I enter in these refresh patterns for certain URLs, does this mean files from sites not included in my refresh pattern do not get cached?

                    1 Reply Last reply Reply Quote 0
                    • N
                      Nachtfalke last edited by

                      @saywhaat:

                      @Nachtfalke:

                      Choose this pattern:

                      refresh_pattern -i .*apple\.com/.*\.(pkg|dmg) 259200 100% 259200 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private;
                      refresh_pattern -i .*microsoft\.com/.*\.(cab|exe|msi|msp) 259200 100% 259200 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private;
                      refresh_pattern -i .*windowsupdate\.com/.*\.(cab|exe|msi|msp) 259200 100% 259200 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private;
                      refresh_pattern -i .*ubuntu\.com/.*\.(tar|bz|bz2|gpg|gz|zip|deb) 259200 100% 259200 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private;
                      range_offset_limit 100 MB;
                      quick_abort_pct 60;
                      
                      

                      They are working for me with Windows XP updates and Windows 7 updates.

                      Thanks, this worked..at least with files I download directly from the Microsoft site. I'm having the techs run automatic updates today, keeping my fingers crossed. I also added in a couple more MS subdomains. So if I enter in these refresh patterns for certain URLs, does this mean files from sites not included in my refresh pattern do not get cached?

                      Files will always be cached if the site allows to cache them. If the header says "do not cache" then squid will not cache these files by default. But we can override these settings - that's why we use the refresh_pattern with the aadditional commands:

                      override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private;
                      

                      Probably MS does not want you to cache updates  ;)

                      1 Reply Last reply Reply Quote 0
                      • First post
                        Last post

                      Products

                      • Platform Overview
                      • TNSR
                      • pfSense
                      • Appliances

                      Services

                      • Training
                      • Professional Services

                      Support

                      • Subscription Plans
                      • Contact Support
                      • Product Lifecycle
                      • Documentation

                      News

                      • Media Coverage
                      • Press
                      • Events

                      Resources

                      • Blog
                      • FAQ
                      • Find a Partner
                      • Resource Library
                      • Security Information

                      Company

                      • About Us
                      • Careers
                      • Partners
                      • Contact Us
                      • Legal
                      Our Mission

                      We provide leading-edge network security at a fair price - regardless of organizational size or network sophistication. We believe that an open-source security model offers disruptive pricing along with the agility required to quickly address emerging threats.

                      Subscribe to our Newsletter

                      Product information, software announcements, and special offers. See our newsletter archive to sign up for future newsletters and to read past announcements.

                      © 2021 Rubicon Communications, LLC | Privacy Policy