Not caching any thing at all?

saywhaat

Hi guys, I've been tasked to create a cache server for Windows Updates. I don't have much experience with Squid but it seems like it works pretty well (from what I have read).

So I downloaded pfSense VMWare image and booted up the machine, installed the Squid package and adjusted a couple things.

My test network is as so…(I am double NATing with the pfSense in place but I don't think it should matter..)

Internet(Office LAN to be accurate)-->pfSenseWAN-->pfSenseLAN(DHCP Server)-->Workstations

Now i've tried to set it up as a transparent proxy/non transparent proxy..nothing seems to happen. Proxy bind to either WAN/LAN..nothing happens.

My custom Options:

refresh_pattern ([^.]+.|)(download|(windows|)update|).(microsoft.|)com/.*\.(cab|exe|msi|msp) 4320 100% 43200 reload-into-ims;range_offset_limit -1; 
refresh_pattern ([^.]+\.)?(download|(windows)?update)\.(microsoft\.)?com/.*\.(cab|exe|msi|msp|psf) 4320 100% 43200 reload-into-ims; range_offset_limit -1;

I have both examples from the wiki site in there just to make sure I get it right..

Hard Disk Cache size is 10000MB
Min Obj Size is 1024KB
Maximum Obj Size is 512000KB

So like I said, I don't have much experience with Squid but from the looks of it, it didn't seem too hard to be able to set up? A division of my company has computers coming and going which all need updates after being reloaded, that is the purpose of this. Is getting this thing to work just a matter of adjusting some Squid settings and going? So far, no good =( My test is, I will download an IE install package from here http://windows.microsoft.com/en-US/internet-explorer/downloads/ie-9/worldwide-languages on one machine, then download the same package on another. So far, nothing to indicate I'm pulling down from the Squid Cache itself. Same T1 speed when I re-download it. Any help is much appreciated!

marcelloc

Microsoft wsus isn't a better way to manager Windows updates?

saywhaat

@marcelloc:

Microsoft wsus isn't a better way to manager Windows updates?

Much better yes, but we service at least 30 new PCs per day (all different/unique) that will not be part of a domain. We don't want to make registry changes to alter the update path. I was hoping to get Squid working as it seems pretty painless and transparent once it works.

phil.davis

WSUS has to run on a Windows Server. With client systems coming and going, you would have to point the systems at the Windows Server WSUS (run a reg script with the settings) and then restore the settings at the end (so that traveling systems will get new updates from Microsoft or wherever they are going to be put in the network).

So, if there is no Windows Server close by, or you don't want to bother changing and restoring Windows Update settings on each machine that comes through, then it can be useful to have some other cache of updates.

marcelloc

Enable squid logs to see if it is working.

You may need to increase cache total size and object size too.

saywhaat

@marcelloc:

Enable squid logs to see if it is working.

You may need to increase cache total size and object size too.

Here is a snippet from cache.log..

2012/03/08 20:08:14| Reconfiguring Squid Cache (version 2.7.STABLE9)...
2012/03/08 20:08:14| FD 14 Closing HTTP connection
2012/03/08 20:08:14| FD 15 Closing HTTP connection
2012/03/08 20:08:14| FD 17 Closing HTCP socket
2012/03/08 20:08:14| FD 18 Closing SNMP socket
2012/03/08 20:08:14| logfileClose: closing log /var/squid/logs/access.log
2012/03/08 20:08:14| Including Configuration File: /usr/local/etc/squid/squid.conf (depth 0)
2012/03/08 20:08:14| Cache dir '/var/squid/cache' size remains unchanged at 10240000 KB
2012/03/08 20:08:14| squid.conf line 67: refresh_pattern ([^.]+.|)(download|(windows|)update|).(microsoft.|)com/.*\.(cab|exe|msi|msp) 4320 100% 43200 reload-into-ims
2012/03/08 20:08:14| parse_refreshpattern: Invalid regular expression '([^.]+.|)(download|(windows|)update|).(microsoft.|)com/.*\.(cab|exe|msi|msp)': empty (sub)expression
2012/03/08 20:08:14| Initialising SSL.
2012/03/08 20:08:14| logfileOpen: opening log /var/squid/logs/access.log
2012/03/08 20:08:14| Store logging disabled
2012/03/08 20:08:14| Referer logging is disabled.
2012/03/08 20:08:14| DNS Socket created at 0.0.0.0, port 32268, FD 12
2012/03/08 20:08:14| Adding domain endeavor from /etc/resolv.conf
2012/03/08 20:08:14| Adding nameserver 10.1.1.5 from /etc/resolv.conf
2012/03/08 20:08:14| Adding nameserver 8.8.8.8 from /etc/resolv.conf
2012/03/08 20:08:14| Accepting proxy HTTP connections at 10.1.1.15, port 3128, FD 14.
2012/03/08 20:08:14| Accepting transparently proxied HTTP connections at 127.0.0.1, port 3128, FD 15.
2012/03/08 20:08:14| Accepting HTCP messages on port 4827, FD 17.
2012/03/08 20:08:14| Accepting SNMP messages on port 3401, FD 18.
2012/03/08 20:08:14| WCCP Disabled.
2012/03/08 20:08:14| Loaded Icons.
2012/03/08 20:08:14| Ready to serve requests.

I downloaded the same MS file on two different workstations after I made the configuration changes (changed my min size to 2MB and max size to 1GB)..no entries in the log..no indication it's caching. The "Invalid Regular Expression", is that something to take note of?

Nachtfalke

Choose this pattern:

refresh_pattern -i .*apple\.com/.*\.(pkg|dmg) 259200 100% 259200 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private;
refresh_pattern -i .*microsoft\.com/.*\.(cab|exe|msi|msp) 259200 100% 259200 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private;
refresh_pattern -i .*windowsupdate\.com/.*\.(cab|exe|msi|msp) 259200 100% 259200 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private;
refresh_pattern -i .*ubuntu\.com/.*\.(tar|bz|bz2|gpg|gz|zip|deb) 259200 100% 259200 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private;
range_offset_limit 100 MB;
quick_abort_pct 60;

They are working for me with Windows XP updates and Windows 7 updates.

saywhaat

@Nachtfalke:

Choose this pattern:

refresh_pattern -i .*apple\.com/.*\.(pkg|dmg) 259200 100% 259200 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private;
refresh_pattern -i .*microsoft\.com/.*\.(cab|exe|msi|msp) 259200 100% 259200 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private;
refresh_pattern -i .*windowsupdate\.com/.*\.(cab|exe|msi|msp) 259200 100% 259200 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private;
refresh_pattern -i .*ubuntu\.com/.*\.(tar|bz|bz2|gpg|gz|zip|deb) 259200 100% 259200 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private;
range_offset_limit 100 MB;
quick_abort_pct 60;

They are working for me with Windows XP updates and Windows 7 updates.

Thanks, this worked..at least with files I download directly from the Microsoft site. I'm having the techs run automatic updates today, keeping my fingers crossed. I also added in a couple more MS subdomains. So if I enter in these refresh patterns for certain URLs, does this mean files from sites not included in my refresh pattern do not get cached?

Nachtfalke

@saywhaat:

@Nachtfalke:
Choose this pattern:
refresh_pattern -i .*apple\.com/.*\.(pkg|dmg) 259200 100% 259200 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private;
refresh_pattern -i .*microsoft\.com/.*\.(cab|exe|msi|msp) 259200 100% 259200 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private;
refresh_pattern -i .*windowsupdate\.com/.*\.(cab|exe|msi|msp) 259200 100% 259200 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private;
refresh_pattern -i .*ubuntu\.com/.*\.(tar|bz|bz2|gpg|gz|zip|deb) 259200 100% 259200 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private;
range_offset_limit 100 MB;
quick_abort_pct 60;
They are working for me with Windows XP updates and Windows 7 updates.
Thanks, this worked..at least with files I download directly from the Microsoft site. I'm having the techs run automatic updates today, keeping my fingers crossed. I also added in a couple more MS subdomains. So if I enter in these refresh patterns for certain URLs, does this mean files from sites not included in my refresh pattern do not get cached?

Files will always be cached if the site allows to cache them. If the header says "do not cache" then squid will not cache these files by default. But we can override these settings - that's why we use the refresh_pattern with the aadditional commands:

override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private;

Probably MS does not want you to cache updates ;)

Not caching any thing at all?

Products

Services

Support

News

Resources

Company

Our Mission

Subscribe to our Newsletter