In need of help to solve a bandwidth issue

Nachtfalke

Thanks for that, great news.. Do you at all see any issues when your cache decides to update itself and use up all the bandwidth? That is what mine was doing..

Yes, in the past I had the problem that my cache started downloading files but no client has an open download/connection. So it was just squid which was downloading. I had sometimes a high difference that WAN has downloaded 8GB and LAN only 6GB.

But as I said in my post before I changed the custom options in squid and now it is working as it should: caching files but not downloading "useless" files.

luke240778

Well, i have inserted that code and will see how things go for a couple days.. thanks again!! it will indeed save alot of bandwidth.

Nachtfalke

You could add this for apple updates:

refresh_pattern -i .*apple\.com/.*\.(pkg|dmg) 129600 100% 129600 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private;

and this for ubuntu (an perhaps other linux distris) updates:

refresh_pattern -i .*ubuntu\.com/.*\.(tar|bz|bz2|gpg|gz|zip|deb) 129600 100% 129600 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private;

and perhaps this will save you bandwidth when you want to save some iso files and multimedia:

# refresh_pattern -i /.*\.(iso|wmv|mov|rm|avi|mp4|mpeg|mpg|divx|xvid|swf|flv|x-flv) 2880 100% 2880 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private;

But please be carful with the caching time.
For caching OS updates I am using 90 days.
For caching ISO files and so on just 2 days.

Perhaps you could play with these settings/times.

With these settings you could avoid that squid is caching wrong DNS entries longer than 5 seconds:

negative_ttl 5 second;
negative_dns_ttl 5 second;

Good luck an dpost back how it is working :o)

Nachtfalke

I did some (successful) changes for caching Windows 7 updates but in a way that squid isn't downloading files without any use.

range_offset_limit 20 MB;

This means:
I f you have a download with 100MB and the download is starting at 35MB till 100MB (what windows 7 seems to do) than this file will NOT be cache by squid with my option above because the range which is not downloades (less than 35MB) is bigger than the size in the range_offset_limit.

If the download of the 100MB file starts at 15MB till 100MB, that the range_offset_limit takes effect and squid is downloading the rest of the file before 15MB.

I hope I could explain it with my small english skills.

That this all will work it is neccessary that your "maximum object size" is high enough to cache the files you are downloading

luke240778

I havent seen my bandwidth issue return since adding this code which is a good thing.. but i do see since that my Disk usage has gone from around 8-10% up to around 70%.. With squid running for over 5 months with all my clients it was only using around 10%.. Nos wince adding this code its way up… What can i do to troubleshoot this?

Nachtfalke

Hi,

of course you disk cache is increasing - that's a good sign because this means that squid is caching many files. And you want to cache many files because you would like to save i-net bandwidth.
In "Cache Management" you find "Low- and High watermark" if more than 80% of the squid cache is used than squid starts to throw out "old" files. but this depends on your replacement algorithms.

For memory replacement I am using: Heap GDSF
For cache replacement I am using: Heap LFUDA

luke240778

Ill take a look at this. Out of interest, is there someway that i can see what files are in my cache? For example, not just websites, but would love to see what large files are in there, like music and movies and updates and stuff.. out of interest to see what my clients interests are to maybe try some other bandwidth saving techniques

Nachtfalke

Hi,

I am not sure where to find these information. But you can use cachemgr.cgi to analyze squid more in detail.

And I found this:
http://wiki.squid-cache.org/SquidFaq/SquidLogs#store.log

But you have to enable this in

/usr/local/pkg/squid.inc

store.log is disabled by default. To enable this I think you just have to modify the line 624:
from

cache_store_log none

to

/var/log/store.log

I am NOT sure if the syntax is correct but it should work. After changing the squid.inc you have to click "Save" in "Services -> Proxy Server -> General" so that the squid.conf will be generated with the new options. But then please check syslog if there are any errors.

luke240778

Thanks for that, i have cachmgr.cgi setup, but where in there shows the files in cache and what they are?

Ill play with that other part now also

Nachtfalke

Hi,

I am sorry but I didn't play with this kind of information. Cachemgr.cgi was just to give you some information about how squid performs.

"All Cached Objects" shows you all objects in squid but they are "encrypted". Here are some information about these lines:
14.2.1.26 objects: All Cache Objects
http://tilt.lib.tsinghua.edu.cn/docs/books/squid/TheDefinitiveGuideChinese/0596001622/squid-CHP-14-SECT-2.html