Is cache really improve the connection speed?

jigpe

Ok thanks. Is it okay to set 100000? 160GB HD

jigp
Davao City

josey

well, yes, you can put even 140gb,

pfs uses less than 1gb, and + swap is 1gb

iamthed

i finally get this….
explanation in pfs is wrong then
"Objects larger than the size specified (in kilobytes) will not be saved on disk. If you wish to increase speed more than you want to save bandwidth, this should be set to a low value."

ok i have 2gb ram, 120gb hdd, and i was playing this morning with cache,

setup
HDD cache size 1000MB
Memory cache size 512MB
Minimum object size 100kb
Maximum object size 50 000kb

and after test download, file od 40mb downloaded in 1 sec :)

why you set minimum object size to 100 kb
and max object size 50000 kb?

i think it's mean max object size per packet..
40mb file that u download it's consist of thousand packet i think

but, how to clean cache?
is it automatic? oldest object deleted for new objects? or how ?

thanks

chudy

First of all Squid is the most powerful and high performance cache.

The question is how you setup your squid…
If you only setup squid as default configuration then forget about squid. stick to your browsers cache.

When using squid:
1. you have to set your refresh pattern to override's servers content expiry(this way that content stays longer to your cache drive), reload-into-ims(check if modified)
2. Increase cache_dir(hdd is the limit), minimm object size to zero, maximum object size to 200MB(some porn has greater than 200MB;D)
This would only give you max saving around 25% you can see if you have mrtg to monitor your squid.

When using full potential it would give you max up to 100% or average of 60% (this includes contents html, php or alike which marked not to cache)
1. use StoreUrlRewrite feature (which you can cache youtube and any CDN servers)

Tikimotel

@josey:

thnx guys i figure it up yesterday :)

set up
hard disck cache size 100GB
hard disk cache system UFS
memory cache size 1GB
minimum object size 0
maximum object size 1GB
memory replacement policy Heap LFUDA

now internet is "fast" like i have 100mbps instead 6mbps :D
it works just great

great product guys, keep up with good work

I'm afraid your experience will degrade over time…
For every GB of HDD cache Squid will allocate approx. an extra 10MB of RAM. (like an index)

(my rig has 1GB ram minus some shared video ram)
Here is my cache setup:

Cache management page :
Hard disk cache size = 2000 MB (more HDD space uses more RAM becarefull not set it too big)
Memory cache size = 128 MB
Minimum object size = 0 KB

Maximum object size = 8192 KB (Squid default is 4096 KB)
set this larger if you want too (32MB maybe) it depends on how much files are how large and how often you download these types of files.
Read the squid visolve manual on caching policy (heap LFUDA). This not always stored in memory, it is the "cache" value!!! It can either be RAM or HDD.

Memory replacement policy = heap LFUDA (RAM cache object handling)
Cache replacement policy = heap LFUDA (HDD cache object handling)

General settings page:
Custom Options =
redirect_program /usr/local/libexec/adzap;redirect_children 10;maximum_object_size_in_memory 128 KB;

maximum_object_size_in_memory; I bumped that value up from the default 64KB because some sites (I visit often) use larger object than 64KB and so this raises the chance of objects being cached from RAM in stead of HDD cache. Play with this value a little in combination with the "Memory cache size" value, remember mine is set to 128MB ram being used. (128MB / 128KB = max. number of "large" objects cached directly from squid RAM. RAM is faster than HDD.) Don't over do these options it will do more harm than good over time…performance will suffer.

I also have "Snort package" running and these options balance my memory consumption nicely. (60 -70 percent average)

Tikimotel

@josey:

i finally get this….
explanation in pfs is wrong then
"Objects larger than the size specified (in kilobytes) will not be saved on disk. If you wish to increase speed more than you want to save bandwidth, this should be set to a low value."

ok i have 2gb ram, 120gb hdd, and i was playing this morning with cache,

setup
HDD cache size 1000MB
Memory cache size 512MB
Minimum object size 100kb
Maximum object size 50 000kb

and after test download, file od 40mb downloaded in 1 sec :)

but, how to clean cache?
is it automatic? oldest object deleted for new objects? or how ?

thanks

Cache is cleaned automatically.
How depends on the caching policy that is chosen. (LRU, Heap GDSF, Heap LFUDA, Heap LRU)
When is controlled by the high and low water mark options in the "Cache management" page.
Low-water-mark in % = 90
High-water-mark in % = 95
If the total cache is BIG the 5 % difference between the low- and high-water-mark can be many MBs or GBs of cache (real data).
(Worst case) Your system could be constantly cleaning older objects from cache at 100% HDD speed if the low-water-mark was reached. And caching speed would suffer greatly.

Tikimotel

I forgot to mention, my browsers use very little local cache (temporary internet files).
I set it around 32MB, the rest is cached by pfSense (squid) anyway.

I remember IE used a setting of 10% of local HDD in the older versions, before HDD's were hundreds of GBs large.
I believe the newer versions have setup a more sensible limit.

iamthed

i don't understand why you ppl put a maximum object size very high..
i think the definition maximum object is for the packet.. not for the overall file..
one file that u download 1mb it maybe consist of hundred package..
so why you setting to high??
i think 1024 for maximum object is enough..

and what is Low-water-mark and High-water-mark in % ?

guys are you using bridge mode for transparent proxy? because i can filter anything in bridge mode using transparent proxy.. ???

mhab12

I set the max object size very high to cache windows updates. There is nothing worse than watching people sit at their desk while a 150mb .NET service pack downloads at 300K.

iamthed

@mhab12:

I set the max object size very high to cache windows updates. There is nothing worse than watching people sit at their desk while a 150mb .NET service pack downloads at 300K.

u don't get it what i mean..
cmiiw the file that u download 150mb.. it consist of a thousand packet..
one packet maybe contain arround 256 kb

what i mean is max object size is per packet.. not a whole file..

jahonix

Hay, d-ifyouare, who told you squid would cache IP packets only? It is caching an entire object (aka file) from the likes of .html or .jpg or .png or.xml or …
Looking at your squid cache folders you'll find objects of highly varying sizes. That's the size of the cached file. It wouldn't differ if its only packets.

iamthed

@jahonix:

Hay, d-ifyouare, who told you squid would cache IP packets only? It is caching an entire object (aka file) from the likes of .html or .jpg or .png or.xml or …
Looking at your squid cache folders you'll find objects of highly varying sizes. That's the size of the cached file. It wouldn't differ if its only packets.

u're questioning about "who told me" that's juz my assumption.. that's why i said "CMIIW"
ok so now my question.. someone refer that he's using cache for windows update..
let's say person A he download windows update at speed 5kb and finish the update <–-- squid will save it as a cache right?

and then person B download windows update too.. how aproximately the speed? is it as fast as file sharing?

Tikimotel

If you have a very busy network the file downloaded from windows-update might not be in cache anymore.
If the network isn't very busy chances are the file will be delivered from squid cache (either RAM or HDD).

Squid performance depends greatly on usage, hence the many caching policies and configuration option (not all implemented into the package).
There is no simple configuration to satisfy all usage variables.

josey

@iamthed:

@jahonix:

Hay, d-ifyouare, who told you squid would cache IP packets only? It is caching an entire object (aka file) from the likes of .html or .jpg or .png or.xml or …
Looking at your squid cache folders you'll find objects of highly varying sizes. That's the size of the cached file. It wouldn't differ if its only packets.

u're questioning about "who told me" that's juz my assumption.. that's why i said "CMIIW"
ok so now my question.. someone refer that he's using cache for windows update..
let's say person A he download windows update at speed 5kb and finish the update <–-- squid will save it as a cache right?

and then person B download windows update too.. how aproximately the speed? is it as fast as file sharing?

yes, how is it working for me
comp A download file 500mb large ar speed 100kbs
after finish
comp B download same file at speed 10 000kbs+ :)

so is it worth it?
definitely!

iamthed

yes, how is it working for me
comp A download file 500mb large ar speed 100kbs
after finish
comp B download same file at speed 10 000kbs+ :)

so is it worth it?
definitely!

okay how bout this situation..
comp 1 download file A on website name www.whocares.com
comp 2 download file A but from diffrent website but it has the same contain from www.icares.com

is cache from website whocares and i cares has stored double on var/squid/cache??
i think i started to like cache :P

josey

who cares if it holds double file :D

you (at least i do) have 100 gigs cache :D

but, good question, anybody knows? for surely?
maybe if name is same, and size…

Bern

The cache will have two copies of the same file as they came from different URIs/URLs.

If you're worried about duplicate Windows Updates then you should be using WSUS or something like Heise's offline update.

In a business environment of any importance you should really be using WSUS to manage/track updates anyway.

iamthed

sorry guys i'm newbie.. i wanna se my cache..
but how?
is it go from shell?
what is the command for look it..
thx a lot

jigpe

users used 60gb a day. Is it ok?

jigp
Davao City
1.2.2

chudy

okay how bout this situation..
comp 1 download file A on website name www.whocares.com
comp 2 download file A but from diffrent website but it has the same contain from www.icares.com

is cache from website whocares and i cares has stored double on var/squid/cache??
i think i started to like cache :P

This is how CDN works. It downloads the content from the different servers usually it downloads near to your location.
Thats why squid has storeurl feature to redirect them into 1 file.