Caching not working, screenshots attached
-
Decided to go ahead and install the squid package. After a couple of hours i thought i would check out the logs, and from what i can see nothing is being cached? Could someone please help me get this working? I followed 2 guides side by side and made a few custom changes along the way.
As you can see from the cache settings ive set it to null in hope it would only use RAM caching? my /tmp and /var are also set to RAM memory as i am running this full install on 2 ssd's in a GEOM mirror so trying to reduce read/writes.
Please see screenshots attached..
Thanks in advance.
-
Any ideas on this chaps? looked back over everything this morning and cannot work it out!
Help Help Help!!
-
Can someone please help with this as its been going on for days, people keep viewing this thread but no replies!!
Someone must know something!
-
It's a lot easier for us to see your shots when they're embedded here, and not linked to a 3rd-party site. Look under Attachments and other options.
Why is your Hard Disk Cache System set to null? The default is ufs. Nothing will be cached to disk, only RAM.
Plus, you don't show (or I missed it) how you have squid configured. Is it transparent or explicit? is anyone on your network using the proxy at all to generate log data?
By default, squid's logging is disabled. You need to enable it.
Edit: Do yourself a favour and turn off that ClamAV stuff. Not worth the performance hit IMO.
-
Hi KOM,
Thought it would be easier to attach them as i did so they dont load full screen for everyone, will remember this for future posts.
Hard Disk Cache System is set to null as i am running a a geom mirror of 2 ssd's, so didnt want it caching to the same drive as boot?
Seems that i have missed that image out, but i have it configured as transparent proxy mode. Is there any better way i can show you the config?
Didnt want to enable squids logging as it suggests not to do it if you are low on ram, and my system has 8gb, ive allocated 4gb to cache but it said dont allocate more than half of the system ram so i didnt enable the logging for this reason.
Everyone should be using the proxy on the network.
I ran clamAV as didnt find it made to much difference performance wise and like knwoing its being filtered before coming to the devices. Could you give me some more info on how it effects performance?
Also any recommendations with my 2 ssd's in geom mirror with config would be fantastic!
Cheers!
-
Hard Disk Cache System is set to null as i am running a a geom mirror of 2 ssd's, so didnt want it caching to the same drive as boot?
I'm not aware of any issues caching on SSDs. If it bothers you, add another disk and then change the squid cache location.
Didnt want to enable squids logging as it suggests not to do it if you are low on ram, and my system has 8gb, ive allocated 4gb to cache but it said dont allocate more than half of the system ram so i didnt enable the logging for this reason.
lol that rec is for really tiny systems, like APUs running 256 MB or so. You have 8 GB. Turn on logging.
I ran clamAV as didnt find it made to much difference performance wise and like knwoing its being filtered before coming to the devices. Could you give me some more info on how it effects performance?
Well, it has to scan everything coming in and compare to its lists, and I'm not sure how well-maintained those lists are. When I tried it a few years ago, our throughput took a serious hit. Use AV on the client, not the firewall. I know the onion security approach is best, but I don't have much faith in ClamAV unfortunately.
-
It wasn't so much as issues with caching on SSD's, but mainly with the read and write limits of ssd's? dont want drive failures any time soon?
I see, will turn logging on!
Will also turn ClamAV off, will report back in a couple hours of how i get on!
Thanks for taking the time to get involved with this!
-
Right, ClamAV is disabled, logging is enabled, and local cache is also enabled.
Could you give me some recommended values for local cache size? I have 2 60gb ssd's in a Geom mirror so 60gb working space.
Also how would i test that its working and caching?
I have noticed that this is filling up extremely fast?…
Disk usage ( /var )
45% of 145MiB - ufs in RAMThankyou!
-
The access logs are showing a lot of information so trafic is being routed through it i believe, but by the looks of it nothing is being cached?
Cheers.
-
but mainly with the read and write limits of ssd's? dont want drive failures any time soon?
You likely won't come anywhere near exhausting the write-cycles of todays SSDs so don't worry about it.
Could you give me some recommended values for local cache size? I have 2 60gb ssd's in a Geom mirror so 60gb working space.
Unless you're using alot of site sthat have a lot of static content, it's very likely that you won't be able to cache all that much, or at least a lot less than you would expect. In my experiences here, with about a dozen users I am caching in the 4-7% range. To answer your question, since you have all that space you could ealy do a 5-10 GB cache just to start and see how it works for you.
Personally, I only use squid as the base for squidguard URL filtering.
Also how would i test that its working and caching?
Check Services - Squid Proxy Server - Realtime - Squid Access Table. All requests are shown there. Cache.log only shows you some squid internal messages and isn't all that useful.
Edit; It's working for you as I can see the access.log above your cache.log.
-
Thanks for clearing up my SSD lifetime question, its always been a worry of mine as done lots and lots of reading into running full install of pfsense on ssd's and it killing them, but the articles were a few years old!
Ok great, aslong as its working, hopefully in a couple days once the cache has built up a bit it will take some strain off of the WAN connection, we have good speeds here anyway, but would rather have it running :).
Think i will set cache to 10gb as you suggested, see how it goes, is that a lot of data to cache? i wouldnt have a clue as never ran squid before.
Whats your thoughts on squid guard? is it quite demanding on the system? dont have anything to block out atm but might run it anyway to learn about.
Also, how large should this be? ..
Disk usage ( /var )
98% of 145MiB - ufs in RAMIts filled up in no time!
Edit: …
Squid has now stopped working (Nagios Monitoring System just informed me) .. logged into pfsense and seen .
Disk usage ( /var )
100% of 145MiB - ufs in RAMSo guessing thats why, how do i set it so it overwrites it to stop it shutting down squid?
2nd Edit...
Squid has come to a stop and cannot manually be started, im guessing because /var is at 100%? I also have no logs in system logs, accross any of the options!
-
is that a lot of data to cache?
10GB is pretty large, but it depends on the data being cached and the user requirements so it's hard to say without knowing more. The default cache size is 100MB.
Whats your thoughts on squid guard? is it quite demanding on the system?
It's a simple but effective URL filter. You can do the same thing with pfBlocker I believe (and much more), but squidguard is much lighter on resources.
As for your current problem, the squid cache lives in /var/squid/cache IIRC. II would stop squid if it isn't already and reset your disk and RAM settings back to default. Save them, clear the cache via the button and then start squid again and see how it goes.
-
@KOM:
is that a lot of data to cache?
10GB is pretty large, but it depends on the data being cached and the user requirements so it's hard to say without knowing more. The default cache size is 100MB.
Whats your thoughts on squid guard? is it quite demanding on the system?
It's a simple but effective URL filter. You can do the same thing with pfBlocker I believe (and much more), but squidguard is much lighter on resources.
As for your current problem, the squid cache lives in /var/squid/cache IIRC. II would stop squid if it isn't already and reset your disk and RAM settings back to default. Save them, clear the cache via the button and then start squid again and see how it goes.
Super, will give it a go and report back, would you mind if I was to contact you through PM?
Cheers!
-
would you mind if I was to contact you through PM?
I'd really rather you didn't, thanks. I don't help out here anywhere as much as I used to so you're better off posting it publicly so that anyone can address it.
-
Problem 1: Squid stops running and wont start once ufs in ram gets to 10% and cannot be manually started?
Problem 2: Nothing is being logged in the system logs, anything regarding firewall, VPN etc, just says there are no logs?
-
No idea. At this point I would be sorely tempted to blow it all away, install fresh and then install squid and reconfigure using defaults.
-
I dont understand what the point of running defaults is? i mean my system is more than capable of running 100 times more than the default spec, just dont understand why it would cause this issue?
I mean i thought running squid would be fantastic! might just bin it!
Wanted to run it for its squidguard also which im still yet to install
Edit:….
After looking back over my settings, it appears that i had this selected!.. "Use memory file system for /tmp and /var" so this would make a lot of sense to why my logs arn't showing and why /var was filling up so fast, will restart and post any updates!
-
I dont understand what the point of running defaults is?
After looking back over my settings, it appears that i had this selected!..
** cough **
