4. Squid 3 memory usage

Squid 3 memory usage

  • B

    Afternoon all,
    Is anyone aware of any memory leaks with the current squid packages? As everything from 0.3.7 to the current 0.4.0 appears to consume RAM like its going out of fashion!

    I've tried lots of config changes, from disk cache size, directories, memory cache size, object size, watermarks, ect; and no matter what i do, over the period of a few hours the RAM usage hits 99% and then the swap starts getting used. The issue happens on fresh 2.2.4 installs along with those fresh installs having whatever the latest squid package is at the time. This is on a system with 4Gb RAM/8Gb swap.

    Any ideas? Something obvious im missing here?

    Current config:
    Cache General: Low watermark in % = 60%
    Cache General: High watermark in % = 65%
    HD Cache: HD Cache Size = 4096Mb
    HD Cache: Level 1 Directories = 32
    HD Cache: Minimum Object Size = 0
    HD Cache: Maximum Object Size = 16Mb
    Mem Cache: Memory Cache Size = 128Mb
    Mem Cache: Maximum Object Size in RAM = 512Kb

    Currently doing a test with Memory cache set as 1Mb and its already taken less than 10mins to jump to 78% memory usage.

    Thanks in advance.

    0
  • B

    On first look the only reliable to limit memory usage, it appears, is to use the "cache deny all" variable & set the disk cache to 0 - not the end of the world as i only really need it to filter, but would be nice for the caching to work correctly.

    ##EDIT##
    This may be a bit premature as the memory is still creeping upwards, just not at quick as before.

    ##EDIT##
    Interestingly, the output from "top" shows it going up in 4Mb increments.

    0
  • B

    Number of bug reports about squid having different types of memory leaks:
    http://bugs.squid-cache.org/show_bug.cgi?id=4074
    http://bugs.squid-cache.org/show_bug.cgi?id=4005

    With:
    http://bugs.squid-cache.org/show_bug.cgi?id=4084
    Being the most generic.
    Not sure if these are related to my setup however.

    Anyone got any tips for getting the cache manager working on Squid3? As the top result guide on these forums doesn't appear to work for Squid3…

    0
  • D
    Banned

    Nothing like this will get fixed here. Upstream issues need to get solved upstream.

    0
  • B

    What are the chances of the existing package being updated to the latest release?

    0
  • D
    Banned

    Until 2.3 is released, probably none unless there's a huge security hole somewhere. Noone wants to touch PBI.

    0
  • B

    @doktornotor:

    Until 2.3 is released, probably none unless there's a huge security hole somewhere. Noone wants to touch PBI.

    Thought that'd be the answer. :p

    Time to throw RAM at the issue perhaps…

    0
  • B

    Amusingly, using all the defaults, but with the HDD cache set at 1024Mb seems to make things more stable RAM wise, slowly filling up, but strangely the swap seems to be filling even though there's free RAM ???
    And its seemingly ignoring the watermark % variables too as its way past the watermark i set yet isnt clearing the swap cache.

    0
  • D
    Banned

    @boomam:

    Anyone got any tips for getting the cache manager working on Squid3? As the top result guide on these forums doesn't appear to work for Squid3…

    1/ Run this from Diagnostics - Command Prompt - PHP Execute

    
require_once("/usr/local/pkg/squid.inc");
$cachemgr = "cachemgr.cgi";
symlink(SQUID_BASE . "/bin/{$cachemgr}", "/usr/local/www/{$cachemgr}");

    2/ Check that Squid is set to listen on loopback in General - Proxy Interfaces - otherwise you'll just get nifty timeouts.
    3/ Stick your trusted IP(s) into Local Cache - External Cache Managers Apparently not needed/ignored. The access is allowed with or without this.
    4/ Add this to General - Custom ACLS (Before Auth)

    
cachemgr_passwd none all

    Alternatively some saner ACLs:

    
cachemgr_passwd none 5min
cachemgr_passwd none 60min
cachemgr_passwd none asndb
cachemgr_passwd none authenticator
cachemgr_passwd none cbdata
cachemgr_passwd none client_list
cachemgr_passwd none comm_incoming
cachemgr_passwd none counters
cachemgr_passwd none delay
cachemgr_passwd none digest_stats
cachemgr_passwd none dns
cachemgr_passwd none events
cachemgr_passwd none filedescriptors
cachemgr_passwd none fqdncache
cachemgr_passwd none histograms
cachemgr_passwd none http_headers
cachemgr_passwd none info
cachemgr_passwd none io
cachemgr_passwd none ipcache
cachemgr_passwd none mem
cachemgr_passwd none menu
cachemgr_passwd none netdb
cachemgr_passwd none non_peers
cachemgr_passwd none objects
cachemgr_passwd none pconn
cachemgr_passwd none peer_select
cachemgr_passwd none redirector
cachemgr_passwd none refresh
cachemgr_passwd none server_list
cachemgr_passwd none store_digest
cachemgr_passwd none storedir
cachemgr_passwd none utilization
cachemgr_passwd none via_headers
cachemgr_passwd none vm_objects
cachemgr_passwd disable config
cachemgr_passwd disable offline_toggle
cachemgr_passwd disable reconfigure
cachemgr_passwd disable rotate
cachemgr_passwd disable shutdown

    Now you can browse to http(s)://your.pfsense.ip.or.fqdn/cachemgr.cgi and use the Administrator's Email set up in General tab just click the Continue button to login without password.

    As you can see, this buggy CGI thing is a nice buggy hole into your setup. WTF.

    0
  • B

    Thanks,
    after running the php code, this error generates:
    Warning: symlink(): No such file or directory in /usr/local/www/exec.php(250) : eval()'d code on line 3

    I've done the remainder of the steps and there's just a 404 error.

    0
  • D
    Banned

    You paste ALL the code I posted there into the field. ALL. Only after that you execute. Let me repeat: The WHOLE thing. Sigh. Really. If unable to follow, please, just leave the thing alone.

    0
  • B

    @doktornotor:

    You paste ALL the code I posted there into the field. ALL. Only after that you execute. Let me repeat: The WHOLE thing. Sigh. Really. If unable to follow, please, just leave the thing alone.

    All the code WAS pasted.
    Why would i pick and mix part of the code??? It makes no sense for me to pick parts of the code and ignore other bits does it.

    0
  • D
    Banned

    NFC. Look, this just works. Period. If unable to symlink a file, then tough cookies.

    0
  • B

    Its not a case of unable, its a case of your suggested step didn't work. Simple as.
    I appreciate the help but you shouldn't assume instantly that i've done something wrong when it was so simple a task that was followed verbatim from your steps.

    0
  • D
    Banned

    Sucks to be you. You apparently have more issues than this, such as downloading a bugfixed file having no effect on your box either. Better call ghostbusters.

    0
  • B

    Wow.

    https://forum.pfsense.org/index.php?topic=100167.msg562343#msg562343
    Thanks, but i cant really compensate when the GUI doesnt do what it says it will can i. ;)

    You really need to stop assuming everyones dumber than you. Whilst i'll freely admit that you know more about this subject than a good 90% of this forum, myself included, i really don't see the need to be so angry all the time when all people are asking for is help for something you helped put together. If you dont want to help, don't, but don't berate people for no reason other than frustration when YOU are choosing to help. Its open source, we're meant to help each other.
    Whether that's creating the packages like yourself, or effectively bug reporting like the rest of us.

    Based on the above revelation about the GUI, i'll try your symlink command via SSH and see if that makes a difference. If so, then I'll publish the results here so others can benefit from the findings.

    0
  • D
    Banned

    Sir. Perhaps you would have better luck doing these actions on a box where Squid3 is properly installed. Outta this debate.

    0
  • B

    That would be lovely, if the package worked 100% on its own.
    Which in all fairness, mostly does apart from either the apparently known memory leaks that are reported upstream, or a few bugs here and there in the provided package from Pf's repository.

    So anyway, regardless of method of input, same error:
    Warning: symlink(): No such file or directory in /usr/local/www/exec.php(250) : eval()'d code on line 3

    0
  • D
    Banned

    Yeah. Sucks to be you. If you have no such file, well…. then your Squid install is incomplete. If of course could STILL symlink the thing from shell, but that'd require producing some effort, instead of trying to paste PHP code in there.

    Bye.

    0
  • B

    You mean the PHP code you suggested to use in the first place ???
    I really do not know why you are hostile to everyone.

    I'll try your new code and hopefully that'll work. Thank you for your input.

    0
  • D
    Banned

    The code is absolutely the same… What's added there is to show you that IT WORKS. There's no new relevant code in there that'd fix the ghosts in your browser/pfSense box/god knows what.

    0
  • B

    Same error, (line 5 instead, but its the same code ;) )

    The box is entirely stock apart from:
    A few squid cache settings, Clam pointing at a regional update server, and your code for adding the widget, fixing the widget, adding the extra logs, and this.
    Nothing else, it's a fresh install, and this has been duplicated on two boxes, both with fresh installs.

    As always, thank you for your input and suggestions.

    0
  • B

    For the moment don't worry, i'll spin up another test box, test on that to see if there's something odd.

    0
  • B

    Strangely, two new VM's, one the code worked on, the other it didnt.
    The only difference between the two is post install they were given different external IPs.
    Very odd, ah well.

    0
  • B

    Well the watermark % variables appear to do nothing.
    Swap is now at 100% on one of the boxes, with RAM at about 70% usage.
    Top reporting Squid at 11Gb usage.

    This is with default settings in Squid too.

    0
  • D
    Banned

    There are docs on the watermark stuff. When you look there, they basically say that it's sort of "best effort" configuration and that stuff will get ignored whenever Squid considers it necessary. Let me restate for the last time: upstream issues will not get sorted on pfSense.

    And, heck… the docs are even linked from the GUI.  ::) http://www.squid-cache.org/Doc/config/cache_swap_low/

    Perhaps you are completely missing what "swap" means in Squid context. That's not the OS swap.

    0
  • B

    @doktornotor:

    There are docs on the watermark stuff. When you look there, they basically say that it's sort of "best effort" configuration and that stuff will get ignored whenever Squid considers it necessary. Let me restate for the last time: upstream issues will not get sorted on pfSense.

    And, heck… the docs are even linked from the GUI.  ::) http://www.squid-cache.org/Doc/config/cache_swap_low/

    Perhaps you are completely missing what "swap" means in Squid context. That's not the OS swap.

    Cheers for the links, however im fully aware of this thank you.
    To be clear, i am not asking you to fix upstream issues. I'm aware that there's little to no point with 2.3 on the horizon.
    I'm simply reporting so others who may happen can see this is maybe not limited to them if they are having it.

    Once i've "tuned" it as well as i can i plan on listening what i've done that helps mitigate the issue so others can benefit in the interim.

    0
  • D
    Banned

    Dude. Why are you referring to the watermark and your OS swap here, then? That stuff deals with disk cache object eviction. Not with evicting things from your physical RAM/swap.

    Well the watermark % variables appear to do nothing.
    Swap is now at 100% on one of the boxes, with RAM at about 70% usage.

    This has nothing to do with it. Simply sanitize your cache size if you are running out of memory. Or get more RAM. There are docs on the memory requirements as well.

    0
  • B

    Correct, but its the system swap, used by the kernel, that's getting used up.
    Testing the last week or two has shown improvements to the memory and system swap usage based on tweaking these variables. "Trial and error".

    0
  • D
    Banned

    Yeah. So, either decrease your cache size, or increase your RAM size. Messing with watermarks is completely useless.

    0
  • B

    Cache size has already been experimented with, as has the RAM allocation.
    No difference either way with memory usage.

    0
Log in to reply
 

Products

Services

Support

News

Resources

Company

Our Mission

We provide leading-edge network security at a fair price - regardless of organizational size or network sophistication. We believe that an open-source security model offers disruptive pricing along with the agility required to quickly address emerging threats.

Subscribe to our Newsletter

Product information, software announcements, and special offers. See our newsletter archive for past announcements.

© Copyright 2002 - 2019 Rubicon Communications, LLC | Privacy Policy