Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Squid is eating all my memory FIXED

    Scheduled Pinned Locked Moved pfSense Packages
    8 Posts 3 Posters 12.1k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • S
      Steve Evans
      last edited by

      I notice that squid is consuming a lot of memory, even when the size of the memory cache is set to < 1/2 the physical memory. The service watchdog fortunately restarts squid an squidguard when they run out of memory, about every four days. I see the following.

      Another view (taken from Observium) is a little clearer and shows where I upgraded the memory from 1G to 2G on Fri 16th May. The memory usage of squid was even clearer after this.

      Using top -o size its easy to see that squid is the big memory hog, and how it grows.

      last pid: 99281;  load averages:  0.00,  0.00,  0.00   up 15+21:51:16  19:39:58
      73 processes:  2 running, 67 sleeping, 4 zombie
      CPU:  0.4% user,  0.0% nice,  0.4% system,  0.0% interrupt, 99.3% idle
      Mem: 242M Active, 234M Inact, 229M Wired, 1684K Cache, 112M Buf, 1275M Free
      Swap: 
      
        PID USERNAME       THR PRI NICE   SIZE    RES STATE    TIME   WCPU COMMAND
       6715 root             6  44    0   333M  8896K uwait    0:20  0.00% radiusd
      88143 proxy           17  44    0   120M   107M ucond    0:10  0.00% squid
      35763 root             1  64   20 89424K  7644K nanslp   3:14  0.00% php
      74686 root             1  54    0 83920K 27796K accept   0:27  0.00% php
      92912 root             1  60    0 78160K 19336K accept   0:04  0.00% php
      
      

      I read that squid a rather leak ridden. Anybody else seen this?

      Steve
      ![Screen Shot 2014-06-01 at 19.31.43.png_thumb](/public/imported_attachments/1/Screen Shot 2014-06-01 at 19.31.43.png_thumb)
      ![Screen Shot 2014-06-01 at 19.31.43.png](/public/imported_attachments/1/Screen Shot 2014-06-01 at 19.31.43.png)
      ![Screen Shot 2014-06-01 at 19.30.33.png](/public/imported_attachments/1/Screen Shot 2014-06-01 at 19.30.33.png)
      ![Screen Shot 2014-06-01 at 19.30.33.png_thumb](/public/imported_attachments/1/Screen Shot 2014-06-01 at 19.30.33.png_thumb)

      1 Reply Last reply Reply Quote 0
      • jimpJ
        jimp Rebel Alliance Developer Netgate
        last edited by

        How large is your disk cache?

        IIRC, the RAM specified for the memory cache is a minimum, not a maximum. It will use more depending on your other settings.

        Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

        Need help fast? Netgate Global Support!

        Do not Chat/PM for help!

        1 Reply Last reply Reply Quote 0
        • S
          Steve Evans
          last edited by

          Hi,

          I've reduced the cache size to 512M. The webpage describes it thus, so I think it's a max amount to use.

          This is the amount of physical RAM (in megabytes) to be used for negative cache and in-transit objects. This value should not exceed more than 50% of the installed RAM. The minimum value is 1MB.

          After making the above post I noticed the following message in /var/log/squid/cache.log, repeated endlessly.

          2014-06-01 18:41:19 [50045] (squidGuard): can't write to logfile /var/log/squidGuard.log
          

          Having changed the file owner from root to proxy things seem to have improved. The memory usage has more or less stabilised.

          last pid:  2272;  load averages:  0.05,  0.06,  0.02   up 18+00:01:32  21:50:14
          77 processes:  4 running, 69 sleeping, 4 zombie
          CPU:  3.4% user,  1.1% nice,  5.2% system,  0.4% interrupt, 89.9% idle
          Mem: 874M Active, 742M Inact, 229M Wired, 56M Cache, 112M Buf, 81M Free
          Swap: 
          
            PID USERNAME       THR PRI NICE   SIZE    RES STATE    TIME   WCPU COMMAND
          88143 proxy           17  44    0   744M   730M ucond    1:36  0.00% squid
           6715 root             6  44    0   333M  9176K uwait    0:26  0.00% radiusd
          35763 root             1  64   20 89424K  7644K nanslp   3:39  0.00% php
          28756 root             1  96    0 86992K 31364K RUN      0:24  0.00% php
          74075 root             1  45    0 78160K 19392K accept   0:09  0.00% php
          40552 root             1  76    0 77136K  7988K wait     0:00  0.00% php
          36424 root             1  47    0 77136K  6804K wait     0:00  0.00% php
          55104 proxy            1  44    0 15564K  4084K sbwait   0:00  0.00% squidGuard
          62286 proxy            1  44    0 15564K  3956K sbwait   0:00  0.00% squidGuard
          60459 proxy            1  44    0 15564K  3892K sbwait   0:00  0.00% squidGuard
          88094 proxy            1  76    0 13756K  7300K wait     0:00  0.00% squid
           5500 root             3  44    0 13020K  4016K ucond    5:30  0.00% bacula-fd
          
          
          1 Reply Last reply Reply Quote 0
          • S
            Steve Evans
            last edited by

            I believe I'm getting somewhere now. Having set the cache to 1024M and used wget to haul down a large number of web pages very fast, I'm seeing that the reported memory usage from squid to be within range, but the memory consumption of it's process to be roughly double this.

            Cache information for squid:
            	Hits as % of all requests:	5min: 0.0%, 60min: 0.5%
            	Hits as % of bytes sent:	5min: 0.7%, 60min: 0.7%
            	Memory hits as % of hit requests:	5min: 0.0%, 60min: 16.9%
            	Disk hits as % of hit requests:	5min: 0.0%, 60min: 71.8%
            	Storage Swap size:	2829518 KB
            	Storage Swap capacity:	27.6% used, 72.4% free
            	Storage Mem size:	444924 KB
            	Storage Mem capacity:	42.4% used, 57.6% free
            	Mean Object Size:	53.03 KB
            
            last pid: 88230;  load averages:  0.05,  0.05,  0.07   up 18+02:49:17  00:37:59
            71 processes:  2 running, 65 sleeping, 4 zombie
            CPU:  1.1% user,  0.0% nice,  1.1% system,  0.4% interrupt, 97.4% idle
            Mem: 1065M Active, 591M Inact, 249M Wired, 268K Cache, 112M Buf, 78M Free
            Swap: 
            
              PID USERNAME       THR PRI NICE   SIZE    RES STATE    TIME   WCPU COMMAND
            53129 proxy           17  44    0   941M   928M RUN      0:35  0.98% squid
            52037 root             1  45    0 84944K 25744K accept   0:21  0.00% php
            74075 root             1  76    0 78160K 17900K accept   0:09  0.00% php
            82484 root             1  64   20  8224K  8244K select  34:10  0.00% ntpd
             6715 root             6  44    0   333M  8036K uwait    0:27  0.00% radiusd
            
            

            And then waiting a couple of minutes we get.

            Cache information for squid:
            	Hits as % of all requests:	5min: 0.0%, 60min: 0.5%
            	Hits as % of bytes sent:	5min: 0.7%, 60min: 0.7%
            	Memory hits as % of hit requests:	5min: 0.0%, 60min: 16.9%
            	Disk hits as % of hit requests:	5min: 0.0%, 60min: 71.8%
            	Storage Swap size:	2862396 KB
            	Storage Swap capacity:	28.0% used, 72.0% free
            	Storage Mem size:	478360 KB
            	Storage Mem capacity:	45.6% used, 54.4% free
            	Mean Object Size:	52.86 KB
            
            last pid: 75787;  load averages:  0.18,  0.08,  0.07   up 18+02:50:54  00:39:36
            71 processes:  1 running, 66 sleeping, 4 zombie
            CPU:  0.0% user,  0.0% nice,  2.2% system,  1.9% interrupt, 95.9% idle
            Mem: 1138M Active, 530M Inact, 239M Wired, 31M Cache, 112M Buf, 45M Free
            Swap: 
            
              PID USERNAME       THR PRI NICE   SIZE    RES STATE    TIME   WCPU COMMAND
            53129 proxy           17  44    0  1010M   996M ucond    0:38  1.95% squid
            52037 root             1  45    0 84944K 25616K accept   0:21  0.00% php
            74075 root             1  76    0 78160K 17900K accept   0:09  0.00% php
            82484 root             1  64   20  8224K  8244K select  34:10  0.00% ntpd
            40552 root             1  73    0 77136K  7984K wait     0:00  0.00% php
            83319 root             1  44    0  9460K  7768K select   3:11  0.00% bsnmpd
            52679 proxy            1  76    0 13756K  7684K wait     0:00  0.00% squid
            

            So, an extra 34M or so of cached data takes up another 68M or so of resident memory.

            Looks like I need to be halving my cache size down to nearer 512M.

            Steve

            1 Reply Last reply Reply Quote 0
            • jimpJ
              jimp Rebel Alliance Developer Netgate
              last edited by

              You'll probably find this most helpful: http://wiki.squid-cache.org/SquidFaq/SquidMemory

              Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

              Need help fast? Netgate Global Support!

              Do not Chat/PM for help!

              1 Reply Last reply Reply Quote 0
              • S
                Steve Evans
                last edited by

                Thanks for that; I'd been looking at that and similar pages before comparing the General Runtime Information from cachemgr.cgi with the output of top -o res, as shown below. With the cache size limit set to 512M it has now stabilised using 1084M of resident memory when the memory cache is full.

                Cache information for squid:
                	Hits as % of all requests:	5min: 1.9%, 60min: 1.2%
                	Hits as % of bytes sent:	5min: 1.1%, 60min: 0.8%
                	Memory hits as % of hit requests:	5min: 10.0%, 60min: 30.3%
                	Disk hits as % of hit requests:	5min: 90.0%, 60min: 62.8%
                	Storage Swap size:	2978610 KB
                	Storage Swap capacity:	29.1% used, 70.9% free
                	Storage Mem size:	521188 KB
                	Storage Mem capacity:	99.4% used,  0.6% free
                	Mean Object Size:	52.09 KB
                	Requests given to unlinkd:	0
                
                last pid:  4916;  load averages:  0.03,  0.08,  0.02   up 18+23:40:34  21:29:16
                72 processes:  1 running, 67 sleeping, 4 zombie
                CPU:  5.6% user,  0.0% nice,  3.4% system,  9.0% interrupt, 82.0% idle
                Mem: 1206M Active, 408M Inact, 227M Wired, 53M Cache, 112M Buf, 88M Free
                Swap: 
                
                  PID USERNAME       THR PRI NICE   SIZE    RES STATE    TIME   WCPU COMMAND
                53129 proxy           17  44    0  1135M  1084M ucond    2:26  1.95% squid
                
                

                The recommendation to not exceed 50% of available RAM appears bogus and I believe should be changed to 25%.

                Steve

                1 Reply Last reply Reply Quote 0
                • M
                  mhab12
                  last edited by

                  I am having similar issues and would like to troubleshoot further.  What command line option did you run to get your squid cache stats?  I also found that there were memory leak fixes in two Squid releases immediately after the build that the pfSense package uses (3.1.20).  Perhaps we can get the package rebuilt to the final 3.1 release (3.1.23).

                  Here are the release notes for the memory leak issues, one fix in 3.1.21 and one in 3.1.22:
                  http://www.squid-cache.org/Versions/v3/3.1/changesets/SQUID_3_1_22.html
                  http://www.squid-cache.org/Versions/v3/3.1/changesets/SQUID_3_1_21.html

                  1 Reply Last reply Reply Quote 0
                  • S
                    Steve Evans
                    last edited by

                    See HERE for how to get Cache Manager stats. Select General Runtime Information from the menu to get the stats above.

                    Steve

                    1 Reply Last reply Reply Quote 0
                    • First post
                      Last post
                    Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.