Squid cache performance: bad



  • G'evening  :-*

    I have Squid3 running. Soho, 6 users, transparent proxy (SquidGuard on top to block porn facebook ( ;D )).

    I notice strange stuff:

    PC1:
    1. Go to a site in Firefox. It loads.
    2. Click a link on that site; it loads relatively faster. Assumption: Squid3 is indeed caching certain web site elements (logos and stuff) and only needs to to fetch what it doesn't have. Good.
    3. Visit that very same site with a different browser (Opera, IE, Chrome): horribly slow. Assumption: it doesn't use any cache at all. Now I can understand the site has different pages to serve depending on the browser, but: horribly slow or a little more slow. There's a difference.
    4. "And now, for something completely different": PC2.

    PC2:
    1. Go to that very same site, again with Firefox, and watch it also not being fast (re: PC1: point 3).
    2. You would however, given the same browser Firefox, expect it to be as fast (re: PC1: point 2).

    Are there any custom settings I need to add to actually make it work?

    Thank you in advance for any help *:-

    Bye,**



  • Are those other browsers actually using the proxy?  Have you blocked off ports 80/443 to enforce proxy use?

    Are there any custom settings I need to add to actually make it work?

    Sure, but there is no one-size-fits-all.  Squid is not a simple thing.

    http://www.squid-cache.org/

    Lots of docs there.

    When you run your tests, you should also check the squid stats through the squidclient.  Shell in and run:

    squiclient -h LAN_IP -p 3128 mgr:info

    Look at your Cache information for Squid and Median Service Times.

    http://wiki.squid-cache.org/KnowledgeBase/PerformanceAnalysis



  • @KOM:

    Are those other browsers actually using the proxy?  Have you blocked off ports 80/443 to enforce proxy use?

    Thanks KOM  :-*

    No, I did not. I only enabled 'transparent proxy', should I also block 80/443 on the LAN(?)

    @KOM:

    Are there any custom settings I need to add to actually make it work?

    Sure, but there is no one-size-fits-all.  Squid is not a simple thing.

    http://www.squid-cache.org/

    Lots of docs there.

    When you run your tests, you should also check the squid stats through the squidclient.  Shell in and run:

    squiclient -h LAN_IP -p 3128 mgr:info

    Look at your Cache information for Squid and Median Service Times.

    http://wiki.squid-cache.org/KnowledgeBase/PerformanceAnalysis

    I recall you looked at my output before and thought everything was fine there. Do you perhaps see something wrong here?

    
     Sending HTTP request ... done.
    HTTP/1.1 200 OK
    Server: squid
    Mime-Version: 1.0
    Date: Wed, 22 Jul 2015 07:24:19 GMT
    Content-Type: text/plain
    Expires: Wed, 22 Jul 2015 07:24:19 GMT
    Last-Modified: Wed, 22 Jul 2015 07:24:19 GMT
    X-Cache: MISS from squid
    X-Cache-Lookup: MISS from squid:3128
    Connection: close
    
    Squid Object Cache: Version 3.4.10
    Build Info:
    Start Time:     Tue, 21 Jul 2015 22:00:15 GMT
    Current Time:   Wed, 22 Jul 2015 07:24:19 GMT
    Connection information for squid:
            Number of clients accessing cache:      3
            Number of HTTP requests received:       2592
            Number of ICP messages received:        0
            Number of ICP messages sent:    0
            Number of queued ICP replies:   0
            Number of HTCP messages received:       0
            Number of HTCP messages sent:   0
            Request failure ratio:   0.00
            Average HTTP requests per minute since start:   4.6
            Average ICP messages per minute since start:    0.0
            Select loop called: 3355056 times, 10.088 ms avg
    Cache information for squid:
            Hits as % of all requests:      5min: 20.5%, 60min: 18.2%
            Hits as % of bytes sent:        5min: 42.9%, 60min: 30.6%
            Memory hits as % of hit requests:       5min: 0.0%, 60min: 39.5%
            Disk hits as % of hit requests: 5min: 0.0%, 60min: 0.6%
            Storage Swap size:      3951416 KB
            Storage Swap capacity:  23.6% used, 76.4% free
            Storage Mem size:       27696 KB
            Storage Mem capacity:    0.7% used, 99.3% free
            Mean Object Size:       69.83 KB
            Requests given to unlinkd:      18
    Median Service Times (seconds)  5 min    60 min:
            HTTP Requests (All):   0.08729  0.27332
            Cache Misses:          0.10281  0.28853
            Cache Hits:            0.00000  0.16775
            Near Hits:             0.00000  0.37825
            Not-Modified Replies:  0.05633  0.05633
            DNS Lookups:           0.03223  0.02683
            ICP Queries:           0.00000  0.00000
    Resource usage for squid:
            UP Time:        33844.128 seconds
            CPU Time:       206.444 seconds
            CPU Usage:      0.61%
            CPU Usage, 5 minute avg:        0.93%
            CPU Usage, 60 minute avg:       1.74%
            Maximum Resident Size: 409504 KB
            Page faults with physical i/o: 0
    Memory accounted for:
            Total accounted:        39672 KB
            memPoolAlloc calls:    878252
            memPoolFree calls:     886566
    File descriptor usage for squid:
            Maximum number of file descriptors:   58977
            Largest file desc currently in use:    149
            Number of file desc currently in use:   41
            Files queued for open:                   0
            Available number of file descriptors: 58936
            Reserved number of file descriptors:   100
            Store Disk files open:                   0
    Internal Data Structures:
             56668 StoreEntries
              1191 StoreEntries with MemObjects
              1190 Hot Object Cache Items
             56589 on-disk objects
    


  • should I also block 80/443 on the LAN(?)

    No, not if you're using transparent mode.

    Do you perhaps see something wrong here?

    No, but all your cache hits are coming from memory and not disk.

    How are you handling HTTPS when squid is transparent?  Have you installed a pfSense cert on your clients?



  • @KOM:

    Do you perhaps see something wrong here?

    No, but all your cache hits are coming from memory and not disk.

    How are you handling HTTPS when squid is transparent?  Have you installed a pfSense cert on your clients?

    Thanks KOM  ;D

    No, I have not (didn't know I needed to). But it also happens with HTTP, so without the s.



  • I'm starting to believe that disk caching is not very useful anymore due to low hit rates, and squid is only good now for URL filtering with squidguard.