Squid3 only reports TCP_MISS, never TCP_HIT

rocketdog

squidclient -h pfSense_LAN_IP_Address -p 3128 mgr:info

Hi!

This command gives me a bunch of html-code, and when I run```
squidclient -h 192.168.0.1 -p 3128 mgr:info >> test.php

KOM

Just SSH in and run the command as given or do it via Diagnostics - Command Prompt.

rocketdog

@KOM:

Just SSH in and run the command as given or do it via Diagnostics - Command Prompt.

I've tried that too, it is as if squid is blocking the LAN ip.

Edit: I believe I used the wrong IP earlier, however, now when I am using the new IP it results:

[2.2-RELEASE][admin@gate.ninya.org]/usr/local/www: squidclient -h 192.168.0.1 -p 3128 mgr:info >> test.php
client: ERROR: Cannot connect to 192.168.0.1:3128: Operation timed out

I've doubled checked, and the port is correct.

Edit 2:

Even this times out:

squidclient mgr:info
client: ERROR: Cannot connect to [::1]:3128: Operation timed out

KOM

You're sure that Squid is running??

rocketdog

@KOM:

You're sure that Squid is running??

AFAIK, yes. And according to ps -aux.

KOM

I've never seen that before. You have Squid listening on port 3128? Any thing of note in your System log?

rocketdog

Yes, it's on 3128.. forgot to say that it is listening on a bridged interface. Could that be the reason it's "time outing"? I haven't opened any ports or so.
The only rules I have is to let the Bridge to WIFI and LAN net.

KOM

I can't imagine that bridged mode would affect the squidclient running on the local server.

Do you have anything in Services - Proxy server - Local Cache - External Cache-Managers? If not, try putting 127.0.0.1; LAN_IP_Address there, save and see if it makes a difference. For example, I use:

127.0.0.1; 10.10.4.1

where 10.10.4.1 is my pfSense LAN IP address.

rocketdog

@KOM:

I can't imagine that bridged mode would affect the squidclient running on the local server.

Do you have anything in Services - Proxy server - Local Cache - External Cache-Managers? If not, try putting 127.0.0.1; LAN_IP_Address there, save and see if it makes a difference. For example, I use:

127.0.0.1; 10.10.4.1

where 10.10.4.1 is my pfSense LAN IP address.

External Cache-Manager was empty, added my IPs now tho..

I've noticed that when I google something, and hit a link, it takes +30 seconds to leave Google for the new link.. What could be the reason for this?

Edit: And how do I get rid of the local hits on "Real Time"? I have added the WAN and LAN IP at "Do not cache", but it still floods 'squid_monitor_data.php'

KOM

Usually long delays in the cache process are due to DNS issues. Shell in and run:

squidclient -h Your_pfSense_LAN-IP -p 3128 mgr:info

For for Median Service Times section. Look for long delays relative to the other counters.

I don't have an answer to your real-time question.

rocketdog

@KOM:

Usually long delays in the cache process are due to DNS issues. Shell in and run:
squidclient -h Your_pfSense_LAN-IP -p 3128 mgr:info
For for Median Service Times section. Look for long delays relative to the other counters.

I don't have an answer to your real-time question.

This is the result:

Median Service Times (seconds)  5 min    60 min:
        HTTP Requests (All):   0.05331  0.02069
        Cache Misses:          0.15048  0.10857
        Cache Hits:            0.00000  0.00000
        Near Hits:             0.03622  0.03829
        Not-Modified Replies:  0.00000  0.00000
        DNS Lookups:           0.02231  0.02231

I dont know if this is good or bad? Do you want to look for more info?
Some sites (like Google, and Wikipedia in particular, as I said before) can really be a b*tch sometimes..

rocketdog

I am bumping the post above:

Is the result of that info good or bad? I am not really that familiar with squid3 yet, and not sure how to tweak it.

KOM

Nothing really out of the ordinary. It really depends on what you're loading. For instance, you can go to a site like CacheFly and download one of their 1 or 10 MB test files, and do it again and watch it come out of the cache. But for websites these days there is a lot of dynamic content that is generated on the fly, with file names that change based on hashes, or web mechanics that force reloading or prevent caching, etc. I must admit that I've never seen a zero before for hits if there is web activity for a reasonable amount of time.

If you really want to know, check your access.log(s) in /var/log/squid and search for hits there. The manager info display only shows the last hour of data.

periko

I have seen cases like this one, the fix have been most of the time re-create cache dirs…

Before first stop squid, next delete cache dirs and finally create again the cache dirs.

squid -k

Test, test, test.

TIP: Don't use UFS is really old use AUFS.

Tikimotel

@rocketdog:

Hi!

I recently installed squid3 (pfSense 2.2).

I've been following some guides, tweaks etc, but when I look at the logs, and Realtime (Servces->Proxy Server->Realtime), all requests are TCP_MISS…Not one single TCP_HIT.

This is how it is configurated at the moment:
General
Proxy Interfaces: Bridge (Lan+Wifi)
Allow users on interface: On

Transparent Proxy interface(s): Bridge (Lan+Wifi)
Transparent Proxy: On

Local cache
Low-water-mark in %: 90
High-water-mark in %: 95
Enable offline mode: Off
Hard disk cache size: 5000 (mb)
Hard disk cache system: UFS
Minimum object size: 0
Maximum object size: 512000 (kb)

Memory cache size: 0
Maximum object size in RAM: 32 (kb)
Memory replacement policy: Head GDSF
Cache Dynamic Content: On

All the other stuff available I havent touched at all, such as: Antivirus, ACL, Auth, Traffic MGMT etc.

Help would be much appreciated.

This is a sample of /var/squid/logs/access.log

425297689.414  10051 192.168.0.10 TCP_MISS/200 1435 GET http://svl4.bredbandskollen.se/ulinfo/3043210925.txt - ORIGINAL_DST/192.36.135.235 text/plain
1425297689.986    104 192.168.0.10 TCP_MISS/200 457 POST http://www.bredbandskollen.se/api.php - ORIGINAL_DST/5.178.76.98 text/html
1425297691.343   1265 192.168.0.10 TCP_MISS/200 1405 GET http://www.bredbandskollen.se/api.php? - ORIGINAL_DST/5.178.76.98 text/xml
1425297691.353   1276 192.168.0.10 TCP_MISS/200 506 GET http://www.google-analytics.com/__utm.gif? - ORIGINAL_DST/77.53.0.154 image/gif
1425297691.384     12 192.168.0.10 TCP_MISS/200 506 GET http://www.google-analytics.com/__utm.gif? - ORIGINAL_DST/77.53.0.154 image/gif
1425297691.417     18 192.168.0.10 TCP_MISS/200 575 GET http://www.bredbandskollen.se/api.php? - ORIGINAL_DST/5.178.76.98 text/xml
1425297692.391  12969 192.168.0.10 TCP_MISS_ABORTED/000 0 POST http://svl4.bredbandskollen.se/cgi/upload.cgi? - ORIGINAL_DST/192.36.135.235 -
1425297692.708  13309 192.168.0.10 TCP_MISS_ABORTED/000 0 POST http://svl4.bredbandskollen.se/cgi/upload.cgi? - ORIGINAL_DST/192.36.135.235 -
1425297697.306  15831 192.168.0.10 TCP_MISS/200 339 GET http://svl4.bredbandskollen.se/huinfo/3043210925.txt? - ORIGINAL_DST/192.36.135.235 text/plain
1425297698.037    109 192.168.0.10 TCP_MISS/200 43829 GET http://www.aftonbladet.se/ - ORIGINAL_DST/144.63.250.5 text/html
1425297698.239     76 192.168.0.10 TCP_MISS/200 11944 GET http://www.svd.se/Bilder/Artikelbilder/4371071.svd/binary/a/ab-volvo.jpg - ORIGINAL_DST/144.63.252.10 image/jpeg
1425297698.239     77 192.168.0.10 TCP_MISS/200 4580 GET http://www.svd.se/Bilder/Artikelbilder/4374735.svd/binary/a/sasen.jpg - ORIGINAL_DST/144.63.252.10 image/jpeg
1425297698.282     97 192.168.0.10 TCP_MISS/200 2740 GET http://gfx.aftonbladet-cdn.se/image/20395294/90/fourByThree/bcf1ec93c88b6/ccb8b687-72cc-46b3-9d57-97240362fa44 - ORIGINAL_DST/193.240.120.254 image/jpeg
1425297698.282     97 192.168.0.10 TCP_MISS/200 2634 GET http://gfx.aftonbladet-cdn.se/image/20398521/90/fourByThree/b41f6d2c30187/917d6bd9-2e7c-42a6-912e-56b8604d28ca - ORIGINAL_DST/193.240.120.254 image/jpeg
1425297698.282     97 192.168.0.10 TCP_MISS/200 3350 GET http://gfx.aftonbladet-cdn.se/image/20397871/90/fourByThree/ce3cbafa23f78/saskastrup.jpg - ORIGINAL_DST/193.240.120.254 image/jpeg
1425297698.282     97 192.168.0.10 TCP_MISS/200 1873 GET http://gfx.aftonbladet-cdn.se/image/20340566/90/fourByThree/5e23f894626f1/Norwegian-B787-Dreamliner.jpg - ORIGINAL_DST/193.240.120.254 image/jpeg
1425297698.282     97 192.168.0.10 TCP_MISS/200 3022 GET http://gfx.aftonbladet-cdn.se/image/20398488/90/fourByThree/6e9d0823f1227/d745cce8-e54b-4afe-b8dc-0bed11ce33b3 - ORIGINAL_DST/193.240.120.254 image/jpeg
1425297698.283     97 192.168.0.10 TCP_MISS/200 5371 GET http://gfx.aftonbladet-cdn.se/image/20397871/133/fourByThree/09bf5af2f8f78/saskastrup.jpg - ORIGINAL_DST/193.240.120.254 image/jpeg
1425297698.283     97 192.168.0.10 TCP_MISS/200 7224 GET http://gfx.aftonbladet-cdn.se/image/20399180/315/imageColumnDouble/c85630966175b/68792-a03382fd1617eb0e0044fec87692be7e401fae8b.jpg - ORIGINAL_DST/193.240.120.254 image/jpeg
1425297698.283     97 192.168.0.10 TCP_MISS/200 12365 GET http://gfx2.aftonbladet-cdn.se/image/20400044/485/normal/3adce41c36813/duscha.jpg - ORIGINAL_DST/193.240.120.254 image/jpeg
1425297698.283     97 192.168.0.10 TCP_MISS/200 14586 GET http://gfx2.aftonbladet-cdn.se/image/20400499/485/imageColumnDouble/e29b8aa4fe0b2/hahnepuff.jpg - ORIGINAL_DST/193.240.120.254 image/jpeg
1425297698.283     98 192.168.0.10 TCP_MISS/200 19474 GET http://gfx2.aftonbladet-cdn.se/image/20400420/475/normal/c28476356e2a4/blavitt.jpg - ORIGINAL_DST/193.240.120.254 image/jpeg
1425297698.287    102 192.168.0.10 TCP_MISS/200 39910 GET http://gfx.aftonbladet-cdn.se/image/19911491/655/normal/b5dd20d77b6a9/asp.jpg - ORIGINAL_DST/193.240.120.254 image/jpeg
1425297698.309     90 192.168.0.10 TCP_MISS/200 338 GET http://wwwapp.aftonbladet.se/eae-logger/Logger? - ORIGINAL_DST/192.71.240.35 image/gif
1425297698.375     27 192.168.0.10 TCP_MISS/200 3559 GET http://gfx.aftonbladet-cdn.se/image/20398152/90/fourByThree/11838d90acef2/SK%C3%84RMAVBILD-2015-03-01-KL.-15.44.15.jpg - ORIGINAL_DST/193.240.120.254 image/jpeg
1425297698.408    137 192.168.0.10 TCP_MISS/200 8777 GET http://gfx.omni-cdn.se/images/62da823f-2224-4571-81f1-e650ba96543f? - ORIGINAL_DST/193.240.120.254 image/jpeg
1425297698.789    139 192.168.0.10 TCP_REFRESH_UNMODIFIED/200 16812 GET http://cdn.mxpnl.com/libs/mixpanel-2.2.min.js - ORIGINAL_DST/23.46.124.19 application/x-javascript
1425297699.024    121 192.168.0.10 TCP_MISS/200 523 GET http://api.mixpanel.com/track/? - ORIGINAL_DST/159.122.4.74 application/json
1425297699.045   8969 192.168.0.10 TCP_MISS_ABORTED/000 0 GET http://www.bredbandskollen.se/api.php? - ORIGINAL_DST/5.178.76.98 -
1425297699.627  20108 192.168.0.10 TCP_MISS/200 350 GET http://priscilla.lrfmedia.se/socket.io/1/xhr-polling/d-7j4zW-Qt0OZ4Ctq-6w? - ORIGINAL_DST/185.20.14.15 text/plain
1425297700.114  20741 192.168.0.10 TCP_MISS/502 3470 POST http://svl4.bredbandskollen.se/cgi/upload.cgi? - ORIGINAL_DST/192.36.135.235 text/html
1425297700.117  20659 192.168.0.10 TCP_MISS/502 3470 POST http://svl4.bredbandskollen.se/cgi/upload.cgi? - ORIGINAL_DST/192.36.135.235 text/html
1425297700.445    999 192.168.0.10 TCP_MISS/200 1515 GET http://gfx.aftonbladet-cdn.se/assets/gfx/streamers/extra/extra-655.gif? - ORIGINAL_DST/193.240.120.254 image/gif

Memory cache size: 0
This is the amount of physical RAM (in megabytes) to be used for negative cache and in-transit objects. This value should not exceed more than 50% of the installed RAM. The minimum value is 1MB.

Up this to at least some value other than 0.

rocketdog

@Tikimotel:

Memory cache size: 0
This is the amount of physical RAM (in megabytes) to be used for negative cache and in-transit objects. This value should not exceed more than 50% of the installed RAM. The minimum value is 1MB.

Up this to at least some value other than 0.

Hello again, and sorry if I bump this thread, but I can't get this out of my head.
I just recently installed a 120GB HDD, using it for squid-cache. I changed some settings to;

Low-water-mark in %: 90
High-water-mark in %: 95
Enable offline mode: Off
Hard disk cache size: 95000 (mb) (was 5000)
Hard disk cache system: UFS
Minimum object size: 0
Maximum object size: 2048 (kb) (was 512000)

Memory cache size: 1647 mb (was 0 mb)
Maximum object size in RAM: 32 (kb)
Memory replacement policy: Head LFUDA (was HEAD GDSF)
Cache Dynamic Content: On

When I do cat /var/squid/logs/access.log | grep HIT , 99,99% of the HITs are http://www.google.com/cast. It feels like I missed something important.

Xeboc

@rocketdog:

Edit: And how do I get rid of the local hits on "Real Time"? I have added the WAN and LAN IP at "Do not cache", but it still floods 'squid_monitor_data.php'

I was able to stop this by adding a proxy exception for the firewall IP on the windows computer I'm using.
(Internet Properties -> Connections -> LAN settings -> Advanced)

As for no HITS, I found that binding squid to localhost caused it to MISS everything. Removing the localhost binding caused squid to start functioning correctly again. No idea why….

I also found that squid didn't like to use the disk cache at all until it was rebuilt. I probably re-booted the router while the initial creating of directories was happening... I used:


squid -k shutdown
squid -z -S