[SOLVED]squid wont cache squid tcp_miss200 & access denied

Shreeyashattal

Hello, I'm new to squid and have been trying my luck to get it working. I've checked the logs and as you can see below everything is either a TCP_MISS/200 or NEGATIVE_HIT


1448563212.979    276 192.168.1.35 TCP_MISS/204 365 GET http://b.scorecardresearch.com/b? - DIRECT/184.86.250.10 -
1448563219.889    179 192.168.1.35 TCP_MISS/200 4424 GET http://tile-service.weather.microsoft.com/en-GB/livetile/preinstall? - DIRECT/23.35.61.218 text/xml
1448563220.196    533 192.168.1.35 TCP_MISS/200 1633 GET http://cdn.content.prod.cms.msn.com/singletile/summary/alias/experiencebyname/today? - DIRECT/184.86.250.25 text/xml
1448563220.397    738 192.168.1.35 TCP_MISS/200 1574 GET http://finance.services.appex.bing.com/Market.svc/AppTileV3? - DIRECT/184.86.250.8 application/xml
1448563229.403     37 192.168.1.35 TCP_MISS/200 1197 GET http://radarfeed.moneycontrol.com/mcradar/processing.php? - DIRECT/124.153.64.106 text/html
1448563249.413     47 192.168.1.35 TCP_MISS/200 1178 GET http://radarfeed.moneycontrol.com/mcradar/processing.php? - DIRECT/124.153.64.106 text/html
1448563269.406     40 192.168.1.35 TCP_MISS/200 1178 GET http://radarfeed.moneycontrol.com/mcradar/processing.php? - DIRECT/124.153.64.106 text/html
1448563289.400     38 192.168.1.35 TCP_MISS/200 1178 GET http://radarfeed.moneycontrol.com/mcradar/processing.php? - DIRECT/124.153.64.106 text/html
1448563309.408     44 192.168.1.35 TCP_MISS/200 1178 GET http://radarfeed.moneycontrol.com/mcradar/processing.php? - DIRECT/124.153.64.106 text/html
1448563322.167    502 192.168.1.35 TCP_MISS/200 1546 GET http://cdn.content.prod.cms.msn.com/singletile/summary/alias/experiencebyname/today? - DIRECT/184.86.250.25 text/xml
1448563322.186    521 192.168.1.35 TCP_MISS/200 1709 GET http://finance.services.appex.bing.com/Market.svc/AppTileV3? - DIRECT/184.86.250.8 application/xml
1448563322.187    522 192.168.1.35 TCP_MISS/200 1914 GET http://finance.services.appex.bing.com/Market.svc/AppTileV3? - DIRECT/184.86.250.16 application/xml
1448563322.192    527 192.168.1.35 TCP_MISS/200 1907 GET http://finance.services.appex.bing.com/Market.svc/AppTileV3? - DIRECT/184.86.250.8 application/xml
1448563322.380    739 192.168.1.35 TCP_MISS/200 1862 GET http://finance.services.appex.bing.com/Market.svc/AppTileV3? - DIRECT/184.86.250.8 application/xml
1448563322.385    720 192.168.1.35 TCP_MISS/200 1875 GET http://finance.services.appex.bing.com/Market.svc/AppTileV3? - DIRECT/184.86.250.16 application/xml
1448563322.410    770 192.168.1.35 TCP_MISS/200 1895 GET http://finance.services.appex.bing.com/Market.svc/AppTileV3? - DIRECT/184.86.250.16 application/xml
1448563322.434    770 192.168.1.35 TCP_MISS/200 1871 GET http://finance.services.appex.bing.com/Market.svc/AppTileV3? - DIRECT/184.86.250.16 application/xml
1448563322.447    783 192.168.1.35 TCP_MISS/200 2061 GET http://finance.services.appex.bing.com/Market.svc/AppTileV3? - DIRECT/184.86.250.8 application/xml
1448563323.280   1614 192.168.1.35 TCP_MISS/200 1877 GET http://finance.services.appex.bing.com/Market.svc/AppTileV3? - DIRECT/184.86.250.16 application/xml
1448563325.890      0 192.168.1.1 TCP_DENIED/403 1392 GET cache_object://192.168.1.1/info - NONE/- text/html
1448563329.407     39 192.168.1.35 TCP_MISS/200 1178 GET http://radarfeed.moneycontrol.com/mcradar/processing.php? - DIRECT/124.153.64.106 text/html
1448563349.475    109 192.168.1.35 TCP_MISS/200 1178 GET http://radarfeed.moneycontrol.com/mcradar/processing.php? - DIRECT/124.153.64.106 text/html
1448563369.398     41 192.168.1.35 TCP_MISS/200 1178 GET http://radarfeed.moneycontrol.com/mcradar/processing.php? - DIRECT/124.153.64.106 text/html
1448563380.068      0 192.168.1.41 TCP_DENIED/400 1490 NONE NONE:// - NONE/- text/html
1448563389.406     38 192.168.1.35 TCP_MISS/200 1178 GET http://radarfeed.moneycontrol.com/mcradar/processing.php? - DIRECT/124.153.64.106 text/html
1448563409.455     90 192.168.1.35 TCP_MISS/200 1178 GET http://radarfeed.moneycontrol.com/mcradar/processing.php? - DIRECT/124.153.64.106 text/html
1448563429.405     41 192.168.1.35 TCP_MISS/200 1178 GET http://radarfeed.moneycontrol.com/mcradar/processing.php? - DIRECT/124.153.64.106 text/html
1448563449.407     43 192.168.1.35 TCP_MISS/200 1178 GET http://radarfeed.moneycontrol.com/mcradar/processing.php? - DIRECT/124.153.64.106 text/html
1448563469.410     41 192.168.1.35 TCP_MISS/200 1178 GET http://radarfeed.moneycontrol.com/mcradar/processing.php? - DIRECT/124.153.64.106 text/html
1448563489.407     39 192.168.1.35 TCP_MISS/200 1178 GET http://radarfeed.moneycontrol.com/mcradar/processing.php? - DIRECT/124.153.64.106 text/html
1448563509.410     37 192.168.1.35 TCP_MISS/200 1178 GET http://radarfeed.moneycontrol.com/mcradar/processing.php? - DIRECT/124.153.64.106 text/html
1448563530.415     45 192.168.1.35 TCP_MISS/200 1178 GET http://radarfeed.moneycontrol.com/mcradar/processing.php? - DIRECT/124.153.64.106 text/html
1448563550.399     33 192.168.1.35 TCP_MISS/200 1178 GET http://radarfeed.moneycontrol.com/mcradar/processing.php? - DIRECT/124.153.64.106 text/html
1448563570.422     54 192.168.1.35 TCP_MISS/200 1197 GET http://radarfeed.moneycontrol.com/mcradar/processing.php? - DIRECT/124.153.64.106 text/html
1448563590.407     34 192.168.1.35 TCP_MISS/200 1178 GET http://radarfeed.moneycontrol.com/mcradar/processing.php? - DIRECT/124.153.64.106 text/html
1448563611.411     40 192.168.1.35 TCP_MISS/200 1178 GET http://radarfeed.moneycontrol.com/mcradar/processing.php? - DIRECT/124.153.64.106 text/html
1448563631.406     35 192.168.1.35 TCP_MISS/200 1197 GET http://radarfeed.moneycontrol.com/mcradar/processing.php? - DIRECT/124.153.64.106 text/html
1448563651.404     39 192.168.1.35 TCP_MISS/200 1197 GET http://radarfeed.moneycontrol.com/mcradar/processing.php? - DIRECT/124.153.64.106 text/html
1448563671.420     47 192.168.1.35 TCP_MISS/200 1178 GET http://radarfeed.moneycontrol.com/mcradar/processing.php? - DIRECT/124.153.64.106 text/html

squidclient -h 192.168.1.1 -p mgr:info

results in


HTTP/1.0 403 Forbidden
Server: squid/2.7.STABLE9
Date: Thu, 26 Nov 2015 18:42:05 GMT
Content-Type: text/html
Content-Length: 1083
X-Squid-Error: ERR_ACCESS_DENIED 0
X-Cache: MISS from localhost
X-Cache-Lookup: NONE from localhost:3128
Via: 1.0 localhost:3128 (squid/2.7.STABLE9)
Connection: close

<title>ERROR: The requested URL could not be retrieved</title>

# ERROR

## The requested URL could not be retrieved

* * *

While trying to retrieve the URL:
[cache_object://192.168.1.1/info](cache_object://192.168.1.1/info)

The following error was encountered:

*   **Access Denied.** 

    Access control configuration prevents your request from
    being allowed at this time.  Please contact your service provider if
    you feel this is incorrect.

Your cache administrator is [admin@localhost](mailto:admin@localhost).

* * *

<address>
Generated Thu, 26 Nov 2015 18:42:05 GMT by localhost (squid/2.7.STABLE9)
</address>

any pointers?

KOM

Are you using an older version of pfSense or Squid?

Any URL that includes a ? seems to not cache. As for your access denied error, you need to add your LAN IP as well as loopback IP to Squid's External Cache Managers via Services - Squid Proxy Server|Proxy Server - Local Cache - Squid cache general settings. You also need to specify the port if you're going to use -p:

squidclient -h 192.168.1.1 -p 3128 mgr:info

Shreeyashattal

Hi thanx for pointing that out.. but i still have no luck.. he're with the new squid version and latest stable build of pfsense

Sending HTTP request ... done.
HTTP/1.1 403 Forbidden
Server: squid/3.4.10
Mime-Version: 1.0
Date: Thu, 26 Nov 2015 19:53:32 GMT
Content-Type: text/html
Content-Length: 3096
X-Squid-Error: ERR_ACCESS_DENIED 0
Vary: Accept-Language
Content-Language: af
X-Cache: MISS from localhost
X-Cache-Lookup: NONE from localhost:3128
Via: 1.1 localhost (squid/3.4.10)
Connection: close

<title>FOUT: Die aangevraagde URL kon nie verkry word nie</title>

# FOUT

## Die aangevraagde URL kon nie verkry word nie

* * *

Die volgende fout is teëgekom tydens verkryging van die URL: [cache_object://192.168.1.1/info](cache_object://192.168.1.1/info)

> **Toegang geweier.**

Die opstelling van toegangsbeheer keer dat u navraag nou toegelaat kan word. Kontak gerus u diensverskaffer indien u voel dit is verkeerd.

Die kasbediener se administrateur is [admin@localhost](mailto:admin@localhost?subject=CacheErrorInfo%20-%20ERR_ACCESS_DENIED&body=CacheHost%3A%20localhost%0D%0AErrPage%3A%20ERR_ACCESS_DENIED%0D%0AErr%3A%20%5Bnone%5D%0D%0ATimeStamp%3A%20Thu,%2026%20Nov%202015%2019%3A53%3A32%20GMT%0D%0A%0D%0AClientIP%3A%20192.168.1.1%0D%0A%0D%0AHTTP%20Request%3A%0D%0AGET%20%2Finfo%20HTTP%2F1.0%0AHost%3A%20192.168.1.1%0D%0AUser-Agent%3A%20squidclient%2F3.4.10%0D%0AAccept%3A%20*%2F*%0D%0AConnection%3A%20close%0D%0A%0D%0A%0D%0A).

* * *

Gegenereer op Thu, 26 Nov 2015 19:53:32 GMT deur localhost (squid/3.4.10)

I added my lan ip <192.168.1.1> & loopback ip <127.0.0.1> to external cache managers as u suggested, yet no luck

1448567990.574    136 192.168.1.35 TCP_MISS/302 803 GET http://googleads.g.doubleclick.net/pagead/viewthroughconversion/953868795/? - ORIGINAL_DST/216.58.220.34 image/gif
1448567990.611     89 192.168.1.35 TCP_MISS/302 997 GET http://www.google.com/ads/user-lists/928062791/? - ORIGINAL_DST/74.125.200.103 text/html
1448567990.626     88 192.168.1.35 TCP_MISS/200 1424 GET http://cdn.chuknu.sokrati.com/global/meta-data-tracker.js - ORIGINAL_DST/54.230.174.246 application/javascript
1448567990.666    427 192.168.1.35 TCP_MISS/200 1061 GET http://sg-pl.vizury.com/analyze/analyze.php? - ORIGINAL_DST/119.81.97.114 text/html
1448567990.697     80 192.168.1.35 TCP_MISS/200 474 GET http://www.google.co.in/ads/user-lists/928062791/? - ORIGINAL_DST/74.125.200.94 image/gif
1448567990.757    195 192.168.1.35 TCP_MISS/302 997 GET http://www.google.com/ads/user-lists/968961100/? - ORIGINAL_DST/74.125.200.103 text/html
1448567990.771    202 192.168.1.35 TCP_MISS/302 997 GET http://www.google.com/ads/user-lists/954607281/? - ORIGINAL_DST/74.125.200.103 text/html
1448567990.794    213 192.168.1.35 TCP_MISS/302 995 GET http://www.google.com/ads/user-lists/953868795/? - ORIGINAL_DST/74.125.200.103 text/html
1448567991.000    326 192.168.1.35 TCP_MISS/200 2611 GET http://d3701cc9l7v9a6.cloudfront.net/js/widget/we-conversion-helper-min-v-1.0.js? - ORIGINAL_DST/54.230.174.224 text/javascript
1448567991.160    650 192.168.1.35 TCP_MISS/200 457 POST http://c.webengage.com/l3.jpg - ORIGINAL_DST/23.21.204.198 image/jpeg
1448567991.315    701 192.168.1.35 TCP_MISS/200 607 GET http://geoservice.webengage.com/geoip/? - ORIGINAL_DST/54.225.223.128 application/x-javascript
1448567991.737     80 192.168.1.35 TCP_MISS/200 474 GET http://www.google.co.in/ads/user-lists/968961100/? - ORIGINAL_DST/74.125.200.94 image/gif
1448567991.820    159 192.168.1.35 TCP_MISS/200 474 GET http://www.google.co.in/ads/user-lists/954607281/? - ORIGINAL_DST/74.125.200.94 image/gif
1448567991.827    164 192.168.1.35 TCP_MISS/200 474 GET http://www.google.co.in/ads/user-lists/953868795/? - ORIGINAL_DST/74.125.200.94 image/gif
1448567991.849     96 192.168.1.35 TCP_MISS/200 20384 GET http://d3701cc9l7v9a6.cloudfront.net/js/jquery/jquery-1.3.2.min.js - ORIGINAL_DST/54.230.174.224 text/javascript
1448567991.978     32 192.168.1.35 TCP_MISS/200 1242 GET http://d3701cc9l7v9a6.cloudfront.net/css/responsive/assets/css/font/webengage/widget-font.css? - ORIGINAL_DST/54.230.174.224 text/css
1448567992.032     69 192.168.1.35 TCP_MISS/200 2447 GET http://d3701cc9l7v9a6.cloudfront.net/css/webengage/notification/~184fc0b7-notification-base.css? - ORIGINAL_DST/54.230.174.224 text/css
1448567992.131     55 192.168.1.35 TCP_MISS/200 7449 GET http://d3701cc9l7v9a6.cloudfront.net/js/widget/we-notification-widget-v-4.1.js? - ORIGINAL_DST/54.230.174.224 text/javascript
1448567992.172    452 192.168.1.35 TCP_MISS/200 918 GET http://tracking.sokrati.com/site? - ORIGINAL_DST/176.34.224.244 image/gif
1448567992.927    601 192.168.1.35 TCP_MISS/200 2367 GET http://notification.webengage.com/json/notification.html? - ORIGINAL_DST/23.21.87.60 application/x-javascript
1448567993.064    114 192.168.1.35 TCP_MISS/200 2795 GET http://d3701cc9l7v9a6.cloudfront.net/js/widget/publisher-notification-layout-~184fc0b7-min-v-4.1.js? - ORIGINAL_DST/54.230.174.224 text/javascript
1448567993.253     39 192.168.1.35 TCP_MISS/200 4605 GET http://d3701cc9l7v9a6.cloudfront.net/css/responsive/assets/css/font/webengage/fonts6/tpi_font.woff? - ORIGINAL_DST/54.230.174.224 text/plain
1448567993.492    307 192.168.1.35 TCP_MISS/200 372 GET http://notification.webengage.com/json/notification.html? - ORIGINAL_DST/23.21.87.60 application/x-javascript
1448567993.509    310 192.168.1.35 TCP_MISS/200 140462 GET http://rtm.ebaystatic.com/203/RTMS/Image/HomeCover_26Nov20155.jpg - ORIGINAL_DST/115.112.4.5 image/jpeg
1448567993.691    137 192.168.1.35 TCP_MISS/200 43272 GET http://rtm.ebaystatic.com/203/RTMS/Image/BlackFriday_RFURB_770x270_18Nov2015.jpg - ORIGINAL_DST/115.112.4.5 image/jpeg
1448567993.771    199 192.168.1.35 TCP_MISS/200 48125 GET http://rtm.ebaystatic.com/203/RTMS/Image/BlackFriday_ML_770x270_18Nov2015_1447940668393.jpg - ORIGINAL_DST/115.112.4.5 image/jpeg
1448567993.794    609 192.168.1.35 TCP_MISS/200 457 POST http://c.webengage.com/l3.jpg - ORIGINAL_DST/23.21.204.198 image/jpeg
1448567993.870    161 192.168.1.35 TCP_MISS/200 44141 GET http://rtm.ebaystatic.com/203/RTMS/Image/770x270_Set_top_boxes_133_22june.jpg - ORIGINAL_DST/115.112.4.5 image/jpeg
1448567994.006    471 192.168.1.35 TCP_MISS/200 561 GET http://rover.ebay.in/idmap/0? - ORIGINAL_DST/66.135.216.173 text/json
1448567994.181    394 192.168.1.35 TCP_MISS/200 61246 GET http://rtm.ebaystatic.com/203/RTMS/Image/BlackFriday2_770x270_10Nov2015.jpg - ORIGINAL_DST/115.112.4.5 image/jpeg
1448567994.231    606 192.168.1.35 TCP_MISS/200 52393 GET http://rtm.ebaystatic.com/203/RTMS/Image/BlackFriday2_Camera_770x270_10Nov2015123.jpg - ORIGINAL_DST/115.112.4.5 image/jpeg
1448567994.292    404 192.168.1.35 TCP_MISS/200 33021 GET http://rtm.ebaystatic.com/203/RTMS/Image/Arya-uWear_570x270_24Nov2015.jpg - ORIGINAL_DST/115.112.4.5 image/jpeg
1448567994.387    775 192.168.1.35 TCP_MISS/200 47489 GET http://rtm.ebaystatic.com/203/RTMS/Image/cleanHome_770x270_14nov2015.jpg - ORIGINAL_DST/115.112.4.5 image/jpeg
1448567994.496    913 192.168.1.35 TCP_MISS/200 67989 GET http://rtm.ebaystatic.com/203/RTMS/Image/BlackFriday2_Tablets_770x270_10Nov201511.jpg - ORIGINAL_DST/115.112.4.5 image/jpeg
1448567994.688   1090 192.168.1.35 TCP_MISS/200 84858 GET http://rtm.ebaystatic.com/203/RTMS/Image/WeddingCampain_770x270_17Nov2015.jpg - ORIGINAL_DST/115.112.4.5 image/jpeg
1448567996.848    284 192.168.1.35 TCP_MISS/200 1432 GET http://srx.in.ebayrtm.com/rtm? - ORIGINAL_DST/66.211.180.42 image/gif
1448567997.266    738 192.168.1.35 TCP_MISS/200 25000 GET http://www.ebay.in/ - ORIGINAL_DST/23.35.71.55 text/html
1448568000.835    284 192.168.1.35 TCP_MISS/200 1353 GET http://srx.in.ebayrtm.com/rtm? - ORIGINAL_DST/66.211.180.42 image/gif
1448568004.839    279 192.168.1.35 TCP_MISS/200 1366 GET http://srx.in.ebayrtm.com/rtm? - ORIGINAL_DST/66.211.180.42 image/gif
1448568008.834    281 192.168.1.35 TCP_MISS/200 1376 GET http://srx.in.ebayrtm.com/rtm? - ORIGINAL_DST/66.211.180.42 image/gif
1448568037.757     88 192.168.1.35 TCP_MISS/200 2455 GET http://indices.moneycontrol.co.in/sensex_nifty/radtkr.json? - ORIGINAL_DST/115.112.3.12 text/plain

KOM

I can't help you with the non-caching content, but here is a screen of how I have my ext cache mgr setting configured. It seems to work for me.

cachemgr.png_thumb

CLI_output.png_thumb

Shreeyashattal

thanks a lot.. I will try figure it out else just do a clean install. I figure it will take me less time to set it up again rather than to figure this out.

Shreeyashattal

Ok, I just clean installed pfsense latest (stable) verified everything was working. Didn't do any changes, as stock as it can be. ONLY installed squid3 0.4.4 from Packages. Configured it, restarted pfsense.

It still won't cache. All i receive are "tcp_miss/200"

I have confirmed cache directory and all the folder have been created.

Any suggestions? I've strictly followed the wiki https://doc.pfsense.org/index.php/Setup_Squid_as_a_Transparent_Proxy

I'm running pfsense on a intel 1007u 1.5Ghz with 6gb RAM & a 80gb SATA hdd.

Could anybody please help? I built this system only to use squid, broadband is highly capped and expensive where I am

Shreeyashattal

I did a clean install of pfsense, downgraded to 2.2.3 and now it seems to work. Lightsquid is showing 2% hits. Although the hit rate isnt impressive, but atleast it's working. I'd suggest everyone having this problem (and there are a lot of unresolved out there) two check few things..

1. Check if your cache directories are generated properly, if yes.. Ignore TCP_MISS/200, rather focus on improving your hit rates
2. Use lightsquid to check hit rates, a lot of times all we see in the log is MISS, lightsquid will complie that easier for you
3. DO NOT enable dynamic cache, it will break your config
4. use```
cat /var/squid/logs/access.log | grep HIT


Big thanks to KOM for helping out.

anybody has pointers on how to improve HIT rating?

JonathanLee

@KOM like this??
Screenshot 2024-03-17 at 21.43.53.png

KOM

@JonathanLee My original post was from 8 years ago and I don't use squid to cache. Haven't for many years now. I only use it as a base for squidguard.

JonathanLee

@KOM thanks for the reply, I love this program, again not many people play around with storeID so I think I have replied to some older posts on it. The caching part of squid is amazing. It’s the $5 or static parts of the text files for the program that lack information on what they do. There is really no explanation on why the database files use that.