Waiting for Proxy Tunnel…
-
I am running pfsense 2.3.4-RELEASE and Squid version 0.4.37. Recently there was an upgrade to squid which I did to get it to the current version but I am having latency problems after the upgrade with webpages loading slow now. In google chrome I notice at the bottom that it says "Waiting for Proxy Tunnel…" and just hangs there for 30 - 60 seconds before it finally loads the website. Then there are times that the website will just load normally with no wait time. I am not sure where I am getting the delay from and I am not sure howto diagnose it. If I turn off the proxy server there is no delay which I expected. Any suggestions would be greatly appreciated.
[2.3.4-RELEASE][root@_]/root: iostat -x 3
device r/s w/s kr/s kw/s qlen svc_t %b
md0 0.0 0.0 0.0 0.0 0 0.0 0
ada0 0.0 1.7 0.0 3.5 0 4.4 1
ada1 0.0 1.3 0.0 18.7 0 0.6 0
ada2 0.0 1.7 0.0 3.5 0 3.9 1
pass0 0.0 0.0 0.0 0.0 0 0.0 0
pass1 0.0 0.0 0.0 0.0 0 0.0 0
pass2 0.0 0.0 0.0 0.0 0 0.0 04:02PM up 6 days, 3:07, 2 users, load averages: 0.15, 0.40, 0.37
-
Go to console and run this:
squidclient -h localhost mgr:infoCheck the Median Service Times section for any large numbers. Here is mine as an example:
Median Service Times (seconds) 5 min 60 min:
HTTP Requests (All): 0.04277 0.05331
Cache Misses: 0.04277 0.05331
Cache Hits: 0.00000 0.00000
Near Hits: 0.00000 0.00091
Not-Modified Replies: 0.00000 0.00000
DNS Lookups: 0.03696 0.02683
ICP Queries: 0.00000 0.00000Note that my numbers may look weird because I only use squid as a platform for squidguard. I don't actually do any caching.
-
So I am assuming that this is a permission error and I need to grant myself permission?
[2.3.4-RELEASE][root@]/root: squidclient -h localhost mgr:info
ERROR: Cannot connect to [::1]:3128 -
So I got it to work. Had to specify 127.0.0.1 for some reason.
[2.3.4-RELEASE][root@]/usr/local/etc/squid: squidclient -h 127.0.0.1 mgr:info
HTTP/1.1 200 OK
Server: squid
Mime-Version: 1.0
Date: Wed, 28 Jun 2017 19:58:56 GMT
Content-Type: text/plain;charset=utf-8
Expires: Wed, 28 Jun 2017 19:58:56 GMT
Last-Modified: Wed, 28 Jun 2017 19:58:56 GMT
X-Cache: MISS from ***X-Cache-Lookup: MISS from ****:3128
Via: 1.1 ****(squid)
Connection: closeSquid Object Cache: Version 3.5.26
Build Info:
Service Name: squid
Start Time: Tue, 27 Jun 2017 02:49:28 GMT
Current Time: Wed, 28 Jun 2017 19:58:56 GMT
Connection information for squid:
Number of clients accessing cache: 71
Number of HTTP requests received: 212543
Number of ICP messages received: 0
Number of ICP messages sent: 0
Number of queued ICP replies: 0
Number of HTCP messages received: 0
Number of HTCP messages sent: 0
Request failure ratio: 0.00
Average HTTP requests per minute since start: 86.1
Average ICP messages per minute since start: 0.0
Select loop called: 23528130 times, 6.298 ms avg
Cache information for squid:
Hits as % of all requests: 5min: 6.5%, 60min: 3.3%
Hits as % of bytes sent: 5min: 1.8%, 60min: 0.6%
Memory hits as % of hit requests: 5min: 55.0%, 60min: 44.4%
Disk hits as % of hit requests: 5min: 37.5%, 60min: 9.5%
Storage Swap size: 6031788 KB
Storage Swap capacity: 5.9% used, 94.1% free
Storage Mem size: 2401200 KB
Storage Mem capacity: 93.8% used, 6.2% free
Mean Object Size: 49.42 KB
Requests given to unlinkd: 0
Median Service Times (seconds) 5 min 60 min:
HTTP Requests (All): 0.30459 0.19742
Cache Misses: 0.18699 0.18699
Cache Hits: 0.01556 0.01235
Near Hits: 0.12106 0.16775
Not-Modified Replies: 0.00000 0.01387
DNS Lookups: 0.07618 0.06657
ICP Queries: 0.00000 0.00000
Resource usage for squid:
UP Time: 148168.467 seconds
CPU Time: 2363.238 seconds
CPU Usage: 1.59%
CPU Usage, 5 minute avg: 1.47%
CPU Usage, 60 minute avg: 1.89%
Maximum Resident Size: 29699280 KB
Page faults with physical i/o: 8
Memory accounted for:
Total accounted: 128446 KB
memPoolAlloc calls: 130050837
memPoolFree calls: 132943492
File descriptor usage for squid:
Maximum number of file descriptors: 470817
Largest file desc currently in use: 222
Number of file desc currently in use: 138
Files queued for open: 0
Available number of file descriptors: 470679
Reserved number of file descriptors: 100
Store Disk files open: 0
Internal Data Structures:
122110 StoreEntries
65675 StoreEntries with MemObjects
65674 Hot Object Cache Items
122058 on-disk objects -
I too suffer from this issue. Most of the time pages take on average around 15 seconds to start loading into the browser, the "waiting for proxy tunnel" message in chrome is present the entire time. Once it appears as if the page has loaded the "waiting for proxy tunnel" message stays around and the page appears to continue loading for at least another 45 seconds as some images have yet to appear.
I recently rebooted my pfSense box within the last half hour so there isn't much data for the statistics, however, it appears as if squid is not doing very much caching?
(I too had to specify 127.0.0.1 instead of localhost)
HTTP/1.1 200 OK Server: squid Mime-Version: 1.0 Date: Wed, 25 Oct 2017 17:56:33 GMT Content-Type: text/plain;charset=utf-8 Expires: Wed, 25 Oct 2017 17:56:33 GMT Last-Modified: Wed, 25 Oct 2017 17:56:33 GMT X-Cache: MISS from localhost X-Cache-Lookup: MISS from localhost:3128 Via: 1.1 localhost (squid) Connection: close Squid Object Cache: Version 3.5.27 Build Info: Service Name: squid Start Time: Wed, 25 Oct 2017 17:41:27 GMT Current Time: Wed, 25 Oct 2017 17:56:33 GMT Connection information for squid: Number of clients accessing cache: 2 Number of HTTP requests received: 383 Number of ICP messages received: 0 Number of ICP messages sent: 0 Number of queued ICP replies: 0 Number of HTCP messages received: 0 Number of HTCP messages sent: 0 Request failure ratio: 0.00 Average HTTP requests per minute since start: 25.3 Average ICP messages per minute since start: 0.0 Select loop called: 122172 times, 7.420 ms avg Cache information for squid: Hits as % of all requests: 5min: 0.0%, 60min: 0.0% Hits as % of bytes sent: 5min: 0.0%, 60min: 1.5% Memory hits as % of hit requests: 5min: 0.0%, 60min: 0.0% Disk hits as % of hit requests: 5min: 0.0%, 60min: 0.0% Storage Swap size: 2540 KB Storage Swap capacity: 0.0% used, 100.0% free Storage Mem size: 216 KB Storage Mem capacity: 0.0% used, 100.0% free Mean Object Size: 22.28 KB Requests given to unlinkd: 0 Median Service Times (seconds) 5 min 60 min: HTTP Requests (All): 57.44813 0.39928 Cache Misses: 0.00000 0.22004 Cache Hits: 0.00000 0.00000 Near Hits: 0.00000 0.00000 Not-Modified Replies: 0.00000 0.00000 DNS Lookups: 0.04237 0.04048 ICP Queries: 0.00000 0.00000 Resource usage for squid: UP Time: 906.534 seconds CPU Time: 4.662 seconds CPU Usage: 0.51% CPU Usage, 5 minute avg: 0.56% CPU Usage, 60 minute avg: 0.51% Maximum Resident Size: 369744 KB Page faults with physical i/o: 11 Memory accounted for: Total accounted: 3375 KB memPoolAlloc calls: 162269 memPoolFree calls: 165217 File descriptor usage for squid: Maximum number of file descriptors: 469782 Largest file desc currently in use: 120 Number of file desc currently in use: 81 Files queued for open: 0 Available number of file descriptors: 469701 Reserved number of file descriptors: 100 Store Disk files open: 0 Internal Data Structures: 166 StoreEntries 52 StoreEntries with MemObjects 51 Hot Object Cache Items 114 on-disk objectsThe way I have this setup it should be caching as much as possible?
-
Your cache settings are massive. With large cache structures, you can lose time while it scans the massive tree for cached content hits. An 80G cache with thousands of subfolders is excessive IMO. Also, caching the dynamic web is quite difficult these days. Then there's the fact that you can't cache HTTPS without MitM, and you start to realize that caching isn't very effective anymore.
If you REALLY want to try, start by reading up on dynamic content and refresh_patterns.
http://www.squid-cache.org/Versions/v3/3.5/cfgman/refresh_pattern.html
-
I don't bother caching dynamic content, youtube, microsoft updates, etc.. seemed pointless. I'll definitely give that a read to better understand what all these settings really do. Thank you kindly!
In the meantime I've turned down the Hard Disk Cache Size to 32768 to use 32 gb, and the Level 1 Directories to 32 for 1024 folders, and cleared the cache.
I'm using a samsung SSD on a SATA3 port, assumed the SSD speed would mitigate any issues with so many level 1 directories and the large cache size.. You know what they say about assuming though ;)
-
I don't bother caching dynamic content, youtube, microsoft updates, etc.. seemed pointless.
Most of the web is dynamic these days. I stopped caching two years ao when I realized that my hit rate was in the area of 3-7%, and it wasn't worth the hassle.
