Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Waiting for Proxy Tunnel…

    Scheduled Pinned Locked Moved Cache/Proxy
    8 Posts 3 Posters 5.5k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • H
      haaser
      last edited by

      I am running pfsense 2.3.4-RELEASE and Squid version 0.4.37. Recently there was an upgrade to squid which I did to get it to the current version but I am having latency problems after the upgrade with webpages loading slow now. In google chrome I notice at the bottom that it says "Waiting for Proxy Tunnel…" and just hangs there for 30 - 60 seconds before it finally loads the website. Then there are times that the website will just load normally with no wait time. I am not sure where I am getting the delay from and I am not sure howto diagnose it. If I turn off the proxy server there is no delay which I expected. Any suggestions would be greatly appreciated.

      [2.3.4-RELEASE][root@_]/root: iostat -x 3
      device    r/s  w/s    kr/s    kw/s qlen svc_t  %b
      md0        0.0  0.0    0.0    0.0    0  0.0  0
      ada0      0.0  1.7    0.0    3.5    0  4.4  1
      ada1      0.0  1.3    0.0    18.7    0  0.6  0
      ada2      0.0  1.7    0.0    3.5    0  3.9  1
      pass0      0.0  0.0    0.0    0.0    0  0.0  0
      pass1      0.0  0.0    0.0    0.0    0  0.0  0
      pass2      0.0  0.0    0.0    0.0    0  0.0  0

      4:02PM  up 6 days,  3:07, 2 users, load averages: 0.15, 0.40, 0.37

      1 Reply Last reply Reply Quote 0
      • KOMK
        KOM
        last edited by

        Go to console and run this:

        squidclient -h localhost mgr:info
        

        Check the Median Service Times section for any large numbers.  Here is mine as an example:

        Median Service Times (seconds)  5 min    60 min:
                HTTP Requests (All):  0.04277  0.05331
                Cache Misses:          0.04277  0.05331
                Cache Hits:            0.00000  0.00000
                Near Hits:            0.00000  0.00091
                Not-Modified Replies:  0.00000  0.00000
                DNS Lookups:          0.03696  0.02683
                ICP Queries:          0.00000  0.00000

        Note that my numbers may look weird because I only use squid as a platform for squidguard.  I don't actually do any caching.

        1 Reply Last reply Reply Quote 0
        • H
          haaser
          last edited by

          So I am assuming that this is a permission error and I need to grant myself permission?

          [2.3.4-RELEASE][root@]/root: squidclient -h localhost mgr:info
          ERROR: Cannot connect to [::1]:3128

          1 Reply Last reply Reply Quote 0
          • H
            haaser
            last edited by

            So I got it to work. Had to specify 127.0.0.1 for some reason.

            [2.3.4-RELEASE][root@]/usr/local/etc/squid: squidclient -h 127.0.0.1 mgr:info
            HTTP/1.1 200 OK
            Server: squid
            Mime-Version: 1.0
            Date: Wed, 28 Jun 2017 19:58:56 GMT
            Content-Type: text/plain;charset=utf-8
            Expires: Wed, 28 Jun 2017 19:58:56 GMT
            Last-Modified: Wed, 28 Jun 2017 19:58:56 GMT
            X-Cache: MISS from ***X-Cache-Lookup: MISS from ****:3128
            Via: 1.1 ****(squid)
            Connection: close

            Squid Object Cache: Version 3.5.26
            Build Info:
            Service Name: squid
            Start Time:    Tue, 27 Jun 2017 02:49:28 GMT
            Current Time:  Wed, 28 Jun 2017 19:58:56 GMT
            Connection information for squid:
                    Number of clients accessing cache:      71
                    Number of HTTP requests received:      212543
                    Number of ICP messages received:        0
                    Number of ICP messages sent:    0
                    Number of queued ICP replies:  0
                    Number of HTCP messages received:      0
                    Number of HTCP messages sent:  0
                    Request failure ratio:  0.00
                    Average HTTP requests per minute since start:  86.1
                    Average ICP messages per minute since start:    0.0
                    Select loop called: 23528130 times, 6.298 ms avg
            Cache information for squid:
                    Hits as % of all requests:      5min: 6.5%, 60min: 3.3%
                    Hits as % of bytes sent:        5min: 1.8%, 60min: 0.6%
                    Memory hits as % of hit requests:      5min: 55.0%, 60min: 44.4%
                    Disk hits as % of hit requests: 5min: 37.5%, 60min: 9.5%
                    Storage Swap size:      6031788 KB
                    Storage Swap capacity:  5.9% used, 94.1% free
                    Storage Mem size:      2401200 KB
                    Storage Mem capacity:  93.8% used,  6.2% free
                    Mean Object Size:      49.42 KB
                    Requests given to unlinkd:      0
            Median Service Times (seconds)  5 min    60 min:
                    HTTP Requests (All):  0.30459  0.19742
                    Cache Misses:          0.18699  0.18699
                    Cache Hits:            0.01556  0.01235
                    Near Hits:            0.12106  0.16775
                    Not-Modified Replies:  0.00000  0.01387
                    DNS Lookups:          0.07618  0.06657
                    ICP Queries:          0.00000  0.00000
            Resource usage for squid:
                    UP Time:        148168.467 seconds
                    CPU Time:      2363.238 seconds
                    CPU Usage:      1.59%
                    CPU Usage, 5 minute avg:        1.47%
                    CPU Usage, 60 minute avg:      1.89%
                    Maximum Resident Size: 29699280 KB
                    Page faults with physical i/o: 8
            Memory accounted for:
                    Total accounted:      128446 KB
                    memPoolAlloc calls: 130050837
                    memPoolFree calls:  132943492
            File descriptor usage for squid:
                    Maximum number of file descriptors:  470817
                    Largest file desc currently in use:    222
                    Number of file desc currently in use:  138
                    Files queued for open:                  0
                    Available number of file descriptors: 470679
                    Reserved number of file descriptors:  100
                    Store Disk files open:                  0
            Internal Data Structures:
                    122110 StoreEntries
                    65675 StoreEntries with MemObjects
                    65674 Hot Object Cache Items
                    122058 on-disk objects

            1 Reply Last reply Reply Quote 0
            • kklouzalK
              kklouzal
              last edited by

              I too suffer from this issue. Most of the time pages take on average around 15 seconds to start loading into the browser, the "waiting for proxy tunnel" message in chrome is present the entire time. Once it appears as if the page has loaded the "waiting for proxy tunnel" message stays around and the page appears to continue loading for at least another 45 seconds as some images have yet to appear.

              I recently rebooted my pfSense box within the last half hour so there isn't much data for the statistics, however, it appears as if squid is not doing very much caching?

              (I too had to specify 127.0.0.1 instead of localhost)

              HTTP/1.1 200 OK
              Server: squid
              Mime-Version: 1.0
              Date: Wed, 25 Oct 2017 17:56:33 GMT
              Content-Type: text/plain;charset=utf-8
              Expires: Wed, 25 Oct 2017 17:56:33 GMT
              Last-Modified: Wed, 25 Oct 2017 17:56:33 GMT
              X-Cache: MISS from localhost
              X-Cache-Lookup: MISS from localhost:3128
              Via: 1.1 localhost (squid)
              Connection: close
              
              Squid Object Cache: Version 3.5.27
              Build Info:
              Service Name: squid
              Start Time:	Wed, 25 Oct 2017 17:41:27 GMT
              Current Time:	Wed, 25 Oct 2017 17:56:33 GMT
              Connection information for squid:
              	Number of clients accessing cache:	2
              	Number of HTTP requests received:	383
              	Number of ICP messages received:	0
              	Number of ICP messages sent:	0
              	Number of queued ICP replies:	0
              	Number of HTCP messages received:	0
              	Number of HTCP messages sent:	0
              	Request failure ratio:	 0.00
              	Average HTTP requests per minute since start:	25.3
              	Average ICP messages per minute since start:	0.0
              	Select loop called: 122172 times, 7.420 ms avg
              Cache information for squid:
              	Hits as % of all requests:	5min: 0.0%, 60min: 0.0%
              	Hits as % of bytes sent:	5min: 0.0%, 60min: 1.5%
              	Memory hits as % of hit requests:	5min: 0.0%, 60min: 0.0%
              	Disk hits as % of hit requests:	5min: 0.0%, 60min: 0.0%
              	Storage Swap size:	2540 KB
              	Storage Swap capacity:	 0.0% used, 100.0% free
              	Storage Mem size:	216 KB
              	Storage Mem capacity:	 0.0% used, 100.0% free
              	Mean Object Size:	22.28 KB
              	Requests given to unlinkd:	0
              Median Service Times (seconds)  5 min    60 min:
              	HTTP Requests (All):  57.44813  0.39928
              	Cache Misses:          0.00000  0.22004
              	Cache Hits:            0.00000  0.00000
              	Near Hits:             0.00000  0.00000
              	Not-Modified Replies:  0.00000  0.00000
              	DNS Lookups:           0.04237  0.04048
              	ICP Queries:           0.00000  0.00000
              Resource usage for squid:
              	UP Time:	906.534 seconds
              	CPU Time:	4.662 seconds
              	CPU Usage:	0.51%
              	CPU Usage, 5 minute avg:	0.56%
              	CPU Usage, 60 minute avg:	0.51%
              	Maximum Resident Size: 369744 KB
              	Page faults with physical i/o: 11
              Memory accounted for:
              	Total accounted:         3375 KB
              	memPoolAlloc calls:    162269
              	memPoolFree calls:     165217
              File descriptor usage for squid:
              	Maximum number of file descriptors:   469782
              	Largest file desc currently in use:    120
              	Number of file desc currently in use:   81
              	Files queued for open:                   0
              	Available number of file descriptors: 469701
              	Reserved number of file descriptors:   100
              	Store Disk files open:                   0
              Internal Data Structures:
              	   166 StoreEntries
              	    52 StoreEntries with MemObjects
              	    51 Hot Object Cache Items
              	   114 on-disk objects
              

              The way I have this setup it should be caching as much as possible?

              1 Reply Last reply Reply Quote 0
              • KOMK
                KOM
                last edited by

                Your cache settings are massive.  With large cache structures, you can lose time while it scans the massive tree for cached content hits.  An 80G cache with thousands of subfolders is excessive IMO.  Also, caching the dynamic web is quite difficult these days.  Then there's the fact that you can't cache HTTPS without MitM, and you start to realize that caching isn't very effective anymore.

                If you REALLY want to try, start by reading up on dynamic content and refresh_patterns.

                http://www.squid-cache.org/Versions/v3/3.5/cfgman/refresh_pattern.html

                1 Reply Last reply Reply Quote 0
                • kklouzalK
                  kklouzal
                  last edited by

                  I don't bother caching dynamic content, youtube, microsoft updates, etc.. seemed pointless. I'll definitely give that a read to better understand what all these settings really do. Thank you kindly!

                  In the meantime I've turned down the Hard Disk Cache Size to 32768 to use 32 gb, and the Level 1 Directories to 32 for 1024 folders, and cleared the cache.

                  I'm using a samsung SSD on a SATA3 port, assumed the SSD speed would mitigate any issues with so many level 1 directories and the large cache size.. You know what they say about assuming though ;)

                  1 Reply Last reply Reply Quote 0
                  • KOMK
                    KOM
                    last edited by

                    I don't bother caching dynamic content, youtube, microsoft updates, etc.. seemed pointless.

                    Most of the web is dynamic these days.  I stopped caching two years ao when I realized that my hit rate was in the area of 3-7%, and it wasn't worth the hassle.

                    1 Reply Last reply Reply Quote 0
                    • First post
                      Last post
                    Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.