Navigation

    Netgate Discussion Forum
    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search

    Squid3 recently very slow

    Cache/Proxy
    2
    13
    3790
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • L
      lucky last edited by

      So I recently updated to the squid 3.4.10_2 pkg 0.2.7 package, and I'm not sure what happened, but it seems to correlate with some really slow/bad web access.

      If I disable squid, all is well. If not, I frequently get websites that load slowly, or even time out (Squid actually reports a timeout connecting to the IP). A refresh usually fixes the problem, but not always. This is happening in all browsers (Chrome/Firefox/IE) and on multiple computers, both wired and wireless. So I am pretty sure the issue lies in the pfSense system (hw or sw). Since nothing else on the system seems to be slow, I am currently ruling out hw. The main config option I use is the transparent proxy setting. Pretty much everything else is default.

      I've removed and re-installed the package. I removed the package, removed every file or directory with "squid" in the name and reinstalled squid3. No go. I've changed some config settings. No go. I'm currently trying to figure out how to get back to original/default settings, since current settings are preserved in the main pfSense config and are not stored with the package.

      I am not using any other Squid-related packages, all other packages I have installed are iperf, mtr-nox11, OpenVPN Client Export Utility, and Service Watchdog.

      Falling back to the "squid" vs "squid3" package restores speedy web access, but does not work with many websites.

      I'm not sure what I can do to either a) troubleshoot (already checked logs, nothing obvious) or b) provide devs with useful info. If anyone has ideas or suggestions, please let me know.

      Thanks

      1 Reply Last reply Reply Quote 0
      • KOM
        KOM last edited by

        Get squid3 working again and then shell in and check the squid client manager:

        squidclient -h pfsense_ip -p 3128 mgr:info

        Look for the Median Service Times section, and check to see that nothing is totally out of whack.

        Next, do a

        tail -f /var/log/squid/access.log

        while browsing and see what's happening in realtime.

        1 Reply Last reply Reply Quote 0
        • L
          lucky last edited by

          Okay, so I went ahead and checked out those things. The Median Service Times were as follows:

          Median Service Times (seconds)  5 min    60 min:
                  HTTP Requests (All):   0.10857  0.02899
                  Cache Misses:          0.10857  0.03829
                  Cache Hits:            0.00000  0.00000
                  Near Hits:             0.00000  0.00000
                  Not-Modified Replies:  0.00000  0.00000
                  DNS Lookups:           0.01210  0.01046
                  ICP Queries:           0.00000  0.00000
          
          

          Which looks fine to me. I didn't do many requests, but I did enough so I got some that went through okay, some that were slow, and some that failed. I investigated one that failed, and this is what I found in the squid access.log:

          1429108291.521  60287 192.168.200.104 TCP_MISS/503 4502 GET http://here.com/traffic/usa/washington-dc - ORIGINAL_DST/66.54.66.154 text/html
          1429108293.842     59 192.168.200.104 TCP_MISS/301 785 GET http://here.com/favicon.ico - ORIGINAL_DST/66.54.66.154 text/html
          
          

          In the browser, Squid returned an error indicating that there was a timeout contacting the site. Any time this happens, if I just refresh it usually loads just fine. The sites are not down, since at the same time I can access them just fine from other computers or browsers not configured to use Squid.

          1 Reply Last reply Reply Quote 0
          • L
            lucky last edited by

            Also, I already had the option mentioned here https://forum.pfsense.org/index.php?topic=52735.msg284810#msg284810 turned on. Doesn't seem to make a difference.

            1 Reply Last reply Reply Quote 0
            • KOM
              KOM last edited by

              No hits at all.  I wonder if your cache folder hierarchy needs to be rebuilt?

              IIRC, there are some issues with squid3 in transparent mode at the moment:

              https://forum.pfsense.org/index.php?topic=91894.0

              https://forum.pfsense.org/index.php?topic=89315.0

              You might be hitting those problems as well.  I've spent some time studying squid3, squidguard, sarg and lightsquid.  I've gotten everything running on a Ubuntu Server 14.10 box.  I will be installing a standalone proxy once Ubuntu Server 15.04 comes out next week.  I've come to the conclusion that it's best to separate extra services from the basic routing firewall, so bye-bye to all packages except reporting, like bandwidthd.

              1 Reply Last reply Reply Quote 0
              • L
                lucky last edited by

                Thanks for pointing out those links. I did have transparent mode on. I just turned it off and manually configured a browser to use the proxy. Also, yesterday I deleted the entire Squid cache folder structure via shell on pfSense. Still seem to have the same problem.

                1 Reply Last reply Reply Quote 0
                • KOM
                  KOM last edited by

                  After you deleted it, I assume you rebuilt it with squid3 -z?

                  1 Reply Last reply Reply Quote 0
                  • L
                    lucky last edited by

                    I didn't, though after the delete, I removed the entire squid3 package and re-installed it…which I assume will do the rebuild?

                    1 Reply Last reply Reply Quote 0
                    • KOM
                      KOM last edited by

                      Should, but you should do it just to be sure.

                      1 Reply Last reply Reply Quote 0
                      • L
                        lucky last edited by

                        Okay, I stopped Squid, ran this:

                        [2.2.1-RELEASE][root@fw]/root: /usr/local/sbin/squid -z
                        [2.2.1-RELEASE][root@fw]/root: 2015/04/15 16:28:15 kid1| Creating missing swap directories
                        
                        

                        And restarted Squid. Still getting the same bad performance.

                        1 Reply Last reply Reply Quote 0
                        • KOM
                          KOM last edited by

                          I don't know what else to tell you.

                          1 Reply Last reply Reply Quote 0
                          • L
                            lucky last edited by

                            Heh, np…I think my next experiment will be to set myself up to get some pcaps, on the client and on the server (both internal and WAN), to see what's happening on the network.

                            1 Reply Last reply Reply Quote 0
                            • L
                              lucky last edited by

                              So I am still not sure exactly what the heck is going on. In some cases, it does appear that SYNs are not being responded to. I am not sure why. Then shortly after, it works…???

                              I added the following to my Squid config, on the General tab in the "Custom ACLS (Before_Auth)" section, and this is helping a lot...though still not good enough for "production":

                              connect_timeout 2
                              forward_max_tries 2
                              connect_retries 2
                              
                              1 Reply Last reply Reply Quote 0
                              • First post
                                Last post