Major slowdown through squid



  • Just in the past couple of days we have seen a major drop in throughput while using squid.  We used to get nearly our full bandwidth (3mb/s) both directions, but now we are seeing more in the range of 300 kbps or less downstream.  The upstream has not suffered as much, usually around 1.4 mbps.  Our bandwidth is fine if you bypass pfsense, fine if you bypass squid, and fine via other protocols in pfsense.  I do not know what the problem could be.

    Just checked our cache.log and found these lines at startup…
    2007/12/18 11:25:17| WARNING: '10.21.1.0/255.255.255.0' is a subnetwork of '0.0.0.0/0.0.0.0'
    2007/12/18 11:25:17| WARNING: because of this '0.0.0.0/0.0.0.0' is ignored to keep splay tree searching predictable
    2007/12/18 11:25:17| WARNING: You should probably remove '10.21.1.0/255.255.255.0' from the ACL named 'localnet'

    All our local users are on 10.21.1.*

    Also, I just checked our pkg_info - we have multiple version of several packages, see below.  Does this create an issue or do different pfsense packages require different versions of perl etc.?

    freetype2-2.2.1_1  A free and portable TrueType font rendering engine
    freetype2-2.2.1_2  A free and portable TrueType font rendering engine
    perl-5.8.8          Practical Extraction and Report Language
    perl-5.8.8_1        Practical Extraction and Report Language
    pkg-config-0.21    A utility to retrieve information about installed libraries
    pkg-config-0.22    A utility to retrieve information about installed libraries
    png-1.2.14          Library for manipulating PNG images
    png-1.2.22          Library for manipulating PNG images

    Any help would be greatly appreciated as this is our production machine…



  • This is beginning to worry me.  I have just done a fresh install and I am seeing the same results.  As soon as I disable squid I am back up to full speed.



  • I have now tried this with different hardware.  Could someone please confirm this. 
    Clean install - change IPs - add DNS servers - install squid package - set browser to use proxy or use transparent checkbox on squid setup page.  When I follow these procedures I am seeing dramatically reduced bandwidth when using squid, bypassing squid the speed is just fine.



  • There is now a bounty to fix this issue.  Suggestions so far have been related to write caching, though that did not seem to fix the problem.  Could squid be starting twice?  I've posted my cache.log on the bounty page as throughput to the pfSense box and HDs is fine.
    http://forum.pfsense.org/index.php/topic,7281.0.html

    Thanks everyone.



  • I'm able to max out my 20 Mb/s bandwidth through Squid on a 1 GHz box with Realtek NICs (FX5620 platform).  As such I'm not able to confirm your experiences.



  • Any chance you would be able to setup another box with a recent snapshot and latest squid package?  We too have a box with 1.2rc3 built on 11/24/07 that is able to exhaust our lowly 3mb bandwidth.  The issue was introduced not too long ago, as recent installs now show this symptom.  We had to revert to an old install as a temporary workaround.  Heiko was able to duplicate the issue.  I really appreciate your comments and any testing you can do related to this.



  • I am running the latest published Squid package (2.6.5_1-p15), though I'm still on 1.2-beta2.  I don't have spare hardware to set up another box (this is at home) and I'm not planning on upgrading the platform until 1.2 goes final, sorry.



  • I'm seeing similar results as I just installed 1.2rc3 from the mirrors, not the snapshot and the problem went away.  There must be an issue with something in the pfsense code.



  • To Fix Squid

    add this to the /boot/loader.conf

    kern.ipc.nmbclusters=32768
    kern.maxfiles=65536
    kern.maxfilesperproc=32768
    net.inet.ip.portrange.last=65535

    or just delete it and replace with

    autoboot_delay="1"
    #kern.ipc.nmbclusters="0"
    hint.apic.0.disabled=1
    kern.hz=100
    #for squid
    kern.ipc.nmbclusters="32768"
    kern.maxfiles="65536"
    kern.maxfilesperproc="32768"
    net.inet.ip.portrange.last="65535"

    you might ask why squid is so slow? its because default configuration of pfsense is router not as a server
    thats why kern.ipc.nmbclusters="0" <- is set to zero. if you just simply remove this squid will be just fine.

    but to tune the squid i add this
    kern.ipc.nmbclusters: 32768
    kern.maxfiles=65536
    kern.maxfilesperproc=32768
    net.inet.ip.portrange.last: 65535

    i just figure out why squid is slow. but i don't like the binary package of squid. i'll be using the squid HEAD bec of the store_rewrite feature for caching youtubes videos and other video files and mp3.


Log in to reply