PfSense 2.2.3 - Internet is very slow via Squid3



  • Hello all,

    I had several crash issues after upgrading to pfsense 2.2.3 from 2.2.2. I was able to resolve from of the packages issues by reinstalling and reboot. However the internet via squid3 was not working. Though my setting were still the same. Nothing changed but the pfsense upgrade

    I decided to do a fresh install of pfsense 2.2.3. Afterwards i installed Squid3, SquidGuard, LightSquid, Sarg, Snort, VHosts, Cron, bandwidthD. ntopng

    I had several package install time-outs. i.e. snort, ntopng but was able to get them to fully install after several retries. I had no issues installing Squid3 and SquidGuard.

    As of now i currently have my settings for both Squid3 and SquidGuard set as i had them in pfsense 2.2.2 but the internet is very slow. DNS is set to internal dns, google and opendns in Squid3.

    Is anyone else having this issue? Any ideas of what might be causing it?

    FYI: I am only using pfsense as a web content filter via squid3, squidGuard and snort. Firewall filtering is currently turned off along with WAN disabled. DMZ and LAN are bridged.



  • fterwards i installed Squid3, SquidGuard, LightSquid, Sarg, Snort, VHosts, Cron,bandwidthD. ntopng

    none of the above are known for their stability and reliability (especially, short after a release).

    I would start with one package at a time and see what the culprit is.

    most likely culprits: LightSquid, Sarg, SquidGuard, VHosts, bandwidthD



  • Focus on squid3 and snort.  I can't possibly imagine how a log analyzer like Lightsquid or Sarg would slow down his web browsing since they only run ever now and then.



  • I have removed all of the packages listed and tried installing squid 2 but i'm still experiencing the same lag.

    Is it possible that it could be a disk read error issue?


  • Rebel Alliance Developer Netgate

    Some have found that the disk changes we made for sync mode have slowed down squid by default.

    If you want to risk the possibility of disk corruption but gain speed, edit the ,sync out of /etc/fstab for the root slice and/or run

    mount -o nosync /
    


  • Squid2 is crap and very old.  Stick to squid3.



  • @jimp:

    Some have found that the disk changes we made for sync mode have slowed down squid by default.

    If you want to risk the possibility of disk corruption but gain speed, edit the ,sync out of /etc/fstab for the root slice and/or run

    mount -o nosync /
    

    I forgot to mention that the pfsense install is on a RAID 1

    but that command seems to have did it. HTTP requests seems to be more responsive! THANK YOU SIR! ;-)



  • How do i make this mount option stick on fstab? After reboot the mount -o nosync option goes away.


  • Banned

    @gdsnytech:

    How do i make this mount option stick on fstab? After reboot the mount -o nosync option goes away.

    Would help to re-read the post quoted. It's already written there.



  • That is what i did. For some reason it wasn't sticking after rebooting. I had to keep going into /etc/fstab to make the change. But it seems to be sticking now.



  • @jimp:

    Some have found that the disk changes we made for sync mode have slowed down squid by default.

    If you want to risk the possibility of disk corruption but gain speed, edit the ,sync out of /etc/fstab for the root slice and/or run

    mount -o nosync /
    

    Could this change to disk sync also be the issue for my listening queue problem with squid? –> https://forum.pfsense.org/index.php?topic=95873.0

    At least as soon as I disable disk sync, the listen queue size drops to zero. I compared my /etc/fstab from backup with current one and this sync options wasn't present before.

    Maybe this option creates big performance impact on RAID systems. My secondary CARP node - also running 2.2.3 and sync option enabled - does not have this problem, but doesn't have RAID either.



  • @hbc:

    Could this change to disk sync also be the issue for my listening queue problem with squid? –> https://forum.pfsense.org/index.php?topic=95873.0

    At least as soon as I disable disk sync, the listen queue size drops to zero. I compared my /etc/fstab from backup with current one and this sync options wasn't present before.

    Maybe this option creates big performance impact on RAID systems. My secondary CARP node - also running 2.2.3 and sync option enabled - does not have this problem, but doesn't have RAID either.

    It is definitely a RAID issue. Squid caching performance suck on a RAID. Especially with the 'mount sync' option. Just edit your /etc/fstab file with mount option 'nosync' with 'Edit File' in 'Diagnostics'. Save and reboot the box. When it comes back check fstab to make sure that the change is still there.

    Other mod changes that i made were to 'Systsm> Advance> System Tunables>'

    Tunable name                    Value
    vfs.read_max            from 32 to 128

    https://doc.pfsense.org/index.php/Squid_Package_Tuning

    Created
    kern.ipc.nmbclusters        32768

    created the above ONLY if you are using the 'diskd' 'Hard disk cache system' setup in Squid. Which requires you to copy ipcs and ipcrm from a FreeBSD 10.1 ISO /usr/bin/ to pfsense /usr/local/bin

    also reboot and check after reboot to make sure that the settings are still there.

    Squid is 'flying right now'. Well sort of. But the 'nosync' boot option in fstab does make a BIG different.

    I am running squid in a production environment.



  • i have already this problem but i dont have any raid config
    everything was good working until i update from 2.2.2 to 2.2.3 and intennet put so slow now i am finding solution i already tried to reinstall squid and squidguard but nothing is solved now i dont have any filter
    i installed new one with fresh install pfsense 2.2.3 but with fresh install i have problem too after restart pfsense squid and squidguard stops and cant start again (squid-1): The redirector helpers are crashing too rapidly, need help!
    my system was squid3 and squidguard-dev amd64 squid non transparent (with wpad)
    i hope to solve
    second time i did big problems (first when i update to 2.2.2 rebooting time delay so much) and 2.2.3 squid problems
    i think it is time to change FW with others



  • @mesro09:

    i have already this problem but i dont have any raid config
    everything was good working until i update from 2.2.2 to 2.2.3 and intennet put so slow now i am finding solution i already tried to reinstall squid and squidguard but nothing is solved now i dont have any filter
    i installed new one with fresh install pfsense 2.2.3 but with fresh install i have problem too after restart pfsense squid and squidguard stops and cant start again (squid-1): The redirector helpers are crashing too rapidly, need help!
    my system was squid3 and squidguard-dev amd64 squid non transparent (with wpad)
    i hope to solve
    second time i did big problems (first when i update to 2.2.2 rebooting time delay so much) and 2.2.3 squid problems
    i think it is time to change FW with others

    Just try the suggestion above and make the change on fstab.



  • @jimp:

    Some have found that the disk changes we made for sync mode have slowed down squid by default.

    If you want to risk the possibility of disk corruption but gain speed, edit the ,sync out of /etc/fstab for the root slice and/or run

    mount -o nosync /
    

    My squid3 is not fast either. But I don't like risking disk corruption, possibly destroying pfSense. That is: what are the odds disk corruption will appear, Jim? Relevant variables? A thumb number (0,005% or 60%)?



  • DNS issues can also make squid look slow.  Shell in and run:

    squidclient -h LAN_IP_Address -p 3128 mgr:info

    Then read the report, paying special attention to the Median Service Times section.  Look for anything that seems large as compared to the others.



  • @KOM:

    DNS issues can also make squid look slow.  Shell in and run:

    squidclient -h LAN_IP_Address -p 3128 mgr:info

    Then read the report, paying special attention to the Median Service Times section.  Look for anything that seems large as compared to the others.

    I do not want to hijack this thread, so if I have to create a new thread (similar problem), I will, just let me know  :-[

    [quote]
    squidclient -h localhost  -p 3128 mgr:info
    Sending HTTP request … done.
    HTTP/1.1 200 OK
    Server: squid
    Mime-Version: 1.0
    Date: Tue, 07 Jul 2015 15:23:33 GMT
    Content-Type: text/plain
    Expires: Tue, 07 Jul 2015 15:23:33 GMT
    Last-Modified: Tue, 07 Jul 2015 15:23:33 GMT
    X-Cache: MISS from squid
    X-Cache-Lookup: MISS from squid:3128
    Connection: close

    Squid Object Cache: Version 3.4.10
    Build Info:
    Start Time:    Tue, 07 Jul 2015 13:39:21 GMT
    Current Time:  Tue, 07 Jul 2015 15:23:33 GMT
    Connection information for squid:
            Number of clients accessing cache:      2
            Number of HTTP requests received:      1037
            Number of ICP messages received:        0
            Number of ICP messages sent:    0
            Number of queued ICP replies:  0
            Number of HTCP messages received:      0
            Number of HTCP messages sent:  0
            Request failure ratio:  0.00
            Average HTTP requests per minute since start:  10.0
            Average ICP messages per minute since start:    0.0
            Select loop called: 609439 times, 10.259 ms avg
    Cache information for squid:
            Hits as % of all requests:      5min: 23.7%, 60min: 12.2%
            Hits as % of bytes sent:        5min: 37.0%, 60min: 4.2%
            Memory hits as % of hit requests:      5min: 0.0%, 60min: 8.9%
            Disk hits as % of hit requests: 5min: 0.0%, 60min: 8.9%
            Storage Swap size:      6668 KB
            Storage Swap capacity:  0.0% used, 100.0% free
            Storage Mem size:      4620 KB
            Storage Mem capacity:    0.2% used, 99.8% free
            Mean Object Size:      12.97 KB
            Requests given to unlinkd:      0
    Median Service Times (seconds)  5 min    60 min:
            HTTP Requests (All):  0.10857  0.32154
            Cache Misses:          0.12783  0.37825
            Cache Hits:            0.00000  0.07014
            Near Hits:            0.00000  0.22004
            Not-Modified Replies:  0.05633  0.05633
            DNS Lookups:          0.01940  0.02683
            ICP Queries:          0.00000  0.00000
    Resource usage for squid:
            UP Time:        6252.187 seconds
            CPU Time:      73.719 seconds
            CPU Usage:      1.18%
            CPU Usage, 5 minute avg:        0.89%
            CPU Usage, 60 minute avg:      1.27%
            Maximum Resident Size: 178544 KB
            Page faults with physical i/o: 0
    Memory accounted for:
            Total accounted:        7844 KB
            memPoolAlloc calls:    246629
            memPoolFree calls:    255812
    File descriptor usage for squid:
            Maximum number of file descriptors:  58977
            Largest file desc currently in use:    51
            Number of file desc currently in use:  32
            Files queued for open:                  0
            Available number of file descriptors: 58945
            Reserved number of file descriptors:  100
            Store Disk files open:                  0
    Internal Data Structures:
              569 StoreEntries
              537 StoreEntries with MemObjects
              536 Hot Object Cache Items
              514 on-disk objects

    What would you make of this, KOM?

    Thank you  :P



  • Everything looks normal.  I think your problem is disk-related as you suspected.



  • @Mr.:

    @jimp:

    Some have found that the disk changes we made for sync mode have slowed down squid by default.

    If you want to risk the possibility of disk corruption but gain speed, edit the ,sync out of /etc/fstab for the root slice and/or run

    mount -o nosync /
    

    My squid3 is not fast either. But I don't like risking disk corruption, possibly destroying pfSense. That is: what are the odds disk corruption will appear, Jim? Relevant variables? A thumb number (0,005% or 60%)?

    I have made the change in fstab both for work (two sites) and home setup and it is working fine. Just make the change in fstab and then reboot.



  • hello
    let me explain you this changes for squid caches but i dont use squid for cache contens my config is 0
    so can i make this changes ?¿?



  • Hi!. I'm having this same issue, when the proxy is enabled the webpages takes a lot of time to load and, once loaded it becomes more responsive but, still, is very slow.
    I have installed Squid3+SquidGuard Transparent proxy enabled and SSL filtering on. I've disabled squidguard (just to make sure) and the issue keeps there. Then, when I configured the lannet to bypass the proxy, the issue dissapeared so I think I've missed something important with Squid3.
    I tried to edit the /etc/fstab from this:

    # Device		Mountpoint	FStype	Options		Dump	Pass#
    /dev/ufsid/558c431cbd7f951e		/		ufs	rw,sync		1	1
    /dev/label/swap0		none		swap	sw		0	0
    
    

    removing the ,sync so the file now reads:

    # Device		Mountpoint	FStype	Options		Dump	Pass#
    /dev/ufsid/558c431cbd7f951e		/		ufs	rw		1	1
    /dev/label/swap0		none		swap	sw		0	0
    
    

    I ran the command

    mount -o nosync /
    

    and restarted the box…
    The issue? still there...
    Am I doing something wrong?

    Thanks in advance

    -------Update------------

    I got tired so did a full restauration from a file that I've backed up previously. I didn't knew what was causing my trouble, but now it is gone. I suggest the young adventurers that they do, at leas one time at week, a full backup of their configurations so, if they experience some isssue of this kind, have where to "run"... It is easier and faster.



  • Did you try my much earlier suggestion of running squidclient and then checking the numbers for outliers?



  • This might be a bit of a "basic" answer, but to me squid3 was painfully slow when using c-icap antivirus integration.
    The clam process just ate my cpu and the sites took ages to load.

    In clam's defense my system runs on a VIA C7 1,5Ghz + 512MB ram… getting too old for all of this.



  • squid3 was painfully slow when using c-icap antivirus integration.

    Of course the addition of either ClamAV or HAVP is going to cause a lot of overhead and will slow down everything.  I've always recommended using a client-based AV instead of having it on the firewall.



  • Hmm, When I ran

    squidclient -h 192.168.1.1 -p 3128 mgr:info

    I got

    
    Sending HTTP request ... done.
    HTTP/1.1 403 Forbidden
    Server: squid/3.4.10
    Mime-Version: 1.0
    Date: Sat, 19 Sep 2015 00:23:59 GMT
    Content-Type: text/html
    Content-Length: 3094
    X-Squid-Error: ERR_ACCESS_DENIED 0
    Vary: Accept-Language
    Content-Language: en
    X-Cache: MISS from localhost
    X-Cache-Lookup: NONE from localhost:3128
    Via: 1.1 localhost (squid/3.4.10)
    Connection: close
    
    <title>ERROR: The requested URL could not be retrieved</title>
    
    # ERROR
    
    ## The requested URL could not be retrieved
    
    * * *
    
    The following error was encountered while trying to retrieve the URL: [cache_obj                                            ect://192.168.1.1/info](cache_object://192.168.1.1/info)
    
    > **Access Denied.**
    
    Access control configuration prevents your request from being allowed at this time. Please contact your service provider                                             if you feel this is incorrect.
    
    Your cache administrator is [admin@localhost](mailto:admin@localhost?subject=CacheErrorInfo%20-%20ERR_ACCESS_DENIED&body=Cache                                            Host%3A%20localhost%0D%0AErrPage%3A%20ERR_ACCESS_DENIED%0D%0AErr%3A%20%5Bnone%5D%0D%0ATimeStamp%3A%20Sat,%2019%20Sep%202015%                                            2000%3A23%3A59%20GMT%0D%0A%0D%0AClientIP%3A%20192.168.1.1%0D%0A%0D%0AHTTP%20Request%3A%0D%0AGET%20%2Finfo%20HTTP%2F1.0%0AHos                                            t%3A%20192.168.1.1%0D%0AUser-Agent%3A%20squidclient%2F3.4.10%0D%0AAccept%3A%20*%2F*%0D%0AConnection%3A%20close%0D%0A%0D%0A%0                                            D%0A).
    
    * * *
    
    Generated Sat, 19 Sep 2015 00:23:59 GMT by localhost (squid/3.4.10)
    
    


  • I setup squid today and I believe I am experiencing this issue as well. I'll try that fstab thing. I did notice that using "links" in shell on pfSense seemed to have the same slow download speed that I have when going through squid, in case that helps any.



  • aGeekHere, are you sure you got it right?  You get that HTML spew when there is an error.



  • aGeekHere, are you sure you got it right?  You get that HTML spew when there is an error.

    Oh no, Well I ssh in and ran

    squidclient -h 192.168.1.1 -p 3128 mgr:info
    

    In the root folder.

    Ok trouble shooting time, where do I start?



  • On Squid's config page, look for External cache-managers and set it to 127.0.0.1, 192.168.1.1.  Save and try again.



  • When I add 127.0.0.1;192.168.1.1 to External cache-managers I now get.

    Sending HTTP request ... done.
    HTTP/1.1 403 Forbidden
    Expires: Thu, 24 Sep 2015 02:14:08 GMT
    Cache-Control: max-age=180000
    Content-Type: text/html
    Date: Tue, 22 Sep 2015 00:14:08 GMT
    Server: lighttpd/1.4.35
    X-Cache: MISS from localhost
    X-Cache-Lookup: MISS from localhost:3128
    Via: 1.1 localhost (squid/3.4.10)
    Connection: close
    
    ### Request denied by pfSense proxy: 403 Forbidden
    
     **Reason:** 
    
    * * *
    
     **Client address:** 192.168.1.1 
    
     **Client name:** pfsense.mydomain.local 
    
     **Client group:** default 
    
     **Target group:** in-addr 
    
     **URL:** cache_object://192.168.1.1/info192.168.1.1/pfsense.mydomain.local-GET 
    
    * * *
    
    


  • Weird.  Check your System logs and squid logs.  I haven't seen that error before.



  • Ok some logs

    When I stop and start squid I get

    Sep 22 10:27:31	squid[22754]: Squid Parent: (squid-1) process 23039 started
    Sep 22 10:27:31	squid[22754]: Squid Parent: will start 1 kids
    Sep 22 10:27:22	php-fpm[84775]: /status_services.php: The command '/usr/local/etc/rc.d/squid.sh stop' returned exit code '1', the output was '2015/09/22 10:27:17| Warning: empty ACL: acl throttle_exts urlpath_regex -i "/var/squid/acl/throttle_exts.acl" squid: No running copy'
    Sep 22 10:26:48	php-fpm[67812]: /status_services.php: The command '/usr/local/etc/rc.d/squid.sh stop' returned exit code '1', the output was '2015/09/22 10:26:42| Warning: empty ACL: acl throttle_exts urlpath_regex -i "/var/squid/acl/throttle_exts.acl"
    

    In squid real time if I do squidclient -h 192.168.1.1 -p 3128 mgr:info
    I get

    22.09.2015 10:33:03	192.168.1.244	TCP_DENIED/403	127.0.0.1:59243	-	-
    22.09.2015 10:32:12	192.168.1.244	TCP_DENIED/403	127.0.0.1:59243	-	-
    22.09.2015 10:32:01	192.168.1.1	TCP_MISS/403	cache_object://192.168.1.1/info	-	192.168.1.1
    22.09.2015 10:31:46	192.168.1.1	TCP_MISS/403	cache_object://192.168.1.1/info	-	192.168.1.1
    22.09.2015 10:31:43	192.168.1.1	TCP_MISS/403	cache_object://192.168.1.1/info	-	192.168.1.1
    22.09.2015 10:31:40	192.168.1.1	TCP_MISS/403	cache_object://192.168.1.1/info	-	192.168.1.1
    22.09.2015 10:31:22	192.168.1.1	TCP_MISS/403	cache_object://192.168.1.1/info	-	192.168.1.1
    22.09.2015 10:29:59	192.168.1.244	TCP_DENIED/403	127.0.0.1:59243	-	-
    22.09.2015 10:26:28	192.168.1.244	TCP_DENIED/403	127.0.0.1:59243	-	-
    

Log in to reply