4. Lag on Http/Https Sites every now and then?

Lag on Http/Https Sites every now and then?

  • K

    Hi,
    I was wondering if someone could assist or guide me on whats going on? Currently Running OpenVPN, squid3, squidguard, Ntopng,Sarg, lightsquid, and WPAD.The internet service has 28mbps down and up 15mbps with ping 31ms
    Many users been complaining for the past weeks about slow lag internet I really did not notice until I tried navigating a few sites and all of sudden it would not load the website. I want to first check if there is something wrong with the settings of Squid before seeing if theres an issue with the cables or the switches. All the users are forcefully navigating though WPAD because I blocked them on ports 443-80 Without transparent proxy. See pictures Below

    I ran a test running```
    squidclient -h 192.168.1.254 -p 3128 mgr:info

    and I get the median service time seems good. (maybe i might be wrong)
Thank you

    Squid Object Cache: Version 3.4.10
    Build Info:
    Start Time:    Wed, 10 Aug 2016 23:13:45 GMT
    Current Time:  Fri, 02 Sep 2016 21:26:44 GMT
    Connection information for squid:
            Number of clients accessing cache:      61
            Number of HTTP requests received:      6476964
            Number of ICP messages received:        0
            Number of ICP messages sent:    0
            Number of queued ICP replies:  0
            Number of HTCP messages received:      0
            Number of HTCP messages sent:  0
            Request failure ratio:  0.00
            Average HTTP requests per minute since start:  196.2
            Average ICP messages per minute since start:    0.0
            Select loop called: 202241612 times, 9.794 ms avg
    Cache information for squid:
            Hits as % of all requests:      5min: 0.0%, 60min: 1.1%
            Hits as % of bytes sent:        5min: 0.1%, 60min: 1.3%
            Memory hits as % of hit requests:      5min: 0.0%, 60min: 15.6%
            Disk hits as % of hit requests: 5min: 0.0%, 60min: 11.2%
            Storage Swap size:      2474120 KB
            Storage Swap capacity:  4.8% used, 95.2% free
            Storage Mem size:      1000 KB
            Storage Mem capacity:  97.7% used,  2.3% free
            Mean Object Size:      39.29 KB
            Requests given to unlinkd:      7903
    Median Service Times (seconds)  5 min    60 min:
            HTTP Requests (All):  10.77596  0.05046
            Cache Misses:        10.77596  0.05046
            Cache Hits:            0.00000  0.00000
            Near Hits:            0.00000  0.07825
            Not-Modified Replies:  0.00000  0.00000
            DNS Lookups:          0.05815  0.05815
            ICP Queries:          0.00000  0.00000
    Resource usage for squid:
            UP Time:        1980778.633 seconds
            CPU Time:      3366.378 seconds
            CPU Usage:      0.17%
            CPU Usage, 5 minute avg:        0.09%
            CPU Usage, 60 minute avg:      0.23%
            Maximum Resident Size: 326896 KB
            Page faults with physical i/o: 18
    Memory accounted for:
            Total accounted:        16482 KB
            memPoolAlloc calls: 1457121328
            memPoolFree calls:  1464779995
    File descriptor usage for squid:
            Maximum number of file descriptors:  107559
            Largest file desc currently in use:    172
            Number of file desc currently in use:  132
            Files queued for open:                  0
            Available number of file descriptors: 107427
            Reserved number of file descriptors:  100
            Store Disk files open:                  0
    Internal Data Structures:
            63083 StoreEntries
              226 StoreEntries with MemObjects
              162 Hot Object Cache Items
            62967 on-disk objects

    
and the WPAD config

    function FindProxyForURL(url, host)
    {
        if (isPlainHostName(host) ||
            shExpMatch(host, "*.local") ||
            isInNet(dnsResolve(host), "192.168.1.0",  "255.255.255.0"))
            return "DIRECT";

    if (isInNet(dnsResolve(host), "192.168.10.0",  "255.255.255.0" ))
            { return "DIRECT"; }
            if (isInNet(dnsResolve(host), "192.168.115.0",  "255.255.255.0" ))
            { return "DIRECT"; }
            if (isInNet(dnsResolve(host), "10.10.1.0",  "255.255.252.0" ))
            { return "DIRECT"; }
            if (isInNet(dnsResolve(host), "10.10.10.1",  "255.255.255.255" ))
            { return "DIRECT"; }
    if (isInNet(dnsResolve(host), "192.168.1.206",  "255.255.255.255" ))
            { return "DIRECT"; }
            if (isInNet(dnsResolve(host), "200.1.XX.XXX",  "255.255.255.255" ))
            { return "DIRECT"; }
            if (isInNet(dnsResolve(host), "181.143.XXX.XX",  "255.255.255.255" ))
            { return "DIRECT"; }
    if (dnsDomainIs(host, "dane.gov.co") || dnsDomainIs(host, "www.dane.gov.co"))
            return "DIRECT";;
    if (dnsDomainIs(host, "cafesalud.com.co") || dnsDomainIs(host, "www.cafesalud.com.co"))
            return "DIRECT";;
            if (dnsDomainIs(host, "heon.com.co") || dnsDomainIs(host, "www.heon.com.co"))
            return "DIRECT";;
    if (dnsDomainIs(host, "saviasaludeps.com") || dnsDomainIs(host, "www.saviasaludeps.com"))
            return "DIRECT";;
            if (dnsDomainIs(host, "saviasaludeps.com") || dnsDomainIs(host, "www.saviasaludeps.com"))
            return "DIRECT";;
            if (dnsDomainIs(host, "outlook.office365.com") || dnsDomainIs(host, "www.outlook.office365.com"))
            return "DIRECT";;

    return "PROXY 192.168.1.254:3128";
    }

    ![Clipboarder.2016.09.02.png_thumb](/public/_imported_attachments_/1/Clipboarder.2016.09.02.png_thumb)
![Clipboarder.2016.09.02-002.png](/public/_imported_attachments_/1/Clipboarder.2016.09.02-002.png)
![Clipboarder.2016.09.02-002.png_thumb](/public/_imported_attachments_/1/Clipboarder.2016.09.02-002.png_thumb)
![Clipboarder.2016.09.02-003.png](/public/_imported_attachments_/1/Clipboarder.2016.09.02-003.png)
![Clipboarder.2016.09.02-003.png_thumb](/public/_imported_attachments_/1/Clipboarder.2016.09.02-003.png_thumb)
![Issue proxy.jpg](/public/_imported_attachments_/1/Issue proxy.jpg)
![Issue proxy.jpg_thumb](/public/_imported_attachments_/1/Issue proxy.jpg_thumb)
    0
  • A

    Try this..

    Clear disk cache

    Hard disk cache system - aufs
    Level 1 dir - 64
    Memory cache size - 1024 MB (I would go for 1.5Gb)

    0
  • K

    Thank you for the reply, I will immediately adjust it on your specification, I guess right now is wait until a user complains about it or is there a way I could check it?

    Thank you

    0
  • A

    It shouldn't do it again. The ufs cache system most likely was causing the lag along with just 1MB mem cache size as that was way too low.

    0
  • K

    Hi,
    Thank you for the reply, so unfortunately some users still complain about the lag not sure if its the proxy or the switches itself?

    This is a packet capture

    https://mega.nz/#!A9JhgCYS!i3OTZpKbvSpGN4ZWR617IkqO45bSDGKGXWRUFbAZW5k

    Thank you

    0
  • M

    Been having the same issue since yesterday with our web server wich is a shared hosting server.

    I've been getting connection timeout errors and lag when using the squid. I tested at the same time with another browser without proxy and the website loads inmediately.

    I've tried disabling caches, deleting cache, disabling squidguard. Also using WPAD. No other server seems affected by the lag.

    Using virtual pfsense 2.3 on xenserver 6.2, squid 3.5.19 with squidguard 1.4.15.

    0

  • I think I remember in the past that this issue could be fixed in some cases by checking the Resolve IPv4 DNS First checkbox in Squid's General Settings.

    0
  • M

    @KOM:

    I think I remember in the past that this issue could be fixed in some cases by checking the Resolve IPv4 DNS First checkbox in Squid's General Settings.

    Yes, this was enabled since i installed the proxy.

    0
  • A

    I see you are on an earlier version of PF and most likely Squid as well. If you can sustain some down time or have a temporary backup, I would recommend you start out with a fresh clean install of the latest version of PF and Squid. Backup the config but do not restore the old config file. Instead, use the backed up config file for reference while manually setting up the new install.

    Not sure why the SWAP space is being used since you have ample amount of free RAM. Also change MBUF Usage by setting kern.ipc.nmbclusters to 1000000 even though the current usage is not close to the max.

    0
  • K

    Thank you for the reply, As I have working on this for a while i thought to do some digging a packet capture on pfSense and then analyze on wireshark, I saw many TCP spurious Retransmission's, TCP Keep alive, TCP previous segment not capture and at one point shows tcp Zero window. Im guessing that the network card of the pfSense box cannot handled this much of traffic. I will keep checking what could be the issue. I will change the MBUF to see if it can handle worst case scenario buy a network card with more capacity?

    Thank you

    new packet capture–---  https://mega.nz/#!RsAh1CKA!gxYChuCWavhXIe-C9oBD50SLuq7XGscR4tm0MRJhb_I

    0
  • K

    Update: So i realized that unchecking auto detect proxy seems to speed everything up but then comes the dilemma of the WPAD….:(

    0
  • M

    As last option i just bypassed the proxy when connecting with my web server (which i'm having issues with) using the wpad configuration with isInNet(dnsResolve(host), "IP", "255.255.255.255").

    So the issue is gone but i hope it doesn't happen with other sites.

    0
  • K

    Thats true the WPAD ignores that IP goes though for a moment I thought it was WPAD now that I have been doing a deep analyze its not it. Not even sure what to do now…

    0
 

Products

Applications

Support

Services

News

Resources

Company

Our Mission

As host of the pfSense open source firewall project, Netgate believes in enhancing network connectivity that maintains both security and privacy. We also believe everyone should be able to afford it.

Subscribe to our Newsletter

Product information, software announcements, and special offers. See our newsletter archive for past announcements.

© Copyright 2002 - 2019 Rubicon Communications, LLC | Privacy Policy