Web browsing through proxy filter is slow.

shehan31 · Mar 30, 2015, 4:51 PM

Dear all;
Web browsing through proxy filter is slow. I have edited some parameters in loader.conf but the problem still exist. It works fine for the last few weeks. CPU usage 1%, RAM 12%, others are not in high levels.

KOM · Mar 30, 2015, 5:11 PM

You probably have a cache that is way too big or too full perhaps. Searching the cache in conjunction with slow disks can make it very slow.

shehan31 · Mar 31, 2015, 4:34 AM

@KOM:

You probably have a cache that is way too big or too full perhaps. Searching the cache in conjunction with slow disks can make it very slow.

Thank you for the steps. What are the actions that I have to take ?

aaronouthier · Mar 31, 2015, 6:10 AM

In your squid proxy, decrease your cache size to less than 1000 MB in the local cache tab.

aaronouthier · Mar 31, 2015, 6:12 AM

Either that, or use faster storage media. Ie flash drive/ssd or a ram drive.

shehan31 · Mar 31, 2015, 11:32 AM

@aaronouthier:

Either that, or use faster storage media. Ie flash drive/ssd or a ram drive.

Thank you.
I have attached a copy of the netstat result. Once again thank you for the reply.

tcp:
304 packets sent
39093 data packets (0 bytes)
241931 data packets (98 bytes) retransmitted
1812 data packets unnecessarily retransmitted
14017447709277307556 resends initiated by MTU discovery
84638 ack-only packets (899675460 delayed)
4435 URG only packets
584458 window probe packets
229439 window update packets
39588 packets received
64556 packets (51151 bytes) received in-sequence
34013 completely duplicate packets (0 bytes)
2 packets with some dup. data (0 bytes duped)
0 packets (278 bytes) of data after window
95915 window probes
251541 packets received after close
34015 discarded due to memory problems
33954 connection requests
67891 connection accepts
15771278854058137566 ignored RSTs in the windows
2 connections established (including accepts)
0 connections closed (including 70482 drops)
10455658960388767176 connections updated cached ssthresh on close
494440 embryonic connections dropped
2132047 segments updated rtt (of 0 attempts)
1095951 retransmit timeouts
1861736 connections dropped by rexmit timeout
99650 keepalive timeouts
730415 keepalive probes sent
659 connections dropped by keepalive
13975273677249250040 cache overflow
9447660539828814644 reset
14382196397542896495 stale
10415049268109197920 bucket overflow
14017127141508569354 segment rexmits in SACK recovery episodes
15771278854058137566 SACK options (SACK blocks) received
92678 times ECN reduced the congestion window
37903 packets with tcp-md5 signature mismatch
54675 packets without expected tcp-md5 signature received
udp:
92678 datagrams received
37903 with no checksum
54675 dropped due to no socket
38003 delivered
ip:
2063109 total packets received
1709686 fragments dropped (dup or out of space)
9935062108823646716 packets received for unknown multicast group
13982004571698895900 packets sent from this host
4322644478532166374 packets sent with fabricated ip header
3763181339498447134 output packets dropped due to no bufs, etc.
3353279232 tunneling packets that can't find gif
15123517903393656705 datagrams with bad address in header
icmp:
Output histogram:
#21: 49143
#23: 39
#25: 16381
Input histogram:
#6: 3353833896
igmp:
ipsec:
15623621353018023376 mbufs coalesced during clone
ah:
esp:
ESP output histogram:
#144: 13970725578518182393
#183: 14017576450920059965
#184: 90916420720095106
#185: 14017288524902475071
#186: 14017588219130456320
#199: 13982054496398737408
#200: 14017576300596204288
#212: 4653429
#222: 13225296049490403542
#225: 13982054496398737408
#226: 13982054530758475776
#227: 13926660241597923328
#228: 14017576450920059965
#229: 13926660001079754752
#230: 13926659953835114496
#232: 9933185634353611520
#233: 10300876702227540576
#234: 17186974817
#235: 10290303248659032544
ipcomp:
pim:
carp:
9936297415751491037 discarded for bad authentication
13970721033390186350 packets sent (IPv6)
13970710875792240003 send failed due to mbuf memory error
pfsync:
arp:
50 ARP requests sent
330463 ARP replies sent
330495 ARP requests received
13970568232391281693 total packets dropped due to no ARP entry
ip6:
1466 total packets received
1458 fragments dropped after timeout
2798 redirects sent
531 packets sent from this host
927 packets sent with fabricated ip header
Input histogram:
#133: 539
#184: 9936671086217113856
#185: 13982033055922002972
#187: 219269010100049079
#194: 10317562612467892224
#195: 14401378290073598272
#196: 2689667654128019066
#199: 14401677130544971775
#234: 14432174665777162880
#237: 8611822540677810517
#240: 14512025312710459653
#241: 9935093338570383376
#246: 14297378938709464032
#247: 2306243267480881493
#248: 1505842663203899749
Mbuf statistics:
0 one mbuf
two or more mbuf:
bge0= 15623621353018023376
0 one ext mbuf
0 two or more ext mbuf
Source addresses selection rule applied:
icmp6:
Output histogram:
#66: 2
#70: 7
#197: 8
Input histogram:
#11: 75153822024116259
#15: 6497534803
#16: 13970692081026072364
#17: 13970691050234995888
#18: 6476669557
#19: 9494657502203544344
#25: 936
#60: 539
#159: 927
#227: 9936671086217113856
#228: 13982033055922002972
#236: 10317562584808987840
#238: 14382396320287652084
#241: 14578148495700733228
#242: 3360462816
Histogram of error messages to be generated:
13970705240808508624 bad router advertisement messages
13982034725758923820 bad redirect messages
ipsec6:
rip6:
15889638843442104661 multicast messages dropped due to no socket
2557105230267446955 delivered
9012800619012389776 datagrams output
pfkey:
histogram by message type:
#169: 14398254302656753664
#177: 109327836762070
#179: 13982051764799537152
#180: 90916420720095071
#221: 13970727304052882864
#234: 4653461

KOM · Mar 31, 2015, 1:07 PM

Squid can be slow for other reasons, like DNS issues. SSH in and run:

squidclient -h pfSense_LAN_IP -p 3128 mgr:info

and look at the Median Service Times. Anything look too big compared to the others? Here is my output when things are working well:

Median Service Times (seconds) 5 min 60 min:
HTTP Requests (All): 0.00000 0.00000
Cache Misses: 0.02592 0.03241
Cache Hits: 0.00000 0.00091
Near Hits: 0.00091 0.09219
Not-Modified Replies: 0.00000 0.00000
DNS Lookups: 0.05078 0.06364
ICP Queries: 0.00000 0.00000

shehan31 · Apr 1, 2015, 8:55 AM

@KOM:

Squid can be slow for other reasons, like DNS issues. SSH in and run:

squidclient -h pfSense_LAN_IP -p 3128 mgr:info

and look at the Median Service Times. Anything look too big compared to the others? Here is my output when things are working well:

Median Service Times (seconds) 5 min 60 min:
HTTP Requests (All): 0.00000 0.00000
Cache Misses: 0.02592 0.03241
Cache Hits: 0.00000 0.00091
Near Hits: 0.00091 0.09219
Not-Modified Replies: 0.00000 0.00000
DNS Lookups: 0.05078 0.06364
ICP Queries: 0.00000 0.00000

Thank you for the reply. I get the following message which does not display as yours. I have reconfigured DNS and seems like it has got little bit better. But I need more time to finalize this.

128 mgr:info
Sending HTTP request … done.
HTTP/1.1 403 Forbidden
Server: squid/3.4.10
Mime-Version: 1.0
Date: Wed, 01 Apr 2015 04:01:50 GMT
Content-Type: text/html
Content-Length: 3112
X-Squid-Error: ERR_ACCESS_DENIED 0
Vary: Accept-Language
Content-Language: az
X-Cache: MISS from localhost
X-Cache-Lookup: NONE from localhost:3128
Via: 1.1 localhost (squid/3.4.10)
Connection: close

<title>SƏHV: Sorğulanan URL ilə əlaqə yaradılmadı</title>

SƏHV

Sorğulanan URL ilə əlaqə yaradılmadı

The following error was encountered while trying to retrieve the URL: <a href ="" ="">cache_object://192.168.0.235/info</a>

Giriş qadağandır.

Access control configuration prevents your request from being allowed at this time. Please contact your service provider if you feel this is incorrect.

Your cache administrator is [admin@localhost](mailto:admin@localhost?subject=CacheErro rInfo%20-%20ERR_ACCESS_DENIED&body=CacheHost%3A%20localhost%0D%0AErrPage%3A% 20ERR_ACCESS_DENIED%0D%0AErr%3A%20%5Bnone%5D%0D%0ATimeStamp%3A%20Wed,%2001%20Apr %202015%2004%3A01%3A50%20GMT%0D%0A%0D%0AClientIP%3A%20192.168.0.238%0D%0A%0D%0AH TTP%20Request%3A%0D%0AGET%20%2Finfo%20HTTP%2F1.0%0AHost%3A%20192.168.0.235%0D%0A User-Agent%3A%20squidclient%2F3.4.10%0D%0AAccept%3A%20*%2F*%0D%0AConnection%3A%2 0close%0D%0A%0D%0A%0D%0A).

Generated Wed, 01 Apr 2015 04:01:50 GMT by localhost (squid/3.4.10)

KOM · Apr 1, 2015, 1:28 PM

Huh???

1. Go to pfSense console.

2. Press 8 for Shell. See image.

3. Type the command I gave you and parse the output.

periko · Apr 1, 2015, 6:26 PM

Are using squidguard in your setup?

shehan31 · Apr 2, 2015, 11:39 AM

@periko:

Are using squidguard in your setup?

Yes.

periko · Apr 2, 2015, 4:39 PM

By default squid will only allow 5 connections to squidguard if u have more than 5 users at the same time u are doom.
Now that u are in your shell check cache.log and see if u have something like this:

2013/10/25 09:44:24| WARNING: All redirector processes are busy.
2013/10/25 09:44:24| Consider increasing the number of redirector processes in your config file.

If this is your case, just remember a formula:

of Squidguard threats x memory size each one=RAM memory for squidguard.

Because squidguard depends on RAM that is why is fast.

Hope this is your case and hope this helps!!!