Squid slowness issues - also a bounty post
-
It saddens me that I'm forced to post another bounty for a squid package that had been stable for over a year, but such is life. Recently something changed in the code that has slowed squid to a crawl. I've posted about the issue twice ( http://forum.pfsense.org/index.php/topic,6808.0.html and http://forum.pfsense.org/index.php/topic,7186.0.html ) but nobody seems interested. I'm sure as more people reinstall the package it will become a bigger problem. I've also created ticket #1557. My only guess is that the issues are linked to the addition of the RFC1918 options, the custom option box, or other changes made by mfuchs. I suppose it could also be something in the pfSense code, but that seems unlikely. I do not have the expertise to back out these changes and test for throughput. CMB, Sullrich, dvserg, or databeestje, this seems like something you guys could help with.
As a workaround, we had to bring out an outdated pfSense box with an older version of the Squid package. All is working well on that machine at the moment.
If you're interested in fixing this, pick a price and we'll see where we go from there.
-
-
I'll give it a try and report back. Thanks so much for the prompt reply.
-
I never had this problem with Squid, but I did have a few others:
For about 25 minutes one day anyone connected to the network was unable to reach Google (other sites worked fine). The error was:
The following error was encountered:
Unable to determine IP address from host name for www.google.com
The dnsserver returned:
Timeout
Then over a couple days time Google wouldn't load upon the first page load every once in a while. There would just be a blank page with a normal 'Generated by pfSense (squid)' message at the bottom. If you refreshed the page it would then load Google without any problems.
We had to remove the Squid package because it affected our employees trying to use the internet. I'd love to use Squid on pfSense again, but I have to wait until it's stable again.
If someone is interested in fixing this, pick a price.
-
Ben, as the error clearly said, the problem was DNS related - not Squid's fault. The correct solution at the time would have been to chose a working DNS server (or set of).
-
I can duplicate this behavoiur, did you use the transparent squid settings? For me it was a problem with the squid and the caching mode, after i set squid to use the transparent mode, all was OK. It is not a DNS Problem.
-
I can duplicate this behavoiur, did you use the transparent squid settings? For me it was a problem with the squid and the caching mode, after i set squid to use the transparent mode, all was OK. It is not a DNS Problem.
Yes, Squid was using transparent mode (transparent proxy checkbox was on).
-
did you also checked the (RFC 1918) Checkbox?
-
did you also checked the (RFC 1918) Checkbox?
No. I didn't think that was needed for my network.
-
please check this box and test again
-
The problem is not the squid package itself.
We turned off write caching on hard drives which is surely causing your slowdowns:sysctl hw.ata.wc
hw.ata.wc: 0
Edit /etc/sysctl.conf and find this value and change to:
hw.ata.wc=1I've played with this setting and have not seen any change in throughput, however mine is a SCSI box so perhaps the command is different. Do you know off hand how to re-enable write caching for SCSI disks?
I've also noticed that proxy throughput is fine for a few seconds just after reboot. After reading through the system log, it appears that squid may be starting twice. Please find below an excerpt from my cache.log
2007/12/31 10:36:35| Starting Squid Cache version 2.6.STABLE5 for i386-portbld-freebsd6.1... 2007/12/31 10:36:35| Process ID 948 2007/12/31 10:36:35| With 7232 file descriptors available 2007/12/31 10:36:35| Using kqueue for the IO loop 2007/12/31 10:36:35| DNS Socket created at 0.0.0.0, port 9650, FD 5 2007/12/31 10:36:35| Adding nameserver 65.106.1.196 from /etc/resolv.conf 2007/12/31 10:36:35| Adding nameserver 65.106.7.196 from /etc/resolv.conf 2007/12/31 10:36:35| Unlinkd pipe opened on FD 10 2007/12/31 10:36:35| Swap maxSize 102400 KB, estimated 7876 objects 2007/12/31 10:36:35| Target number of buckets: 393 2007/12/31 10:36:35| Using 8192 Store buckets 2007/12/31 10:36:35| Max Mem size: 8192 KB 2007/12/31 10:36:35| Max Swap size: 102400 KB 2007/12/31 10:36:35| Store logging disabled 2007/12/31 10:36:35| Rebuilding storage in /var/squid/cache (CLEAN) 2007/12/31 10:36:35| Using Least Load store dir selection 2007/12/31 10:36:35| Current Directory is /tmp 2007/12/31 10:36:35| Loaded Icons. 2007/12/31 10:36:35| Accepting proxy HTTP connections at 10.21.1.24, port 3128, FD 12. 2007/12/31 10:36:35| Accepting proxy HTTP connections at 192.168.1.1, port 3128, FD 13. 2007/12/31 10:36:35| Accepting transparently proxied HTTP connections at 127.0.0.1, port 80, FD 14. 2007/12/31 10:36:35| WCCP Disabled. 2007/12/31 10:36:35| Ready to serve requests. 2007/12/31 10:36:35| Done reading /var/squid/cache swaplog (162 entries) 2007/12/31 10:36:35| Finished rebuilding storage from disk. 2007/12/31 10:36:35| 162 Entries scanned 2007/12/31 10:36:35| 0 Invalid entries. 2007/12/31 10:36:35| 0 With invalid flags. 2007/12/31 10:36:35| 162 Objects loaded. 2007/12/31 10:36:35| 0 Objects expired. 2007/12/31 10:36:35| 0 Objects cancelled. 2007/12/31 10:36:35| 0 Duplicate URLs purged. 2007/12/31 10:36:35| 0 Swapfile clashes avoided. 2007/12/31 10:36:35| Took 0.5 seconds ( 317.8 objects/sec). 2007/12/31 10:36:35| Beginning Validation Procedure 2007/12/31 10:36:35| Completed Validation Procedure 2007/12/31 10:36:35| Validated 162 Entries 2007/12/31 10:36:35| store_swap_size = 568k 2007/12/31 10:36:36| storeLateRelease: released 0 objects 2007/12/31 10:36:40| Starting Squid Cache version 2.6.STABLE5 for i386-portbld-freebsd6.1... 2007/12/31 10:36:40| Process ID 1000 2007/12/31 10:36:40| With 7232 file descriptors available 2007/12/31 10:36:40| Using kqueue for the IO loop 2007/12/31 10:36:40| DNS Socket created at 0.0.0.0, port 3353, FD 5 2007/12/31 10:36:40| Adding nameserver 65.106.1.196 from /etc/resolv.conf 2007/12/31 10:36:40| Adding nameserver 65.106.7.196 from /etc/resolv.conf 2007/12/31 10:36:40| Unlinkd pipe opened on FD 10 2007/12/31 10:36:40| Swap maxSize 102400 KB, estimated 7876 objects 2007/12/31 10:36:40| Target number of buckets: 393 2007/12/31 10:36:40| Using 8192 Store buckets 2007/12/31 10:36:40| Max Mem size: 8192 KB 2007/12/31 10:36:40| Max Swap size: 102400 KB 2007/12/31 10:36:40| Store logging disabled 2007/12/31 10:36:40| Rebuilding storage in /var/squid/cache (DIRTY) 2007/12/31 10:36:40| Using Least Load store dir selection 2007/12/31 10:36:40| Current Directory is /tmp 2007/12/31 10:36:40| Loaded Icons. 2007/12/31 10:36:40| Accepting proxy HTTP connections at 10.21.1.24, port 3128, FD 12. 2007/12/31 10:36:40| Accepting proxy HTTP connections at 192.168.1.1, port 3128, FD 13. 2007/12/31 10:36:40| Accepting transparently proxied HTTP connections at 127.0.0.1, port 80, FD 14. 2007/12/31 10:36:40| WCCP Disabled. 2007/12/31 10:36:40| Ready to serve requests. 2007/12/31 10:36:40| Done reading /var/squid/cache swaplog (162 entries) 2007/12/31 10:36:40| Finished rebuilding storage from disk. 2007/12/31 10:36:40| 162 Entries scanned 2007/12/31 10:36:40| 0 Invalid entries. 2007/12/31 10:36:40| 0 With invalid flags. 2007/12/31 10:36:40| 162 Objects loaded. 2007/12/31 10:36:40| 0 Objects expired. 2007/12/31 10:36:40| 0 Objects cancelled. 2007/12/31 10:36:40| 0 Duplicate URLs purged. 2007/12/31 10:36:40| 0 Swapfile clashes avoided. 2007/12/31 10:36:40| Took 0.3 seconds ( 554.1 objects/sec). 2007/12/31 10:36:40| Beginning Validation Procedure 2007/12/31 10:36:40| Completed Validation Procedure 2007/12/31 10:36:40| Validated 162 Entries 2007/12/31 10:36:40| store_swap_size = 568k 2007/12/31 10:36:41| storeLateRelease: released 0 objects
I appreciate all the help thus far…
-
I am able to get 50+ mb/s when copying files to and from the pfSense box via WinSCP. Perhaps this isn't a write caching issue after all. I'm still ready to pay a bounty for a solution to this problem as it is impacting our business.
-
I just downloaded and installed 1.2RC3 release, not the latest snapshot and speeds were fine after installing squid via the GUI. The issue must be in the pfsense code somewhere…
-
I am having this problem as well.
I have a total of 6 pFsense boxen running 1.2 rc4 with the latest available Squid version via Packages. All 6 boxen are experiencing this issue.
When going through the proxy pages load about 30% to 40% slower. I can test at 2800kbps on a speed test site without going through the proxy and only 2200kbps going through it. I can reproduce this every time.
I am using local authentication in normal mode, NOT transparent mode.
I didn't try using Squid until RC4 was out, so cannot confirm if its a problem on an older version.
-
Hello!
Are there any news with this issue?
-
Yes, please make a squid package update
-
-
Hi, same problem here, 4 Mb downstream syncronous.
pfsense 1.2-RC3, and really serious problems with downstream using proxy, traffic sharper disabled.
squid version 2.6.18
can anyone help? is there a solution? i'm working with pfsense since a year ago and really proud of it, but no clue how to solve this issue :/
kind regards
-
Please add to our bounty to fix the problem!
-
With 1.2 release and the latest Squid package (and traffic shaper enabled) I'm seeing download speeds much the same as before the upgrades - between 16 and 17 Mb/s on my 20 Mb/s line.