PfSense 2.2.3 - Slow UPLOAD Speed via Squid3
-
Hello all,
I have notice that the internet upload speed has been nearly 'cut in half' when traffic is pushed through squid.
Now i am on a 'up to' 100mbps up/down connection.
speed test shows about around 90+ down and barely over 30 up when ran on a machine that is using squid. On a machine that is not using squid i get about 50mbps for up.
With the pfSense box totally out of the picture i get far closer to the 100mbps for the up speed.
The internet is still relatively slow but a lot much more responsive after making the 'nosync' change in fstab.
Does anybody have any idea as to what is going on?
FYI: I am using pfsense in a production environment strictly as a web content filter via squid3 and SquidGuard. The firewall has been disabled. LAN and DMZ (connected to Juniper FW) are in bridged mode. So the only filtering relatively going on is through Squid3 and SquidGuard.
Thank you for your help…if any.
-
If you're only using it as an URL filter, have to disabled the hard disk cache? What kind of box are you running on?
-
I'm using an HP proliant server with RAID 1, 6GB RAM, Xeon processor, 4 NICS (only two are being used which are bridged). HDD cache is enabled as i would like some caching done as well.
-
Did you set up some monstrously large number of level-1 subdirs in conjunction with a very large cache size?
-
@KOM:
Did you set up some monstrously large number of level-1 subdirs in conjunction with a very large cache size?
Level-1 = 16
I have 100GB set for HDD cache, 2GB for Mem cache.
Also using diskd. ipcs and ipcrm were copied from a FreeBSD 10.1 distro to pfsense /usr/local/bin.
-
How full is that 100GB? As a test you cold try blowing the cache away, recreate and then see if there is any performance difference.
-
@KOM:
How full is that 100GB? As a test you cold try blowing the cache away, recreate and then see if there is any performance difference.
It is nowhere near full. In fact i have cleared it numerous times today.
-
There is something completely off ass download speeds hit 85+ while upload speeds hit around 30 - 40. It doesn't make sense.
-
I might suggest disabling the disk cache altogether and see if it is disk-related or not. Set the size to minimum (0 or 1) and the file system to null.
-
@KOM:
I might suggest disabling the disk cache altogether and see if it is disk-related or not. Set the size to minimum (0 or 1) and the file system to null.
I have made the changes, rebooted and it is still clocking around the same. Nothing changed pretty much.
I have even disabled SquidGuard and the upload speed is still hitting around 30 and 40. More like 30 and 35. It hardly ever hits 40. And this was tested on multiple machines by the way.
-
I don't know what to tell you. I also have a 100Mb fibre connection and we don't have those speed issues.
-
I don't know what to tell you. I also have a 100Mb fibre connection and we don't have those speed issues.
Would you tell us your hardware parts please?
We where also very disappointed about our Squid throughput indeed! So we want now
setting up a the new pfSense box and then we want to set behind the firewall (pfSense)
a stand alone Squid Proxy & SquidGuard, based on the following numbers:
Linux Based Squid 3- RAID1 (2 HDD/SSD)
- RAID5 or RAID10 (4 SSDs) Cache
- Intel Xeon E3-1286v3
- Intel Quad port NIC
- 32 GB ECC GB RAM
Also Snort sensors and a dedicated Snort Server would freeing the firewall from more load.
-
a stand alone Squid Proxy & SquidGuard
I was going to do this but I ran into the problem of every major distro on Earth packaging old versions of Squid. Even the latest Ubuntu server ships with squid 3.3.8(!) and has a broken squidguard:
https://bugs.launchpad.net/ubuntu/+source/squid3/+bug/1448149
I tried a few others and it was even worse, with squid 3.2 being the favourite shipping version in the top distros. I would rather not have to compile from source and then keep up on it like that. When pfSense 2.2 came out with better squid support, I ended up staying with it instead of spinning off a separate Squid server.
-
The speed within the last hour is hitting at 83 for down and 20 for up! :-\
This is in an environment with less than 100 users. :o
-
This is in an environment with less than 100 users.
Perhaps pending on the hardware you are using?
-
The culprit i have found it in fact to be the "Antivirus", ClamAv and I-cap. The speeds are nearly identical, up above 90/70 with it turned off.
With it on upload in particular is hitting around the 30 mark. Barely over at times.
Any suggestion(s) in tuning ClamAV and I-Cap?
