PfSense 2.2.3 - ClamAV-ICap for Squid3 inpact on througput (HELP)
-
I have finally figure out the culprit in regards to the slowness of my internet speed and just over all disruption of smooth web browsing. Particularly the negative effect on Upload speed.
It is the anti-virus, ClamAV, I-Cap combo for Squid3.
After extensive testing of turning off the anti-virus the net speeds sustained at a relative, almost equal rate as it should. Giving that i am working with a 100up/100down speep.
With the anti virus off down and up were hitting around 85-90+/60+-75+.
I have tried adjusting the some setting in ClamAV, i.e. disabling scanning of media files. And with I-Cap disabling time outs, increasing startup/max servers and threads but Upload speed in particular is barely hitting 30 or over at times.
Can anyone help with tuning these pair?
Thanks
-
I have finally figure out the culprit in regards to the slowness of my internet speed and just over all disruption of smooth web browsing. Particularly the negative effect on Upload speed.
Could it be that your hardware is capable to handle some things but not AV Scanning on top?
With the anti virus off down and up were hitting around 85-90+/60+-75+.
And there is only the AV off? Not the Squid, AV and snort?
I have tried adjusting the some setting in ClamAV, i.e. disabling scanning of media files.
Could also be that a proper Squid, snort and AV tuning will be helping you to speed up the one or another
thing but not really changing the entire workload.Can anyone help with tuning these pair?
Could you perhaps provide some more informations about the used hardware such as;
- CPU
- RAM
- Cache space
- HDD/SDD/mSATA
-
Thanks for the reply. But i can without of a shadow of a doubt say that it is the AV.
With it enabled Uploads speed barely hits 30 on a "up to 100/mbps" speed. With it off it clocks in the 70's.
I have since replaced Snort with Suricata to make use of the multi-threading feature.
Squid is configured properly and working great along with Squidguard. Up 100GB hard drive space for caching and 2GB ram for caching.
With all other services disabled, i.e. bandwidthd, ntopng, snort/suricata, squidguard and just Squid enabled along with the AV i still get the same results.
In one environment box is an HP server with a xeon processor, 6GB ram, RAID 1 300GB SAS (less than 100 users). Probably more in network devices
In the other (less than 40 users) core i5 cpu, 4GB ram, 500GB single HDD. In this environment, net speed is up to 50/50. With AV on upload test gets barley 20mbps. With it off it clocks in the low 40's.
So yeah, it is definitely the AV.
I have tried giving HAVP "another shot" but the service still doesn't start. Even after making the changes suggested in the link below. I just uninstalled it.
https://forum.pfsense.org/index.php?topic=90706.0
FYI: With HAVP set as parent of proxy web pages doesn't load. I suppose because the service doesn't start.
-
What I found is that i-Cap and ClamAV are having HDD writes extensively.
-
What I found is that i-Cap and ClamAV are having HDD writes extensively.
What is your cache for ClamAV scans?
What kind of drive is it? (mSATA, SSD, HDD)
Is the OS also installed on this drive? -
@BlueKobold:
What I found is that i-Cap and ClamAV are having HDD writes extensively.
What is your cache for ClamAV scans?
What kind of drive is it? (mSATA, SSD, HDD)
Is the OS also installed on this drive?my pfSense with Squid3, using HDD. I used 'top' to see these 2 have extensively WRITES, I heard HDD noise UNTIL I disabled them (Anti-Virus), then every thing kept quiet again. I have un-installed Squid3 by now, not very useful for me.
-
Is the OS also installed on this drive?
This could be then to much load for the entire drive.
-
@BlueKobold:
Is the OS also installed on this drive?
This could be then to much load for the entire drive.
Don't think that would be one of the reason.
-
my pfSense with Squid3, using HDD. I used 'top' to see these 2 have extensively WRITES, I heard HDD noise UNTIL I disabled them (Anti-Virus), then every thing kept quiet again.
@BlueKobold:
This could be then to much load for the entire drive.
Don't think that would be one of the reason.
Logic does not seem to be your forte… :o ;D
-
my pfSense with Squid3, using HDD. I used 'top' to see these 2 have extensively WRITES, I heard HDD noise UNTIL I disabled them (Anti-Virus), then every thing kept quiet again.
@BlueKobold:
This could be then to much load for the entire drive.
Don't think that would be one of the reason.
Logic does not seem to be your forte… :o ;D
not quite get this, i-Cap and ClamAV were having extensively writes to HDD (250Gb), is because of OS (FreeBSD, pfSense) also installed on the same drive?
Edit: forgot mention that Squid Cache was disabled.
-
Regardless, your throughput is going to stink if you have a virus scanner in the mix. You're better off running an acceptable AV package on your clients instead of loading the firewall with extraneous stuff that's guaranteed to slow the flow.
