How many users can I pass through from squidguard via pfsense

nearones

Hi,
I had been using Pfsense from 2 years, but since 4 to 5 months I need to constantly restart my pfsense server.
I had tired by just restarting squid and squidguard services, but it did not succeeded.
My issue is suddenly internet browsing stops for end users. So if I restart my pfsense, it works for around 3 to 4 hrs.
My current setting of cache as follows for 500+ users

General
–--------------
Log rotate - 30
What to do with requests that have whitespace characters in the URI - strip

Cache management

Hard disk cache size - 4096
Hard disk cache system - ufs
Memory cache size - 1024 (Server is of 2gb RAM)
Minimum object size - 0
Maximum object size - 131072
Maximum object size in RAM - 8192
Level 1 subdirectories - 16
Memory replacement policy - Heap GDSF
Cache replacement policy - Heap LFUDA
Low-water-mark in % - 90
High-water-mark in % - 95

Traffic Management

Maximum download size - 5120000
Maximum upload size - 20480
Overall bandwidth throttling - 0
Per-host throttling - 0
Finish transfer if less than x KB remaining - 0
Abort transfer if more than x KB remaining - 0
Finish transfer if more than x % finished - 0

Thanks & Regards,
Prashant

Nachtfalke

I would say the amount of available RAM ist not the best related to the fact you at minimum are using 1Gb for squid RAM caching. The 1GB is not a maximum limit for squid - it is a minimum limit. HDD cache will need some RAM for index and so on. squidguard will need RAM, too.

You could check this after your system is running for some hours or days and check the RAM usage and check if there is SWAP file usage. If there is SWAP file usage you should lower the squid RAM to lets say 512MB or 768MB.

Further you could try to lower the watermarks to 85 - 90%.

And why do you have to restart pfsense? What does not work anymore? Or is speed slow?
Did you check under "System activity" the CPU usage? RRD graphs will also tell you the RAM and CPU usage and you should look at the times when it is slow.

What I had in my environment ist that the number of redirectors for squid is to low. squid2 and squidguard users only 5 redirectors. I modified this in the following file:

/usr/local/pkg/squidguard_configurator.inc

Before:

define('REDIRECTOR_PROCESS_COUNT', '5'); # redirector processes count will started

After:

define('REDIRECTOR_PROCESS_COUNT', '75'); # redirector processes count will started

Then click "Save" and "Apply" on squidguard to make the changes take effect.

nearones

@Nachtfalke:

I would say the amount of available RAM ist not the best related to the fact you at minimum are using 1Gb for squid RAM caching. The 1GB is not a maximum limit for squid - it is a minimum limit. HDD cache will need some RAM for index and so on. squidguard will need RAM, too.

You could check this after your system is running for some hours or days and check the RAM usage and check if there is SWAP file usage. If there is SWAP file usage you should lower the squid RAM to lets say 512MB or 768MB.

Further you could try to lower the watermarks to 85 - 90%.

And why do you have to restart pfsense? What does not work anymore? Or is speed slow?
Did you check under "System activity" the CPU usage? RRD graphs will also tell you the RAM and CPU usage and you should look at the times when it is slow.

What I had in my environment ist that the number of redirectors for squid is to low. squid2 and squidguard users only 5 redirectors. I modified this in the following file:
/usr/local/pkg/squidguard_configurator.inc
Before:
define('REDIRECTOR_PROCESS_COUNT', '5'); # redirector processes count will started
After:
define('REDIRECTOR_PROCESS_COUNT', '75'); # redirector processes count will started
Then click "Save" and "Apply" on squidguard to make the changes take effect.

Thanks for your reply, I had made changes as you had mentioned.
But this changes are for how many users?
As my pfsense is been installed with 2GB of RAM, and when the users are not able to browse internet via proxy that time CPU usage of PFsense is between 25 to 30 % and RAM shows arround 80% of 2GB.
But after restarting the server RAM usage moves to 30% of 2GB and internet starts working.

Thanks again

nearones

@nearones:

@Nachtfalke:
I would say the amount of available RAM ist not the best related to the fact you at minimum are using 1Gb for squid RAM caching. The 1GB is not a maximum limit for squid - it is a minimum limit. HDD cache will need some RAM for index and so on. squidguard will need RAM, too.

You could check this after your system is running for some hours or days and check the RAM usage and check if there is SWAP file usage. If there is SWAP file usage you should lower the squid RAM to lets say 512MB or 768MB.

Further you could try to lower the watermarks to 85 - 90%.

And why do you have to restart pfsense? What does not work anymore? Or is speed slow?
Did you check under "System activity" the CPU usage? RRD graphs will also tell you the RAM and CPU usage and you should look at the times when it is slow.

What I had in my environment ist that the number of redirectors for squid is to low. squid2 and squidguard users only 5 redirectors. I modified this in the following file:
/usr/local/pkg/squidguard_configurator.inc
Before:
define('REDIRECTOR_PROCESS_COUNT', '5'); # redirector processes count will started
After:
define('REDIRECTOR_PROCESS_COUNT', '75'); # redirector processes count will started
Then click "Save" and "Apply" on squidguard to make the changes take effect.
Thanks for your reply, I had made changes as you had mentioned.
But this changes are for how many users?
As my pfsense is been installed with 2GB of RAM, and when the users are not able to browse internet via proxy that time CPU usage of PFsense is between 25 to 30 % and RAM shows arround 80% of 2GB.
But after restarting the server RAM usage moves to 30% of 2GB and internet starts working.

Thanks again

Hi,
Just now while I was posting message to u Internet had stopped working on the floor here is the screen shot i had attached of the dashboard

PFsense.jpg_thumb

Nachtfalke

You should check what process is using that much CPU time.

That the RAM usage decreases after a reboot is normal because squid is caching files in RAM and after a reboot all the files will be lost. That's normal and ok.

The redirecter processes - don't know for how manz users this will work. When checking the cachemgr.cgi of squid I can see that there are maximum 25 redirectors used. I just increased it that high because I have the RAM and so it doesn't hurt me ;-)

nearones

@Nachtfalke:

You should check what process is using that much CPU time.

That the RAM usage decreases after a reboot is normal because squid is caching files in RAM and after a reboot all the files will be lost. That's normal and ok.

The redirecter processes - don't know for how manz users this will work. When checking the cachemgr.cgi of squid I can see that there are maximum 25 redirectors used. I just increased it that high because I have the RAM and so it doesn't hurt me ;-)

Can I know how many users are working via PFsense at your place ?
And what is the pfsense server config you are using for the same

Thanks

Nachtfalke

Not more than 180 at maximum.
Around 15Mbit/s downloading from the internet.

Xeon with quad core 2.1GHz
8GB RAM with Intel Gigabit NICs.
72GB HDD 10k SAS RAID1

Most time the CPU usage is around 2-3%. Some time increases up to 10%. RAM usage around 85%. Swap space 0%.

Latest Squid2.7
Squid 40GB HDD cache size
2GB RAM for caching
Squidguard with www.shallalist.de as blacklist