How many users can I pass through from squidguard via pfsense



  • Hi,
    I had been using Pfsense from 2 years, but since 4 to 5 months I need to constantly restart my pfsense server.
    I had tired by just restarting squid and squidguard services, but it did not succeeded.
    My issue is suddenly internet browsing stops for end users. So if I restart my pfsense, it works for around 3 to 4 hrs.
    My current setting of cache as follows for 500+ users

    General
    ā€“--------------
    Log rotate - 30
    What to do with requests that have whitespace characters in the URI - strip

    Cache management

    Hard disk cache size - 4096
    Hard disk cache system - ufs
    Memory cache size - 1024 (Server is of 2gb RAM)
    Minimum object size - 0
    Maximum object size - 131072
    Maximum object size in RAM - 8192
    Level 1 subdirectories - 16
    Memory replacement policy - Heap GDSF
    Cache replacement policy - Heap LFUDA
    Low-water-mark in % - 90
    High-water-mark in % - 95

    Traffic Management

    Maximum download size - 5120000
    Maximum upload size - 20480
    Overall bandwidth throttling - 0
    Per-host throttling - 0
    Finish transfer if less than x KB remaining - 0
    Abort transfer if more than x KB remaining - 0
    Finish transfer if more than x % finished - 0

    Thanks & Regards,
    Prashant



  • I would say the amount of available RAM ist not the best related to the fact you at minimum are using 1Gb for squid RAM caching. The 1GB is not a maximum limit for squid - it is a minimum limit. HDD cache will need some RAM for index and so on. squidguard will need RAM, too.

    You could check this after your system is running for some hours or days and check the RAM usage and check if there is SWAP file usage. If there is SWAP file usage you should lower the squid RAM to lets say 512MB or 768MB.

    Further you could try to lower the watermarks to 85 - 90%.

    And why do you have to restart pfsense? What does not work anymore? Or is speed slow?
    Did you check under "System activity" the CPU usage? RRD graphs will also tell you the RAM and CPU usage and you should look at the times when it is slow.

    What I had in my environment ist that the number of redirectors for squid is to low. squid2 and squidguard users only 5 redirectors. I modified this in the following file:

    /usr/local/pkg/squidguard_configurator.inc
    

    Before:

    define('REDIRECTOR_PROCESS_COUNT', '5'); # redirector processes count will started
    

    After:

    define('REDIRECTOR_PROCESS_COUNT', '75'); # redirector processes count will started
    

    Then click "Save" and "Apply" on squidguard to make the changes take effect.



  • @Nachtfalke:

    I would say the amount of available RAM ist not the best related to the fact you at minimum are using 1Gb for squid RAM caching. The 1GB is not a maximum limit for squid - it is a minimum limit. HDD cache will need some RAM for index and so on. squidguard will need RAM, too.

    You could check this after your system is running for some hours or days and check the RAM usage and check if there is SWAP file usage. If there is SWAP file usage you should lower the squid RAM to lets say 512MB or 768MB.

    Further you could try to lower the watermarks to 85 - 90%.

    And why do you have to restart pfsense? What does not work anymore? Or is speed slow?
    Did you check under "System activity" the CPU usage? RRD graphs will also tell you the RAM and CPU usage and you should look at the times when it is slow.

    What I had in my environment ist that the number of redirectors for squid is to low. squid2 and squidguard users only 5 redirectors. I modified this in the following file:

    /usr/local/pkg/squidguard_configurator.inc
    

    Before:

    define('REDIRECTOR_PROCESS_COUNT', '5'); # redirector processes count will started
    

    After:

    define('REDIRECTOR_PROCESS_COUNT', '75'); # redirector processes count will started
    

    Then click "Save" and "Apply" on squidguard to make the changes take effect.

    Thanks for your reply, I had made changes as you had mentioned.
    But this changes are for how many users?
    As my pfsense is been installed with 2GB of RAM, and when the users are not able to browse internet via proxy that time CPU usage of PFsense is between 25 to 30 % and RAM shows arround 80% of 2GB.
    But after restarting the server RAM usage moves to 30% of 2GB and internet starts working.

    Thanks again



  • @nearones:

    @Nachtfalke:

    I would say the amount of available RAM ist not the best related to the fact you at minimum are using 1Gb for squid RAM caching. The 1GB is not a maximum limit for squid - it is a minimum limit. HDD cache will need some RAM for index and so on. squidguard will need RAM, too.

    You could check this after your system is running for some hours or days and check the RAM usage and check if there is SWAP file usage. If there is SWAP file usage you should lower the squid RAM to lets say 512MB or 768MB.

    Further you could try to lower the watermarks to 85 - 90%.

    And why do you have to restart pfsense? What does not work anymore? Or is speed slow?
    Did you check under "System activity" the CPU usage? RRD graphs will also tell you the RAM and CPU usage and you should look at the times when it is slow.

    What I had in my environment ist that the number of redirectors for squid is to low. squid2 and squidguard users only 5 redirectors. I modified this in the following file:

    /usr/local/pkg/squidguard_configurator.inc
    

    Before:

    define('REDIRECTOR_PROCESS_COUNT', '5'); # redirector processes count will started
    

    After:

    define('REDIRECTOR_PROCESS_COUNT', '75'); # redirector processes count will started
    

    Then click "Save" and "Apply" on squidguard to make the changes take effect.

    Thanks for your reply, I had made changes as you had mentioned.
    But this changes are for how many users?
    As my pfsense is been installed with 2GB of RAM, and when the users are not able to browse internet via proxy that time CPU usage of PFsense is between 25 to 30 % and RAM shows arround 80% of 2GB.
    But after restarting the server RAM usage moves to 30% of 2GB and internet starts working.

    Thanks again

    Hi,
    Just now while I was posting message to u Internet had stopped working on the floor here is the screen shot i had attached of the dashboard




  • You should check what process is using that much CPU time.

    That the RAM usage decreases after a reboot is normal because squid is caching files in RAM and after a reboot all the files will be lost. That's normal and ok.

    The redirecter processes - don't know for how manz users this will work. When checking the cachemgr.cgi of squid I can see that there are maximum 25 redirectors used. I just increased it that high because I have the RAM and so it doesn't hurt me ;-)



  • @Nachtfalke:

    You should check what process is using that much CPU time.

    That the RAM usage decreases after a reboot is normal because squid is caching files in RAM and after a reboot all the files will be lost. That's normal and ok.

    The redirecter processes - don't know for how manz users this will work. When checking the cachemgr.cgi of squid I can see that there are maximum 25 redirectors used. I just increased it that high because I have the RAM and so it doesn't hurt me ;-)

    Can I know how many users are working via PFsense at your place ?
    And what is the pfsense server config you are using for the same

    Thanks



  • Not more than 180 at maximum.
    Around 15Mbit/s downloading from the internet.

    Xeon with quad core 2.1GHz
    8GB RAM with Intel Gigabit NICs.
    72GB HDD 10k SAS RAID1

    Most time the CPU usage is around 2-3%. Some time increases up to 10%. RAM usage around 85%. Swap space 0%.

    Latest Squid2.7
    Squid 40GB HDD cache size
    2GB RAM for caching
    Squidguard with www.shallalist.de as blacklist