Squid use all memory ram

[remzej]

I want to share my simple PHP code to monitor the memory and SWAP usage every 5 minutes using cron.

This PHP code will automatically stop and restart squid services when memory and SWAP usage goes beyond 90% and 75% respectively without deleting the existing hard disk cache and swap.state file. The swapstate_check.php script will handle it if swap.state file goes beyond 1GB size.

Attached here is the monitor_memory_usage.txt file you can download and save to your preferred location on your pfSense box. Don't forget to change the file extention from .txt to .php.

In my case I saved it in /usr/local/pkg/ directory. Don't forget to set the file permission to rwxr-xr-r.

Next thing to do, is to add a cron job that will look like this: (Be sure to install cron package to be able to do this)

*/5 * * * * root /usr/local/pkg/monitor_memory_usage.php

then save it. That's it! I hope this simple hard work and research will help some of you that has high memory usage problem.

Thanks!
remzej

monitor_memory_usage.txt

[marcelloc]

Thanks for the contribution. :)

[gersonofstone]

@marcelloc:

Thanks for the contribution. :)

this change was in last update for squid?

[marcelloc]

@:

this change was in last update for squid?

Not sure, it was updated recently. Did you updated the package?

[gersonofstone]

Update squid to 1.16.2

And make changes

• SystemAdvancedFirewall & NAT
  Firewall Optimization Options Agrresive

• IP Do-Not-Fragment compatibility
  Clear invalid DF bits instead of dropping the packets

*Disable Firewall Scrub
Disables the PF scrubbing option which can sometimes interfere with NFS traffic.

• SystemAdvancedNetworking

Hardware Checksum Offloading *Disable
Hardware TCP Segmentation Offloading *Disable
Hardware Large Receive Offloading *Disable

this for NIC realteck

:)

[remzej]

Latest squid package v0.4.37 already fixed this bug.

[Gravitator]

Hello!

Version PFSense:

2.3.4-RELEASE (i386)
built on Wed May 03 15:22:11 CDT 2017
FreeBSD 10.3-RELEASE-p19

Latest squid package v0.4.37 with SquidGuard 1.16.2 in transparent mode with SSL interception (strip all).

Gradually increasing memory consumption, the overflow resets.

@:

Update squid to 1.16.2

And make changes

• SystemAdvancedFirewall & NAT
  Firewall Optimization Options Agrresive

• IP Do-Not-Fragment compatibility
  Clear invalid DF bits instead of dropping the packets

*Disable Firewall Scrub
Disables the PF scrubbing option which can sometimes interfere with NFS traffic.

• SystemAdvancedNetworking

Hardware Checksum Offloading *Disable
Hardware TCP Segmentation Offloading *Disable
Hardware Large Receive Offloading *Disable

this for NIC realteck

:)

These settings did not help.

Will the transition to x64?

[emax4]

Hello gravitator

With the update of Squid is enough for the solution of the problem, greetings!

[Gravitator]

@emax4:

Hello gravitator

With the update of Squid is enough for the solution of the problem, greetings!

Hello…

Unfortunately, you have the latest version of package squid - 0.4.37. The problem is not resolved.
Pfsense is also updated to the version from 20.07.2017.

[emax4]

@Gravitator:

@emax4:

Hello gravitator

With the update of Squid is enough for the solution of the problem, greetings!

Hello…

Unfortunately, you have the latest version of package squid - 0.4.37. The problem is not resolved.
Pfsense is also updated to the version from 20.07.2017.

Hello

I have version 0.4.37 of squid and 2.3.4 of Pfsense, I no longer have the problem.

With the update resolved

[Sko]

@Impatient:

I have had the issue since pfSense version 2.3 and the only way to help alleviate the issue was to set the Maximum Object size in Ram back to 256 kb.

Thank you, this helped me a lot. I was fiddling with squid settings, but it always ate all the RAM eventually. Setting 'Maximum Object Size in RAM' to default stopped it.

pfSense 2.3.4_p1 (amd64)
squid 0.4.37

[Gravitator]

Hello!
Рroblem solved!

This problem is observed only on i386. On amd-64 squid is working fine. Memory consumption was normal.

PFSense 2.3.4-1 (x64).
Squid packages 0.4.37.

[doktornotor]

If you are seeing this with HTTPS, see http://bugs.squid-cache.org/show_bug.cgi?id=4005

No fix for 3.5.x. Complain upstream.

Note: Setting "SSL Certificate Deamon Children" to 1 (or just something lower that the default 5) should help due to the way how dynamic_cert_mem_cache_size is configured by the package, but obviously will cause issues with busy proxies.

[beauw]

Okay - remzej's script saved me so much grief.

I run PFsense with about 1200 clients and the squid memory continually increases over the day until it locks up with no reboot.  Even on the newest versions.

I adjusted the script to reboot only on memory (65% for me) and it works like a charm.

Once I got the cron running, it checks every 15 mins.  At the most, the reboot results in 15 seconds downtime for the network which is palatable.

Thanks again remzej for your contribution….

[gersonofstone]

@beauw:

Okay - remzej's script saved me so much grief.

I run PFsense with about 1200 clients and the squid memory continually increases over the day until it locks up with no reboot.  Even on the newest versions.

I adjusted the script to reboot only on memory (65% for me) and it works like a charm.

Once I got the cron running, it checks every 15 mins.  At the most, the reboot results in 15 seconds downtime for the network which is palatable.

Thanks again remzej for your contribution….

Do You have rule this similar it?