Problem with Squid + HTTPS/SSL interception consuming all memory

alesilvam

Hi Guys,

The pfSense of my work is reaching 99% of its memory consumption, at the beginning of the day the memory consumption is 5...6%, the consumption starts to increase during the day, until the half of the day is already at 45...50% and after a few minutes shoot 55...60...70...85...99% in a matter of seconds, if you leave the pfSense collapses and crashes.

For this problem, the squid is the service that is consuming more memory. I have noticed with the tests that the villain is related with the option "HTTPS/SSL Interception: Enable SSL filtering" + "SSL/MITM Mode: Splice All", I leaved this option disabled for 2 days for testing and the memory consumption was stable between 5...10%.

One way to avoid the crash is to click "Clear Disk Cache NOW" from the menu Services / Squid Proxy Server / Local Cache / Squid Hard Disk Cache Settings, with that memory consumption returns to its normal.

I don't know what else to do, i tried several suggestions of cache settings and nothing solves it, also tried to do a new installation of pfSense and did not solve the problem.

SETTINGS:

pfSense 2.4.4-RELEASE (amd64)
squid 0.4.44_6
squidGuard 1.16.18_1
Lightsquid 3.0.6_4

CPU Type:
Intel(R) Core(TM) i5-7400 CPU @ 3.00GHz
Current: 3000 MHz, Max: 3001 MHz
4 CPUs: 1 package(s) x 4 core(s)
AES-NI CPU Crypto: Yes (inactive)

Memory: 16GB
Hard Disk: 1TB

SQUID PROXY SERVER
Transparent HTTP Proxy: Enable
HTTPS/SSL Interception: Enable SSL filtering
SSL/MITM Mode: Splice All

LOCAL CACHE
== Squid Cache General Settings ==
Cache Replacement Policy: Heap LFUDA
Low-Water Mark in %: 60
High-Water Mark in %: 65

== Squid Hard Disk Cache Settings ==
Hard Disk Cache Size: 100
Hard Disk Cache System: ufs
Level 1 Directories: 16
Minimum Object Size: 0
Maximum Object Size: 4

== Squid Memory Cache Settings ==
Memory Cache Size: 64
Maximum Object Size in RAM: 256
Memory Replacement Policy: Heap GDSF

Can anyone help me, please?

GixD

same issue, its solved?

alesilvam

Unfortunately no, we are thinking of replacing pfSense.
An interim solution was to install the Cron package and set up to restart the Squid service 5 times during the day:

/usr/local/etc/rc.d/squid.sh restart

GixD

i see, im try to disable access.log on webconfig and its helped for decreasing ram for this issue.

alesilvam

I disabled the log of Squid and Squidguard by Webconfig and in my case I noticed little difference in memory consumption.

atom1983

I have the same problem as this problem and I have no chance to help

GixD

@atom1983 lets pray togther

beauw

All..... The script came from user Remzej. I have it on a cron job to check every 5 minutes (we are a busy proxy environment)...

*/2 * * * * root /usr/bin/nice -n20 /usr/local/bin/php-cgi -f /usr/local/pkg/monitor_memory_usage.php

#!/usr/local/bin/php-cgi -f
<?php
/*

monitor_memory_usage.php
part of pfSense (https://www.pfsense.org)
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
require_once('config.inc');
require_once('util.inc');
require_once('squid.inc');
global $config;

// Monitor memory usage by remzej
// Get SWAP usage funtion
function swap_usage() {
exec("/usr/sbin/swapinfo", $swap_info);
$swap_used = "";
foreach ($swap_info as $line) {
if (preg_match('/(\d+)%$/', $line, $matches)) {
$swap_used = $matches[1];
break;
}
}
return $swap_used;
}

// Get memory usage function
function mem_usage() {
$memory = "";
exec("/sbin/sysctl -n vm.stats.vm.v_page_count vm.stats.vm.v_inactive_count " .
"vm.stats.vm.v_cache_count vm.stats.vm.v_free_count", $memory);
```
 $totalMem = $memory[0];
 $availMem = $memory[1] + $memory[2] + $memory[3];
 $usedMem = $totalMem - $availMem;
 $memUsage = round(($usedMem * 100) / $totalMem, 0);

 return $memUsage;
```
}

// Get memory and SWAP usage value
$memusage_pct = mem_usage();
$swapusage_pct = swap_usage();

// Display memory usage
echo "Memory Usage: " . $memusage_pct . "%" . PHP_EOL;
echo "SWAP Usage: " . $swapusage_pct . "%" . PHP_EOL;

// If memory usage is above 90%, stop and restart squid services.
if (($memusage_pct > 90) or ($swapusage_pct > 80)) {
squid_stop_monitor();
if (is_service_running('squid')) {
stop_service("squid");
}
squid_restart_services();
log_error(gettext(sprintf("[squid] Memory usage is $memusage_pct percent, Swap Usage is $swap_usage percent, stopping and restarting services.")));
}
log_error(gettext(sprintf("[squid] Memory usage is $memusage_pct percent and Swap Usage is $swapusage_pct")));
?>