Squid use all memory ram

Impatient

I have been well pleased with 2.4 beta and so far memory usage has been much better.

Currently after 7 day's since reboot I have 4gb's out of 8gb's allocated for Squid still free.

When I installed pfSense 2.4 I used the ZFS file system and I used diskd for Squid with 128
Level 1 Directories.

The only package's I have installed is PfblockerNG,Snort on wan interface,and of course Squid.

gersonofstone

I enable into squid "debug_options all,2" and find this messages.

"clientPeekAndSpliceSSL: SSL_accept failed"

this cause used all memory RAM

bbassotti

Hello, I'v two identical pfsense 2.3.2: one with squid 3.5.19 and one with 3.5.23. The first one have no problem, the second one consume all memory.

[2.3.2-RELEASE][root@fw1]/root: squid -v
Squid Cache: Version 3.5.19

[2.3.2-RELEASE][root@fw2]/root: squid -v
Squid Cache: Version 3.5.23

the main difference between two are compile options:

3.5.23:
'build_alias=amd64-portbld-freebsd10.3'
'CC=cc' 'CPPFLAGS=-I/usr/local/include'
'CXX=c++'
'CXXFLAGS=-O2 -pipe -I/usr/local/include -I/usr/local/include -fstack-protector -DLDAP_DEPRECATED -fno-strict-aliasing -Wno-unknown-warning-option -Wno-undefined-bool-conversion -Wno-tautological-undefined-compare -Wno-dynamic-class-memaccess'
'CPP=cpp'
–enable-ltdl-convenience

3.5.19:
'--build=amd64-portbld-freebsd10.3'
'build_alias=amd64-portbld-freebsd10.3'
'CC=cc'
'CPPFLAGS=-I/usr/local/include'
'CXX=c++'
'CXXFLAGS=-O2 -pipe -I/usr/local/include -I/usr/local/include -fstack-protector -DLDAP_DEPRECATED -fno-strict-aliasing '
'CPP=cpp'
–enable-ltdl-convenience

Can I rollback to 3.5.19? if yes how? thanx.

gersonofstone

Posted by: bbassotti

your box pfsense with Squid Cache: Version 3.5.19 is filtering https?

bbassotti

@:

Posted by: bbassotti

your box pfsense with Squid Cache: Version 3.5.19 is filtering https?

yes.

emax4

Hello

Actually I have the problem of lato consumption of RAM, verify and is the Squid. Restarting the service returns to normal.

Ttengo installed Pfsense 2.3.2 with Squid 3.5.23

I could not solve the problem.

Any solution?

moodian

Hi guys.

Me too…
Actually, I have the same problem...
I think is the MITM.
I work in a college and we are using pfsense on the latest version, with squid 3.5.23.
Our server has 8GB of RAM and my impression is that, when my network is set without MITM filtering, the consuming of RAM is around 15%. If I activate MITM filtering for SSL package interception, the consuming of RAM slowly grows up to 100% and our PFsense system goes down.
I've tried to change the settings of Local Cache, but I haven't found any conclusive results.
If I restart the squid service the consuming of RAM decreases.

Any idea?
Thank you for help.

emax4

Where is the MITM configuration?

moodian

Hello

SSL MITM is a acronym for SSL Man In The Midle Filtering.
It is when we enable SSL filtering for the PFSense analise the HTTPS traffic beyond HTTP.

Look MITM configuration in: Services > Squid Proxy Server > General > SSL Man In the Middle Filtering

A provisory solution that I found was to create 2 cron jobs. The first to stop and the second for start the squid 10 seconds after stop.

For exemple:
30 * * * * root /usr/local/sbin/squid -k shutdown
30 * * * * root sleep 10 && /usr/local/sbin/squid

emax4

Hello…

Ummmm .... yes, the problem is that if I disable the HTTPS / SSL Interception, the squidguard will not filter me the sites with ssl (https) certificate.

On the other hand, it is interesting to enable the cron that you indicate me.

I'll try those cron

miquim

I have this problem too!
any one has a update or downgread do works version?

Tks!

emax4

No one has solved the problem?

gersonofstone

I still have the problem

miquim

Me too,

i just configure the cron for evey 30 min stop and start the squid service.

You can install the Cron package and there you can add 2 news jobs

*/30 * * * * root /usr/local/sbin/squid -k shutdown
*/30 * * * * root sleep 10 && /usr/local/sbin/squid

its work for me but is not the best solution.

remzej

The swapstate_check.php won't execute because there is no cache partition mounted found in the filesystem. I checked the source code line by line and by creating a test.php from there I can tell that it will never execute because of the conditions doesn't meet. We can modify the swapstate_check.php to monitor the swap.state file size and clean the cache if this file exceeds the specified amount we set in the script.

emax4

Yes, for now I think the best solution is to use a cron to stop and knit the squid, as says miquim. I think we have to wait for an update to see if they solve the problem. If someone finds the solution, please advise

remzej

It's hard to monitor the swap.state filesize and the SWAP usage percentage because it grows dynamically. In my case I modified the swapstate_check.php code to execute if the cache folder size reach 250MB or the swap.state filesize reach 640KB.

TomS

I'm also seeing this memory exhaustion problem on my box. I do not use any disk cache and only want to be prepared to be able to block web sites just in case I'm told to do so by government authorities or law enforcement. Currently, I'm not blocking any websites. One more thing I'm currently doing with Squid is creating log files (i.e. which web sites have been opened by the users)
The system is an anonymous WiFi hotspot with approx. 300 users per day.
For some reason, the memory consumption is slowly increasing. At first only the memory itself, but later also the swap space until all the free memory is occupied and the pfSense is crashing.

I saw bbassotti's post and the configure differences in the squid versions… I'm using squid V 3.5.24 which has exactly the same configure options as V3.5.23 and it shows the same behavior... Could this be the root cause?
What else could be the reason for this strange behavior?
Is there anyone working on this issue? If there's a way I can contribute or help finding and fixing this issue, please just let me know.

Best,
TomS

remzej

@TomS:

I'm also seeing this memory exhaustion problem on my box. I do not use any disk cache and only want to be prepared to be able to block web sites just in case I'm told to do so by government authorities or law enforcement. Currently, I'm not blocking any websites. One more thing I'm currently doing with Squid is creating log files (i.e. which web sites have been opened by the users)
The system is an anonymous WiFi hotspot with approx. 300 users per day.
For some reason, the memory consumption is slowly increasing. At first only the memory itself, but later also the swap space until all the free memory is occupied and the pfSense is crashing.

I saw bbassotti's post and the configure differences in the squid versions… I'm using squid V 3.5.24 which has exactly the same configure options as V3.5.23 and it shows the same behavior... Could this be the root cause?
What else could be the reason for this strange behavior?
Is there anyone working on this issue? If there's a way I can contribute or help finding and fixing this issue, please just let me know.

Best,
TomS

This was posted as a bug for pfSense v2.3.x I read someone posted that this issue doesn't exist in pfSense v2.4 beta. We hope they will release an update soon to fix this problem.

emax4

@remzej:

@TomS:

I'm also seeing this memory exhaustion problem on my box. I do not use any disk cache and only want to be prepared to be able to block web sites just in case I'm told to do so by government authorities or law enforcement. Currently, I'm not blocking any websites. One more thing I'm currently doing with Squid is creating log files (i.e. which web sites have been opened by the users)
The system is an anonymous WiFi hotspot with approx. 300 users per day.
For some reason, the memory consumption is slowly increasing. At first only the memory itself, but later also the swap space until all the free memory is occupied and the pfSense is crashing.

I saw bbassotti's post and the configure differences in the squid versions… I'm using squid V 3.5.24 which has exactly the same configure options as V3.5.23 and it shows the same behavior... Could this be the root cause?
What else could be the reason for this strange behavior?
Is there anyone working on this issue? If there's a way I can contribute or help finding and fixing this issue, please just let me know.

Best,
TomS

This was posted as a bug for pfSense v2.3.x I read someone posted that this issue doesn't exist in pfSense v2.4 beta. We hope they will release an update soon to fix this problem.

Hopefully they will soon solve the problem, if anyone knows anything please let me know