Memory Leak Issue after Upgrade to 2.4

scream88

Hi

Since I upgraded my pfSense from 2.3.4 to 2.4 I got this (see Screenshot).
My memory is growing until the pfsense stops working completly.
They does not forwarding some traffic, no webGUI, no SSH … :(

Any Ideas what the cause of this issue can be?

Other pfSense installations working fine.. so I think it is something that I've configured :)

![2017-10-14 10_27_57-Observium.png](/public/imported_attachments/1/2017-10-14 10_27_57-Observium.png)
![2017-10-14 10_27_57-Observium.png_thumb](/public/imported_attachments/1/2017-10-14 10_27_57-Observium.png_thumb)

johnpoz

What packages do you have installed? What hardware is it on? Or is it vm?

Derelict

Hmm. Have not seen that in any testing. Might be a package.

Diagnostics > System Activity might show the process that is growing. Pay attention to the SIZE and RES columns.

ps axlww might show the process that is growing if that does not. Look particularly at VSZ and RSS.

scream88

It is a VM on a ESX Host.

Installed Packages:
Cron
Nut
Open-VM-Tools
Openvpn-client-export
PfBlockerNG
Squid
SquidGuard
SystemPatches

I‘m trying to monitor the process that is growing… but not so easy, when it just stops responding.

johnpoz

"PfBlockerNG"

Just my guess… We taking bets to which package it is.. ;) Only others that could really be would be squid stuff.

Derelict

@scream88:

I‘m trying to monitor the process that is growing… but not so easy, when it just stops responding.

It shouldn't stop responding until it actually uses all available memory. It should be fairly obvious what process it is at a much earlier point than that.

Grimson

Did you reinstall it using ZFS? That will also increase it's memory usage, not by too much, but enough if it was already close to the limit before.

scream88

atm it looks like it is bsnmpd.
Memory usage is growed from 64MB to 100MB in about 3h.

   0 63739     1   0  52  0 133032 84552 g_waitfo Ds    -   64:31.82 /usr/sbin/bsnmpd -c /var/etc/snmpd.conf -p /var/
   0 63739     1   0  96  0 149416 100976 -        Rs    -   154:07.87 /usr/sbin/bsnmpd -c /var/etc/snmpd.conf -p /var/

but I think I need to wait some more hours to get a better view.

No reinstall, upgraded within the WebGUI.

Don't know "pfBlockerNG" doesn't have a own process I think. But maybe I'm wrong?
Squid uses a lot of memory… but looks not really growing...
If I can't find out I will try to remove pfBlockerNG.

Derelict

It should become obvious at about the 1GB point….

scream88

Maybe I found a issue… (https://redmine.pfsense.org/issues/7868)

My VM has a "CD/DVD Drive" attached. SNMP uses constantly about 70-80% CPU if I show it under System Activity.
Now I removed the Drive from the VM and rebooted. CPU usage is now normal.. (around 0.01%).

Will report back later if this resolves the memory issue :)

Thank you all for your support :D

BBcan177

scream88

Update: It looks like the issue is solved by removing the cd/dvd drive from the VM. I will monitor it anyway and report back tomorrow.

Update2: Issue is solved for me as you can see in the attached Screenshot.

![2017-10-15 14_45_24-Observium.png](/public/imported_attachments/1/2017-10-15 14_45_24-Observium.png)
![2017-10-15 14_45_24-Observium.png_thumb](/public/imported_attachments/1/2017-10-15 14_45_24-Observium.png_thumb)

bbrendon

I have this as well. Memory related.

The system isn't 100% crashed as it is moving packets but the GUI is unresponsive and so is the console.

After digging through some history, it appears memory related.

It's an ESXi (vmxnet3) VM and the CDROM isn't connected. The system ran for about 36 hours before dying.

I do have bsnmpd running so I'm going to watch that and see if it's also the culprit. There is nothing in the logs about bsnmpd other than the package upgrade.

Does anyone know if there is a bug report for the bsnmpd memory leak yet?

pfcrash.png_thumb

MiB

@bbrendon:

I have this as well. Memory related.

The system isn't 100% crashed as it is moving packets but the GUI is unresponsive and so is the console.

After digging through some history, it appears memory related.

It's an ESXi (vmxnet3) VM and the CDROM isn't connected. The system ran for about 36 hours before dying.

I do have bsnmpd running so I'm going to watch that and see if it's also the culprit. There is nothing in the logs about bsnmpd other than the package upgrade.

Does anyone know if there is a bug report for the bsnmpd memory leak yet?

It's a known issue until the patch comes out. They're on it!
https://redmine.pfsense.org/issues/6882

For now you either have to remove the CD drive completely from the VM's hardware profile (it's not enough for the CD media to be disconnected), or uncheck Host Resources in the SNMP modules.

bbrendon

Mib- Yea. I saw that but it didn't mention a memory leak so I dismissed it. I'm guessing then it's related.

bbrendon

Ah ha! Michael Knowles has it figured out in #10.
https://redmine.pfsense.org/issues/6882#note-10

Rebuild the snmpd.conf and you're fixed.

klou

On a bare metal install, had the same issue (though I noticed it as odd CPU activity).

Slightly contrary to the notes on the Redmine ticket, I had "Host Resources" available on the GUI (and active before and after upgrade). Unchecking it at Services -> SNMP and ensuring that SNMP has restarted fixed the issue for me.