High memory use ?

Guest

How does pfSense uses memory please?

It is pending on more then only one point, that said here, but;

likes you were configure it (tunings, pimps, and so on)
like you were installing packets, the amount of packets and their memory need or plain usage will be important too
based on the turned on options, offered services and used protocols this may be very different from each
pfSense installation to another one, also on the same hardware it can be very different too!

Our home network has approx 30 clients.

What are this clients doing, using or producing for network flow or load?
As an example, mailing and surfing is not the same as streaming, game console playing and heavy downloading
bigger files and at least also through an VPN tunnel it might be then needing more the you can imagine.

The firewall has autobackup installed and pfBlockerNG installed.

The backup is not really the matter or bad case here in that game play as I see it right!
But coming to pfBlockerNG, I was reading here many different posts about, how much memory it is needing!
In case you can be really happy with 2 GB of RAM and on the other side 32 GB will be hard in usage depending
on the subscribed lists! So that can only answered by your self and not us, what you have configured to use.

It is running 2.4.1 stable. It uses 3 client VPN’s.
That’s it.

If you were configure pfSense like that;

mbuf size is on 1000000
if all network load goes through the vpn tunnel only
num.queues = not narrowed down to 1, 2 or 4 queues
highing up the send and receive buffer fo OpenVPN tunnels might be also a hidden memory user too

Soms nothing compared to the config’s I read about yet it reports that 84 % of 8 GB is in use.

If the pfBlockerNG is using many lists and many IP addreses and/or urls it might be that it comes from there.

Does FreeBSD and pfSense use all available memory in an efficient way or is this an indication of a problem ?

No wrong with it as I am right informed.

Is there a way to see what eats all this memory ?

I personally would be have a look on the following things;

mbuf size amount
num.queues amount
pfBlockerNG subscribed lists
OpenVPN send & receive buffer
DHCP and DNS entries must be stored to be able for caching so it might be also helping if you will
be setting up a static IP address for all your ~30+/- devices as well, not big but working.

Koent

Thanks guys. Based on your info, I started looking into the pfBlockerNG config.
The culprit was the experimental feature ‘TLD’ that is clearly marked beta.
I turned it on to see if it would help, and then completely forgot about it.

Memory is now back to 2 GB used, which feels normal with the lists I’ve loaded.

arch113

Didn't help my problem, before the upgrade I was only using 20% of 8 gig, now I'm using over 94% plus hitting swap file.

kejianshi

packages?

Guest

@arch113:

Didn't help my problem, before the upgrade I was only using 20% of 8 gig, now I'm using over 94% plus hitting swap file.

Providing to us some more detailed information at first, would be nice!

Likes asked before, how many and what kind of packages did you installing?
And on top of this which version of pfSense you are running?
How many NICs or LAN ports the pfSense box own or has?
Is this a VM or bare metal installation?
Any VLANs at the WAN?
Is SNMP activated?

arch113

Rebooted last night and as of right not its sitting at 16%

Qotom J1900 barebones with 8 gig of ram 128 gig ssd, 4 intel nics, 1 WAN, 1 LAN, 1 used as "DMZ"
Following packages installed:
Squid
Squidguard
Lightsquid
OpenVPN
Snort
pfBlockerNG

Home use Cable modem 400/40 connection ~30+ clients

Guest

Rebooted last night and as of right not its sitting at 16%

Rebooting mostly flush the entire DNS, DHCP and other caches so it might be based on something in that direction
as I see it.

Qotom J1900 barebones with 8 gig of ram 128 gig ssd, 4 intel nics, 1 WAN, 1 LAN, 1 used as "DMZ"

You could try out, if it turns once more again so high;

Enable PowerD (high adaptive) if not done but not really realted to this cirumstance
set the num.queues to 1, 2 or 4 for each NIC
set the mbuf size to 125000, 250000, 500000 or 1000000

Following packages installed:
Squid

Was there the base or default amount of memory high up?

OpenVPN

Was there set up the send and receive buffer higher then normal?

Snort

How many rules you where performing and setting up?

pfBlockerNG

Is DNSBL and/or TLD activated?

Home use Cable modem 400/40 connection ~30+ clients

Is this a real and pure modem or a router?

kejianshi

Squid, Snortpf, BlockerNG = The usual suspects when memory starts blowing up. You have them all running. This should be fun.

arch113

2 Days and 10 hours since last reboot, using 21-27% memory, hopefully it was just a fluke.

Answer some questions above:
Cable Modem (not router): ARRIS CM3200
PowerD is on, I didnt have it on when I was running 2.3.4
pfBlockerNG: DNSBL=on TLD=On
Snort: WAN Interface only, still messing with this, when i have them all on, it sure is block happy :)

Thats all I have for now…

Guest

now I'm using over 94% plus hitting swap file.

1st

Rebooted last night and as of right not its sitting at 16%

2nd

2 Days and 10 hours since last reboot, using 21-27% memory, hopefully it was just a fluke.

It is uprising be careful and wathing out what is coming next.

pfBlockerNG: DNSBL=on TLD=On

TLD can be eating much more pending on the used or subscribed IP addresses.

arch113

2 Days and 10 hours since last reboot, using 21-27% memory, hopefully it was just a fluke.

It is uprising be careful and wathing out what is coming next.

I expect to rise a little with squid caching turned on and Memory set to 1024, on 2.3.4 i never saw over 40%

pfBlockerNG: DNSBL=on TLD=On

TLD can be eating much more pending on the used or subscribed IP addresses.

Hasn't been a problem when I was on 2.3.4