Change in memory management/usage?
-
When a FreeBSD based system runs out of memory, the kernel can move sleeping or inactive processes into swap area. A dedicated Swap partition goes a long way to avoid system freeze but if you notice you are running out of RAM or your applications are consuming too much of it then you may want to own a swap partition that is let us call it buffering down that behaviour.
Many years ago, the rule of thumb for the amount of swap space that should be allocated was 2X the amount of RAM installed in the computer.
I have on board soldered RAM of 4 GB and pfSense was
using nearly 67 % to over 90 % of it, now I am on something around 22% RAM / 24% swap on 23.05 RC
and 18% RAM / 24% Swap on 2.7 Devel now it can be
used (the free RAM) for other things, such mbuf size,
queues amount and size and others. -
@dobby_ So have you noticed any change in swap usage between 23.01 and 23.05 RC?
Are there any kernel parameters in FreeBSD, that control the "sensitivity" to start swapping?
-
@pfsjap said in Change in memory management/usage?:
@dobby_ So have you noticed any change in swap usage between 23.01 and 23.05 RC?
Something about my situation, I have two boxes with
onboard soldered RAM, so I am not able to change it
it if needed to a greater size by installing a greater RAM
module with let us say 8 GB or 16 GB like other users
will be able to do. And so it comes that I am watching
the RAM and Swap usage a little bit more then others.In former days together with pfSense CE 2.6 and pfSense+ 23.01 if I updated or upgraded something and must reboot the hardware I saw at a fresh boot up where
all the snort rules, pfBlocker-NG feeds and the ClamAV
av signatures were fresh loaded (common) that the RAM
were spiking out at something between 60% to over 90%
and the entire swap partition was not in usage. But now on
pfSense 23.05 RC (APU6B4) and pfSense 2.7 Devel
(APU4D4) the entire system will be not like before in 2 - 4 days needing to level off, so that swap and ram will be used together much faster and more let us call it "weighted" and this in or after only some hours.
Please see the added pictures belowpfSense+ (Plus) 23.05 RC
Fresh upgraded and rules, feeds and signatures will be fresh loaded
pfSense 2.7 Devel
After some hours it is really fast level off to use both
Swap and RAM like shown below
Are there any kernel parameters in FreeBSD, that control the "sensitivity" to start swapping?
I can`t anything about that, but the reaction time and
also the art and wise of taking advantage from the
available swap in the system is going more fast and moderate as I was recognizing. And for sure with
onboard soldered ram I will be much more happy
then perhaps others with that behaviour! -
I have 8GB RAM and so far have never seen my system use swap, though not running pfblocker etc.
After upgrades, the memory usage jumps to somewhere between 25-35%, but after a second reboot it goes back to the normal 6-7%pfSense Plus 24.03-RELEASE (amd64)
Dell Optiplex 7040 SFF
Core i5-6500, 8GB RAM, 2x 240GB SSD (ZFS Mirror)
HPE 561T (X540-AT2), 2-port 10Gb RJ45
HPE 562SFP+ (X710-DA2), 2-port 10Gb SFP+ -
@pfsjap I agree with the above and that swap usage like you are experiencing is not normal, especially with 8GB on Netgate hardware.
I also run a 6100 Max and I guess you are more familiar with the memory use shown on mine:
Mem: 125M Active, 398M Inact, 622M Wired, 40K Buf, 6635M Free ARC: 223M Total, 48M MFU, 167M MRU, 1490K Header, 6081K Other 191M Compressed, 518M Uncompressed, 2.72:1 Ratio Swap: 1024M Total, 1024M FreeThere has to be a ripple in that particular load, which is good data for Netgate. Did you try a full restart from cold and log those details too?
️ -
@robbiett Swap usage starts at 0% after restart, but increases later on. I haven't taken up any figures, but after restart memory usage might be 40-50%, but then goes down. Upgraded earlier today to 23.05.r.20230521.0305 and now memory usage is 23% and swap usage 2%.
I have these packages installed:
pfBlocker has 23 IP and DNSBL groups configured. IPS policy is set to Security in Snort with 22 categories enabled in ET Open Rules.
-
@pfsjap said in Change in memory management/usage?:
parameters in FreeBSD
See https://redmine.pfsense.org/issues/14030 and
https://forum.netgate.com/topic/177886/23-1-using-more-ramWhen disk intensive activities occur (such as software update), the zfs file system cache increases. The cache will be released if required but not immediately forcing use of swap. The solution is to limit the cache size down from the very generous default.
-
@patch In 23.01 I had set vfs.zfs.arc.max to 419430400. With 23.05 RC I had already decreased that and at the moment it's 400000.
-
@pfsjap Could you run
top -aS -o resfor us so we can look at how the memory is being used?️ -
@robbiett Sure, looks like Snort is using a lot of memory:
last pid: 85727; load averages: 0.25, 0.55, 0.47 up 0+09:12:12 18:28:07 93 processes: 2 running, 89 sleeping, 2 waiting CPU: 0.1% user, 0.2% nice, 0.1% system, 0.0% interrupt, 99.6% idle Mem: 1520M Active, 2240M Inact, 124M Laundry, 756M Wired, 3148M Free ARC: 257M Total, 164M MFU, 83M MRU, 920K Anon, 2009K Header, 7051K Other 223M Compressed, 546M Uncompressed, 2.44:1 Ratio Swap: 2048M Total, 43M Used, 2005M Free, 2% Inuse PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 83927 root 3 52 20 1421M 1079M bpf 3 1:16 0.23% /usr/local/bin/snort -R _34675 -D -q --suppress-config-log --daq pcap --daq-mode passive --treat-drop-as-alert - 13140 root 2 52 20 1421M 1063M bpf 0 0:23 0.05% /usr/local/bin/snort -R _8486 -D -q --suppress-config-log --daq pcap --daq-mode passive --treat-drop-as-alert -l 6934 unbound 4 20 0 822M 756M kqread 3 0:11 0.26% /usr/local/sbin/unbound -c /var/unbound/unbound.conf 28523 root 1 68 0 147M 69M accept 0 0:26 0.00% php-fpm: pool nginx (php-fpm) 72038 root 1 44 0 153M 65M accept 3 0:43 0.00% php-fpm: pool nginx (php-fpm) 1040 root 1 68 0 147M 62M accept 0 0:48 0.00% php-fpm: pool nginx (php-fpm) 1041 root 1 68 0 147M 55M accept 0 0:38 0.00% php-fpm: pool nginx (php-fpm) 85623 root 1 20 0 47M 37M bpf 0 0:00 0.00% /usr/local/sbin/arpwatch -Z -f /usr/local/arpwatch/arp_igc2.dat -i igc2 -w digger909@hotmail.com 85048 root 1 20 0 47M 37M bpf 2 0:00 0.00% /usr/local/sbin/arpwatch -Z -f /usr/local/arpwatch/arp_igc0.dat -i igc0 -w digger909@hotmail.com 93605 root 1 20 0 72M 36M piperd 3 0:01 0.01% /usr/local/bin/php_pfb -f /usr/local/pkg/pfblockerng/pfblockerng.inc filterlog 1039 root 1 20 0 112M 17M kqread 1 0:01 0.00% php-fpm: master process (/usr/local/lib/php-fpm.conf) (php-fpm) 3716 dhcpd 1 20 0 25M 10M select 0 0:01 0.01% /usr/local/sbin/dhcpd -user dhcpd -group _dhcp -chroot /var/dhcpd -cf /etc/dhcpd.conf -pf /var/run/dhcpd.pid igc -
@mvikman said in Change in memory management/usage?:
I have 8GB RAM and so far have never seen my system use swap, though not running pfblocker etc.
I run something nearly a fully UTM and you run a
pure firewall there is for sure a difference as I see it.- Squid & SquidGuard (DNSBL)
- pfBlocker-NG (Feeds)
- ClamAV (AV Signatures)
- Snort (Rules)
This will be fresh load after a reboot, and in my eyes there
is nothing wrong with and so I am happy that swap will be used in such cases.After upgrades, the memory usage jumps to somewhere between 25-35%,
But from your 8 GB will 35% are 2.8 GB and this is nearly
70% of my RAM you have forgotten in your counting.but after a second reboot it goes back to the normal 6-7%
Only after several hours here, look at the picture from today morning (11 hours) and from now (below) all is balanced fine.
-
The original poster asked about swap usage / change in swap usage from 23.01 to 23.05RC, also didn't specify what packages were running in system.
I just provided my experience with my system, which is that I have never seen swap usage in it.
I haven't bothered to change the ZFS ARC cache limits because it hasn't been a problem in my system.Of course you can directly compare systems with 4GB and 8GB RAM.
-
All of this (and more) is covered in recent docs updates:
https://docs.netgate.com/pfsense/en/latest/hardware/memory.html
