• Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login
Netgate Discussion Forum
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login

Change in memory management/usage?

Scheduled Pinned Locked Moved Plus 23.05 Development Snapshots (Retired)
14 Posts 6 Posters 2.0k Views
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • P
    pfsjap
    last edited by May 19, 2023, 8:49 PM

    23.05RC seems to use some swap space in Netgate 6100MAX while 23.01 with the same configuration did not. It may not be aggressive usage, but I would rather not have it use swap at all.

    Is this swap usage a symptom of some memory management problem in some service? This is after upgrading to 23.05.r.20230519.0600:

    c7a3dc78-62e7-46b4-9551-7450795e1402-image.png

    874ae4e9-e664-49ec-acf1-afe159db889a-image.png

    R 1 Reply Last reply May 21, 2023, 11:02 AM Reply Quote 0
    • D
      Dobby_
      last edited by May 20, 2023, 5:55 AM

      When a FreeBSD based system runs out of memory, the kernel can move sleeping or inactive processes into swap area. A dedicated Swap partition goes a long way to avoid system freeze but if you notice you are running out of RAM or your applications are consuming too much of it then you may want to own a swap partition that is let us call it buffering down that behaviour.

      Many years ago, the rule of thumb for the amount of swap space that should be allocated was 2X the amount of RAM installed in the computer.

      I have on board soldered RAM of 4 GB and pfSense was
      using nearly 67 % to over 90 % of it, now I am on something around 22% RAM / 24% swap on 23.05 RC
      and 18% RAM / 24% Swap on 2.7 Devel now it can be
      used (the free RAM) for other things, such mbuf size,
      queues amount and size and others.

      #~. @Dobby

      Turris Omnia - 4 Ports - 2 GB RAM / TurrisOS 7 Release (Btrfs)
      PC Engines APU4D4 - 4 Ports - 4 GB RAM / pfSense CE 2.7.2 Release (ZFS)
      PC Engines APU6B4 - 4 Ports - 4 GB RAM / pfSense+ (Plus) 24.03_1 Release (ZFS)

      P 1 Reply Last reply May 21, 2023, 6:12 AM Reply Quote 0
      • P
        pfsjap @Dobby_
        last edited by May 21, 2023, 6:12 AM

        @dobby_ So have you noticed any change in swap usage between 23.01 and 23.05 RC?

        Are there any kernel parameters in FreeBSD, that control the "sensitivity" to start swapping?

        D P 2 Replies Last reply May 21, 2023, 8:20 AM Reply Quote 0
        • D
          Dobby_ @pfsjap
          last edited by May 21, 2023, 8:20 AM

          @pfsjap said in Change in memory management/usage?:

          @dobby_ So have you noticed any change in swap usage between 23.01 and 23.05 RC?

          Something about my situation, I have two boxes with
          onboard soldered RAM, so I am not able to change it
          it if needed to a greater size by installing a greater RAM
          module with let us say 8 GB or 16 GB like other users
          will be able to do. And so it comes that I am watching
          the RAM and Swap usage a little bit more then others.

          In former days together with pfSense CE 2.6 and pfSense+ 23.01 if I updated or upgraded something and must reboot the hardware I saw at a fresh boot up where
          all the snort rules, pfBlocker-NG feeds and the ClamAV
          av signatures were fresh loaded (common) that the RAM
          were spiking out at something between 60% to over 90%
          and the entire swap partition was not in usage. But now on
          pfSense 23.05 RC (APU6B4) and pfSense 2.7 Devel
          (APU4D4) the entire system will be not like before in 2 - 4 days needing to level off, so that swap and ram will be used together much faster and more let us call it "weighted" and this in or after only some hours.
          Please see the added pictures below

          pfSense+ (Plus) 23.05 RC
          Fresh upgraded and rules, feeds and signatures will be fresh loaded
          23.05RC swap ram.jpg

          pfSense 2.7 Devel
          After some hours it is really fast level off to use both
          Swap and RAM like shown below

          2.7 Devel latest swap ram.jpg

          Are there any kernel parameters in FreeBSD, that control the "sensitivity" to start swapping?

          I can`t anything about that, but the reaction time and
          also the art and wise of taking advantage from the
          available swap in the system is going more fast and moderate as I was recognizing. And for sure with
          onboard soldered ram I will be much more happy
          then perhaps others with that behaviour!

          #~. @Dobby

          Turris Omnia - 4 Ports - 2 GB RAM / TurrisOS 7 Release (Btrfs)
          PC Engines APU4D4 - 4 Ports - 4 GB RAM / pfSense CE 2.7.2 Release (ZFS)
          PC Engines APU6B4 - 4 Ports - 4 GB RAM / pfSense+ (Plus) 24.03_1 Release (ZFS)

          1 Reply Last reply Reply Quote 0
          • M
            mvikman
            last edited by May 21, 2023, 9:59 AM

            I have 8GB RAM and so far have never seen my system use swap, though not running pfblocker etc.
            After upgrades, the memory usage jumps to somewhere between 25-35%, but after a second reboot it goes back to the normal 6-7%

            pfSense Plus 24.11-RELEASE (amd64)
            Dell Optiplex 7040 SFF
            Core i5-6500, 8GB RAM, 2x 240GB SSD (ZFS Mirror)
            HPE 561T (X540-AT2), 2-port 10Gb RJ45
            HPE 562SFP+ (X710-DA2), 2-port 10Gb SFP+

            D 1 Reply Last reply May 21, 2023, 7:12 PM Reply Quote 0
            • R
              RobbieTT @pfsjap
              last edited by May 21, 2023, 11:02 AM

              @pfsjap I agree with the above and that swap usage like you are experiencing is not normal, especially with 8GB on Netgate hardware.

              I also run a 6100 Max and I guess you are more familiar with the memory use shown on mine:

              Mem: 125M Active, 398M Inact, 622M Wired, 40K Buf, 6635M Free
              ARC: 223M Total, 48M MFU, 167M MRU, 1490K Header, 6081K Other
                   191M Compressed, 518M Uncompressed, 2.72:1 Ratio
              Swap: 1024M Total, 1024M Free
              

              There has to be a ripple in that particular load, which is good data for Netgate. Did you try a full restart from cold and log those details too?

              ☕️

              P 1 Reply Last reply May 21, 2023, 12:26 PM Reply Quote 0
              • P
                pfsjap @RobbieTT
                last edited by May 21, 2023, 12:26 PM

                @robbiett Swap usage starts at 0% after restart, but increases later on. I haven't taken up any figures, but after restart memory usage might be 40-50%, but then goes down. Upgraded earlier today to 23.05.r.20230521.0305 and now memory usage is 23% and swap usage 2%.

                I have these packages installed:
                ec7ac3e8-e5a9-4058-bd68-e4f60251c254-image.png

                pfBlocker has 23 IP and DNSBL groups configured. IPS policy is set to Security in Snort with 22 categories enabled in ET Open Rules.

                1 Reply Last reply Reply Quote 0
                • P
                  Patch @pfsjap
                  last edited by Patch May 21, 2023, 12:32 PM May 21, 2023, 12:29 PM

                  @pfsjap said in Change in memory management/usage?:

                  parameters in FreeBSD

                  See https://redmine.pfsense.org/issues/14030 and
                  https://forum.netgate.com/topic/177886/23-1-using-more-ram

                  When disk intensive activities occur (such as software update), the zfs file system cache increases. The cache will be released if required but not immediately forcing use of swap. The solution is to limit the cache size down from the very generous default.

                  P 1 Reply Last reply May 21, 2023, 12:45 PM Reply Quote 0
                  • P
                    pfsjap @Patch
                    last edited by May 21, 2023, 12:45 PM

                    @patch In 23.01 I had set vfs.zfs.arc.max to 419430400. With 23.05 RC I had already decreased that and at the moment it's 400000.

                    R 1 Reply Last reply May 21, 2023, 12:54 PM Reply Quote 0
                    • R
                      RobbieTT @pfsjap
                      last edited by RobbieTT May 21, 2023, 12:55 PM May 21, 2023, 12:54 PM

                      @pfsjap Could you run top -aS -o res for us so we can look at how the memory is being used?

                      ☕️

                      P 1 Reply Last reply May 21, 2023, 3:31 PM Reply Quote 0
                      • P
                        pfsjap @RobbieTT
                        last edited by May 21, 2023, 3:31 PM

                        @robbiett Sure, looks like Snort is using a lot of memory:

                        last pid: 85727;  load averages:  0.25,  0.55,  0.47                                                                                                            up 0+09:12:12  18:28:07
                        93 processes:  2 running, 89 sleeping, 2 waiting
                        CPU:  0.1% user,  0.2% nice,  0.1% system,  0.0% interrupt, 99.6% idle
                        Mem: 1520M Active, 2240M Inact, 124M Laundry, 756M Wired, 3148M Free
                        ARC: 257M Total, 164M MFU, 83M MRU, 920K Anon, 2009K Header, 7051K Other
                             223M Compressed, 546M Uncompressed, 2.44:1 Ratio
                        Swap: 2048M Total, 43M Used, 2005M Free, 2% Inuse
                        
                          PID USERNAME    THR PRI NICE   SIZE    RES STATE    C   TIME    WCPU COMMAND
                        83927 root          3  52   20  1421M  1079M bpf      3   1:16   0.23% /usr/local/bin/snort -R _34675 -D -q --suppress-config-log --daq pcap --daq-mode passive --treat-drop-as-alert -
                        13140 root          2  52   20  1421M  1063M bpf      0   0:23   0.05% /usr/local/bin/snort -R _8486 -D -q --suppress-config-log --daq pcap --daq-mode passive --treat-drop-as-alert -l
                         6934 unbound       4  20    0   822M   756M kqread   3   0:11   0.26% /usr/local/sbin/unbound -c /var/unbound/unbound.conf
                        28523 root          1  68    0   147M    69M accept   0   0:26   0.00% php-fpm: pool nginx (php-fpm)
                        72038 root          1  44    0   153M    65M accept   3   0:43   0.00% php-fpm: pool nginx (php-fpm)
                         1040 root          1  68    0   147M    62M accept   0   0:48   0.00% php-fpm: pool nginx (php-fpm)
                         1041 root          1  68    0   147M    55M accept   0   0:38   0.00% php-fpm: pool nginx (php-fpm)
                        85623 root          1  20    0    47M    37M bpf      0   0:00   0.00% /usr/local/sbin/arpwatch -Z -f /usr/local/arpwatch/arp_igc2.dat -i igc2 -w digger909@hotmail.com
                        85048 root          1  20    0    47M    37M bpf      2   0:00   0.00% /usr/local/sbin/arpwatch -Z -f /usr/local/arpwatch/arp_igc0.dat -i igc0 -w digger909@hotmail.com
                        93605 root          1  20    0    72M    36M piperd   3   0:01   0.01% /usr/local/bin/php_pfb -f /usr/local/pkg/pfblockerng/pfblockerng.inc filterlog
                         1039 root          1  20    0   112M    17M kqread   1   0:01   0.00% php-fpm: master process (/usr/local/lib/php-fpm.conf) (php-fpm)
                         3716 dhcpd         1  20    0    25M    10M select   0   0:01   0.01% /usr/local/sbin/dhcpd -user dhcpd -group _dhcp -chroot /var/dhcpd -cf /etc/dhcpd.conf -pf /var/run/dhcpd.pid igc
                        
                        
                        1 Reply Last reply Reply Quote 0
                        • D
                          Dobby_ @mvikman
                          last edited by May 21, 2023, 7:12 PM

                          @mvikman said in Change in memory management/usage?:

                          I have 8GB RAM and so far have never seen my system use swap, though not running pfblocker etc.

                          I run something nearly a fully UTM and you run a
                          pure firewall there is for sure a difference as I see it.

                          • Squid & SquidGuard (DNSBL)
                          • pfBlocker-NG (Feeds)
                          • ClamAV (AV Signatures)
                          • Snort (Rules)

                          This will be fresh load after a reboot, and in my eyes there
                          is nothing wrong with and so I am happy that swap will be used in such cases.

                          After upgrades, the memory usage jumps to somewhere between 25-35%,

                          But from your 8 GB will 35% are 2.8 GB and this is nearly
                          70% of my RAM you have forgotten in your counting.

                          but after a second reboot it goes back to the normal 6-7%

                          Only after several hours here, look at the picture from today morning (11 hours) and from now (below) all is balanced fine.

                          pfSense 23.05 RC swap ram 2.jpg

                          #~. @Dobby

                          Turris Omnia - 4 Ports - 2 GB RAM / TurrisOS 7 Release (Btrfs)
                          PC Engines APU4D4 - 4 Ports - 4 GB RAM / pfSense CE 2.7.2 Release (ZFS)
                          PC Engines APU6B4 - 4 Ports - 4 GB RAM / pfSense+ (Plus) 24.03_1 Release (ZFS)

                          M 1 Reply Last reply May 21, 2023, 8:03 PM Reply Quote 0
                          • M
                            mvikman @Dobby_
                            last edited by May 21, 2023, 8:03 PM

                            @dobby_

                            The original poster asked about swap usage / change in swap usage from 23.01 to 23.05RC, also didn't specify what packages were running in system.

                            I just provided my experience with my system, which is that I have never seen swap usage in it.
                            I haven't bothered to change the ZFS ARC cache limits because it hasn't been a problem in my system.

                            Of course you can directly compare systems with 4GB and 8GB RAM.

                            pfSense Plus 24.11-RELEASE (amd64)
                            Dell Optiplex 7040 SFF
                            Core i5-6500, 8GB RAM, 2x 240GB SSD (ZFS Mirror)
                            HPE 561T (X540-AT2), 2-port 10Gb RJ45
                            HPE 562SFP+ (X710-DA2), 2-port 10Gb SFP+

                            1 Reply Last reply Reply Quote 0
                            • J
                              jimp Rebel Alliance Developer Netgate
                              last edited by May 22, 2023, 1:53 PM

                              All of this (and more) is covered in recent docs updates:

                              https://docs.netgate.com/pfsense/en/latest/hardware/memory.html

                              Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

                              Need help fast? Netgate Global Support!

                              Do not Chat/PM for help!

                              1 Reply Last reply Reply Quote 1
                              14 out of 14
                              • First post
                                14/14
                                Last post
                              Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.
                                This community forum collects and processes your personal information.
                                consent.not_received