Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Upgrade to 23.01 3x memory usage

    General pfSense Questions
    7
    21
    1.4k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • S
      scottlindner
      last edited by

      This post is deleted!
      S 1 Reply Last reply Reply Quote 0
      • S
        SteveITS Rebel Alliance @scottlindner
        last edited by

        @scottlindner There are a few long threads actually. There’s a patch to eliminate some unneeded cron jobs that cause ZFS to use RAM.

        https://forum.netgate.com/topic/178023/1100-upgrade-22-05-23-01-high-mem-usage/59

        ZFS should give it up. Not sure if it’s the above but there are threads discussing limiting ZFS cache size.

        Pre-2.7.2/23.09: Only install packages for your version, or risk breaking it. Select your branch in System/Update/Update Settings.
        When upgrading, allow 10-15 minutes to restart, or more depending on packages and device speed.
        Upvote 👍 helpful posts!

        S 1 Reply Last reply Reply Quote 1
        • S
          scottlindner @SteveITS
          last edited by

          @steveits Thanks for the reply. I rebooted again and it seems like the memory usage dropped to normal. I'm going to watch it to see if it creeps back up again. And also I already learned that doing a clean install rather than upgrade has more reliable results so I might do that too after a bit. I'm gonna track that thread. Thank you for that.

          johnpozJ S 2 Replies Last reply Reply Quote 0
          • johnpozJ
            johnpoz LAYER 8 Global Moderator @scottlindner
            last edited by

            @scottlindner said in Upgrade to 23.01 3x memory usage:

            clean install rather than upgrade has more reliable results

            This really should not be the case - I have done a few clean installs, and mostly upgrades.. The only time I have done cleans in recent memory is when moved from UFS to ZFS, and then later did another clean install because they changed the zfs layout..

            The last 2 have been from upgrades, from 22.01 to 22.05, from 22.05 to 23.01

            And I have no seen any issues that worth mentioning.. I am not really showing anything of concern with memory use as of yet.

            Here going back a month.. You can see the blip where I updated to 23.01 6 days ago.. But I am not seeing anything of concern.

            mem.jpg

            I have not done any of the patches related to the mem use thing, or adjusted anything with my zfs max, etc.. If I zoom in I see the little bumps at 3am etc..

            bump.jpg

            An intelligent man is sometimes forced to be drunk to spend time with his fools
            If you get confused: Listen to the Music Play
            Please don't Chat/PM me for help, unless mod related
            SG-4860 24.03 | Lab VMs 2.7.2, 24.03

            1 Reply Last reply Reply Quote 0
            • S
              SteveITS Rebel Alliance @scottlindner
              last edited by

              @scottlindner The issue with the 3am cron job is only the first day after which the memory is already in the cache. It doesn’t increase subsequent days. And as noted it’s supposed to release it so is more or less cosmetic.

              I’ve also had no issues with upgrading over the last 15ish years :) aside from the current 1100/2100 boot/EFI issue on devices not using ZFS. Doesn’t hurt to reinstall and restore from backup but normally not necessary.

              Pre-2.7.2/23.09: Only install packages for your version, or risk breaking it. Select your branch in System/Update/Update Settings.
              When upgrading, allow 10-15 minutes to restart, or more depending on packages and device speed.
              Upvote 👍 helpful posts!

              1 Reply Last reply Reply Quote 0
              • bingo600B
                bingo600
                last edited by bingo600

                Not all memory usage is "bad usage" ...
                I seem to have allocated a lot of "Inactive", and according to Doc.
                That mem will be released if the OS asks "nicely".

                This is my Qotom "home box" , upgraded to 23.01
                And have been running for 1d11h.

                c80f755f-bcbb-48d8-b154-ea668a228312-image.png

                43bb7dea-e657-49fd-910f-ab37d7ba499d-image.png

                I did some debugging on my "Test Hystou box 23.01", and found out, that if i stopped NtopNG.
                The Inactive mem dropped drastically, and free mem would increase by almost the same amount.
                On the Hystou i was using ~40% of available mem (dashboard) , and it dropped to around 18% (dashboard).

                I seem to remember that on 2.6.0 (or was it 2.5.1) , the usage was around 18% even with NtopNG running.
                So something (NtopNG) is allocating free mem for bufferusage, but i'm not too concerned.
                1: It is Inactive, and would be released if something else needs it.
                2: It stops around 40%, even when running for 170 days. So not indicating an uncontrolled leak.

                This (inactive) memory utilization might even be a good thing:
                1: Underutilized memory would be a waste.
                2: Buffering could save you diskwrites (SSD life)

                A Top on my Qotom (Home prod)
                6044e6b1-348e-4bde-bb2a-3271cceddc3e-image.png

                As you can see NtopNG is a "hungry fella" , both CPU time wise & Memory wise.

                I'm actually beginning to grumble about why i have it running permanently.
                But at some point i liked to be able to use it for creating a "flow view", but primarily for a "Now snapshot" ... So i could just start it, when needed.

                But : It doesn't interfere with performance (aka i can still fill my 1Gb connections") , and i have 8G Ram, so maybe SSD wear should be my concern.

                My Samsung EVO 870 250GB stats

                SMART Attributes Data Structure revision number: 1
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  5 Reallocated_Sector_Ct   0x0033   100   100   010    Pre-fail  Always       -       0
  9 Power_On_Hours          0x0032   097   097   000    Old_age   Always       -       11655
 12 Power_Cycle_Count       0x0032   099   099   000    Old_age   Always       -       33
177 Wear_Leveling_Count     0x0013   090   090   000    Pre-fail  Always       -       222
179 Used_Rsvd_Blk_Cnt_Tot   0x0013   100   100   010    Pre-fail  Always       -       0
181 Program_Fail_Cnt_Total  0x0032   100   100   010    Old_age   Always       -       0
182 Erase_Fail_Count_Total  0x0032   100   100   010    Old_age   Always       -       0
183 Runtime_Bad_Block       0x0013   100   100   010    Pre-fail  Always       -       0
187 Uncorrectable_Error_Cnt 0x0032   100   100   000    Old_age   Always       -       0
190 Airflow_Temperature_Cel 0x0032   059   045   000    Old_age   Always       -       41
195 ECC_Error_Rate          0x001a   200   200   000    Old_age   Always       -       0
199 CRC_Error_Count         0x003e   100   100   000    Old_age   Always       -       0
235 POR_Recovery_Count      0x0012   099   099   000    Old_age   Always       -       29
241 Total_LBAs_Written      0x0032   099   099   000    Old_age   Always       -       124046637227

SMART Error Log Version: 1
No Errors Logged

                What do you do with NtopNG ?

                Looking at the SMART data, i decided to stop NtopNG from running permanently.
                I'll post a pict of mem usage in a while.

                Here's my calculated Writes , using the SMART data above ..
                I suppose it's NtopNG doing most of it (syslog is remote)

                TBW
                (124051195843 * 512) ÷ 1024^4 = 57,76 TB Written

                OnTime
                11656÷24 = 485,66 Days

                Writes/day
                57,765839548÷485,666666667 = 0,12 TB/day
                57,765839548÷485,666666667×1024 = 121,8 GB/day

                du -h of /var/db/ntopng
                556M ntopng/

                * Warrantied TBW for 870 EVO: 150 TBW for 250 GB model, 300 TBW for 500 GB model, 600 TBW for 1 TB model, 1,200 TBW for 2 TB model and 2,400 TBW for 4 TB model.

* 5-years or TBW, whichever comes first. For more information on the warranty, please find the enclosed warranty statement in the package.

                Seems like i have used 38% of the TBW in 1.33Yr ... 🤒 🤕
                I'm glad i don't use any "Soldered storage"

                Edit:
                Free Mem use after NtopNG has been stopped ... Inactive dropped quite a bit, but really doesn't matter. Since it was inactive memory, that can be reallocated.
                455f574b-303c-47da-aec4-8b101ee5a1b1-image.png

                b4551f58-1acd-4352-976d-083c24d3c7c4-image.png

                Edit:
                Seems like filterdns runs a separate instance for every DNS host (Alias) to be resolved.
                If those all take 39M , it is quite a lot of "?? Wired mem ??", or have i mistaken how that works ?

                74510 root         68    0   127M    39M usem     0   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         20    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
  393 root         68   20    13M  2732K kqread   0   0:00   0.00% check_reload_status: Monitoring daemon of check_reload_status (check_reload_status)
74510 root         68    0   127M    39M usem     0   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
    0 root          8    -     0B  1472K -        0   0:00   0.00% [kernel{inm_free taskq}]
74510 root         68    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
    0 root        -16    -     0B  1472K -        3   0:00   0.00% [kernel{z_rd_iss_5}]
74510 root         68    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     1   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     3   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     0   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     1   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     1   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     3   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     0   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     1   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     0   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     1   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc74510 root         68    0   127M    39M usem     0   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         20    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
  393 root         68   20    13M  2732K kqread   0   0:00   0.00% check_reload_status: Monitoring daemon of check_reload_status (check_reload_status)
74510 root         68    0   127M    39M usem     0   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
    0 root          8    -     0B  1472K -        0   0:00   0.00% [kernel{inm_free taskq}]
74510 root         68    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
    0 root        -16    -     0B  1472K -        3   0:00   0.00% [kernel{z_rd_iss_5}]
74510 root         68    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     1   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     3   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     0   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     1   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     1   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     3   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     0   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     1   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     0   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     1   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     0   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
    6 root         -8    -     0B   928K -        0   0:00   0.00% [zfskern{z_zrele_2}]
74510 root         68    0   127M    39M usem     0   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     0   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     3   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     3   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     1   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     3   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     3   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     0   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     0   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     1   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
67690 root         20    0    13M  2900K nanslp   0   0:00   0.00% /usr/local/libexec/sshg-blocker{sshg-blocker}
74510 root         68    0   127M    39M usem     3   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     1   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     1   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     1   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     0   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     1   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     3   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     3   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     1   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     1   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     0   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     1   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     3   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     3   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     1   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     3   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     3   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     3   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     3   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     0   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
    6 root         -8    -     0B   928K -        0   0:00   0.00% [zfskern{z_zrele_2}]
74510 root         68    0   127M    39M usem     0   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     0   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     3   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     3   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     1   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     3   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     3   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     0   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     0   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     1   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
67690 root         20    0    13M  2900K nanslp   0   0:00   0.00% /usr/local/libexec/sshg-blocker{sshg-blocker}
74510 root         68    0   127M    39M usem     3   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     1   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     1   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     1   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     0   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     1   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     3   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     3   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     1   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     1   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     0   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     1   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     3   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     3   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     1   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     3   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     3   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     3   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     3   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}

                Continuing my ramblings here:

                Re:
                https://forum.netgate.com/post/1089960

                I could reduce my ARC memory by adding this to "System Tunables"
                I had ARC = 1510M above.
                905824eb-003a-49d8-a160-989509e7567c-image.png
                2185c670-6a1a-4965-bb5b-742f1b4e71b2-image.png

                But considering my TBW above, and that i have 8G Ram , i'd like all the buffering i can get.
                So i'll remove that setting again, it was mostly a proof of concept.

                Here like 9 hours after disabling NtopNG + Briefly Limit ZFS ARC to 256MB , and remove the limit again. The system seems to have settled on the new Mem usage ... It dropped by 30%.

                c78ecc54-afa8-4bf0-a2e5-eee3f614300f-image.png

                /Bingo

                If you find my answer useful - Please give the post a 👍 - "thumbs up"

                pfSense+ 23.05.1 (ZFS)

                QOTOM-Q355G4 Quad Lan.
                CPU  : Core i5 5250U, Ram : 8GB Kingston DDR3LV 1600
                LAN  : 4 x Intel 211, Disk  : 240G SAMSUNG MZ7L3240HCHQ SSD

                1 Reply Last reply Reply Quote 0
                • S
                  scottlindner
                  last edited by scottlindner

                  So I'm a pretty basic user. I know some of the names in this thread and know you guys are heavy weights with pfSense. I did install a new hard drive in my SG-2220 and used zfs when I did that upgrade. This is my memory graph. What am I doing wrong for hitting the easy button upgrade?

                  7ec34266-bbd1-46ff-916e-262fdfa5adf1-image.png

                  My ARC (is this the ZFS utilization?) is likely the cause. I could easily calculate it if I knew what it was before the upgrade.
                  72fa52c7-b5b6-4cfd-8b31-eeb668806102-image.png

                  I have a 2GB RAM Netgate, but 807MB of 2GB is roughly 40% and that is the exact amount of "extra" memory usage my unit is showing. It used to hover around 20% (17%-21% seemed normal( and now it's 60%-62% after two days from upgrade. So if all this is is caching for improved performance.. then I LOVE it! Just needed to understand what I'm seeing.

                  S S 2 Replies Last reply Reply Quote 0
                  • S
                    SteveITS Rebel Alliance @scottlindner
                    last edited by

                    @scottlindner ZFS ARC is a cache. It’s supposed to be released if/when needed.

                    The 3am increase is from cron jobs disabled in prior versions. The patch above disables the cron job.
                    https://forum.netgate.com/topic/178279/upgrade-to-23-01-3x-memory-usage/2

                    Pre-2.7.2/23.09: Only install packages for your version, or risk breaking it. Select your branch in System/Update/Update Settings.
                    When upgrading, allow 10-15 minutes to restart, or more depending on packages and device speed.
                    Upvote 👍 helpful posts!

                    S 1 Reply Last reply Reply Quote 0
                    • S SteveITS referenced this topic on
                    • S
                      scottlindner @scottlindner
                      last edited by

                      This post is deleted!
                      1 Reply Last reply Reply Quote 0
                      • S
                        scottlindner @SteveITS
                        last edited by

                        @steveits said in Upgrade to 23.01 3x memory usage:

                        @scottlindner ZFS ARC is a cache. It’s supposed to be released if/when needed.

                        The 3am increase is from cron jobs disabled in prior versions. The patch above disables the cron job.
                        https://forum.netgate.com/topic/178279/upgrade-to-23-01-3x-memory-usage/2

                        I'm not sure I understand how to apply a patch. I see the trouble ticket with an ID to another description. Again, I'm not a power user like you guys are and I'm just not seeing where this stuff all comes together into a fix.

                        S 1 Reply Last reply Reply Quote 0
                        • S scottlindner referenced this topic on
                        • S
                          SteveITS Rebel Alliance @scottlindner
                          last edited by

                          @scottlindner Install the System Patches package, and add using the commit ID. Then fetch the patch and apply.
                          https://docs.netgate.com/pfsense/en/latest/development/system-patches.html

                          Pre-2.7.2/23.09: Only install packages for your version, or risk breaking it. Select your branch in System/Update/Update Settings.
                          When upgrading, allow 10-15 minutes to restart, or more depending on packages and device speed.
                          Upvote 👍 helpful posts!

                          S 1 Reply Last reply Reply Quote 1
                          • F
                            FSC830
                            last edited by

                            See here, how to apply a patch using GUI.

                            Regards

                            1 Reply Last reply Reply Quote 1
                            • S
                              scottlindner @SteveITS
                              last edited by

                              @steveits said in Upgrade to 23.01 3x memory usage:

                              @scottlindner Install the System Patches package, and add using the commit ID. Then fetch the patch and apply.
                              https://docs.netgate.com/pfsense/en/latest/development/system-patches.html

                              That was super easy. Thank you.

                              S 1 Reply Last reply Reply Quote 0
                              • S
                                scottlindner @scottlindner
                                last edited by

                                After applying the patch the memory usage has been very low. Slightly lower than before the upgrade to 23.01, and 22.05 (if I recall the previous version right) always had a slight memory creep of about 1%/month and this new version with the patch applied is consistently lower by a few percentage points and it stays there. Thank you everyone!

                                1 Reply Last reply Reply Quote 2
                                • D
                                  dimitristsilis
                                  last edited by dimitristsilis

                                  I have a similar problem. My RAM usage is ~40% after rebooting my SG1100 and it jumps to ~85% at 3:00 am. Should I apply the aforementioned patch?
                                  d67090b1-b34d-45b9-a59a-86db4a35c486-εικόνα.png

                                  S 1 Reply Last reply Reply Quote 0
                                  • S
                                    scottlindner @dimitristsilis
                                    last edited by

                                    @dimitristsilis said in Upgrade to 23.01 3x memory usage:

                                    I have a similar problem. My RAM usage is ~40% after rebooting my SG1100 and it jumps to ~85% at 3:00 am. Should I apply the aforementioned patch?
                                    d67090b1-b34d-45b9-a59a-86db4a35c486-εικόνα.png

                                    If that started after the upgrade to 23.01, then yes, the patch will fix it. The instructions are in this thread. I didn't know how to patch either. It's super easy.

                                    N 1 Reply Last reply Reply Quote 1
                                    • N
                                      NGUSER6947 @scottlindner
                                      last edited by

                                      My understanding is that this is more of a visual/perception issue than a real problem... am I understanding correctly?

                                      S S 2 Replies Last reply Reply Quote 0
                                      • S
                                        SteveITS Rebel Alliance @NGUSER6947
                                        last edited by

                                        @nguser6947 said in Upgrade to 23.01 3x memory usage:

                                        My understanding is that this is more of a visual/perception issue than a real problem... am I understanding correctly?

                                        Pretty much. See
                                        https://forum.netgate.com/topic/178568/netgate-1100-high-memory-consumption/4 and the ZFS doc linked there.

                                        Pre-2.7.2/23.09: Only install packages for your version, or risk breaking it. Select your branch in System/Update/Update Settings.
                                        When upgrading, allow 10-15 minutes to restart, or more depending on packages and device speed.
                                        Upvote 👍 helpful posts!

                                        1 Reply Last reply Reply Quote 0
                                        • S
                                          scottlindner @NGUSER6947
                                          last edited by

                                          @nguser6947 said in Upgrade to 23.01 3x memory usage:

                                          My understanding is that this is more of a visual/perception issue than a real problem... am I understanding correctly?

                                          No. It is real, but it is also largely harmless too. It is happening due to some default OS Cron jobs starting things pfEense doesn't need. It was an oversight. The patch turns those Cron jobs off the way the pfSense crew intended.

                                          S 1 Reply Last reply Reply Quote 0
                                          • S
                                            SteveITS Rebel Alliance @scottlindner
                                            last edited by

                                            @scottlindner said in Upgrade to 23.01 3x memory usage:

                                            It is happening due to some default OS Cron jobs starting things pfEense doesn't need.

                                            The cron jobs did get enabled again, however, that's just a trigger. It's my understanding any disk activity will grow the ZFS ARC cache as noted ("1/2 RAM or the total RAM minus 1GB, whichever is greater").

                                            Whether that actually causes a problem or is just cosmetic is situation dependent. "ZFS will yield this RAM if other processes require more memory, but it may not give up memory fast enough for every use case."

                                            Pre-2.7.2/23.09: Only install packages for your version, or risk breaking it. Select your branch in System/Update/Update Settings.
                                            When upgrading, allow 10-15 minutes to restart, or more depending on packages and device speed.
                                            Upvote 👍 helpful posts!

                                            S 1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post