Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Upgrade to 23.01 3x memory usage

    General pfSense Questions
    7
    21
    1.4k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • bingo600B
      bingo600
      last edited by bingo600

      Not all memory usage is "bad usage" ...
      I seem to have allocated a lot of "Inactive", and according to Doc.
      That mem will be released if the OS asks "nicely".

      This is my Qotom "home box" , upgraded to 23.01
      And have been running for 1d11h.

      c80f755f-bcbb-48d8-b154-ea668a228312-image.png

      43bb7dea-e657-49fd-910f-ab37d7ba499d-image.png

      I did some debugging on my "Test Hystou box 23.01", and found out, that if i stopped NtopNG.
      The Inactive mem dropped drastically, and free mem would increase by almost the same amount.
      On the Hystou i was using ~40% of available mem (dashboard) , and it dropped to around 18% (dashboard).

      I seem to remember that on 2.6.0 (or was it 2.5.1) , the usage was around 18% even with NtopNG running.
      So something (NtopNG) is allocating free mem for bufferusage, but i'm not too concerned.
      1: It is Inactive, and would be released if something else needs it.
      2: It stops around 40%, even when running for 170 days. So not indicating an uncontrolled leak.

      This (inactive) memory utilization might even be a good thing:
      1: Underutilized memory would be a waste.
      2: Buffering could save you diskwrites (SSD life)

      A Top on my Qotom (Home prod)
      6044e6b1-348e-4bde-bb2a-3271cceddc3e-image.png

      As you can see NtopNG is a "hungry fella" , both CPU time wise & Memory wise.

      I'm actually beginning to grumble about why i have it running permanently.
      But at some point i liked to be able to use it for creating a "flow view", but primarily for a "Now snapshot" ... So i could just start it, when needed.

      But : It doesn't interfere with performance (aka i can still fill my 1Gb connections") , and i have 8G Ram, so maybe SSD wear should be my concern.

      My Samsung EVO 870 250GB stats

      SMART Attributes Data Structure revision number: 1
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  5 Reallocated_Sector_Ct   0x0033   100   100   010    Pre-fail  Always       -       0
  9 Power_On_Hours          0x0032   097   097   000    Old_age   Always       -       11655
 12 Power_Cycle_Count       0x0032   099   099   000    Old_age   Always       -       33
177 Wear_Leveling_Count     0x0013   090   090   000    Pre-fail  Always       -       222
179 Used_Rsvd_Blk_Cnt_Tot   0x0013   100   100   010    Pre-fail  Always       -       0
181 Program_Fail_Cnt_Total  0x0032   100   100   010    Old_age   Always       -       0
182 Erase_Fail_Count_Total  0x0032   100   100   010    Old_age   Always       -       0
183 Runtime_Bad_Block       0x0013   100   100   010    Pre-fail  Always       -       0
187 Uncorrectable_Error_Cnt 0x0032   100   100   000    Old_age   Always       -       0
190 Airflow_Temperature_Cel 0x0032   059   045   000    Old_age   Always       -       41
195 ECC_Error_Rate          0x001a   200   200   000    Old_age   Always       -       0
199 CRC_Error_Count         0x003e   100   100   000    Old_age   Always       -       0
235 POR_Recovery_Count      0x0012   099   099   000    Old_age   Always       -       29
241 Total_LBAs_Written      0x0032   099   099   000    Old_age   Always       -       124046637227

SMART Error Log Version: 1
No Errors Logged

      What do you do with NtopNG ?

      Looking at the SMART data, i decided to stop NtopNG from running permanently.
      I'll post a pict of mem usage in a while.

      Here's my calculated Writes , using the SMART data above ..
      I suppose it's NtopNG doing most of it (syslog is remote)

      TBW
      (124051195843 * 512) ÷ 1024^4 = 57,76 TB Written

      OnTime
      11656÷24 = 485,66 Days

      Writes/day
      57,765839548÷485,666666667 = 0,12 TB/day
      57,765839548÷485,666666667×1024 = 121,8 GB/day

      du -h of /var/db/ntopng
      556M ntopng/

      * Warrantied TBW for 870 EVO: 150 TBW for 250 GB model, 300 TBW for 500 GB model, 600 TBW for 1 TB model, 1,200 TBW for 2 TB model and 2,400 TBW for 4 TB model.

* 5-years or TBW, whichever comes first. For more information on the warranty, please find the enclosed warranty statement in the package.

      Seems like i have used 38% of the TBW in 1.33Yr ... 🤒 🤕
      I'm glad i don't use any "Soldered storage"

      Edit:
      Free Mem use after NtopNG has been stopped ... Inactive dropped quite a bit, but really doesn't matter. Since it was inactive memory, that can be reallocated.
      455f574b-303c-47da-aec4-8b101ee5a1b1-image.png

      b4551f58-1acd-4352-976d-083c24d3c7c4-image.png

      Edit:
      Seems like filterdns runs a separate instance for every DNS host (Alias) to be resolved.
      If those all take 39M , it is quite a lot of "?? Wired mem ??", or have i mistaken how that works ?

      74510 root         68    0   127M    39M usem     0   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         20    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
  393 root         68   20    13M  2732K kqread   0   0:00   0.00% check_reload_status: Monitoring daemon of check_reload_status (check_reload_status)
74510 root         68    0   127M    39M usem     0   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
    0 root          8    -     0B  1472K -        0   0:00   0.00% [kernel{inm_free taskq}]
74510 root         68    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
    0 root        -16    -     0B  1472K -        3   0:00   0.00% [kernel{z_rd_iss_5}]
74510 root         68    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     1   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     3   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     0   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     1   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     1   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     3   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     0   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     1   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     0   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     1   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc74510 root         68    0   127M    39M usem     0   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         20    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
  393 root         68   20    13M  2732K kqread   0   0:00   0.00% check_reload_status: Monitoring daemon of check_reload_status (check_reload_status)
74510 root         68    0   127M    39M usem     0   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
    0 root          8    -     0B  1472K -        0   0:00   0.00% [kernel{inm_free taskq}]
74510 root         68    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
    0 root        -16    -     0B  1472K -        3   0:00   0.00% [kernel{z_rd_iss_5}]
74510 root         68    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     1   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     3   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     0   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     1   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     1   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     3   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     0   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     1   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     0   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     1   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     0   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
    6 root         -8    -     0B   928K -        0   0:00   0.00% [zfskern{z_zrele_2}]
74510 root         68    0   127M    39M usem     0   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     0   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     3   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     3   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     1   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     3   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     3   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     0   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     0   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     1   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
67690 root         20    0    13M  2900K nanslp   0   0:00   0.00% /usr/local/libexec/sshg-blocker{sshg-blocker}
74510 root         68    0   127M    39M usem     3   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     1   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     1   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     1   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     0   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     1   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     3   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     3   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     1   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     1   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     0   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     1   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     3   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     3   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     1   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     3   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     3   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     3   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     3   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     0   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
    6 root         -8    -     0B   928K -        0   0:00   0.00% [zfskern{z_zrele_2}]
74510 root         68    0   127M    39M usem     0   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     0   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     3   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     3   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     1   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     3   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     3   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     0   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     0   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     1   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
67690 root         20    0    13M  2900K nanslp   0   0:00   0.00% /usr/local/libexec/sshg-blocker{sshg-blocker}
74510 root         68    0   127M    39M usem     3   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     1   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     1   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     1   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     0   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     1   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     3   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     3   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     1   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     1   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     0   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     1   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     2   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     3   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     3   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     1   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     3   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     3   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     3   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}
74510 root         68    0   127M    39M usem     3   0:00   0.00% /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1{filterdns}

      Continuing my ramblings here:

      Re:
      https://forum.netgate.com/post/1089960

      I could reduce my ARC memory by adding this to "System Tunables"
      I had ARC = 1510M above.
      905824eb-003a-49d8-a160-989509e7567c-image.png
      2185c670-6a1a-4965-bb5b-742f1b4e71b2-image.png

      But considering my TBW above, and that i have 8G Ram , i'd like all the buffering i can get.
      So i'll remove that setting again, it was mostly a proof of concept.

      Here like 9 hours after disabling NtopNG + Briefly Limit ZFS ARC to 256MB , and remove the limit again. The system seems to have settled on the new Mem usage ... It dropped by 30%.

      c78ecc54-afa8-4bf0-a2e5-eee3f614300f-image.png

      /Bingo

      If you find my answer useful - Please give the post a 👍 - "thumbs up"

      pfSense+ 23.05.1 (ZFS)

      QOTOM-Q355G4 Quad Lan.
      CPU  : Core i5 5250U, Ram : 8GB Kingston DDR3LV 1600
      LAN  : 4 x Intel 211, Disk  : 240G SAMSUNG MZ7L3240HCHQ SSD

      1 Reply Last reply Reply Quote 0
      • S
        scottlindner
        last edited by scottlindner

        So I'm a pretty basic user. I know some of the names in this thread and know you guys are heavy weights with pfSense. I did install a new hard drive in my SG-2220 and used zfs when I did that upgrade. This is my memory graph. What am I doing wrong for hitting the easy button upgrade?

        7ec34266-bbd1-46ff-916e-262fdfa5adf1-image.png

        My ARC (is this the ZFS utilization?) is likely the cause. I could easily calculate it if I knew what it was before the upgrade.
        72fa52c7-b5b6-4cfd-8b31-eeb668806102-image.png

        I have a 2GB RAM Netgate, but 807MB of 2GB is roughly 40% and that is the exact amount of "extra" memory usage my unit is showing. It used to hover around 20% (17%-21% seemed normal( and now it's 60%-62% after two days from upgrade. So if all this is is caching for improved performance.. then I LOVE it! Just needed to understand what I'm seeing.

        S S 2 Replies Last reply Reply Quote 0
        • S
          SteveITS Rebel Alliance @scottlindner
          last edited by

          @scottlindner ZFS ARC is a cache. It’s supposed to be released if/when needed.

          The 3am increase is from cron jobs disabled in prior versions. The patch above disables the cron job.
          https://forum.netgate.com/topic/178279/upgrade-to-23-01-3x-memory-usage/2

          Pre-2.7.2/23.09: Only install packages for your version, or risk breaking it. Select your branch in System/Update/Update Settings.
          When upgrading, allow 10-15 minutes to restart, or more depending on packages and device speed.
          Upvote 👍 helpful posts!

          S 1 Reply Last reply Reply Quote 0
          • S SteveITS referenced this topic on
          • S
            scottlindner @scottlindner
            last edited by

            This post is deleted!
            1 Reply Last reply Reply Quote 0
            • S
              scottlindner @SteveITS
              last edited by

              @steveits said in Upgrade to 23.01 3x memory usage:

              @scottlindner ZFS ARC is a cache. It’s supposed to be released if/when needed.

              The 3am increase is from cron jobs disabled in prior versions. The patch above disables the cron job.
              https://forum.netgate.com/topic/178279/upgrade-to-23-01-3x-memory-usage/2

              I'm not sure I understand how to apply a patch. I see the trouble ticket with an ID to another description. Again, I'm not a power user like you guys are and I'm just not seeing where this stuff all comes together into a fix.

              S 1 Reply Last reply Reply Quote 0
              • S scottlindner referenced this topic on
              • S
                SteveITS Rebel Alliance @scottlindner
                last edited by

                @scottlindner Install the System Patches package, and add using the commit ID. Then fetch the patch and apply.
                https://docs.netgate.com/pfsense/en/latest/development/system-patches.html

                Pre-2.7.2/23.09: Only install packages for your version, or risk breaking it. Select your branch in System/Update/Update Settings.
                When upgrading, allow 10-15 minutes to restart, or more depending on packages and device speed.
                Upvote 👍 helpful posts!

                S 1 Reply Last reply Reply Quote 1
                • F
                  FSC830
                  last edited by

                  See here, how to apply a patch using GUI.

                  Regards

                  1 Reply Last reply Reply Quote 1
                  • S
                    scottlindner @SteveITS
                    last edited by

                    @steveits said in Upgrade to 23.01 3x memory usage:

                    @scottlindner Install the System Patches package, and add using the commit ID. Then fetch the patch and apply.
                    https://docs.netgate.com/pfsense/en/latest/development/system-patches.html

                    That was super easy. Thank you.

                    S 1 Reply Last reply Reply Quote 0
                    • S
                      scottlindner @scottlindner
                      last edited by

                      After applying the patch the memory usage has been very low. Slightly lower than before the upgrade to 23.01, and 22.05 (if I recall the previous version right) always had a slight memory creep of about 1%/month and this new version with the patch applied is consistently lower by a few percentage points and it stays there. Thank you everyone!

                      1 Reply Last reply Reply Quote 2
                      • D
                        dimitristsilis
                        last edited by dimitristsilis

                        I have a similar problem. My RAM usage is ~40% after rebooting my SG1100 and it jumps to ~85% at 3:00 am. Should I apply the aforementioned patch?
                        d67090b1-b34d-45b9-a59a-86db4a35c486-εικόνα.png

                        S 1 Reply Last reply Reply Quote 0
                        • S
                          scottlindner @dimitristsilis
                          last edited by

                          @dimitristsilis said in Upgrade to 23.01 3x memory usage:

                          I have a similar problem. My RAM usage is ~40% after rebooting my SG1100 and it jumps to ~85% at 3:00 am. Should I apply the aforementioned patch?
                          d67090b1-b34d-45b9-a59a-86db4a35c486-εικόνα.png

                          If that started after the upgrade to 23.01, then yes, the patch will fix it. The instructions are in this thread. I didn't know how to patch either. It's super easy.

                          N 1 Reply Last reply Reply Quote 1
                          • N
                            NGUSER6947 @scottlindner
                            last edited by

                            My understanding is that this is more of a visual/perception issue than a real problem... am I understanding correctly?

                            S S 2 Replies Last reply Reply Quote 0
                            • S
                              SteveITS Rebel Alliance @NGUSER6947
                              last edited by

                              @nguser6947 said in Upgrade to 23.01 3x memory usage:

                              My understanding is that this is more of a visual/perception issue than a real problem... am I understanding correctly?

                              Pretty much. See
                              https://forum.netgate.com/topic/178568/netgate-1100-high-memory-consumption/4 and the ZFS doc linked there.

                              Pre-2.7.2/23.09: Only install packages for your version, or risk breaking it. Select your branch in System/Update/Update Settings.
                              When upgrading, allow 10-15 minutes to restart, or more depending on packages and device speed.
                              Upvote 👍 helpful posts!

                              1 Reply Last reply Reply Quote 0
                              • S
                                scottlindner @NGUSER6947
                                last edited by

                                @nguser6947 said in Upgrade to 23.01 3x memory usage:

                                My understanding is that this is more of a visual/perception issue than a real problem... am I understanding correctly?

                                No. It is real, but it is also largely harmless too. It is happening due to some default OS Cron jobs starting things pfEense doesn't need. It was an oversight. The patch turns those Cron jobs off the way the pfSense crew intended.

                                S 1 Reply Last reply Reply Quote 0
                                • S
                                  SteveITS Rebel Alliance @scottlindner
                                  last edited by

                                  @scottlindner said in Upgrade to 23.01 3x memory usage:

                                  It is happening due to some default OS Cron jobs starting things pfEense doesn't need.

                                  The cron jobs did get enabled again, however, that's just a trigger. It's my understanding any disk activity will grow the ZFS ARC cache as noted ("1/2 RAM or the total RAM minus 1GB, whichever is greater").

                                  Whether that actually causes a problem or is just cosmetic is situation dependent. "ZFS will yield this RAM if other processes require more memory, but it may not give up memory fast enough for every use case."

                                  Pre-2.7.2/23.09: Only install packages for your version, or risk breaking it. Select your branch in System/Update/Update Settings.
                                  When upgrading, allow 10-15 minutes to restart, or more depending on packages and device speed.
                                  Upvote 👍 helpful posts!

                                  S 1 Reply Last reply Reply Quote 0
                                  • S
                                    scottlindner @SteveITS
                                    last edited by

                                    @steveits said in Upgrade to 23.01 3x memory usage:

                                    @scottlindner said in Upgrade to 23.01 3x memory usage:

                                    It is happening due to some default OS Cron jobs starting things pfEense doesn't need.

                                    The cron jobs did get enabled again, however, that's just a trigger. It's my understanding any disk activity will grow the ZFS ARC cache as noted ("1/2 RAM or the total RAM minus 1GB, whichever is greater").

                                    Whether that actually causes a problem or is just cosmetic is situation dependent. "ZFS will yield this RAM if other processes require more memory, but it may not give up memory fast enough for every use case."

                                    For sure. I think it's more, "Something changed, is it bad?"

                                    1 Reply Last reply Reply Quote 0
                                    • First post
                                      Last post