Memory usage climbing



  • Hi:

    I have pfsense as my router for the past 1 month. I have never restarted it since installed. I noticed that the memory usage keeps climbing, from the first day 3% to today 20%. I did use bittorrent few times(< 40 hour in total). But other than that, I only have about 4 computers with normal internet browsing. Anyone has a clue on why is this happening? Is this a known issue? Anything I could do?

    Thanks.



  • And do you have packages installed like squid?


  • Netgate Administrator

    How much ram do you have?
    The fact that memory usage is climbing is not necessarily a problem. Unused ram is just taking up space in your box.

    Steve


  • Rebel Alliance Developer Netgate

    From the console run "top -SH" or look at Diagnostics > System Activity and look at the break-down of RAM utilization there.



  • I will take a look the memory break down today.
    I did not install any extra packages beside the default ones there.



  • This is what I got, anything I could do to help on reducing the memory? I have 1GB of ram.
    It looks like tcpdump uses the most, I do know tcpdump is the tool to capture network packets. In pfsense, which feature is it being used?

    last pid:  9692;  load averages:  0.00,  0.00,  0.00  up 14+22:26:54    04:48:15
    117 processes: 3 running, 99 sleeping, 15 waiting

    Mem: 134M Active, 15M Inact, 58M Wired, 1044K Cache, 26M Buf, 755M Free
    Swap: 2048M Total, 2048M Free

    PID USERNAME PRI NICE   SIZE    RES STATE   C   TIME   WCPU COMMAND
      11 root     171 ki31     0K    32K RUN     0 355.6H 100.00% {idle: cpu0}
      11 root     171 ki31     0K    32K CPU1    1 355.0H 100.00% {idle: cpu1}
      12 root     -68    -     0K   240K WAIT    1  75:56  0.39% {irq16: dc0 uhci0}
    11958 root      76    0   102M 21408K piperd  0   0:02  0.10% php
      12 root     -32    -     0K   240K WAIT    1 129:17  0.00% {swi4: clock}
       0 root     -68    0     0K   112K -       0  33:17  0.00% {em0 taskq}
      14 root     -16    -     0K    16K -       1   5:01  0.00% yarrow
    24300 root      44    0 14776K  2796K select  0   4:36  0.00% syslogd
    21240 root      44    0 69112K 62560K bpf     1   3:05  0.00% tcpdump
    21296 root      44    0  5832K  1040K piperd  1   2:04  0.00% logger
    59743 root      76   20  8292K  1700K wait    1   1:17  0.00% sh
    31670 root      64   20  5836K  1508K select  0   1:13  0.00% apinger
    46248 nobody    44    0 10144K  3460K select  1   1:00  0.00% dnsmasq
       0 root      45    0     0K   112K sched   0   0:50  0.00% {swapper}
    40988 dhcpd     44    0 13052K  7276K select  1   0:47  0.00% dhcpd
      25 root      44    -     0K    16K syncer  1   0:46  0.00% syncer
    33972 root      44    0 25768K  5684K kqread  1   0:39  0.00% lighttpd
       2 root      -8    -     0K    16K -       1   0:37  0.00% g_event


  • Netgate Administrator

    To be honest 20% memory use doesn't seem like a problem to me. Unless it's still climbing.
    My own system is usually at around 30% (of 512MB) when idle. See attached RRD graph.
    For comparrsion here is my top -SH output:

    last pid: 46273;  load averages:  0.28,  0.12,  0.04                                                up 37+20:42:52  13:10:27
    108 processes: 2 running, 90 sleeping, 16 waiting
    CPU:  0.0% user,  0.0% nice,  0.7% system,  0.0% interrupt, 99.3% idle
    Mem: 61M Active, 18M Inact, 62M Wired, 680K Cache, 59M Buf, 343M Free
    Swap:
    
      PID USERNAME PRI NICE   SIZE    RES STATE    TIME   WCPU COMMAND
       10 root     171 ki31     0K     8K RUN    876.8H 98.00% idle
       11 root     -32    -     0K   128K WAIT   116:14  0.00% {swi4: clock}
       11 root     -68    -     0K   128K WAIT    52:18  0.00% {irq18: em0 ath0+}
    12511 root      76   20  3656K  1580K wait    23:31  0.00% sh
        0 root     -68    0     0K    88K -       20:41  0.00% {em0 taskq}
     1807 nobody    74  r30  3368K  1484K nanslp  16:27  0.00% LCDd
       11 root     -68    -     0K   128K WAIT    16:24  0.00% {irq17: fxp2 fxp6}
    63939 root      65   20 46428K 17528K nanslp  13:21  0.00% php
       11 root     -44    -     0K   128K WAIT    10:50  0.00% {swi1: netisr 0}
       13 root     -16    -     0K     8K -        6:23  0.00% yarrow
        0 root     -68    0     0K    88K -        4:49  0.00% {ath0 taskq}
    29948 root      64   20  3316K  1336K select   4:01  0.00% apinger
        0 root     -68    0     0K    88K -        4:01  0.00% {em2 taskq}
        0 root     -68    0     0K    88K -        3:52  0.00% {em1 taskq}
       20 root      44    -     0K     8K syncer   3:38  0.00% syncer
       35 root      -8    -     0K     8K mdwait   3:23  0.00% md1
        2 root      -8    -     0K     8K -        2:46  0.00% g_event
       11 root     -68    -     0K   128K WAIT     2:27  0.00% {irq19: fxp0 fxp4}
     2392 dhcpd     44    0  8436K  6012K select   2:11  0.00% dhcpd
        0 root     -16    0     0K    88K sched    2:09  0.00% {swapper}
    17355 root      44    0  6588K  5060K kqread   2:04  0.00% lighttpd
        4 root      -8    -     0K     8K -        1:47  0.00% g_down
       12 root     -16    -     0K     8K sleep    1:47  0.00% ng_queue
       29 root      -8    -     0K     8K mdwait   1:45  0.00% md0
    27610 root      44    0  8464K  4424K select   1:43  0.00% {mpd5}
    47631 root      44    0  5912K  3336K bpf      0:58  0.00% tcpdump
    33556 root      44    0  3352K  1308K select   0:54  0.00% miniupnpd
        3 root      -8    -     0K     8K -        0:54  0.00% g_up
    13826 root      44    0  8464K  4168K select   0:52  0.00% {mpd5}
        7 root     -16    -     0K     8K pftm     0:35  0.00% pfpurge
       11 root     -64    -     0K   128K WAIT     0:34  0.00% {irq14: ata0}
    53459 nobody    44    0  5556K  3116K select   0:26  0.00% dnsmasq
     6151 root      76    0 43356K 18728K accept   0:23  0.00% php
    39429 root      76    0 43356K 18468K accept   0:23  0.00% php
     9777 root      76    0 43356K 16596K accept   0:21  0.00% php
    39767 root      76    0 43356K 17108K accept   0:21  0.00% php
    
    

    Your tcpdump memory use is far higher than mine but that could be nothing to worry about.

    Steve

    Edit: Hmm, that tcpdump value is pretty high! Have you used the packet capture tool at all?




  • Thanks for your reply. Today, it just climbed to 25% off 1G memory. Here is my system summary. I am not using any packet capture. Everything in my pfsense is stock, no any extra package or plugin installed. The only features I use in pfsense if NAT port forwarding and dynamic DNS. Does port forwarding require tcpdump?
    Could anyone suggest on a feature that could possibly be using tcpdump that I could check if mine is enabled?

    last pid: 24302;  load averages:  0.00,  0.00,  0.00  up 16+21:42:07    04:03:28
    117 processes: 3 running, 99 sleeping, 15 waiting

    Mem: 179M Active, 15M Inact, 59M Wired, 1044K Cache, 26M Buf, 711M Free
    Swap: 2048M Total, 2048M Free

    PID USERNAME PRI NICE   SIZE    RES STATE   C   TIME   WCPU COMMAND
      11 root     171 ki31     0K    32K RUN     0 402.4H 100.00% {idle: cpu0}
      11 root     171 ki31     0K    32K CPU1    1 401.5H 100.00% {idle: cpu1}
      12 root     -68    -     0K   240K WAIT    1  98:03  1.46% {irq16: dc0 uhci0}
       0 root     -68    0     0K   112K -       0  42:58  0.29% {em0 taskq}
    11958 root      44    0   102M 24160K piperd  0   0:02  0.10% php
      12 root     -32    -     0K   240K WAIT    1 146:21  0.00% {swi4: clock}
    24300 root      44    0 14776K  2796K select  1   7:24  0.00% syslogd
      14 root     -16    -     0K    16K -       1   6:22  0.00% yarrow
    21240 root      44    0   111M   104M bpf     0   6:00  0.00% tcpdump
    21296 root      44    0  5832K  1040K piperd  0   3:17  0.00% logger
    59743 root      76   20  8292K  1700K wait    1   1:39  0.00% sh
    31670 root      64   20  5836K  1508K select  0   1:36  0.00% apinger
    46248 nobody    44    0 10144K  3460K select  1   1:13  0.00% dnsmasq
    40988 dhcpd     44    0 13052K  7276K select  1   0:54  0.00% dhcpd
      25 root      44    -     0K    16K syncer  1   0:53  0.00% syncer
       0 root      45    0     0K   112K sched   0   0:50  0.00% {swapper}
       9 root      44    -     0K    16K pftm    0   0:44  0.00% pfpurge
    33972 root      44    0 25768K  5684K kqread  1   0:42  0.00% lighttpd

    @stephenw10:

    To be honest 20% memory use doesn't seem like a problem to me. Unless it's still climbing.
    My own system is usually at around 30% (of 512MB) when idle. See attached RRD graph.
    For comparrsion here is my top -SH output:

    last pid: 46273;  load averages:  0.28,  0.12,  0.04                                                up 37+20:42:52  13:10:27
    108 processes: 2 running, 90 sleeping, 16 waiting
    CPU:  0.0% user,  0.0% nice,  0.7% system,  0.0% interrupt, 99.3% idle
    Mem: 61M Active, 18M Inact, 62M Wired, 680K Cache, 59M Buf, 343M Free
    Swap:
    
      PID USERNAME PRI NICE   SIZE    RES STATE    TIME   WCPU COMMAND
       10 root     171 ki31     0K     8K RUN    876.8H 98.00% idle
       11 root     -32    -     0K   128K WAIT   116:14  0.00% {swi4: clock}
       11 root     -68    -     0K   128K WAIT    52:18  0.00% {irq18: em0 ath0+}
    12511 root      76   20  3656K  1580K wait    23:31  0.00% sh
        0 root     -68    0     0K    88K -       20:41  0.00% {em0 taskq}
     1807 nobody    74  r30  3368K  1484K nanslp  16:27  0.00% LCDd
       11 root     -68    -     0K   128K WAIT    16:24  0.00% {irq17: fxp2 fxp6}
    63939 root      65   20 46428K 17528K nanslp  13:21  0.00% php
       11 root     -44    -     0K   128K WAIT    10:50  0.00% {swi1: netisr 0}
       13 root     -16    -     0K     8K -        6:23  0.00% yarrow
        0 root     -68    0     0K    88K -        4:49  0.00% {ath0 taskq}
    29948 root      64   20  3316K  1336K select   4:01  0.00% apinger
        0 root     -68    0     0K    88K -        4:01  0.00% {em2 taskq}
        0 root     -68    0     0K    88K -        3:52  0.00% {em1 taskq}
       20 root      44    -     0K     8K syncer   3:38  0.00% syncer
       35 root      -8    -     0K     8K mdwait   3:23  0.00% md1
        2 root      -8    -     0K     8K -        2:46  0.00% g_event
       11 root     -68    -     0K   128K WAIT     2:27  0.00% {irq19: fxp0 fxp4}
     2392 dhcpd     44    0  8436K  6012K select   2:11  0.00% dhcpd
        0 root     -16    0     0K    88K sched    2:09  0.00% {swapper}
    17355 root      44    0  6588K  5060K kqread   2:04  0.00% lighttpd
        4 root      -8    -     0K     8K -        1:47  0.00% g_down
       12 root     -16    -     0K     8K sleep    1:47  0.00% ng_queue
       29 root      -8    -     0K     8K mdwait   1:45  0.00% md0
    27610 root      44    0  8464K  4424K select   1:43  0.00% {mpd5}
    47631 root      44    0  5912K  3336K bpf      0:58  0.00% tcpdump
    33556 root      44    0  3352K  1308K select   0:54  0.00% miniupnpd
        3 root      -8    -     0K     8K -        0:54  0.00% g_up
    13826 root      44    0  8464K  4168K select   0:52  0.00% {mpd5}
        7 root     -16    -     0K     8K pftm     0:35  0.00% pfpurge
       11 root     -64    -     0K   128K WAIT     0:34  0.00% {irq14: ata0}
    53459 nobody    44    0  5556K  3116K select   0:26  0.00% dnsmasq
     6151 root      76    0 43356K 18728K accept   0:23  0.00% php
    39429 root      76    0 43356K 18468K accept   0:23  0.00% php
     9777 root      76    0 43356K 16596K accept   0:21  0.00% php
    39767 root      76    0 43356K 17108K accept   0:21  0.00% php
    
    

    Your tcpdump memory use is far higher than mine but that could be nothing to worry about.

    Steve

    Edit: Hmm, that tcpdump value is pretty high! Have you used the packet capture tool at all?



  • tcpdump is for the firewall logs, assuming it's running on pflog which it almost certainly is. Whether or not increasing memory usage is something to worry about depends on how it's increasing. If it steadily increases forever, that's an issue. If it stays where it is more or less after it's been up a while, which is typical, then that's fine.



  • It just ate up all my memories: 94% memory, 64% SWAP

    last pid: 47724;  load averages:  0.24,  0.32,  0.34  up 35+00:22:37    06:43:58
    114 processes: 3 running, 96 sleeping, 15 waiting

    Mem: 856M Active, 4316K Inact, 73M Wired, 30M Cache, 52M Buf, 136K Free
    Swap: 2048M Total, 1318M Used, 729M Free, 64% Inuse

    PID USERNAME PRI NICE   SIZE    RES STATE   C   TIME   WCPU COMMAND
      11 root     171 ki31     0K    32K CPU1    1 809.9H 98.19% {idle: cpu1}
      11 root     171 ki31     0K    32K RUN     0 785.4H 90.58% {idle: cpu0}
      19 root      50    -     0K    16K psleep  0 491:41  8.69% pagedaemon
    11183 root      65   20  1477M   852M swread  0  59.3H  1.86% tcpdump
       3 root      -8    -     0K    16K -       0   9:14  0.29% g_up
      12 root     -68    -     0K   240K WAIT    1 262:35  0.20% {irq16: dc0 uhci0}
    3344 root      70    0   104M  8488K piperd  0   0:07  0.10% php
      12 root     -32    -     0K   240K WAIT    1 307:06  0.00% {swi4: clock}
       0 root     -68    0     0K   112K -       0 119:43  0.00% {em0 taskq}
    24300 root      44    0 14776K  1056K select  1  34:59  0.00% syslogd
      14 root      44    -     0K    16K -       1  15:37  0.00% yarrow
    11359 root      64   20  5832K   220K piperd  0   7:44  0.00% logger
    59743 root      76   20  8292K   324K wait    0   5:08  0.00% sh
    29925 root      64   20  5836K   452K select  0   4:14  0.00% apinger
    46248 nobody    44    0 10144K  1000K select  1   3:50  0.00% dnsmasq
       4 root      -8    -     0K    16K -       0   2:28  0.00% g_down
      12 root     -64    -     0K   240K WAIT    1   2:19  0.00% {irq19: uhci2 uhc}
      25 root      44    -     0K    16K syncer  1   2:15  0.00% syncer


  • Netgate Administrator

    Hmm, well that seems clearly wrong.

    Are you experiencing any problems?

    Steve



  • The network is still fine, but when I login to admin dashboard, it is super slow.
    This is obviously tcpdump problem. How do I tell which feature uses tcpdump exactly? I wanted to turn it off.


  • Netgate Administrator

    TBH I don't know. However as Chris said above it's used for firewall logging.
    Have you changed the logging settings at all?
    Do you have a large number of firewall rules?

    Steve



  • No, I only added 2 entries other than the default. Do you know how do I disable logging?


  • Netgate Administrator

    You can disable logging by the default block rule in:
    Status: System logs: Settings:
    I don't know much difference it will make since logging on other rules will still take place. It's worth a try though since it's easy to do.

    Steve


  • Rebel Alliance Developer Netgate

    Disabling logging of the default block doesn't disable that process, but it might lighten its load.

    From the console or Diag > command, run: killall -9 tcpdump, then go to System > General and press Save, that should give tcpdump a kick and make it release all the memory it was holding.

    I added a bit of code in 2.1 today to restart tcpdump from Status > System Logs on the Settings tab when Save is pressed. It was supposed to be restarted from System > General but it appears as though the code/test to match the tcpdump process may be incorrect ( I fixed that too, but I don't think it was matching properly on 2.0.x ).


  • Netgate Administrator

    Any reason it might climb like that though? I've not seen anyone else complaining about it. Some specific circumstance?  :-\

    Steve


  • Rebel Alliance Developer Netgate

    I have yet to reproduce it reliably, so I don't know. I've seen it maybe a half dozen times over the years, from 1.2 to 2.x. It's rare, but it does happen.

    I don't know if it's due to an especially high/sustained rate of logging or something else that puts it over the edge.


Locked