Memory usage climbing

[cpthk]

Thanks for your reply. Today, it just climbed to 25% off 1G memory. Here is my system summary. I am not using any packet capture. Everything in my pfsense is stock, no any extra package or plugin installed. The only features I use in pfsense if NAT port forwarding and dynamic DNS. Does port forwarding require tcpdump?
Could anyone suggest on a feature that could possibly be using tcpdump that I could check if mine is enabled?

last pid: 24302;  load averages:  0.00,  0.00,  0.00  up 16+21:42:07    04:03:28
117 processes: 3 running, 99 sleeping, 15 waiting

Mem: 179M Active, 15M Inact, 59M Wired, 1044K Cache, 26M Buf, 711M Free
Swap: 2048M Total, 2048M Free

PID USERNAME PRI NICE   SIZE    RES STATE   C   TIME   WCPU COMMAND
  11 root     171 ki31     0K    32K RUN     0 402.4H 100.00% {idle: cpu0}
  11 root     171 ki31     0K    32K CPU1    1 401.5H 100.00% {idle: cpu1}
  12 root     -68    -     0K   240K WAIT    1  98:03  1.46% {irq16: dc0 uhci0}
   0 root     -68    0     0K   112K -       0  42:58  0.29% {em0 taskq}
11958 root      44    0   102M 24160K piperd  0   0:02  0.10% php
  12 root     -32    -     0K   240K WAIT    1 146:21  0.00% {swi4: clock}
24300 root      44    0 14776K  2796K select  1   7:24  0.00% syslogd
  14 root     -16    -     0K    16K -       1   6:22  0.00% yarrow
21240 root      44    0   111M   104M bpf     0   6:00  0.00% tcpdump
21296 root      44    0  5832K  1040K piperd  0   3:17  0.00% logger
59743 root      76   20  8292K  1700K wait    1   1:39  0.00% sh
31670 root      64   20  5836K  1508K select  0   1:36  0.00% apinger
46248 nobody    44    0 10144K  3460K select  1   1:13  0.00% dnsmasq
40988 dhcpd     44    0 13052K  7276K select  1   0:54  0.00% dhcpd
  25 root      44    -     0K    16K syncer  1   0:53  0.00% syncer
   0 root      45    0     0K   112K sched   0   0:50  0.00% {swapper}
   9 root      44    -     0K    16K pftm    0   0:44  0.00% pfpurge
33972 root      44    0 25768K  5684K kqread  1   0:42  0.00% lighttpd

@stephenw10:

To be honest 20% memory use doesn't seem like a problem to me. Unless it's still climbing.
My own system is usually at around 30% (of 512MB) when idle. See attached RRD graph.
For comparrsion here is my top -SH output:

last pid: 46273;  load averages:  0.28,  0.12,  0.04                                                up 37+20:42:52  13:10:27
108 processes: 2 running, 90 sleeping, 16 waiting
CPU:  0.0% user,  0.0% nice,  0.7% system,  0.0% interrupt, 99.3% idle
Mem: 61M Active, 18M Inact, 62M Wired, 680K Cache, 59M Buf, 343M Free
Swap:

  PID USERNAME PRI NICE   SIZE    RES STATE    TIME   WCPU COMMAND
   10 root     171 ki31     0K     8K RUN    876.8H 98.00% idle
   11 root     -32    -     0K   128K WAIT   116:14  0.00% {swi4: clock}
   11 root     -68    -     0K   128K WAIT    52:18  0.00% {irq18: em0 ath0+}
12511 root      76   20  3656K  1580K wait    23:31  0.00% sh
    0 root     -68    0     0K    88K -       20:41  0.00% {em0 taskq}
 1807 nobody    74  r30  3368K  1484K nanslp  16:27  0.00% LCDd
   11 root     -68    -     0K   128K WAIT    16:24  0.00% {irq17: fxp2 fxp6}
63939 root      65   20 46428K 17528K nanslp  13:21  0.00% php
   11 root     -44    -     0K   128K WAIT    10:50  0.00% {swi1: netisr 0}
   13 root     -16    -     0K     8K -        6:23  0.00% yarrow
    0 root     -68    0     0K    88K -        4:49  0.00% {ath0 taskq}
29948 root      64   20  3316K  1336K select   4:01  0.00% apinger
    0 root     -68    0     0K    88K -        4:01  0.00% {em2 taskq}
    0 root     -68    0     0K    88K -        3:52  0.00% {em1 taskq}
   20 root      44    -     0K     8K syncer   3:38  0.00% syncer
   35 root      -8    -     0K     8K mdwait   3:23  0.00% md1
    2 root      -8    -     0K     8K -        2:46  0.00% g_event
   11 root     -68    -     0K   128K WAIT     2:27  0.00% {irq19: fxp0 fxp4}
 2392 dhcpd     44    0  8436K  6012K select   2:11  0.00% dhcpd
    0 root     -16    0     0K    88K sched    2:09  0.00% {swapper}
17355 root      44    0  6588K  5060K kqread   2:04  0.00% lighttpd
    4 root      -8    -     0K     8K -        1:47  0.00% g_down
   12 root     -16    -     0K     8K sleep    1:47  0.00% ng_queue
   29 root      -8    -     0K     8K mdwait   1:45  0.00% md0
27610 root      44    0  8464K  4424K select   1:43  0.00% {mpd5}
47631 root      44    0  5912K  3336K bpf      0:58  0.00% tcpdump
33556 root      44    0  3352K  1308K select   0:54  0.00% miniupnpd
    3 root      -8    -     0K     8K -        0:54  0.00% g_up
13826 root      44    0  8464K  4168K select   0:52  0.00% {mpd5}
    7 root     -16    -     0K     8K pftm     0:35  0.00% pfpurge
   11 root     -64    -     0K   128K WAIT     0:34  0.00% {irq14: ata0}
53459 nobody    44    0  5556K  3116K select   0:26  0.00% dnsmasq
 6151 root      76    0 43356K 18728K accept   0:23  0.00% php
39429 root      76    0 43356K 18468K accept   0:23  0.00% php
 9777 root      76    0 43356K 16596K accept   0:21  0.00% php
39767 root      76    0 43356K 17108K accept   0:21  0.00% php

Your tcpdump memory use is far higher than mine but that could be nothing to worry about.

Steve

Edit: Hmm, that tcpdump value is pretty high! Have you used the packet capture tool at all?

[cmb]

tcpdump is for the firewall logs, assuming it's running on pflog which it almost certainly is. Whether or not increasing memory usage is something to worry about depends on how it's increasing. If it steadily increases forever, that's an issue. If it stays where it is more or less after it's been up a while, which is typical, then that's fine.

[cpthk]

It just ate up all my memories: 94% memory, 64% SWAP

last pid: 47724;  load averages:  0.24,  0.32,  0.34  up 35+00:22:37    06:43:58
114 processes: 3 running, 96 sleeping, 15 waiting

Mem: 856M Active, 4316K Inact, 73M Wired, 30M Cache, 52M Buf, 136K Free
Swap: 2048M Total, 1318M Used, 729M Free, 64% Inuse

PID USERNAME PRI NICE   SIZE    RES STATE   C   TIME   WCPU COMMAND
  11 root     171 ki31     0K    32K CPU1    1 809.9H 98.19% {idle: cpu1}
  11 root     171 ki31     0K    32K RUN     0 785.4H 90.58% {idle: cpu0}
  19 root      50    -     0K    16K psleep  0 491:41  8.69% pagedaemon
11183 root      65   20  1477M   852M swread  0  59.3H  1.86% tcpdump
   3 root      -8    -     0K    16K -       0   9:14  0.29% g_up
  12 root     -68    -     0K   240K WAIT    1 262:35  0.20% {irq16: dc0 uhci0}
3344 root      70    0   104M  8488K piperd  0   0:07  0.10% php
  12 root     -32    -     0K   240K WAIT    1 307:06  0.00% {swi4: clock}
   0 root     -68    0     0K   112K -       0 119:43  0.00% {em0 taskq}
24300 root      44    0 14776K  1056K select  1  34:59  0.00% syslogd
  14 root      44    -     0K    16K -       1  15:37  0.00% yarrow
11359 root      64   20  5832K   220K piperd  0   7:44  0.00% logger
59743 root      76   20  8292K   324K wait    0   5:08  0.00% sh
29925 root      64   20  5836K   452K select  0   4:14  0.00% apinger
46248 nobody    44    0 10144K  1000K select  1   3:50  0.00% dnsmasq
   4 root      -8    -     0K    16K -       0   2:28  0.00% g_down
  12 root     -64    -     0K   240K WAIT    1   2:19  0.00% {irq19: uhci2 uhc}
  25 root      44    -     0K    16K syncer  1   2:15  0.00% syncer

[stephenw10]

Hmm, well that seems clearly wrong.

Are you experiencing any problems?

Steve

[cpthk]

The network is still fine, but when I login to admin dashboard, it is super slow.
This is obviously tcpdump problem. How do I tell which feature uses tcpdump exactly? I wanted to turn it off.

[stephenw10]

TBH I don't know. However as Chris said above it's used for firewall logging.
Have you changed the logging settings at all?
Do you have a large number of firewall rules?

Steve

[cpthk]

No, I only added 2 entries other than the default. Do you know how do I disable logging?

[stephenw10]

You can disable logging by the default block rule in:
Status: System logs: Settings:
I don't know much difference it will make since logging on other rules will still take place. It's worth a try though since it's easy to do.

Steve

[jimp]

Disabling logging of the default block doesn't disable that process, but it might lighten its load.

From the console or Diag > command, run: killall -9 tcpdump, then go to System > General and press Save, that should give tcpdump a kick and make it release all the memory it was holding.

I added a bit of code in 2.1 today to restart tcpdump from Status > System Logs on the Settings tab when Save is pressed. It was supposed to be restarted from System > General but it appears as though the code/test to match the tcpdump process may be incorrect ( I fixed that too, but I don't think it was matching properly on 2.0.x ).

[stephenw10]

Any reason it might climb like that though? I've not seen anyone else complaining about it. Some specific circumstance?  :-\

Steve

[jimp]

I have yet to reproduce it reliably, so I don't know. I've seen it maybe a half dozen times over the years, from 1.2 to 2.x. It's rare, but it does happen.

I don't know if it's due to an especially high/sustained rate of logging or something else that puts it over the edge.