Getting a better idea of CPU usage

clarknova

pfsense 2.1 i386
Soekris net5501-70

WAN is vr2 (PPPoE)
LANs are vr0_vlanxx

When I run 'top -SH' while downloading WAN to LAN ~10 Mbps, I see numbers similar to the following:

last pid: 41975;  load averages:  0.57,  0.96,  1.54                                                                                                                                                                up 12+17:34:39  11:11:58
111 processes: 7 running, 91 sleeping, 13 waiting
CPU:  0.8% user,  5.8% nice, 27.8% system, 15.8% interrupt, 49.8% idle
Mem: 77M Active, 23M Inact, 93M Wired, 220K Cache, 59M Buf, 288M Free
Swap:

  PID USERNAME PRI NICE   SIZE    RES STATE    TIME   WCPU COMMAND
   10 root     171 ki31     0K     8K RUN    245.9H 76.95% idle
   11 root     -68    -     0K   112K RUN    363:04  8.98% intr{irq9: vr2}
   11 root     -68    -     0K   112K WAIT   231:05  0.00% intr{irq11: vr0}

A few things jump out at me here.

1. The 15.8% interrupt use doesn't match the 8.98% attributed to intr{irq9: vr2}.
2. intr{irq11: vr0} is reporting 0 usage.

These two points together seem to indicate to me that there is probably 15.8 - 8.98, or ~6.82% interrupt being used by a vlan on vr0, but not being reported by top.

3. 76.95 + 8.98 = 85.93%. Even if we add in the 6.82% phantom vlan usage, we still add up to only 92.75%
4. The idle process is reporting 76.95% while the system is showing 49.8% idle.
5. The system is reporting 27.8% system use, 5.8% nice and 0.8% user.

Why the huge discrepancy between system idle and the idle process? What processes are contributing to the system, user, and nice values? I assume top itself must be generating some usage, but it doesn't appear anywhere in the list.

My conclusion is that top isn't showing all the processes, and it's not attributing vlan-generated interrupts to any process. Can anybody correct my thinking, offer insight into why the system values don't match the process values, or show me some better switches for top to show more info?

jimp

top will show all the threads it can see when using -SH (I prefer -aSH but that wouldn't impact how much it shows, just detail)

vlans still use the IRQs of the card itself, but other thigns use IRQs that may not have dedicated processing threads visible.

From the shell, run:

vmstat -i

and

systat -vmstat

That should give a better picture of the IRQ usage.

clarknova

That helps. Thanks.