Interrupt processor affinity

[jlauser]

I was poking around in my firewall yesterday (2.3.4 on i386/nanobsd), and I noticed that when under heavy load (moving data between internal subnets), the CPU utilization as reported by top would show right around 50% interrupt, but pretty much everything else completely idle. This suggested to me that the routing functionality was only taking advantage of one of my CPU's two cores. Switching to top -SH, it became obvious that all of the interrupts from all of the nics were only being handled by CPU0. Some poking around online pointed me to cpuset, where I verified my findings.

I then used cpuset to pin one of the interrupt handlers to CPU1 and immediately saw the other CPU core take on load and the maximum throughput through the firewall jumped by 70-80%.

So… this raises some questions in my mind...

• Why was my system doing this?

• Is what I did (use cpuset to force processor affinity) an appropriate fix?

• Is this a bug?

Any insight would be appreciated.

[Harvy66]

All of the NIC's interrupted pinned to the same CPU is a strange setup. Could be more of a corner case than a bug, but it does sound undesirable.