CPU usage for interrupts and pf?
-
Hello all,
I have a question here about how pfSense (or FreeBSD really?) behaves when acting as a firewall. I understand the pf process is "giant locked" to a single CPU core when inspecting packets inbound and outbound. I was wondering, how does that manifest when I look at "top -P" on the firewall?
Right now I have a Myricom 10G NIC, and the mxge driver is "multiplexing" interrupt processing across all the CPU cores for speed. So, when the firewall is busy, I see all the cpu cores quite busy processing interrupts (like 70% or more CPU utilization). But, all CPU work seems to be in interrupts. I don't see anything, or very little, in system or user space for CPU utilization. Should the pf process be using some CPU too? If so, how could I tell that? I'm trying to figure out if I'm limited by not having enough CPU to process the interrupts or not enough CPU to process the packet filtering process. Right now it looks like interrupts but I'm not sure.
Does anyone know?
Thanks!
-
pf is in the kernel so it just shows as "system" usage, not in a thread.
Also use "top -SH" to make sure you're seeing all possible threads.
-
OK, great! So, given that I see less than 5% system usage and more than 70% interrupt usage when pushing 2+Gb/s, I can likely add more CPU cores and more interrupt queues and increase my maximum capacity, yes? I would guess my box is doing more work trying to pull packets off the NIC than actually processing pf rules and state table lookups, given that breakdown. Does that seem to be a reasonable assumption?
-
On another note - I have heard from the freebsd-net list that the version of pf in -9 (or -head) has some locking fixes and other goodies. Are you guys by chance backporting that version of pf into pfSense 2.0.3 or 2.1? Word on the street is that it is a lot better performing that what -8 provides.
-
No chance of that being in 2.0.x, nor in 2.1.
pfSense 2.2 may be targeting FreeBSD 10.x and we'll pick it up then, but we need to get the current development version out first before we can pursue that.
-
On another note - I have heard from the freebsd-net list that the version of pf in -9 (or -head) has some locking fixes and other goodies. Are you guys by chance backporting that version of pf into pfSense 2.0.3 or 2.1? Word on the street is that it is a lot better performing that what -8 provides.
Hi, I noticed your posts in -net, here is some info you might find helpful:
- SMP-pf is only available in FreeBSD-HEAD (what will eventually become FreeBSD 10)
- Backporting SMP-pf to 8.x or 9.x seems highly unlikely, because there have been many changes "under the hood" that break kernel APIs
- FreeBSD kernel developmer time would be better spent in porting a newer version of OpenBSD pf into FreeBSD
- a FreeBSD10-based version of pfSense doesn't seem to be coming soon (consider that pfsense 2.1 has been in development for 15+ months)
Assuming you want to stay with pfSense, your best option right now would be to try pfSense v2.1 which is based on FreeBSD 8.3 with several updated drivers for popular NICs and spend some time fine-tuning NIC driver tunables.
