UPDATE — Root cause identified and confirmed
Posting this because I find it frustrating when threads like this go cold without a resolution. Full findings below.
Summary
Unbound was not the root cause—it was the victim. A USB-connected UPS (CyberPower SL700U) was repeatedly cycling connect/disconnect events, leaking kernel DMA bounce buffers at approximately 10 MB/hour. After ~10 days of uptime, the accumulated wired memory (~2,900 MB) left only ~187 MB free, forcing the OOM killer to target Unbound as the largest unprivileged process. Because this leaked memory remained trapped in kernel space, restarting Unbound without rebooting failed every time.
Resolution & Next Steps
The immediate fix was to disconnect the UPS and reboot the system to fully clear the wired memory. My apologies for the red herring and for unintentionally misrepresenting this as a 26.03 issue. Moving forward, I will be filing a bug against the FreeBSD USB HID driver regarding how it handles DMA management during frequent connect/disconnect cycles.
Diagnostic path
No log entries from Unbound even at verbosity level 3. This is characteristic of SIGKILL from the kernel rather than an internal crash — a process cannot log its own OOM kill.
Confirmed OOM kill found in rotated system log:
May 12 12:41:52 pfSense-small kernel: pid 57593 (unbound), jid 0, uid 59, was killed: failed to reclaim memory
Memory state before crash (system had been up 10 days, 17 hours):
Mem: 14M Active, 21M Inact, 198M Laundry, 2900M Wired, 187M Free
2,900 MB of wired memory on a 3.3 GB system is abnormal. Wired memory cannot be paged out and is only released by a kernel restart.
vmstat -m identified the source:
bounce 154820 634142720 154820
busdma 154822 9908992 154822
~605 MB of DMA bounce buffers with a matching busdma descriptor count — both growing. At ~10 MB/hour accumulation rate, projecting back over 10.75 days of uptime yields ~2,917 MB, which matches the observed 2,900 MB of wired memory almost exactly.
dmesg confirmed the USB cycling:
ugen1.2: <CPS SL Series> at usbus1
ugen1.2: <CPS SL Series> at usbus1 (disconnected)
This pair repeated continuously. Each connect/disconnect cycle allocated kernel DMA bounce buffers that were never freed.
Minute-by-minute monitoring confirmed the growth rate and halt:
Pre-disconnect (bounce buffer samples from cron monitor):
07:02 606 MB
07:13 608 MB (+2 MB/11 min = ~9.8 MB/hr)
07:35 612 MB
07:57 616 MB
08:19 619 MB
08:41 623 MB
08:52 624 MB ← USB cable disconnected
09:03 624 MB
09:14 624 MB
09:25 624 MB
09:36 624 MB ← still flat, 44 minutes later
Growth stopped immediately and completely upon disconnecting the USB cable.
Control comparison — Florida system
I have an identical Netgate 2100 running pfSense Plus 26.03 at a separate location (Florida) with a different UPS model and no USB cycling in dmesg. After 57+ hours of uptime:
Bounce buffers: 4,228 items / 16.5 MB — stable, not growing
Wired memory: 545 MB flat
Free memory: ~2,400–2,570 MB, stable throughout
Unbound RSS: plateaued at ~81.7 MB
The contrast with California (155,000+ bounce buffers / 607 MB growing at ~10 MB/hr) is unambiguous.
Why restarting Unbound without rebooting didn't work
The OOM killer reclaims the killed process's user-space heap (~80–90 MB), but the bounce buffers live in kernel space — owned by the DMA subsystem, entirely independent of any user-space process. After the kill, free memory increased only marginally (~80 MB recovered against ~2,900 MB still wired). Restarting Unbound immediately placed it back into the same near-exhausted memory environment and it was killed again almost instantly. A full reboot was the only path to clearing the wired kernel memory.
Is this a 26.03 regression?
Almost certainly not. The CyberPower UPS was connected while still running 25.07.1. The upgrade to 26.03 required a reboot, which happened to clear the accumulated bounce buffers and reset the counter. Had the system stayed on 25.07.1 long enough with the UPS connected, the same outcome would have occurred. The bug is in FreeBSD's USB HID DMA cleanup path — bounce buffers are not freed when a USB device disconnects — not in pfSense application code.
Resolution
Immediate: Disconnect the USB cable from the offending UPS. Bounce buffer accumulation stops instantly.
Clear existing wired memory: Reboot. The 624 MB already accumulated cannot be released any other way.
Underlying bug: FreeBSD USB subsystem does not free DMA bounce buffers on device disconnect when a device cycles repeatedly. This is worth filing upstream with FreeBSD if not already known.
Hope this saves someone else the diagnostic journey. Happy to share monitoring scripts or raw log data if useful.