But simply disabling that should run the CPU at full speed all the time.

I really appreciate your feedback. And you are right.

The problem was a suspected and annoying OOM and/or failing hardware or even a configuration mistake.
For the life of me, I could not find a cause or find any other solution whatsoever by myself (reinstalled, used a backup, removed packages, reinstalled packages clean, etc.).

SpeedShift (HWP) is indeed far faster than SpeedStep (EST). HWP runs in CPU microcode, EST is a software driver. The Xeon D-1521 does not support HWP.

You should only ever use one frequency control method. If your CPU supports both SpeedStep and SpeedShift you should use SpeedShift. It's far far faster reacting. It runs in the CPU directly rather than relying on a software daemon. The controls in pfSense only configure it. It varies with CPU but if you set SpeedShift towards the performance end of the scale it should have no problems. Say at 30%.

If you really want maximum performance and are not worried about heat or power then just disable both. Running multiple custom scripts is almost never a good idea.

Host: pfSense Plus 26.03.1 on Supermicro Xeon D-1521
Date: July 21, 2026
Author: Diagnosis and remediation session

1. Platform Specifications

Hardware

Software

Installed pfSense Packages

2. Symptoms

• Internet speed is fast immediately after reboot, then gradually degrades over days
• System eventually becomes completely unresponsive -- requires physical reset button
• Daily scheduled reboot (/etc/rc.reboot at 02:03) often fails to fire
• No crash dumps written, no kernel panic messages in logs -- system simply stops with a multi-hour silent gap in the log

3. Root Cause 1: CPU Throttled to Half Speed (PRIMARY)

Discovery

The CPU was running at 800-1700 MHz instead of the rated 2401 MHz -- between 33% and 71% of intended speed. After the most recent boot, observed frequencies were:

• 09:37: 1500 MHz (after 50 min uptime)
• 09:43: 1100 MHz (dropped while idle)
• 09:46: 2401 MHz (manually forced)
• 09:46+01s: 1700 MHz (EST driver dropped it back within 1 second)
• 09:48: 2400 MHz (watchdog active, stable)

Why

dmesg: sysctl_register_oid: can't re-use a leaf (machdep.hwpstate_pkg_ctrl)!

Two FreeBSD CPU frequency drivers conflict on this FreeBSD 16.0-CURRENT kernel:

• est (Enhanced SpeedStep) -- driver module, registers first during boot
• hwpstate_intel (Hardware P-State) -- tries to register the same machdep.hwpstate_pkg_ctrl sysctl second, fails

The EST driver then aggressively downclocks the CPU for power saving, dropping to 1.1 GHz at idle (46% of rated speed). Under network or interrupt load, it fails to ramp up fast enough.

The degradation cycle

Boot (fast) --> CPU at 1100-1500 MHz --> NIC interrupts pile up at 31K/sec per queue -->
Network latency increases --> Retransmits generate more interrupts -->
CPU can't catch up at half speed --> Interrupt queue overflows (intr_queue_drops > 0) -->
Packet loss --> Perceived speed degradation -->
System enters feedback loop of more retransmits/less CPU --> Eventually locks up

Evidence

Fix: Three-layer approach

Layer 1: Continuous watchdog (IMMEDIATE)

A background script at /usr/local/bin/cpu_freq_watchdog.sh runs every 5 seconds
and forces all cores to 2401 MHz:

while :; do
  for c in 0 1 2 3 4 5 6 7; do
    sysctl dev.cpu.${c}.freq=2401 > /dev/null 2>&1
  done
  sleep 5
done

Started from /etc/rc.local with & (must background).

Result: CPU stable at 2400-2401 MHz with watchdog running.

Layer 2: Boot-level disable (PERMANENT AFTER REBOOT)

/boot/loader.conf.local:

hint.est.0.disabled=1
hint.acpi_throttle.0.disabled="1"

This prevents the EST driver from loading entirely. loader.conf.local survives
pfSense package updates and config regeneration -- it is NOT overwritten like
/etc/crontab or /var/unbound/unbound.conf.

Layer 3: Per-minute cron (FALLBACK)

* * * * * root for c in 0 1 2 3 4 5 6 7; do sysctl dev.cpu.${c}.freq=2401 > /dev/null 2>&1; done

Added to /etc/crontab as secondary safety net. Note: EST's scaling interval is
sub-second, so per-minute cron alone is NOT sufficient -- the 5-second watchdog
is the critical layer.

4. Root Cause 2: Swap Overcommit (SECONDARY)

Discovery

vm.swap_total:   1,073,741,824 (1.0 GB)
vm.swap_reserved: 3,594,932,224 (3.6 GB)  -->  335% overcommit

The kernel reserves swap space for every memory allocation. When reserved exceeds
total, the kernel has zero headroom to page out under pressure.

Fix

8 GB swap file created via mdconfig:

dd if=/dev/zero of=/swap bs=1m count=8192
chmod 0600 /swap
mdconfig -a -t vnode -f /swap -u 0
swapon /dev/md0

Persisted in /etc/rc.local.

Result: swap_reserved: 3.5 GB well within swap_total: 9.3 GB (38% utilization).

5. Root Cause 3: Unbound Memory Pressure (CONTRIBUTING)

Discovery

unbound RSS: 1,071 MB after only 50 minutes uptime. The pfBlockerNG
python module (pfb_unbound.py) loads entirely inside unbound's process space.
The pfb_py_zone.txt (85 MB) and pfb_py_data.txt (16 MB) DNSBL data files
are parsed into in-memory data structures. Cache tuning does not reduce this
baseline -- it only constrains runtime growth.

Fix

6. Persistence Mechanism

7. Files Deployed

8. Verification

# CPU frequency
sysctl dev.cpu.0.freq        # Expected: 2400-2401

# Watchdog running
ps aux | grep cpu_freq_watchdog

# Swap
swapinfo -h                  # Expected: 9 GB total, 0 used

# Unbound config
grep -E "(outgoing-range|msg-cache-size|cache-min-ttl|prefetch|serve-expired)" /var/unbound/unbound.conf

# pfBlockerNG cron priority
grep pfblockerng /etc/crontab | grep nice

# @reboot entries
cat /etc/cron.d/pfsense_tweaks

# Boot-level EST disable
cat /boot/loader.conf.local

# Link health
ifconfig ix0 | grep media
sysctl dev.ix.0.mac_stats.crc_errs  # Expected: 0

9. Source code of files deployed

rc.local

#!/bin/sh

# Swap file - added Wed Jul  1 14:40:05 CEST 2026
mdconfig -a -t vnode -f /swap -u 0 && swapon /dev/md0

# Unbound memory tunings - re-apply after pfSense regenerates config
sleep 3
sed -i "" "s/^outgoing-range: 4096/outgoing-range: 2048/" /var/unbound/unbound.conf
sed -i "" "s/^msg-cache-size: 10m/msg-cache-size: 5m/" /var/unbound/unbound.conf
sed -i "" "s/^rrset-cache-size: 20m/rrset-cache-size: 10m/" /var/unbound/unbound.conf
sed -i "" "s/^cache-min-ttl: 0/cache-min-ttl: 300/" /var/unbound/unbound.conf
sed -i "" "s/^prefetch: yes/prefetch: no/" /var/unbound/unbound.conf
sed -i "" "s/^prefetch-key: yes/prefetch-key: no/" /var/unbound/unbound.conf
sed -i "" "s/^serve-expired: no/serve-expired: yes/" /var/unbound/unbound.conf

# Restart unbound with tuned config
/bin/kill -9 $(/bin/pgrep unbound) 2>/dev/null
sleep 1
/usr/local/sbin/unbound -c /var/unbound/unbound.conf

# pfBlockerNG cron with low CPU priority
sed -i "" "s|/usr/local/bin/php /usr/local/www/pfblockerng/pfblockerng.php cron|/usr/bin/nice -n 19 /usr/local/bin/php /usr/local/www/pfblockerng/pfblockerng.php cron|" /etc/crontab

# Start CPU frequency watchdog (est/hwpstate conflict workaround)
/usr/local/bin/cpu_freq_watchdog.sh # Lock all CPUs

# Lock all CPUs to max frequency - est/hwpstate conflict on FreeBSD 16-CURRENT
for c in 0 1 2 3 4 5 6 7; do sysctl dev.cpu.${c}.freq=2401 > /dev/null 2>&1; done

pfsense_tweaks.cron

# pfSense post-boot tweaks
# Runs 30s after boot to re-apply settings that pfSense overwrites on boot
@reboot root sleep 10 && for c in 0 1 2 3 4 5 6 7; do /usr/sbin/sysctl dev.cpu.${c}.freq=2401 > /dev/null 2>&1; done && sleep 20 && /bin/sed -i "" "s/^outgoing-range: 4096/outgoing-range: 2048/" /var/unbound/unbound.conf && /bin/sed -i "" "s/^msg-cache-size: 10m/msg-cache-size: 5m/" /var/unbound/unbound.conf && /bin/sed -i "" "s/^rrset-cache-size: 20m/rrset-cache-size: 10m/" /var/unbound/unbound.conf && /bin/sed -i "" "s/^cache-min-ttl: 0/cache-min-ttl: 300/" /var/unbound/unbound.conf && /bin/sed -i "" "s/^prefetch: yes/prefetch: no/" /var/unbound/unbound.conf && /bin/sed -i "" "s/^prefetch-key: yes/prefetch-key: no/" /var/unbound/unbound.conf && /bin/sed -i "" "s/^serve-expired: no/serve-expired: yes/" /var/unbound/unbound.conf && /bin/kill -9 $(/bin/pgrep unbound) 2>/dev/null; sleep 1; /usr/local/sbin/unbound -c /var/unbound/unbound.conf && /bin/sed -i "" "s|/usr/local/bin/php /usr/local/www/pfblockerng/pfblockerng.php cron|/usr/bin/nice -n 19 /usr/local/bin/php /usr/local/www/pfblockerng/pfblockerng.php cron|" /etc/crontab

loader.conf.local

if_ixl_load="YES"
hint.est.0.disabled=1
hint.acpi_throttle.0.disabled="1"

cpu_freq_watchdog.sh

#!/bin/sh
# CPU frequency watchdog - EST/hwpstate conflict workaround
# Forces all CPUs to max frequency every 5 seconds
while :; do
  for c in 0 1 2 3 4 5 6 7; do
    /usr/sbin/sysctl dev.cpu.${c}.freq=2401 > /dev/null 2>&1
  done
  sleep 5
done