Supported CPU C-States

snow

Hi guys,

My current pfsense setup is based on an Intel Celeron G3930 (Kaby Lake).
When using any Linux Distro like Debian 9, Ubuntu 18.x, etc., the deepest CPU C-State C8 (also Package C8) will be used when System is in idle state.

When checking the status in pfsense, seems to be the deepest available C-State is C3:

dev.cpu.0.cx_supported: C1/1/1 C2/2/151 C3/3/256

Does anyone know if it's possible to get the CPU C-States C3+ available as well?

Thanks,
Philipp

stephenw10

Hmm, not sure I've ever seen C-states higher (lower?) than C3 in FreeBSD. Though last time I played around with that stuff it made immeasurably small difference to anything enabling further states.

Is it actually using the states you have currently? What does dev.cpu.0.cx_usage show?

What are you actually hoping to achieve by enabling further states?

Steve

snow

Hi Steve,

As shown below, C3 state will be currently used:

sysctl dev.cpu |grep cx
dev.cpu.1.cx_method: C1/mwait/hwc C2/mwait/hwc C3/mwait/hwc
dev.cpu.1.cx_usage_counters: 1513873 2478992 9943845
dev.cpu.1.cx_usage: 10.86% 17.78% 71.35% last 3386us
dev.cpu.1.cx_lowest: C8
dev.cpu.1.cx_supported: C1/1/1 C2/2/151 C3/3/256
dev.cpu.0.cx_method: C1/mwait/hwc C2/mwait/hwc C3/mwait/hwc
dev.cpu.0.cx_usage_counters: 83485812 108776041 11098
dev.cpu.0.cx_usage: 43.42% 56.57% 0.00% last 102us
dev.cpu.0.cx_lowest: C8
dev.cpu.0.cx_supported: C1/1/1 C2/2/151 C3/3/256

Seems to be the C3 state will be used more on the second cpu core (cpu.1):

dev.cpu.0.cx_usage: 43.42% 56.57% 0.00% last 102us
dev.cpu.1.cx_usage: 10.86% 17.78% 71.35% last 3386us

Concerning the lowest state C8, I set the following in > System> Advanced > System Tunables:

dev.cpu.0.cx_lowest > C8
dev.cpu.1.cx_lowest > C8
hw.acpi.cpu.cx_lowest > C8

The lowest idle power consumption is currently 11.2W (measured with a power meter).
If the same system is running on Debian 9, the lowest idle power consumption is below 8W.

stephenw10

That additional power consumption may not be the CPU of course. 3W seems like a lot for states below C3 which is already very low power. As I said I'm not sure I've seen lower C states reported by FreeBSD. It might not be possible.

You have enabled powerd I assume? Though the provided P-states are swamped by C-state reductions I found.

This page, though now a bit dated, was very useful when I was poking at this stuff.
https://wiki.freebsd.org/TuningPowerConsumption

Steve

mrsunfire

Got an i7-1165G7 but I never get to see C2 or C3. All I get is this:

dev.cpu.3.cx_method: C1/mwait/hwc C2/mwait/hwc C3/mwait/hwc
dev.cpu.3.cx_usage_counters: 46820 0 0
dev.cpu.3.cx_usage: 100.00% 0.00% 0.00% last 171us
dev.cpu.3.cx_lowest: C1
dev.cpu.3.cx_supported: C1/1/1 C2/2/253 C3/3/1048
dev.cpu.2.cx_method: C1/mwait/hwc C2/mwait/hwc C3/mwait/hwc
dev.cpu.2.cx_usage_counters: 46701 0 0
dev.cpu.2.cx_usage: 100.00% 0.00% 0.00% last 141us
dev.cpu.2.cx_lowest: C1
dev.cpu.2.cx_supported: C1/1/1 C2/2/253 C3/3/1048
dev.cpu.1.cx_method: C1/mwait/hwc C2/mwait/hwc C3/mwait/hwc
dev.cpu.1.cx_usage_counters: 46400 0 0
dev.cpu.1.cx_usage: 100.00% 0.00% 0.00% last 149us
dev.cpu.1.cx_lowest: C1
dev.cpu.1.cx_supported: C1/1/1 C2/2/253 C3/3/1048
dev.cpu.0.cx_method: C1/mwait/hwc C2/mwait/hwc C3/mwait/hwc
dev.cpu.0.cx_usage_counters: 56149 0 0
dev.cpu.0.cx_usage: 100.00% 0.00% 0.00% last 237us
dev.cpu.0.cx_lowest: C1
dev.cpu.0.cx_supported: C1/1/1 C2/2/253 C3/3/1048

Did try the settings written here: https://wiki.freebsd.org/TuningPowerConsumption nothing helped. BIOS C-states enabled of course.

What am I missing?

stephenw10

You have to set dev.cpu.X.cx_lowest to C3 if you want it to use them. All 4 cores are restricted to C1 only in that output.

Steve

mrsunfire

@stephenw10 I've set it to Cmax. Is that wrong?

Screenshot 2022-11-20 084515.jpg

mvikman

I added this to System Tunables

"sysctl dev.cpu | grep cx" showes this, so I guess it's working

dev.cpu.3.cx_method: C1/mwait/hwc C2/mwait/hwc C3/mwait/hwc
dev.cpu.3.cx_usage_counters: 128 166 6341
dev.cpu.3.cx_usage: 1.92% 2.50% 95.56% last 2086us
dev.cpu.3.cx_lowest: C3
dev.cpu.3.cx_supported: C1/1/1 C2/2/151 C3/3/256
dev.cpu.2.cx_method: C1/mwait/hwc C2/mwait/hwc C3/mwait/hwc
dev.cpu.2.cx_usage_counters: 174 154 6615
dev.cpu.2.cx_usage: 2.50% 2.21% 95.27% last 32us
dev.cpu.2.cx_lowest: C3
dev.cpu.2.cx_supported: C1/1/1 C2/2/151 C3/3/256
dev.cpu.1.cx_method: C1/mwait/hwc C2/mwait/hwc C3/mwait/hwc
dev.cpu.1.cx_usage_counters: 221 220 16075
dev.cpu.1.cx_usage: 1.33% 1.33% 97.32% last 368us
dev.cpu.1.cx_lowest: C3
dev.cpu.1.cx_supported: C1/1/1 C2/2/151 C3/3/256
dev.cpu.0.cx_method: C1/mwait/hwc C2/mwait/hwc C3/mwait/hwc
dev.cpu.0.cx_usage_counters: 142 210 13944
dev.cpu.0.cx_usage: 0.99% 1.46% 97.53% last 14974us
dev.cpu.0.cx_lowest: C3
dev.cpu.0.cx_supported: C1/1/1 C2/2/151 C3/3/256

mrsunfire

@mvikman Thanks! For pfSense hw.acpi.cpu.cx_lowest is the right command. I've set it to Cmax.

Did you set anything for "kern.hz"? My CPU seems to stay mostly at C1.

dev.cpu.3.cx_method: C1/mwait/hwc C2/mwait/hwc C3/mwait/hwc
dev.cpu.3.cx_usage_counters: 411155 21225 1607
dev.cpu.3.cx_usage: 94.73% 4.89% 0.37% last 145us
dev.cpu.3.cx_lowest: C8
dev.cpu.3.cx_supported: C1/1/1 C2/2/253 C3/3/1048
dev.cpu.2.cx_method: C1/mwait/hwc C2/mwait/hwc C3/mwait/hwc
dev.cpu.2.cx_usage_counters: 427522 21371 1503
dev.cpu.2.cx_usage: 94.92% 4.74% 0.33% last 130us
dev.cpu.2.cx_lowest: C8
dev.cpu.2.cx_supported: C1/1/1 C2/2/253 C3/3/1048
dev.cpu.1.cx_method: C1/mwait/hwc C2/mwait/hwc C3/mwait/hwc
dev.cpu.1.cx_usage_counters: 434369 22616 1595
dev.cpu.1.cx_usage: 94.72% 4.93% 0.34% last 131us
dev.cpu.1.cx_lowest: C8
dev.cpu.1.cx_supported: C1/1/1 C2/2/253 C3/3/1048
dev.cpu.0.cx_method: C1/mwait/hwc C2/mwait/hwc C3/mwait/hwc
dev.cpu.0.cx_usage_counters: 457227 29227 1425
dev.cpu.0.cx_usage: 93.71% 5.99% 0.29% last 191us
dev.cpu.0.cx_lowest: C8
dev.cpu.0.cx_supported: C1/1/1 C2/2/253 C3/3/1048

stephenw10

Mmm, @mvikman's CPU must be very lightly used. Anything I've ever tried that on looks more like your output there, mostly in C1.

fireodo

@mrsunfire said in Supported CPU C-States:

@mvikman Thanks! For pfSense hw.acpi.cpu.cx_lowest is the right command. I've set it to Cmax.

Did you set anything for "kern.hz"? My CPU seems to stay mostly at C1.

If the operating system states:

dev.cpu.0.cx_supported: C1/1/1 C2/2/253 C3/3/1048

why do you think you can go

dev.cpu.0.cx_lowest: C8

below C3?

Just asking out of curiosity :-)

stephenw10

It won't ever go below 3 but I wouldn't expect it to hurt.

mrsunfire

@fireodo C8 is max my CPU is supporting. I just set Cmax in system tunables. So I think the CPU decides the max limit even though FreeBSD only supports C3.

mvikman

@stephenw10 said in Supported CPU C-States:

Mmm, @mvikman's CPU must be very lightly used. Anything I've ever tried that on looks more like your output there, mostly in C1.

LOL yeah, it's just my home router and it's got a i5-6500, just haven't had time to dabble with pfblocker and such things yet

@mrsunfire No, that was the only line I added.

Though I have PowerD set to Adaptive instead of HiAdaptive because HiAdaptive doesn't seem let CPU to throttle speed lower MHz...