Enhanced Intel SpeedStep / Speed Shift - Are they fully supported?
-
pfSense supports higher (lower?) C states if the BIOS passes them:
dev.cpu.0.cx_method: C1/mwait/hwc C2/mwait/hwc C3/mwait/hwc dev.cpu.0.cx_usage_counters: 9066 5086 8391 dev.cpu.0.cx_usage: 40.21% 22.56% 37.22% last 2204us dev.cpu.0.cx_lowest: C8 dev.cpu.0.cx_supported: C1/1/1 C2/2/127 C3/3/1048 -
@stephenw10
That's good to know. Typically, how deep down into the C-states would you expect for a router/firewall, without overly impacting performance?
️ -
For best performance I would disable C states deeper than C1. I have enabled a deep as it could go on some boxes that mostly sit idle and it makes no significant difference to either performance or power consumption beyond C3 as far as I could tell.
-
@tman222 Note vertical scale only goes to 30%, its an overpowered system
~24 hours pre/post upgrade
Pre/current
Edit: I took the opportunity to upgrade the motherboard firmware at the same time as update pfSense too which may have some impact.
-
@q54e3w
Am I reading this correctly - that Speed Shift seems to work the CPU harder?I have noticed that the CPU tends to run at a higher load and frequency but drops pretty quickly, at least on my own system.
️ -
@RobbieTT Yup, it looks like it although how the operating system and pfSense has changed will also impact. Its not an apples to apples comparison exactly because its not on the same operating system and use of IPSec-MB etc. I also upgraded the BIOS at the same time given the box was out of service anyway.
I'm currently trying '60' to see if it makes a noticeable difference to temps, power consumption or performance. -
Speedshift reacts much faster so you will the CPU at higher frequencies more often. Quite how that translates into apparent loading though.... unclear!
-
@stephenw10
I think we may miss stuff with the reporting rate vs actual rate of change. With different frequencies on different cores the values we see may not be truly representative.With my previous 'adaptive' profile I didn't observe any turbo frequencies (does not mean it wasn't happening I guess) but with Speed Shift on 80 it sticks at 799 MHz under varying demand and then rapidly turbos when things get busy:
[23.09-RELEASE][admin@Router-7.redacted.me]/root: powerd -v powerd: unable to determine AC line status CPU frequency is below user-defined minimum; changing frequency to 2700 MHz load 0%, current freq 799 MHz ( 0), wanted freq 4754 MHz load 4%, current freq 799 MHz ( 0), wanted freq 4605 MHz load 94%, current freq 799 MHz ( 0), wanted freq 5400 MHz load 163%, current freq 799 MHz ( 0), wanted freq 5400 MHz load 156%, current freq 799 MHz ( 0), wanted freq 5400 MHz load 144%, current freq 799 MHz ( 0), wanted freq 5400 MHz load 190%, current freq 799 MHz ( 0), wanted freq 5400 MHz load 110%, current freq 799 MHz ( 0), wanted freq 5400 MHz load 136%, current freq 799 MHz ( 0), wanted freq 5400 MHz load 265%, current freq 799 MHz ( 0), wanted freq 5400 MHz load 194%, current freq 799 MHz ( 0), wanted freq 5400 MHz load 143%, current freq 3199 MHz ( 0), wanted freq 5400 MHz load 162%, current freq 3199 MHz ( 0), wanted freq 5400 MHz load 138%, current freq 3199 MHz ( 0), wanted freq 5400 MHz load 125%, current freq 3199 MHz ( 0), wanted freq 5400 MHz load 113%, current freq 3199 MHz ( 0), wanted freq 5400 MHz load 150%, current freq 3199 MHz ( 0), wanted freq 5400 MHz load 135%, current freq 3199 MHz ( 0), wanted freq 5400 MHz load 98%, current freq 3199 MHz ( 0), wanted freq 5400 MHz load 179%, current freq 3199 MHz ( 0), wanted freq 5400 MHz load 169%, current freq 3199 MHz ( 0), wanted freq 5400 MHz load 113%, current freq 3199 MHz ( 0), wanted freq 5400 MHz load 108%, current freq 3199 MHz ( 0), wanted freq 5400 MHz load 125%, current freq 3199 MHz ( 0), wanted freq 5400 MHz load 149%, current freq 3199 MHz ( 0), wanted freq 5400 MHz load 165%, current freq 3199 MHz ( 0), wanted freq 5400 MHz load 234%, current freq 3199 MHz ( 0), wanted freq 5400 MHz load 159%, current freq 3199 MHz ( 0), wanted freq 5400 MHz load 156%, current freq 3199 MHz ( 0), wanted freq 5400 MHz load 99%, current freq 3199 MHz ( 0), wanted freq 5400 MHz load 80%, current freq 3199 MHz ( 0), wanted freq 5400 MHz load 143%, current freq 3199 MHz ( 0), wanted freq 5400 MHz load 176%, current freq 3199 MHz ( 0), wanted freq 5400 MHz load 116%, current freq 3199 MHz ( 0), wanted freq 5400 MHz load 113%, current freq 3199 MHz ( 0), wanted freq 5400 MHz load 107%, current freq 3199 MHz ( 0), wanted freq 5400 MHz load 141%, current freq 3199 MHz ( 0), wanted freq 5400 MHz load 177%, current freq 3199 MHz ( 0), wanted freq 5400 MHz load 101%, current freq 3199 MHz ( 0), wanted freq 5400 MHz load 146%, current freq 3199 MHz ( 0), wanted freq 5400 MHz load 166%, current freq 3199 MHz ( 0), wanted freq 5400 MHz load 105%, current freq 3199 MHz ( 0), wanted freq 5400 MHz load 78%, current freq 3199 MHz ( 0), wanted freq 5400 MHz load 116%, current freq 3199 MHz ( 0), wanted freq 5400 MHz load 184%, current freq 3199 MHz ( 0), wanted freq 5400 MHz load 138%, current freq 3199 MHz ( 0), wanted freq 5400 MHz load 162%, current freq 3199 MHz ( 0), wanted freq 5400 MHz load 129%, current freq 3199 MHz ( 0), wanted freq 5400 MHz load 200%, current freq 3199 MHz ( 0), wanted freq 5400 MHz load 163%, current freq 3199 MHz ( 0), wanted freq 5400 MHz load 175%, current freq 3199 MHz ( 0), wanted freq 5400 MHz load 31%, current freq 3199 MHz ( 0), wanted freq 5400 MHz load 3%, current freq 3199 MHz ( 0), wanted freq 5231 MHz load 0%, current freq 3199 MHz ( 0), wanted freq 5067 MHz load 0%, current freq 3199 MHz ( 0), wanted freq 4908 MHz load 54%, current freq 3199 MHz ( 0), wanted freq 5400 MHz load 41%, current freq 3199 MHz ( 0), wanted freq 5400 MHz load 39%, current freq 3199 MHz ( 0), wanted freq 5400 MHz load 37%, current freq 3199 MHz ( 0), wanted freq 5400 MHz load 89%, current freq 3199 MHz ( 0), wanted freq 5400 MHz load 37%, current freq 3199 MHz ( 0), wanted freq 5400 MHz load 31%, current freq 3199 MHz ( 0), wanted freq 5400 MHz load 47%, current freq 3199 MHz ( 0), wanted freq 5400 MHz load 31%, current freq 3199 MHz ( 0), wanted freq 5400 MHz load 30%, current freq 3199 MHz ( 0), wanted freq 5400 MHz load 39%, current freq 799 MHz ( 0), wanted freq 5400 MHz load 65%, current freq 799 MHz ( 0), wanted freq 5400 MHz load 75%, current freq 799 MHz ( 0), wanted freq 5400 MHz load 22%, current freq 799 MHz ( 0), wanted freq 5231 MHz load 25%, current freq 799 MHz ( 0), wanted freq 5231 MHz load 44%, current freq 799 MHz ( 0), wanted freq 5400 MHz load 110%, current freq 799 MHz ( 0), wanted freq 5400 MHz load 127%, current freq 799 MHz ( 0), wanted freq 5400 MHz load 176%, current freq 799 MHz ( 0), wanted freq 5400 MHz load 138%, current freq 799 MHz ( 0), wanted freq 5400 MHz load 36%, current freq 799 MHz ( 0), wanted freq 5400 MHz load 11%, current freq 799 MHz ( 0), wanted freq 5231 MHz load 81%, current freq 799 MHz ( 0), wanted freq 5400 MHz load 185%, current freq 799 MHz ( 0), wanted freq 5400 MHz load 210%, current freq 799 MHz ( 0), wanted freq 5400 MHz load 197%, current freq 799 MHz ( 0), wanted freq 5400 MHz load 38%, current freq 799 MHz ( 0), wanted freq 5400 MHz load 44%, current freq 799 MHz ( 0), wanted freq 5400 MHz load 19%, current freq 799 MHz ( 0), wanted freq 5231 MHz load 22%, current freq 799 MHz ( 0), wanted freq 5067 MHz load 21%, current freq 799 MHz ( 0), wanted freq 4908 MHz load 58%, current freq 799 MHz ( 0), wanted freq 5400 MHz load 138%, current freq 799 MHz ( 0), wanted freq 5400 MHz load 34%, current freq 799 MHz ( 0), wanted freq 5400 MHz load 25%, current freq 799 MHz ( 0), wanted freq 5400 MHz load 52%, current freq 799 MHz ( 0), wanted freq 5400 MHz load 119%, current freq 799 MHz ( 0), wanted freq 5400 MHz load 17%, current freq 799 MHz ( 0), wanted freq 5231 MHzI presume the above is just the fastest core though.
️ -
I agree. When I was first testing Speedshift I had a very hard time determining if my changes were actually doing anything. I came to the conclusion it's because simply running sysctl to read the cpu state causes it to bump the frequency. For example running
sysctl dev.cpu.0.freq dev.cpu.1.freq dev.cpu.2.freq dev.cpu.3.freqgives the same result as runningsysctl dev.cpu.3.freq dev.cpu.2.freq dev.cpu.1.freq dev.cpu.0.freq. I.e. whichever core you query first gives the lowest result and each subsequent core is running faster. -
@stephenw10 said in Enhanced Intel SpeedStep / Speed Shift - Are they fully supported?:
I came to the conclusion it's because simply running sysctl to read the cpu state causes it to bump the frequency.
Thankfully I don't see that, at least on this cpu:
[23.09-RELEASE]/root: sysctl dev.cpu.0.freq dev.cpu.1.freq dev.cpu.2.freq dev.cpu.3.freq dev.cpu.4.freq dev.cpu.5.freq dev.cpu.6.freq dev.cpu.7.freq dev.cpu.0.freq: 799 dev.cpu.1.freq: 799 dev.cpu.2.freq: 799 dev.cpu.3.freq: 799 dev.cpu.4.freq: 799 dev.cpu.5.freq: 799 dev.cpu.6.freq: 799 dev.cpu.7.freq: 799 [23.09-RELEASE]/root:Heisenberg defeated.
Less luck with PPPoE handling though (920 Mbps download test):
[23.09-RELEASE]/root: sysctl dev.cpu.0.freq dev.cpu.1.freq dev.cpu.2.freq dev.cpu.3.freq dev.cpu.4.freq dev.cpu.5.freq dev.cpu.6.freq dev.cpu.7.freq dev.cpu.0.freq: 3199 dev.cpu.1.freq: 799 dev.cpu.2.freq: 799 dev.cpu.3.freq: 799 dev.cpu.4.freq: 799 dev.cpu.5.freq: 799 dev.cpu.6.freq: 799 dev.cpu.7.freq: 799 [23.09-RELEASE]/root:️ -
Hmm, I guess as long as the load sysctl imposes doesn't push it over whatever the threshold is you wouldn't see that. You CPU is likely a lot more powerful than what I'm seeing that on.
-
Per core working just fine here.
-
BTW here is a great article showing why this is so important.
https://pcper.com/2015/11/intel-speed-shift-tested-significant-user-experience-improvements/
For those who weren't utilizing powerd it's no big deal while those of us who were welcome this update with open arms.
In these examples you can see that it really only take a couple of milliseconds now to ramp clock speed.
Essentially meaning there is no reason to not run this on a modern CPU that supports it when concerned about the best possible performance. HUGE for those of us with power hungry x86 CPU's that are running 24/7.
-
I appear to get higher (better) PPPoE throughput on my Xeon-D, which I didn't expect. I need to think a bit more as to why.
Anyway, just Intel Speed Select Technology to come...
️ -
@RobbieTT What is the model of your CPU?
Were you using PowerD before?
Also any virtualization involved?
-
-
@RobbieTT This is way better than PowerD, PowerD = SpeedStep
Check out my link above.
-
-
After testing / monitoring for another week, I have concluded that a Speed Shift setting of "60" works provides a pretty good performance / efficiency trade off on the Intel Xeon D-1718T CPU in one of my systems. If I increase the value further to "80", I find that the low CPU frequencies become too sticky (i.e. it seems to take too long to ramp up), while not really resulting in incremental power savings. If I lower to "50" the CPU ramps up too quickly to top frequencies, resulting in a temperature increase. What's interesting - on another system with a Intel Core i3-10100 CPU, a setting of "60" appears not conservative enough and the CPU still ramps up very quickly to higher frequencies. Could there be some differences between how different Intel CPU architectures handle / implement Speed Shift?
-
I just completed a number of WAN latency tests on my Xeon D-1718T system and had different results. I had to go down to a setting of 25 for Speed Shift to prevent the router from introducing latency on the WAN connection. 30 might have been ok, but 25 seemed to provide the best throughput and power results. A value of 60 increased the latency back to the same values I had when running on my Atom C3758 based router with PowerD set at Max values.
