Sharing on a 1Gbps Router built(+Tests on clocked at 1.6GHz and Default)

nexusN

Check your logs, make sure that powerd is working correctly.
You may have to change the kernel timecounter.
Powerd has a number of tuning options the govern how quickly it ramps up the cpu speed. These can be tuned to give you better performance/power ratio. See the man page.

Steve

Thanks for the idea,
I will try to see if I can tune to strike a balance.

stephenw10

You can do some testing manually. See my experiences with it here:
http://forum.pfsense.org/index.php/topic,20095.msg161139.html#msg161139

Even at the most conservative setting it should ramp up in a few seconds. How long does that test take? It looks more like it's not running correctly.

Steve

nexusN

@stephenw10:

You can do some testing manually. See my experiences with it here:
http://forum.pfsense.org/index.php/topic,20095.msg161139.html#msg161139

Even at the most conservative setting it should ramp up in a few seconds. How long does that test take? It looks more like it's not running correctly.

Steve

Maybe I can try to "warm it up" before carrying out the test.
The test takes a very short period, like 5 s for downloading and another 5s for uploading, the time maybe too short for the router to response.

nexusN

More on the PowerD tests,
as mentioned, the PowerD should be able to speed up upon the need of performance, I have tried to run a few tests continuously and found the below:

Each pair of column in the network utilization graph refers to a single test, the bench result on the left is referring to the last test(the pair at the right most).

I was wrong on my previous statement, in a pair, the first column should be for uploading and the second column is referring to downloading,
as all the tests are showing a result for a faster downloading and slower uploading.
This is important that, it means the Uploading test triggered the software to speed up the CPU thus a better result is obtained on downloading which follows closely while the uploading itself has been stuck with a lower performance.

Also, from the test it seems the degree the CPU speeds up is quite uncertain, the flat top of each columns states that the "speed up" actually stopped at a certain level during a transfer, and this level may not be at the best performance.

nexusN

Just now I tried to add the two command in loader.conf:

hint.p4tcc.0.disabled=1
hint.acpi_throttle.0.disabled=1

Then it seems with the throttling disabled the clock of CPU is fixed as I have set in BIOS:
1.6GHz

[2.0.1-RELEASE][admin@nchannel.nelsewhere.com]/root(2): sysctl dev.cpu.0.freq_levels
dev.cpu.0.freq_levels: 1600/39000

And upon a test, there is no more performance drop with the same tests with PowerD on.
Here comes another questions, if the CPU clock has been fixed in BIOS already, is there any other pros to leave PowerD on?
I have only one 2.5" harddisk in the box, it can't be turned off for idle even PowerD is on right?
Sad that I don't have a power meter to get the ans.

Thanks for your kind attention.

SlowGrind

That is a great looking rig… And the cost is about the same as an Atom board. I might just build this same setup.

nexusN

@nexusN:

Just now I tried to add the two command in loader.conf:
hint.p4tcc.0.disabled=1
hint.acpi_throttle.0.disabled=1
Then it seems with the throttling disabled the clock of CPU is fixed as I have set in BIOS:
1.6GHz
[2.0.1-RELEASE][admin@nchannel.nelsewhere.com]/root(2): sysctl dev.cpu.0.freq_levels
dev.cpu.0.freq_levels: 1600/39000
And upon a test, there is no more performance drop with the same tests with PowerD on.
Here comes another questions, if the CPU clock has been fixed in BIOS already, is there any other pros to leave PowerD on?
I have only one 2.5" harddisk in the box, it can't be turned off for idle even PowerD is on right?
Sad that I don't have a power meter to get the ans.

Thanks for your kind attention.

Apart from PowerD,
More on PowerSaving(Hopefully):
On some Googles, I have found that pfsense 2.0.1 on default doesn't enable C3 state for CPUs,
and I checked it to be true by:

[2.0.1-RELEASE][admin@nchannel.nelsewhere.com]/root(5): sysctl dev.cpu |grep cx
dev.cpu.0.cx_supported: C1/1 C2/80 C3/104
dev.cpu.0.cx_lowest: C1
dev.cpu.0.cx_usage: 100.00% 0.00% 0.00% last 500us
dev.cpu.1.cx_supported: C1/1 C2/80 C3/104
dev.cpu.1.cx_lowest: C1
dev.cpu.1.cx_usage: 100.00% 0.00% 0.00% last 500us

To get C3 state works, the little tweak is:

# sysctl dev.cpu.0.cx_lowest=C3

then you'll see:

dev.cpu.1.cx_lowest: C1 -> C3

Do this for all cores.
Verify by:

[2.0.1-RELEASE][admin@nchannel.nelsewhere.com]/root(12): sysctl dev.cpu | grep cx
dev.cpu.0.cx_supported: C1/1 C2/80 C3/104
dev.cpu.0.cx_lowest: C3
dev.cpu.0.cx_usage: 2.76% 8.00% 89.22% last 306us
dev.cpu.1.cx_supported: C1/1 C2/80 C3/104
dev.cpu.1.cx_lowest: C3
dev.cpu.1.cx_usage: 2.86% 10.01% 87.12% last 223us

Suggested setting (one with C2 state, other as deep as possible) by vermaden
To permanently set the lowest C state, add the lines in /etc/sysctl.conf.
"Update: I can't make it to run with booting even with the above lines added to sysctl.conf nor /etc/rc.conf,
is there any other ways?
Thanks."

While I am not sure if this really helps, desperately need a power meter…...for fun ;D

Thanks the below links:
http://forums.freebsd.org/showthread.php?t=172
http://wiki.freebsd.org/TuningPowerConsumption

stephenw10

powerd is only the daemon that controls the cpu frequency dependent on load. It does so via the appropriate driver, est in my case.
If your cpu doesn't have driver support then powerd can't control it usefully.
acpi throttling provides no real power saving.

Steve

Edit: Although others seem to have had good results. Check the sysctls for est or your boot logs.

nexusN

@stephenw10:

powerd is only the daemon that controls the cpu frequency dependent on load. It does so via the appropriate driver, est in my case.
If your cpu doesn't have driver support then powerd can't control it usefully.
acpi throttling provides no real power saving.

Steve

Edit: Although others seem to have had good results. Check the sysctls for est or your boot logs.

I see, I disabled throttling for it was having a impact on performance.
If PowerD is working on EST, if I have actually locked the CPU freq at 1.6GHz(which is the lowest),
would PowerD be completely useless in the case?

stephenw10

Yes if you have the cpu locked to 1 frequency then powerd can't switch it so it will be doing nothing.
When it's working correctly you should see a number of levels reported by dev.cpu.0.freq_levels. You board has a full set of bios acpi details as it's also reporting the power level (39,000mW).

Steve

nexusN

@stephenw10:

Yes if you have the cpu locked to 1 frequency then powerd can't switch it so it will be doing nothing.
When it's working correctly you should see a number of levels reported by dev.cpu.0.freq_levels. You board has a full set of bios acpi details as it's also reporting the power level (39,000mW).

Steve

Thanks for the detailed explanation, so I now set it with PowerD off, freq locked at 1.6GHz which is the lowest EST state for this CPU(there won't be issues as it is enough and the task running time is decided by time needed for data transfer), and both core set to have lowest C state as C3.
Hope the above will save me a few watts ::)

dreamslacker

@nexusN:

I see, I disabled throttling for it was having a impact on performance.
If PowerD is working on EST, if I have actually locked the CPU freq at 1.6GHz(which is the lowest),
would PowerD be completely useless in the case?

Yes. PowerD working on EST uses speedstep for power savings. Since EIST basically does multiplier reduction, locking the multiplier or having the lowest multiplier basically means that EST cannot do anything.
The main power savings doesn't come from EIST per se, it's actually something called Idle states (sometimes known as idle64) where the CPU is designed to reduce voltage when the processor is clocked down (by EIST).

You should be able to allow the processor to run at normal speeds and simply enable EIST. PowerD with EST will basically downclock the processor when at very low loads. Similarly, when under stress, the processor should be ramped up to max speed.

nexusN

@dreamslacker:

@nexusN:

I see, I disabled throttling for it was having a impact on performance.
If PowerD is working on EST, if I have actually locked the CPU freq at 1.6GHz(which is the lowest),
would PowerD be completely useless in the case?

Yes. PowerD working on EST uses speedstep for power savings. Since EIST basically does multiplier reduction, locking the multiplier or having the lowest multiplier basically means that EST cannot do anything.
The main power savings doesn't come from EIST per se, it's actually something called Idle states (sometimes known as idle64) where the CPU is designed to reduce voltage when the processor is clocked down (by EIST).

You should be able to allow the processor to run at normal speeds and simply enable EIST. PowerD with EST will basically downclock the processor when at very low loads. Similarly, when under stress, the processor should be ramped up to max speed.

Thanks for pointing out and I was thinking for the same,
would PowerD + EIST actually be able to lower the power consumption to a greater extent than if I lock the multiplier to the smallest value I can in BIOS? I do need to verify this.
And for the idle state, would it be something similar with C state? Pfsense has a default setting at C1 only, I tried to set it to C3, not sure if this is actually saving power though.

dreamslacker

@nexusN:

Thanks for pointing out and I was thinking for the same,
would PowerD + EIST actually be able to lower the power consumption to a greater extent than if I lock the multiplier to the smallest value I can in BIOS? I do need to verify this.
And for the idle state, would it be something similar with C state? Pfsense has a default setting at C1 only, I tried to set it to C3, not sure if this is actually saving power though.

It might, depending on the implementation on the CPU. EIST allows for voltage reduction and disabling it while holding the multiplier at the lowest value only allows you to gain power savings from lower operating frequency. However, enabling EIST will allow the core voltage to be reduced at lower loads, thus allowing additional savings. The savings might be minimal though.

Idle states is similar to C-states. However, C-states above C1E are only really useful if the core can be shut off. That is, if you have multiple cores and only 1 is in use, then the unused cores can be powered off. Alternatively, if the computer is in sleep mode or hibernation, C3/5/6 can be applied to almost completely power off the entire processor.

The key idea of using PowerD with EIST and disabling P4 throttling is that the CPU can ramp up to maximum speed when under load and still slow down to minimal speed when idle. Thus, you can get maximum performance and still save power when idling without manually setting the speeds in BIOS.
If your BIOS allows for it and you only need the CPU operating at minimum frequency, you can try undervolting the CPU in addition to locking the multiplier at the lowest frequency. If possible, applying a VID reduction is better than a fixed lower voltage since it would apply the reduced voltage at all states.

stephenw10

As Dreamslacker says using EIST allows both the CPU frequency and voltage to be changed according to load.
It provided quite a big power saving for me on my Pentium-M box.
You'll just have to get a power meter and try it out! :)

Steve

nexusN

Thanks all for your kind sharing :D
Probably I will restore the Freq control and let PowerD works on it, they are so designed and I believe there are reasons behind.
Still, I will need a power meter to make things clear :-\ Will try to see if I can get one.

nexusN

Just now I got to BIOS and restore everything to default,
after some studies I found that the previous act, locking the freq to 1.6GHz by fixing the Maximum possible multiplier, actually has no conflicts with EIST, as EIST is an separated option on another page.
With the clock locked, it will work always at the lowest step possible thus most power-saving, just no room for it to step up.

Now with the setting reset, the clock levels available becomes:

[2.0.1-RELEASE][admin@nchannel.nelsewhere.com]/root(6): sysctl dev.cpu.0.freq_levels
dev.cpu.0.freq_levels: 2400/65000 2300/61000 2200/58000 2100/55000 2000/51000 1900/48000 1800/45000 1700/42000 1600/39000

Maybe I can just leave it this way as the router will usually stick to the lowest clock with PowerD on, reading from the dashboard.
The difference in power consumption with locked clock should be minimal.

stephenw10

Exactly. Given that your board can route 850Mbps with cycles to spare it should remain at 1.6GHz almost all the time.
You can run powerd -v at the console to see the clock speed in real time.

Steve

nexusN

@stephenw10:

Exactly. Given that your board can route 850Mbps with cycles to spare it should remain at 1.6GHz almost all the time.
You can run powerd -v at the console to see the clock speed in real time.

Steve

Thanks a lot Steve, you have been keeping follow up my questions and offer kind and detailed explanation :D
Also the others who have shared your ideas ;D

Hultner

Could you try testing 1/1Gbs Full Duplex. upload and download simultaneously)? Would be grateful.