intel_pstate: Use the cpu load to determine the PercentPerformance
Subsystems (2)
PATCH 1/2 intel_pstate: Use the cpu load to determine the PercentPerformance
Aperf and Mperf counter are not enough to determine the Target P-state because they measure performance only when the targeted processor is in the C0 state (active state).
Because of that, we were computing the average P-state during the last period which can be very different from the average frequency (or percentage of performance).
As defined in the SDM (section 14.2), the PercentPerformance is defined by:
PercentPerformance = PercentBusy * (delta_aperf / delta_mperf);
The PercentBusy (or load) can be estimated as the ratio of the mperf counter running at a constant frequency only during active periods (C0) and the time stamp counter running at the same frequency but also during idle.
So, PercentBusy = 100 * (delta_mperf / delta_tsc)
and, PercentPerformance = 100 * (delta_mperf / delta_tsc) *
(delta_aperf / delta_mperf)
That can be simplified with:
PercentPerformance = 100 * (delta_aperf / delta_tsc)
--- drivers/cpufreq/intel_pstate.c | 87 +++++++++++++++--------------------------- 1 file changed, 31 insertions(+), 56 deletions(-)
drivers/cpufreq/intel_pstate.c +31 -56
@@ -69,7 +69,7 @@ static inline int ceiling_fp(int32_t x) }struct sample { -int32_t core_pct_busy; +int32_t cpu_load;u64 aperf;u64 mperf;u64 tsc;
@@ -993,21 +993,39 @@ static void intel_pstate_get_cpu_pstates(struct cpudata *cpu)intel_pstate_set_pstate (cpu, cpu->pstate.min_pstate, false); } -static inline void intel_pstate_calc_busy (struct cpudata *cpu) +static inline int32_t intel_pstate_calc_busy (struct cpudata *cpu) {struct sample *sample = &cpu->sample; -int64_t core_pct; +struct pstate_data *pstate = &cpu->pstate; +int64_t core_busy_ratio; - core_pct =int_tofp (sample->aperf) *int_tofp (100 ); - core_pct =div64_u64 (core_pct,int_tofp (sample->mperf)); +/* +* The load can be estimated as the ratio of the mperf counter +* running at a constant frequency only during active periods +* (C0) and the time stamp counter running at the same frequency +* also during C-states. +*/ + sample->cpu_load =div64_u64 (100 * sample->mperf, sample->tsc); + +/* +* The target P-state can be estimated with the following formula: +* PercentPerformance = PercentBusy * (delta_aperf/delta_mperf); +* (see Section 14.2 from Intel Software Developer Manual) +* with PercentBusy = 100 * (delta_mperf / delta_tsc) and +* PercentPerformance can be simplified with: +* (delta_mperf * delta_aperf) / (delta_tsc * delta_mperf) = +* delta_aperf / delta_tsc. Finally, we normalize core_busy_ratio, +* which was our actual percent performance to what we requested +* during the last sample period. The result will be a percentage of +* busy at a specified pstate. +*/ + core_busy_ratio =div64_u64 (int_tofp (100 ) * sample->aperf * + pstate->max_pstate, sample->tsc * pstate->current_pstate); - sample->freq =fp_toint ( -mul_fp (int_tofp ( - cpu->pstate.max_pstate_physical * - cpu->pstate.scaling /100 ), - core_pct)); + sample->freq =div64_u64 (sample->aperf * pstate->max_pstate * + pstate->scaling, sample->mperf); - sample->core_pct_busy = (int32_t )core_pct; +return core_busy_ratio; }static inline void intel_pstate_sample (struct cpudata *cpu)
@@ -1036,8 +1054,6 @@ static inline void intel_pstate_sample(struct cpudata *cpu) cpu->sample.mperf -= cpu->prev_mperf; cpu->sample.tsc -= cpu->prev_tsc; -intel_pstate_calc_busy (cpu); - cpu->prev_aperf = aperf; cpu->prev_mperf = mperf; cpu->prev_tsc = tsc;
@@ -1059,47 +1075,6 @@ static inline void intel_pstate_set_sample_time(struct cpudata *cpu)mod_timer_pinned (&cpu->timer, jiffies + delay); } -static inline int32_t intel_pstate_get_scaled_busy (struct cpudata *cpu) -{ -int32_t core_busy, max_pstate, current_pstate, sample_ratio; -s64 duration_us; -u32 sample_time; - -/* -* core_busy is the ratio of actual performance to max -* max_pstate is the max non turbo pstate available -* current_pstate was the pstate that was requested during -* the last sample period. -* -* We normalize core_busy, which was our actual percent -* performance to what we requested during the last sample -* period. The result will be a percentage of busy at a -* specified pstate. -*/ - core_busy = cpu->sample.core_pct_busy; - max_pstate =int_tofp (cpu->pstate.max_pstate_physical); - current_pstate =int_tofp (cpu->pstate.current_pstate); - core_busy =mul_fp (core_busy,div_fp (max_pstate, current_pstate)); - -/* -* Since we have a deferred timer, it will not fire unless -* we are in C0. So, determine if the actual elapsed time -* is significantly greater (3x) than our sample interval. If it -* is, then we were idle for a long enough period of time -* to adjust our busyness. -*/ - sample_time = pid_params.sample_rate_ms *USEC_PER_MSEC ; - duration_us =ktime_us_delta (cpu->sample.time, - cpu->last_sample_time); -if (duration_us > sample_time *3 ) { - sample_ratio =div_fp (int_tofp (sample_time), -int_tofp (duration_us)); - core_busy =mul_fp (core_busy, sample_ratio); - } - -return core_busy; -} -static inline void intel_pstate_adjust_busy_pstate (struct cpudata *cpu) {int32_t busy_scaled;
@@ -1111,7 +1086,7 @@ static inline void intel_pstate_adjust_busy_pstate(struct cpudata *cpu) from = cpu->pstate.current_pstate; pid = &cpu->pid; - busy_scaled =intel_pstate_get_scaled_busy (cpu); + busy_scaled =intel_pstate_calc_busy (cpu); ctl =pid_calc (pid, busy_scaled);
@@ -1119,7 +1094,7 @@ static inline void intel_pstate_adjust_busy_pstate(struct cpudata *cpu)intel_pstate_set_pstate (cpu, cpu->pstate.current_pstate - ctl, true); sample = &cpu->sample; - trace_pstate_sample(fp_toint(sample->core_pct_busy), + trace_pstate_sample(fp_toint(busy_scaled), fp_toint(busy_scaled), from, cpu->pstate.current_pstate,
Signature · 1 line
1.9.1
PATCH 2/2 intel_pstate: Change the setpoint for the cores
Change the setpoint to 60 accordingly to the new core busy scaled formula. The new formaula is based on the number of cycles per seconds (average frequency) divided by the requested frequency. So, we need to chose a setpoint more aggressive to improve performance.
Measured with this parameter, we noticed an improvement in Browsermark for power and perf compared to the old formula:
Score without the patch: 3517
Power without the patch: 6856 mW
Score with the patch: 3719
Power with the patch: 6265 mW
--- drivers/cpufreq/intel_pstate.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/cpufreq/intel_pstate.c +1 -1
@@ -872,7 +872,7 @@ static struct cpu_defaults core_params = { .pid_policy = { .sample_rate_ms = 10, .deadband = 0, - .setpoint = 97, + .setpoint = 60, .p_gain_pct = 20, .d_gain_pct = 0, .i_gain_pct = 0,
Signature · 1 line
1.9.1
RE: [PATCH v1 2/2] intel_pstate: Change the setpoint for the cores
Quoted 4 lines · On 2015.11.03 01:27 Philippe Longepe wrote:
Change the setpoint to 60 accordingly to the new core busy scaled formula. The new formaula is based on the number of cycles per seconds (average frequency) divided by the requested frequency. So, we need to chose a setpoint more aggressive to improve performance.
Myself, and so as to improve response to some games and such that use
many threads and such but often a lower overall CPU load, I think the setpoint should be set a little lower.
There is a tradeoff in reducing the setpoint further as it increases the noise and tendency to oscillate in the response curve. Ultimately, it may be desirable to introduce a little slope in the load / CPU frequency response curve.
I have a bunch of graphs comparing response curves. [1]
Quoted 3 lines
Measured with this parameter, we noticed an improvement in Browsermark for power and perf compared to the old formula:
I would like to try this test on my system. What is the exact test? Do I understand correctly, that I need a browser to do the test? (my test system is a server, and it doesn't have a browser.)
Quoted 6 lines
Score without the patch: 3517
Power without the patch: 6856 mWScore with the patch: 3719
Power with the patch: 6265 mW
There are some other Phoronix tests that we (the original maintainer and the a couple of others working with him used to use. See [1].
Please be aware that the last time I tried to bring back load based calculations, Kristen tested the proposed solution on some intel "specpower test bed and experienced a regression on haswell based server platforms vs. Dirks algorithm." I don't have any details.
Your response curve, and in particular your step function response time, is different, so it might worth re-testing.
References:
[1] double u double u double u dot smythies dot com/~doug/linux/intel_pstate/philippe_longepe/index.html
Re: [PATCH v1 2/2] intel_pstate: Change the setpoint for the cores
Quoted 8 lines · Doug Smythies
Quoted 4 lines · On 2015.11.03 01:27 Philippe Longepe wrote:
Change the setpoint to 60 accordingly to the new core busy scaled formula. The new formaula is based on the number of cycles per seconds (average frequency) divided by the requested frequency. So, we need to chose a setpoint more aggressive to improve performance.
Myself, and so as to improve response to some games and such that use
many threads and such but often a lower overall CPU load, I think the setpoint should be set a little lower.
I have an idea to address these oscillating workload. I'm testing a patch on Android that detects these use cases (mainly GLThreads migrating are responsible for these oscillation).
I'll submit it as soon as it gives the best power and performance trade-off.
Quoted 5 lines
There is a tradeoff in reducing the setpoint further as it increases the noise and tendency to oscillate in the response curve. Ultimately, it may be desirable to introduce a little slope in the load / CPU frequency response curve.
Quoted 7 lines
I have a bunch of graphs comparing response curves. [1]
Quoted 2 lines
Measured with this parameter, we noticed an improvement in Browsermark for power and perf compared to the old formula:
I would like to try this test on my system. What is the exact test? Do I understand correctly, that I need a browser to do the test? (my test system is a server, and it doesn't have a browser.)
Yes, for browsermark, you can use this link but you need a browser:
http://web.basemark.com/
Else you can try some gaming workloads (I was using CandyCrush on Android) to evaluate the power gain.
Quoted 26 lines
Quoted 6 lines
Score without the patch: 3517
Power without the patch: 6856 mWScore with the patch: 3719
Power with the patch: 6265 mWThere are some other Phoronix tests that we (the original maintainer and the a couple of others working with him used to use. See [1].
Please be aware that the last time I tried to bring back load based calculations, Kristen tested the proposed solution on some intel "specpower test bed and experienced a regression on haswell based server platforms vs. Dirks algorithm." I don't have any details.
Your response curve, and in particular your step function response time, is different, so it might worth re-testing.References:
[1] double u double u double u dot smythies dot com/~doug/linux/intel_pstate/philippe_longepe/index.html
--
To unsubscribe from this list: send the line "unsubscribe linux-pm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 18 lines · Philippe Longepe
Aperf and Mperf counter are not enough to determine the Target P-state because they measure performance only when the targeted processor is in the C0 state (active state).
Because of that, we were computing the average P-state during the last period which can be very different from the average frequency (or percentage of performance).As defined in the SDM (section 14.2), the PercentPerformance is defined by:
PercentPerformance = PercentBusy * (
delta_aperf/delta_mperf);The PercentBusy (or load) can be estimated as the ratio of the mperf counter running at a constant frequency only during active periods (C0) and the time stamp counter running at the same frequency but also during idle.
So, PercentBusy = 100 * (
delta_mperf/delta_tsc)
Quoted 7 lines
and, PercentPerformance = 100 * (
delta_mperf/delta_tsc) *
(delta_aperf/delta_mperf)
That can be simplified with:
PercentPerformance = 100 * (delta_aperf/delta_tsc)Signed-off-by Philippe Longepe <[email protected]>Signed-off-by Stephane Gasparini <[email protected]>
Srinivas, I need your ACK here and for the [2/2].
Thanks,
Rafael
Srinivas Pandruvada
maintainerQuoted 30 lines · Rafael J. Wysocki
Quoted 29 lines · Philippe Longepe
Aperf and Mperf counter are not enough to determine the Target P -state
because they measure performance only when the targeted processor is
in the C0 state (active state).
Because of that, we were computing the average P-state during the last
period which can be very different from the average frequency (or percentage of performance).As defined in the SDM (section 14.2), the PercentPerformance is defined by:
PercentPerformance = PercentBusy * (
delta_aperf/delta_mperf);The PercentBusy (or load) can be estimated as the ratio of the mperf
counter running at a constant frequency only during active periods (C0)
and the time stamp counter running at the same frequency but also during idle.So, PercentBusy = 100 * (
delta_mperf/delta_tsc)and, PercentPerformance = 100 * (
delta_mperf/delta_tsc) *
(delta_aperf/delta_mperf)
That can be simplified with:
PercentPerformance = 100 * (delta_aperf/delta_tsc)
Quoted 6 lines
Quoted 3 lines
Signed-off-by Philippe Longepe <[email protected]>Signed-off-by Stephane Gasparini <Srinivas, I need your ACK here and for the [2/2].
These needs further validation to merge to upstream kernel.
Thanks,
Srinivas
Quoted 3 lines
Thanks,
Rafael
Philippe, Stephane,
The resulting load / CPU frequency response curve has some interesting qualities. The resulting step function load response time is much much faster than the current solution, and yet there is not much tendency for the system to oscillate, or at least the oscillation magnitude is smallish.
I agree that some sort of real load calculation needs to be brought back to the intel_pstate driver.
On 2015.11.06 17:14 Srinivas Pandruvada wrote:
Quoted 23 lines · Rafael J. Wysocki
Quoted 22 lines · Philippe Longepe
Aperf and Mperf counter are not enough to determine the Target P -state
because they measure performance only when the targeted processor is
in the C0 state (active state).
Because of that, we were computing the average P-state during the last
period which can be very different from the average frequency (or percentage of performance).As defined in the SDM (section 14.2), the PercentPerformance is defined by:
PercentPerformance = PercentBusy * (
delta_aperf/delta_mperf);The PercentBusy (or load) can be estimated as the ratio of the mperf
counter running at a constant frequency only during active periods (C0)
and the time stamp counter running at the same frequency but also during idle.
Quoted 6 lines
Quoted 6 lines
So, PercentBusy = 100 * (
delta_mperf/delta_tsc)and, PercentPerformance = 100 * (
delta_mperf/delta_tsc) *
(delta_aperf/delta_mperf)
That can be simplified with:
PercentPerformance = 100 * (delta_aperf/delta_tsc)
Yes, but the last time I tried to bring back actual load calculations In a similar way it was pointed out that one should not do it this way.
[1] SDM also states:
"Only the IA32_APERF/IA32_MPERF ratio is architecturally defined;
software should not attach meaning to the content of the individual of IA32_APERF or IA32_MPERF MSRs."
Note that I have been working on a "legal" way to calculate load [2] and I have been using the method for about a month now. (note based on a different patch set starting point).
Quoted 6 lines
drivers/cpufreq/intel_pstate.c static inline void intel_pstate_set_sample_time(struct cpudata *cpu) · line 1097
Quoted 6 lines
drivers/cpufreq/intel_pstate.c +0
-6- duration_us =ktime_us_delta (cpu->sample.time, - cpu->last_sample_time); -if (duration_us > sample_time *3 ) { - sample_ratio =div_fp (int_tofp (sample_time), -int_tofp (duration_us)); - core_busy =mul_fp (core_busy, sample_ratio);
While the duration stuff has issues that can sometimes result in incorrectly driving the target pstate downwards, rather than deleting it there might still be value in fixing it using the actual load information.
References:
[1] http://marc.info/?l=linux-pm&m=143094005601231&w=2
[2] double u double u double u .dot smythies dot com/~doug/linux/intel_pstate/build58/0006-intel_pstate-Change-utilization-calculation-to-a-leg.txt
Hi Smythies,
Yes, the cpu load gives good results for almost all use cases except IO workloads (it needs an additionnal patch to boost the IOs).
That's why we will enable it only on Atoms first but not yet on core/servers (more validation is needed).
Quoted 40 lines · Doug Smythies
Philippe, Stephane,
The resulting load / CPU frequency response curve has some interesting qualities. The resulting step function load response time is much much faster than the current solution, and yet there is not much tendency for the system to oscillate, or at least the oscillation magnitude is smallish.
I agree that some sort of real load calculation needs to be brought back to the
intel_pstatedriver.On 2015.11.06 17:14 Srinivas Pandruvada wrote:
Quoted 23 lines · Rafael J. Wysocki
Quoted 22 lines · Philippe Longepe
Aperf and Mperf counter are not enough to determine the Target P -state
because they measure performance only when the targeted processor is
in the C0 state (active state).
Because of that, we were computing the average P-state during the last
period which can be very different from the average frequency (or percentage of performance).As defined in the SDM (section 14.2), the PercentPerformance is defined by:
PercentPerformance = PercentBusy * (
delta_aperf/delta_mperf);The PercentBusy (or load) can be estimated as the ratio of the mperf
counter running at a constant frequency only during active periods (C0)
and the time stamp counter running at the same frequency but also during idle.Quoted 6 lines
Quoted 6 lines
So, PercentBusy = 100 * (
delta_mperf/delta_tsc)and, PercentPerformance = 100 * (
delta_mperf/delta_tsc) *
(delta_aperf/delta_mperf)
That can be simplified with:
PercentPerformance = 100 * (delta_aperf/delta_tsc)
Quoted 6 lines
Yes, but the last time I tried to bring back actual load calculations In a similar way it was pointed out that one should not do it this way.
[1] SDM also states:"Only the IA32_APERF/IA32_MPERF ratio is architecturally defined;
software should not attach meaning to the content of the individual ofIA32_APERForIA32_MPERFMSRs."
Individual counters are not architecturally defined but their ratio is defined. MERF and TSC counters are both counting at the same frequency (max non turbo freq) but MPERF is counting only in active state (C0) and TSC is counting all the time. So, delta_mperf/delta_tsc is representing the load.
Quoted 24 lines
Note that I have been working on a "legal" way to calculate load [2] and I have been using the method for about a month now. (note based on a different patch set starting point).
Quoted 6 lines
drivers/cpufreq/intel_pstate.c static inline void intel_pstate_set_sample_time(struct cpudata *cpu) · line 1097
Quoted 6 lines
drivers/cpufreq/intel_pstate.c +0
-6- duration_us =ktime_us_delta (cpu->sample.time, - cpu->last_sample_time); -if (duration_us > sample_time *3 ) { - sample_ratio =div_fp (int_tofp (sample_time), -int_tofp (duration_us)); - core_busy =mul_fp (core_busy, sample_ratio);While the duration stuff has issues that can sometimes result in incorrectly driving the target pstate downwards, rather than deleting it there might still be value in fixing it using the actual load information.
References:
[1] http://marc.info/?l=linux-pm&m=143094005601231&w=2
[2] double u double u double u .dot smythies dotcom/~doug/linux/intel_pstate/build58/0006-intel_pstate-Change-utilization-calculation-to-a-leg.txt--
To unsubscribe from this list: send the line "unsubscribe linux-pm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 40 lines · On 2015.11.23 05:29 plongepe wrote:
Quoted 32 lines · Doug Smythies
On 2015.11.06 17:14 Srinivas Pandruvada wrote:
Quoted 23 lines · Rafael J. Wysocki
Quoted 22 lines · Philippe Longepe
Aperf and Mperf counter are not enough to determine the Target P -state
because they measure performance only when the targeted processor is
in the C0 state (active state).
Because of that, we were computing the average P-state during the last
period which can be very different from the average frequency (or percentage of performance).As defined in the SDM (section 14.2), the PercentPerformance is defined by:
PercentPerformance = PercentBusy * (
delta_aperf/delta_mperf);The PercentBusy (or load) can be estimated as the ratio of the mperf
counter running at a constant frequency only during active periods (C0)
and the time stamp counter running at the same frequency but also during idle.Quoted 6 lines
Quoted 6 lines
So, PercentBusy = 100 * (
delta_mperf/delta_tsc)and, PercentPerformance = 100 * (
delta_mperf/delta_tsc) *
(delta_aperf/delta_mperf)
That can be simplified with:
PercentPerformance = 100 * (delta_aperf/delta_tsc)Quoted 6 lines
Yes, but the last time I tried to bring back actual load calculations In a similar way it was pointed out that one should not do it this way.
[1] SDM also states:"Only the IA32_APERF/IA32_MPERF ratio is architecturally defined;
software should not attach meaning to the content of the individual ofIA32_APERForIA32_MPERFMSRs."
Quoted 7 lines
Individual counters are not architecturally defined but their ratio is defined.
MERF and TSC counters are both counting at the same frequency (max non turbo freq)
but MPERF is counting only in active state (C0) and TSC is counting all the time.
So, delta_mperf/delta_tsc is representing the load.
I agree.
However, Intel should decide once and for all if mperf / tsc is allowed or not. It was used one time, then it was deleted, then I tried to bring it back, then I was told it wasn't allowed, now you are trying to bring it back.
... Doug
Srinivas Pandruvada
maintainerQuoted 5 lines · Doug Smythies
Quoted 4 lines · On 2015.11.23 05:29 plongepe wrote:
Quoted 3 lines · Doug Smythies
On 2015.11.06 17:14 Srinivas Pandruvada wrote:
Quoted 1 line · Rafael J. Wysocki
On Tuesday, November 03, 2015 10:27:19 AM Philippe Longepe wrote:
[CUT.]
Quoted 1 line
However, Intel should decide once and for all if mperf / tsc is allowed or not.
We can't apply this uniformly across all CPUs. So this patchset will not be applied to all CPUs, only where this is allowed and verified. BTW, these patches need cleanup.
Thanks,
Srinivas
Quoted 6 lines
It was used one time, then it was deleted, then I tried to bring it back, then I was told it wasn't allowed, now you are trying to bring it back.
... Doug