cpufreq: intel_pstate: diverge of current_pstate and actual P state

Hi all,

Currently, we use the current P state to calculate the busy_scaled factor and then the next P state.

We also read the MSR_TURBO_RATIO_LIMIT to get the turbo ratio limit as the turbo_pstate. But, we always read bits 7:0 ("Maximum turbo ratio limit of 1 core active").

So, in processor families that have different turbo ratio limit depending on active cores the current P state as it's considered by the driver might be different from the actual current P state.

For example, I use an i7-3770 which reports as maximum turbo ratio limits with 1/2/3/4 actives cores the values 39/39/38/37. So, in some cases we will calculate as the next P state the value 39. If the active cores at that time was 3 or 4 the actual P state will be 38 or 37. The current_pstate variable will have the value 39 and this will lead to wrong calculation at the next sampling interval.

Trying to find a solution to the above I couldn't find an MSR that we could use to get the number of active cores and use the respective turbo ratio limit.

I also thought to use the IA32_PERF_STATUS to get the current P state and use it in the calculations, but its scope is per core and not per thread.

Am I missing something? If the above is correct, any idea how this could be resolved?

Thanks in advance,
Stratos

Quoted 28 lines · Stratos Karafotis

Hi all,

Currently, we use the current P state to calculate the busy_scaled factor and then the next P state.

We also read the MSR_TURBO_RATIO_LIMIT to get the turbo ratio limit as the turbo_pstate. But, we always read bits 7:0 ("Maximum turbo ratio limit of 1 core active").

So, in processor families that have different turbo ratio limit depending on active cores the current P state as it's considered by the driver might be different from the actual current P state.

For example, I use an i7-3770 which reports as maximum turbo ratio limits with 1/2/3/4 actives cores the values 39/39/38/37. So, in some cases we will calculate as the next P state the value 39. If the active cores at that time was 3 or 4 the actual P state will be 38 or 37. The current_pstate variable will have the value 39 and this will lead to wrong calculation at the next sampling interval.

Trying to find a solution to the above I couldn't find an MSR that we could use to get the number of active cores and use the respective turbo ratio limit.

I also thought to use the IA32_PERF_STATUS to get the current P state and use it in the calculations, but its scope is per core and not per thread.

Quoted 2 lines

Am I missing something? If the above is correct, any idea how this could be resolved?

The above is correct except the requested pstate is calculated using max_pstate and turbo_pstate is used as the upper limit when calling intel_pstate_set_pstate.

The value written to (MSR_IA32_PPERF_CTL is a request that is processed by the CPU and is clipped internally to the current state of the CPU. Whether or not any turbo is available is decided by the CPU asking for the top turbo bin says give me all that is available.

Asking for more than the CPU has to give ATM is harmless.

Quoted 29 lines · Dirk Brandewie

Quoted 28 lines · Stratos Karafotis

Hi all,

Currently, we use the current P state to calculate the busy_scaled factor and then the next P state.

We also read the MSR_TURBO_RATIO_LIMIT to get the turbo ratio limit as the turbo_pstate. But, we always read bits 7:0 ("Maximum turbo ratio limit of 1 core active").

So, in processor families that have different turbo ratio limit depending on active cores the current P state as it's considered by the driver might be different from the actual current P state.

For example, I use an i7-3770 which reports as maximum turbo ratio limits with 1/2/3/4 actives cores the values 39/39/38/37. So, in some cases we will calculate as the next P state the value 39. If the active cores at that time was 3 or 4 the actual P state will be 38 or 37. The current_pstate variable will have the value 39 and this will lead to wrong calculation at the next sampling interval.

Trying to find a solution to the above I couldn't find an MSR that we could use to get the number of active cores and use the respective turbo ratio limit.

I also thought to use the IA32_PERF_STATUS to get the current P state and use it in the calculations, but its scope is per core and not per thread.

Quoted 6 lines

Quoted 2 lines

Am I missing something? If the above is correct, any idea how this could be resolved?

The above is correct except the requested pstate is calculated using max_pstate and turbo_pstate is used as the upper limit when calling intel_pstate_set_pstate.

Thanks for your prompt reply!

But when we call intel_pstate_set_pstate we also set the current_pstate to the requested pstate (which it may be, for example, 39).

Quoted 7 lines

The value written to (MSR_IA32_PPERF_CTL is a request that is processed by the CPU and is clipped internally to the current state of the CPU. Whether or not any turbo is available is decided by the CPU asking for the top turbo bin says give me all that is available.

Asking for more than the CPU has to give ATM is harmless.

Then the CPU clips internally, as you said, to the current actual state. If all cores are active the current state (in CPU) will be 37. Driver will still consider the current_pstate as 39.

In next sampling interval, in intel_pstate_get_scaled_busy we calculate the core_busy as core_busy * max_pstate / current_pstate.

So, in the above example we have:
core_busy = core_busy * 34 / 39
and not
core_busy = core_busy * 34 / 37
as it should be.

Stratos

Quoted 54 lines · Stratos Karafotis

Quoted 29 lines · Dirk Brandewie

Quoted 28 lines · Stratos Karafotis

Hi all,

Currently, we use the current P state to calculate the busy_scaled factor and then the next P state.

We also read the MSR_TURBO_RATIO_LIMIT to get the turbo ratio limit as the turbo_pstate. But, we always read bits 7:0 ("Maximum turbo ratio limit of 1 core active").

So, in processor families that have different turbo ratio limit depending on active cores the current P state as it's considered by the driver might be different from the actual current P state.

For example, I use an i7-3770 which reports as maximum turbo ratio limits with 1/2/3/4 actives cores the values 39/39/38/37. So, in some cases we will calculate as the next P state the value 39. If the active cores at that time was 3 or 4 the actual P state will be 38 or 37. The current_pstate variable will have the value 39 and this will lead to wrong calculation at the next sampling interval.

Trying to find a solution to the above I couldn't find an MSR that we could use to get the number of active cores and use the respective turbo ratio limit.

I also thought to use the IA32_PERF_STATUS to get the current P state and use it in the calculations, but its scope is per core and not per thread.

Quoted 6 lines

Quoted 2 lines

Am I missing something? If the above is correct, any idea how this could be resolved?

The above is correct except the requested pstate is calculated using max_pstate and turbo_pstate is used as the upper limit when calling intel_pstate_set_pstate.

Thanks for your prompt reply!

But when we call intel_pstate_set_pstate we also set the current_pstate to the requested pstate (which it may be, for example, 39).

Quoted 7 lines

The value written to (MSR_IA32_PPERF_CTL is a request that is processed by the CPU and is clipped internally to the current state of the CPU. Whether or not any turbo is available is decided by the CPU asking for the top turbo bin says give me all that is available.

Asking for more than the CPU has to give ATM is harmless.

Then the CPU clips internally, as you said, to the current actual state. If all cores are active the current state (in CPU) will be 37. Driver will still consider the current_pstate as 39.

Quoted 8 lines

In next sampling interval, in intel_pstate_get_scaled_busy we calculate the core_busy as core_busy * max_pstate / current_pstate.

So, in the above example we have:
core_busy = core_busy * 34 / 39
and not
core_busy = core_busy * 34 / 37
as it should be.

Yes and I know of no good way around it. In practice I haven't seen the oscillation in the upper turbo range (which is what this may cause). Do you have a workload where this is hurting performance?

Keep in mind you may have gotten 3.8385 Ghz or any value in the turbo range based on the state of the processor. The processor updates the effective frequency a lot faster than we sample.