Avoid calling cpu_pm functions for retention idle states
Subsystems (2)
v2 cover Avoid calling cpu_pm functions for retention idle states
CPU_PM_CPU_IDLE_ENTER() treats all idle states whose idx != 0 as a state that loses some context, but we can have deeper idle states that doesn't lose any software context. If a CPU is entering such a low power idle state where it retains the context, then there is no need to call cpu_pm_enter()/cpu_pm_exit().
Add a new macro(CPU_PM_CPU_IDLE_ENTER_RETENTION) to be used by cpuidle drivers when they are entering retention state. By not calling cpu_pm_enter and cpu_pm_exit we reduce the latency involved in entering and exiting the retention states.
On ARM64 based Qualcomm server platform we measured below overhead for for calling cpu_pm_enter and cpu_pm_exit for retention states.
workload: stress --hdd #CPUs --hdd-bytes 32M -t 30 Overhead of cpu_pm_enter - 1.2us(Average), 6.5us(Max) Overhead of cpu_pm_exit - 3.1us(Average), 11.1us(Max)
Listed below are 5 functions that were notified on ENTER/EXIT on the test platform:
gic_cpu_pm_notifier
arch_timer_cpu_pm_notify
cpu_pm_pmu_notify
hyp_init_cpu_pm_notifier
fpsimd_cpu_pm_notifier
Prashanth Prakash (2):
cpuidle: Add new macro to enter a retention idle state ARM64 / cpuidle: Use new cpuidle macro for entering retention state
Changes in v2:
- Reordered cpuidle.h macros for better readablity (Rafael)
arch/arm64/kernel/cpuidle.c | 8 +++++++- include/linux/cpuidle.h | 40 ++++++++++++++++++++++++---------------- 2 files changed, 31 insertions(+), 17 deletions(-)
Signature · 3 lines
Qualcomm Datacenter Technologies on behalf of Qualcomm Technologies, Inc. Qualcomm Technologies, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project.
PATCH v2 1/2 cpuidle: Add new macro to enter a retention idle state
If a CPU is entering a low power idle state where it doesn't lose any context, then there is no need to call cpu_pm_enter()/cpu_pm_exit(). Add a new macro(CPU_PM_CPU_IDLE_ENTER_RETENTION) to be used by cpuidle drivers when they are entering retention state. By not calling cpu_pm_enter and cpu_pm_exit we reduce the latency involved in entering and exiting the retention idle states.
On ARM64 based Qualcomm Server Platform we measured below overhead for for calling cpu_pm_enter and cpu_pm_exit for retention states.
workload: stress --hdd #CPUs --hdd-bytes 32M -t 30
Average overhead of cpu_pm_enter - 1.2us Average overhead of cpu_pm_exit - 3.1us
--- include/linux/cpuidle.h | 40 ++++++++++++++++++++++++---------------- 1 file changed, 24 insertions(+), 16 deletions(-)
include/linux/cpuidle.h +24 -16
@@ -257,22 +257,30 @@ static inline int cpuidle_register_governor(struct cpuidle_governor *gov) {return 0 ;}#endif -#define CPU_PM_CPU_IDLE_ENTER (low_level_idle_enter, idx) \ -({ \ - int __ret; \ - \ - if (!idx) { \ - cpu_do_idle(); \ - return idx; \ - } \ - \ - __ret = cpu_pm_enter(); \ - if (!__ret) { \ - __ret = low_level_idle_enter(idx); \ - cpu_pm_exit(); \ - } \ - \ - __ret ? -1 : idx; \ +#define __CPU_PM_CPU_IDLE_ENTER (low_level_idle_enter, idx, is_retention) \ +({ \ + int __ret = 0; \ + \ + if (!idx) { \ + cpu_do_idle(); \ + return idx; \ + } \ + \ + if (!is_retention) \ + __ret = cpu_pm_enter(); \ + if (!__ret) { \ + __ret = low_level_idle_enter(idx); \ + if (!is_retention) \ + cpu_pm_exit(); \ + } \ + \ + __ret ? -1 : idx; \ }) +#define CPU_PM_CPU_IDLE_ENTER (low_level_idle_enter, idx) \ + __CPU_PM_CPU_IDLE_ENTER(low_level_idle_enter, idx, 0) + +#define CPU_PM_CPU_IDLE_ENTER_RETENTION (low_level_idle_enter, idx) \ + __CPU_PM_CPU_IDLE_ENTER(low_level_idle_enter, idx, 1) +#endif /* _LINUX_CPUIDLE_H */
Signature · 3 lines
Qualcomm Datacenter Technologies on behalf of Qualcomm Technologies, Inc. Qualcomm Technologies, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project.
PATCH v2 1/2 cpuidle: Add new macro to enter a retention idle state
Quoted 7 lines · Prashanth Prakash
If a CPU is entering a low power idle state where it doesn't lose any context, then there is no need to call cpu_pm_enter()/cpu_pm_exit(). Add a new
macro(CPU_PM_CPU_IDLE_ENTER_RETENTION)to be used by cpuidle drivers when they are entering retention state. By not callingcpu_pm_enterandcpu_pm_exitwe reduce the latency involved in entering and exiting the retention idle states.
Quoted 7 lines
On ARM64 based Qualcomm Server Platform we measured below overhead for for calling
cpu_pm_enterandcpu_pm_exitfor retention states.workload: stress --hdd #CPUs --hdd-bytes 32M -t 30
Average overhead ofcpu_pm_enter- 1.2us Average overhead ofcpu_pm_exit- 3.1us
Just add a description to inform that if retention is enabled, it's assumed to be full retention(IOW no state needs to be saved/restored) and hence the CPU PM notifiers are not called. We may need to change this in case we need to support partial retention states in future.
Other than that,
Signature · 2 lines
Regards, Sudeep
PATCH v2 2/2 ARM64 / cpuidle: Use new cpuidle macro for entering retention state
In reply to Prashanth PrakashCPU_PM_CPU_IDLE_ENTER_RETENTION skips calling cpu_pm_enter() and cpu_pm_exit(). By not calling cpu_pm functions in idle entry/exit paths we can reduce the latency involved in entering and exiting the low power idle state.
On ARM64 based Qualcomm server platform we measured below overhead for calling cpu_pm_enter and cpu_pm_exit for retention states.
workload: stress --hdd #CPUs --hdd-bytes 32M -t 30
Average overhead of cpu_pm_enter - 1.2us Average overhead of cpu_pm_exit - 3.1us
--- arch/arm64/kernel/cpuidle.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)
arch/arm64/kernel/cpuidle.c +7 -1
@@ -47,6 +47,8 @@ int arm_cpuidle_suspend(int index)#include <acpi/processor.h> +#define ARM64_LPI_IS_RETENTION_STATE (arch_flags) (!(arch_flags)) +int acpi_processor_ffh_lpi_probe (unsigned int cpu) {return arm_cpuidle_init (cpu);
@@ -54,6 +56,10 @@ int acpi_processor_ffh_lpi_probe(unsigned int cpu)int acpi_processor_ffh_lpi_enter (struct acpi_lpi_state *lpi) { -return CPU_PM_CPU_IDLE_ENTER (arm_cpuidle_suspend, lpi->index); +if (ARM64_LPI_IS_RETENTION_STATE (lpi->arch_flags)) +return CPU_PM_CPU_IDLE_ENTER_RETENTION (arm_cpuidle_suspend, + lpi->index); +else +return CPU_PM_CPU_IDLE_ENTER (arm_cpuidle_suspend, lpi->index); }#endif
Signature · 3 lines
Qualcomm Datacenter Technologies on behalf of Qualcomm Technologies, Inc. Qualcomm Technologies, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project.
PATCH v2 2/2 ARM64 / cpuidle: Use new cpuidle macro for entering retention state
Quoted 22 lines · Prashanth Prakash
CPU_PM_CPU_IDLE_ENTER_RETENTIONskips callingcpu_pm_enter()andcpu_pm_exit(). By not callingcpu_pmfunctions in idle entry/exit paths we can reduce the latency involved in entering and exiting the low power idle state.On ARM64 based Qualcomm server platform we measured below overhead for calling
cpu_pm_enterandcpu_pm_exitfor retention states.workload: stress --hdd #CPUs --hdd-bytes 32M -t 30
Average overhead ofcpu_pm_enter- 1.2us Average overhead ofcpu_pm_exit- 3.1usSigned-off-by Prashanth Prakash <[email protected]>--- arch/arm64/kernel/cpuidle.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)arch/arm64/kernel/cpuidle.c +0
-0
arch/arm64/kernel/cpuidle.c int arm_cpuidle_suspend(int index) · line 51
Quoted 5 lines
arch/arm64/kernel/cpuidle.c +2
-0@@ -47,6 +47,8 @@ int arm_cpuidle_suspend(int index)#include <acpi/processor.h> +#define ARM64_LPI_IS_RETENTION_STATE (arch_flags) (!(arch_flags)) +
This is fine, but just to be safer, is it better to check for all flags to be set as we can't/don't support any partial retention modes.
Just curious, how is retention handled on mobile parts. I guess mainline is not important on those parts, but add a note in the commit message that we can make it ACPI agnostic and PSCI param dependent if needed.
Signature · 2 lines
Regards, Sudeep
PATCH v2 2/2 ARM64 / cpuidle: Use new cpuidle macro for entering retention state
Quoted 25 lines · Sudeep Holla
Quoted 23 lines · Prashanth Prakash
CPU_PM_CPU_IDLE_ENTER_RETENTIONskips callingcpu_pm_enter()andcpu_pm_exit(). By not callingcpu_pmfunctions in idle entry/exit paths we can reduce the latency involved in entering and exiting the low power idle state.On ARM64 based Qualcomm server platform we measured below overhead for calling
cpu_pm_enterandcpu_pm_exitfor retention states.workload: stress --hdd #CPUs --hdd-bytes 32M -t 30
Average overhead ofcpu_pm_enter- 1.2us Average overhead ofcpu_pm_exit- 3.1usSigned-off-by Prashanth Prakash <[email protected]>--- arch/arm64/kernel/cpuidle.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)arch/arm64/kernel/cpuidle.c +0
-0@@ -47,6 +47,8 @@ int arm_cpuidle_suspend(int index)
Quoted 6 lines
arch/arm64/kernel/cpuidle.c int arm_cpuidle_suspend(int index) · line 51
Quoted 4 lines
arch/arm64/kernel/cpuidle.c +2
-0#include <acpi/processor.h> +#define ARM64_LPI_IS_RETENTION_STATE (arch_flags) (!(arch_flags)) +This is fine, but just to be safer, is it better to check for all flags to be set as we can't/don't support any partial retention modes.
I am not sure I completely understand.
If any bit is set in arch_flags, then we lose the context corresponding to that bit, so a full retention state will have no bit set, so that's what we are checking for. Or am i missing something?
Thanks,
Prashanth
PATCH v2 2/2 ARM64 / cpuidle: Use new cpuidle macro for entering retention state
Quoted 34 lines · Prashanth
Quoted 25 lines · Sudeep Holla
Quoted 23 lines · Prashanth Prakash
CPU_PM_CPU_IDLE_ENTER_RETENTIONskips callingcpu_pm_enter()andcpu_pm_exit(). By not callingcpu_pmfunctions in idle entry/exit paths we can reduce the latency involved in entering and exiting the low power idle state.On ARM64 based Qualcomm server platform we measured below overhead for calling
cpu_pm_enterandcpu_pm_exitfor retention states.workload: stress --hdd #CPUs --hdd-bytes 32M -t 30
Average overhead ofcpu_pm_enter- 1.2us Average overhead ofcpu_pm_exit- 3.1usSigned-off-by Prashanth Prakash <[email protected]>--- arch/arm64/kernel/cpuidle.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)arch/arm64/kernel/cpuidle.c +0
-0@@ -47,6 +47,8 @@ int arm_cpuidle_suspend(int index)Quoted 6 lines
arch/arm64/kernel/cpuidle.c int arm_cpuidle_suspend(int index) · line 51
Quoted 4 lines
arch/arm64/kernel/cpuidle.c +2
-0#include <acpi/processor.h> +#define ARM64_LPI_IS_RETENTION_STATE (arch_flags) (!(arch_flags)) +This is fine, but just to be safer, is it better to check for all flags to be set as we can't/don't support any partial retention modes.
Quoted 6 lines
I am not sure I completely understand.
If any bit is set in
arch_flags, then we lose the context corresponding to that bit, so a full retention state will have no bit set, so that's what we are checking for. Or am i missing something?
Ah you are right, I somehow misinterpreted the flags exactly to be opposite(i.e. state retained) rather than state lost, sorry for the noise.
Signature · 2 lines
Regards, Sudeep