arm64: Fix cpuidle/suspend with pseudo-NMI enabled
Subsystems (3)
PATCH v2 4/4 arm64: suspend: Use cpuidle context helpers in cpu_suspend()
In reply to Marc ZyngierUse cpuidle context helpers to switch to using DAIF.IF instead of PMR to mask interrupts, ensuring that we suspend with interrupts being able to reach the CPU interface.
--- arch/arm64/kernel/suspend.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-)
arch/arm64/kernel/suspend.c +11 -1
@@ -7,6 +7,7 @@#include <asm/alternative.h> #include <asm/cacheflush.h> #include <asm/cpufeature.h> +#include <asm/cpuidle.h> #include <asm/daifflags.h> #include <asm/debug-monitors.h> #include <asm/exec.h>
@@ -91,6 +92,7 @@ int cpu_suspend(unsigned long arg, int (*fn)(unsigned long))int ret =0 ;unsigned long flags;struct sleep_stack_data state; +struct arm_cpuidle_irq_context context;/* Report any MTE async fault before going to suspend */ mte_suspend_enter ();
@@ -103,12 +105,18 @@ int cpu_suspend(unsigned long arg, int (*fn)(unsigned long)) flags =local_daif_save ();/* -* Function graph tracer state gets incosistent when the kernel +* Function graph tracer state gets inconsistent when the kernel * calls functions that never return (aka suspend finishers) hence * disable graph tracing during their execution. */ pause_graph_tracing (); +/* +* Switch to using DAIF.IF instead of PMR in order to reliably +* resume if we're using pseudo-NMIs. +*/ +arm_cpuidle_save_irq_context (&context); +if (__cpu_suspend_enter (&state)) {/* Call the suspend finisher */ ret =fn (arg);
@@ -126,6 +134,8 @@ int cpu_suspend(unsigned long arg, int (*fn)(unsigned long))RCU_NONIDLE (__cpu_suspend_exit ()); } +arm_cpuidle_restore_irq_context (&context); +unpause_graph_tracing ();/*
Signature · 7 lines
2.30.2 _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
PATCH v2 2/4 arm64: Convert cpu_do_idle() to using cpuidle context helpers
In reply to Marc ZyngierNow that we have helpers that are aware of the pseudo-NMI feature, introduce them to cpu_do_idle(). This allows for some nice cleanup.
No functional change intended.
--- arch/arm64/kernel/process.c | 41 ++++++++----------------------------- 1 file changed, 9 insertions(+), 32 deletions(-)
arch/arm64/kernel/process.c +9 -32
@@ -46,9 +46,9 @@#include <linux/prctl.h> #include <asm/alternative.h> -#include <asm/arch_gicv3.h> #include <asm/compat.h> #include <asm/cpufeature.h> +#include <asm/cpuidle.h> #include <asm/cacheflush.h> #include <asm/exec.h> #include <asm/fpsimd.h>
@@ -74,33 +74,6 @@ EXPORT_SYMBOL_GPL(pm_power_off);void (*arm_pm_restart)(enum reboot_mode reboot_mode,const char *cmd); -static void noinstr __cpu_do_idle(void) -{ -dsb (sy); -wfi (); -} - -static void noinstr __cpu_do_idle_irqprio(void) -{ -unsigned long pmr; -unsigned long daif_bits; - - daif_bits =read_sysreg (daif); -write_sysreg (daif_bits |PSR_I_BIT |PSR_F_BIT , daif); - -/* -* Unmask PMR before going idle to make sure interrupts can -* be raised. -*/ - pmr =gic_read_pmr (); -gic_write_pmr (GIC_PRIO_IRQON |GIC_PRIO_PSR_I_SET ); - -__cpu_do_idle (); - -gic_write_pmr (pmr); -write_sysreg (daif_bits, daif); -} -/* * cpu_do_idle() *
@@ -112,10 +85,14 @@ static void noinstr __cpu_do_idle_irqprio(void)*/ void noinstr cpu_do_idle(void) { -if (system_uses_irq_prio_masking ()) -__cpu_do_idle_irqprio (); -else -__cpu_do_idle (); +struct arm_cpuidle_irq_context context; + +arm_cpuidle_save_irq_context (&context); + +dsb (sy); +wfi (); + +arm_cpuidle_restore_irq_context (&context); }/*
Signature · 7 lines
2.30.2 _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Re: [PATCH v2 3/4] PSCI: Use cpuidle context helpers in psci_cpu_suspend_enter()
In reply to Marc ZyngierQuoted 11 lines · Marc Zyngier
The PSCI CPU suspend code isn't aware of the PMR vs DAIF game, resulting in a system that locks up if entering CPU suspend with GICv3 pNMI enabled.
To save the day, teach the suspend code about our new cpuidle context helpers, which will do everything that's required just like the usual WFI cpuidle code.
This fixes my Altra system, which would otherwise lock-up at boot time when booted with irqchip.gicv3_pseudo_nmi=1.
--
Regards,
Sudeep
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
v2 cover arm64: Fix cpuidle/suspend with pseudo-NMI enabled
This is a follow-up on the series posted at [1], with some added fixes. The main difference is the covering of system suspend, although it is untested (the GICv3 boxes I have don't support system suspend...).
Tested on an Ampere Altra system.
* From v1:
- Moved the wrapping of cpu_suspend() into the body of the function, now covering system suspend as well
- Added RBs, TBs.
[1] https://lore.kernel.org/r/[email protected]
Marc Zyngier (4):
arm64: Add cpuidle context save/restore helpers
arm64: Convert cpu_do_idle() to using cpuidle context helpers
PSCI: Use cpuidle context helpers in psci_cpu_suspend_enter()
arm64: suspend: Use cpuidle context helpers in cpu_suspend()
arch/arm/include/asm/cpuidle.h | 5 ++++ arch/arm64/include/asm/cpuidle.h | 35 +++++++++++++++++++++++++++ arch/arm64/kernel/process.c | 41 +++++++------------------------- arch/arm64/kernel/suspend.c | 12 +++++++++- drivers/firmware/psci/psci.c | 9 +++++-- 5 files changed, 67 insertions(+), 35 deletions(-)
Signature · 7 lines
2.30.2 _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
PATCH v2 3/4 PSCI: Use cpuidle context helpers in psci_cpu_suspend_enter()
The PSCI CPU suspend code isn't aware of the PMR vs DAIF game, resulting in a system that locks up if entering CPU suspend with GICv3 pNMI enabled.
To save the day, teach the suspend code about our new cpuidle context helpers, which will do everything that's required just like the usual WFI cpuidle code.
This fixes my Altra system, which would otherwise lock-up at boot time when booted with irqchip.gicv3_pseudo_nmi=1.
--- drivers/firmware/psci/psci.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)
drivers/firmware/psci/psci.c +7 -2
@@ -335,10 +335,15 @@ int psci_cpu_suspend_enter(u32 state) {int ret; -if (!psci_power_state_loses_context (state)) +if (!psci_power_state_loses_context (state)) { +struct arm_cpuidle_irq_context context; + +arm_cpuidle_save_irq_context (&context); ret = psci_ops.cpu_suspend (state,0 ); -else +arm_cpuidle_restore_irq_context (&context); + }else { ret =cpu_suspend (state, psci_suspend_finisher); + }return ret; }
Signature · 7 lines
2.30.2 _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
PATCH v2 1/4 arm64: Add cpuidle context save/restore helpers
In reply to Marc ZyngierAs we need to start doing some additional work on all idle paths, let's introduce a set of macros that will perform the work related to the GICv3 pseudo-NMI idle entry exit.
Stubs are introduced to 32bit ARM for compatibility. As these helpers are currently unused, there is no functional change.
--- arch/arm/include/asm/cpuidle.h | 5 +++++ arch/arm64/include/asm/cpuidle.h | 35 ++++++++++++++++++++++++++++++++ 2 files changed, 40 insertions(+)
arch/arm/include/asm/cpuidle.h +5 -0
@@ -49,4 +49,9 @@ extern int arm_cpuidle_suspend(int index);extern int arm_cpuidle_init (int cpu); +struct arm_cpuidle_irq_context { }; + +#define arm_cpuidle_save_irq_context (c) (void)c +#define arm_cpuidle_restore_irq_context (c) (void)c +#endif
arch/arm64/include/asm/cpuidle.h +35 -0
@@ -18,4 +18,39 @@ static inline int arm_cpuidle_suspend(int index)return -EOPNOTSUPP ; }#endif + +#ifdef CONFIG_ARM64_PSEUDO_NMI +#include <asm/arch_gicv3.h> + +struct arm_cpuidle_irq_context { +unsigned long pmr; +unsigned long daif_bits; +}; + +#define arm_cpuidle_save_irq_context (__c) \ + do { \ + struct arm_cpuidle_irq_context *c = __c; \ + if (system_uses_irq_prio_masking()) { \ + c->daif_bits = read_sysreg(daif); \ + write_sysreg(c->daif_bits | PSR_I_BIT | PSR_F_BIT, \ + daif); \ + c->pmr = gic_read_pmr(); \ + gic_write_pmr(GIC_PRIO_IRQON | GIC_PRIO_PSR_I_SET); \ + } \ + } while (0) + +#define arm_cpuidle_restore_irq_context (__c) \ + do { \ + struct arm_cpuidle_irq_context *c = __c; \ + if (system_uses_irq_prio_masking()) { \ + gic_write_pmr(c->pmr); \ + write_sysreg(c->daif_bits, daif); \ + } \ + } while (0) +#else +struct arm_cpuidle_irq_context { }; + +#define arm_cpuidle_save_irq_context (c) (void)c +#define arm_cpuidle_restore_irq_context (c) (void)c +#endif #endif
Signature · 7 lines
2.30.2 _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Re: [PATCH v2 4/4] arm64: suspend: Use cpuidle context helpers in cpu_suspend()
In reply to Marc ZyngierQuoted 4 lines · Marc Zyngier
Use cpuidle context helpers to switch to using DAIF.IF instead of PMR to mask interrupts, ensuring that we suspend with interrupts being able to reach the CPU interface.
--
Regards,
Sudeep
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Quoted 8 lines · Marc Zyngier
This is a follow-up on the series posted at [1], with some added fixes. The main difference is the covering of system suspend, although it is untested (the GICv3 boxes I have don't support system suspend...).
Tested on an Ampere Altra system.
[...]
Applied to arm64 (for-next/cpuidle), thanks!
[1/4] arm64: Add cpuidle context save/restore helpers
https://git.kernel.org/arm64/c/8848f0665b3c [2/4] arm64: Convert cpu_do_idle() to using cpuidle context helpers https://git.kernel.org/arm64/c/d4dc10277255 [3/4] PSCI: Use cpuidle context helpers in psci_cpu_suspend_enter() https://git.kernel.org/arm64/c/c9223b616298 [4/4] arm64: suspend: Use cpuidle context helpers in cpu_suspend() https://git.kernel.org/arm64/c/77345ef70445
Cheers,
Signature · 10 lines
Will https://fixes.arm64.dev https://next.arm64.dev https://will.arm64.dev _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel