Skip to contentSkip to conversation

arm64: Fix cpuidle/suspend with pseudo-NMI enabled

PATCH Applied Marc Zyngier 3 participants 8 messages Jun 15, 2021 – Jun 17, 2021

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

v2 cover arm64: Fix cpuidle/suspend with pseudo-NMI enabled

This is a follow-up on the series posted at [1], with some added fixes. The main difference is the covering of system suspend, although it is untested (the GICv3 boxes I have don't support system suspend...).

Tested on an Ampere Altra system.

* From v1:
- Moved the wrapping of cpu_suspend() into the body of the function, now covering system suspend as well
- Added RBs, TBs.

[1] https://lore.kernel.org/r/[email protected]

Marc Zyngier (4):
arm64: Add cpuidle context save/restore helpers
arm64: Convert cpu_do_idle() to using cpuidle context helpers
PSCI: Use cpuidle context helpers in psci_cpu_suspend_enter()
arm64: suspend: Use cpuidle context helpers in cpu_suspend()

 arch/arm/include/asm/cpuidle.h   |  5 ++++
 arch/arm64/include/asm/cpuidle.h | 35 +++++++++++++++++++++++++++
 arch/arm64/kernel/process.c      | 41 +++++++-------------------------
 arch/arm64/kernel/suspend.c      | 12 +++++++++-
 drivers/firmware/psci/psci.c     |  9 +++++--
 5 files changed, 67 insertions(+), 35 deletions(-)
Signature · 7 lines
2.30.2


_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
#1 lore raw

PATCH v2 1/4 arm64: Add cpuidle context save/restore helpers

As we need to start doing some additional work on all idle paths, let's introduce a set of macros that will perform the work related to the GICv3 pseudo-NMI idle entry exit.

Stubs are introduced to 32bit ARM for compatibility. As these helpers are currently unused, there is no functional change.

---
 arch/arm/include/asm/cpuidle.h   |  5 +++++
 arch/arm64/include/asm/cpuidle.h | 35 ++++++++++++++++++++++++++++++++
 2 files changed, 40 insertions(+)
arch/arm/include/asm/cpuidle.h +5 -0
index 0d67ed682e07..dc8f53f1a219 100644
@@ -49,4 +49,9 @@ extern int arm_cpuidle_suspend(int index);
 
 extern int arm_cpuidle_init(int cpu);
 
+struct arm_cpuidle_irq_context { };
+
+#define arm_cpuidle_save_irq_context(c)		(void)c
+#define arm_cpuidle_restore_irq_context(c)	(void)c
+
 #endif
arch/arm64/include/asm/cpuidle.h +35 -0
index 3c5ddb429ea2..14a19d1141bd 100644
@@ -18,4 +18,39 @@ static inline int arm_cpuidle_suspend(int index)
 	return -EOPNOTSUPP;
 }
 #endif
+
+#ifdef CONFIG_ARM64_PSEUDO_NMI
+#include <asm/arch_gicv3.h>
+
+struct arm_cpuidle_irq_context {
+	unsigned long pmr;
+	unsigned long daif_bits;
+};
+
+#define arm_cpuidle_save_irq_context(__c)				\
+	do {								\
+		struct arm_cpuidle_irq_context *c = __c;		\
+		if (system_uses_irq_prio_masking()) {			\
+			c->daif_bits = read_sysreg(daif);		\
+			write_sysreg(c->daif_bits | PSR_I_BIT | PSR_F_BIT, \
+				     daif);				\
+			c->pmr = gic_read_pmr();			\
+			gic_write_pmr(GIC_PRIO_IRQON | GIC_PRIO_PSR_I_SET); \
+		}							\
+	} while (0)
+
+#define arm_cpuidle_restore_irq_context(__c)				\
+	do {								\
+		struct arm_cpuidle_irq_context *c = __c;		\
+		if (system_uses_irq_prio_masking()) {			\
+			gic_write_pmr(c->pmr);				\
+			write_sysreg(c->daif_bits, daif);		\
+		}							\
+	} while (0)
+#else
+struct arm_cpuidle_irq_context { };
+
+#define arm_cpuidle_save_irq_context(c)		(void)c
+#define arm_cpuidle_restore_irq_context(c)	(void)c
+#endif
 #endif
Signature · 7 lines
2.30.2


_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
#2 lore raw

PATCH v2 2/4 arm64: Convert cpu_do_idle() to using cpuidle context helpers

In reply to Marc Zyngier

Now that we have helpers that are aware of the pseudo-NMI feature, introduce them to cpu_do_idle(). This allows for some nice cleanup.

No functional change intended.

---
 arch/arm64/kernel/process.c | 41 ++++++++-----------------------------
 1 file changed, 9 insertions(+), 32 deletions(-)
arch/arm64/kernel/process.c +9 -32
index b4bb67f17a2c..b715c6b2558f 100644
@@ -46,9 +46,9 @@
 #include <linux/prctl.h>
 
 #include <asm/alternative.h>
-#include <asm/arch_gicv3.h>
 #include <asm/compat.h>
 #include <asm/cpufeature.h>
+#include <asm/cpuidle.h>
 #include <asm/cacheflush.h>
 #include <asm/exec.h>
 #include <asm/fpsimd.h>
@@ -74,33 +74,6 @@ EXPORT_SYMBOL_GPL(pm_power_off);
 
 void (*arm_pm_restart)(enum reboot_mode reboot_mode, const char *cmd);
 
-static void noinstr __cpu_do_idle(void)
-{
-	dsb(sy);
-	wfi();
-}
-
-static void noinstr __cpu_do_idle_irqprio(void)
-{
-	unsigned long pmr;
-	unsigned long daif_bits;
-
-	daif_bits = read_sysreg(daif);
-	write_sysreg(daif_bits | PSR_I_BIT | PSR_F_BIT, daif);
-
-	/*
-	 * Unmask PMR before going idle to make sure interrupts can
-	 * be raised.
-	 */
-	pmr = gic_read_pmr();
-	gic_write_pmr(GIC_PRIO_IRQON | GIC_PRIO_PSR_I_SET);
-
-	__cpu_do_idle();
-
-	gic_write_pmr(pmr);
-	write_sysreg(daif_bits, daif);
-}
-
 /*
  *	cpu_do_idle()
  *
@@ -112,10 +85,14 @@ static void noinstr __cpu_do_idle_irqprio(void)
  */
 void noinstr cpu_do_idle(void)
 {
-	if (system_uses_irq_prio_masking())
-		__cpu_do_idle_irqprio();
-	else
-		__cpu_do_idle();
+	struct arm_cpuidle_irq_context context;
+
+	arm_cpuidle_save_irq_context(&context);
+
+	dsb(sy);
+	wfi();
+
+	arm_cpuidle_restore_irq_context(&context);
 }
 
 /*
Signature · 7 lines
2.30.2


_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
#3 lore raw

PATCH v2 3/4 PSCI: Use cpuidle context helpers in psci_cpu_suspend_enter()

In reply to Marc Zyngier

The PSCI CPU suspend code isn't aware of the PMR vs DAIF game, resulting in a system that locks up if entering CPU suspend with GICv3 pNMI enabled.

To save the day, teach the suspend code about our new cpuidle context helpers, which will do everything that's required just like the usual WFI cpuidle code.

This fixes my Altra system, which would otherwise lock-up at boot time when booted with irqchip.gicv3_pseudo_nmi=1.

---
 drivers/firmware/psci/psci.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)
drivers/firmware/psci/psci.c +7 -2
index 3c1c5daf6df2..e3da38e15c5b 100644
@@ -335,10 +335,15 @@ int psci_cpu_suspend_enter(u32 state)
 {
 	int ret;
 
-	if (!psci_power_state_loses_context(state))
+	if (!psci_power_state_loses_context(state)) {
+		struct arm_cpuidle_irq_context context;
+
+		arm_cpuidle_save_irq_context(&context);
 		ret = psci_ops.cpu_suspend(state, 0);
-	else
+		arm_cpuidle_restore_irq_context(&context);
+	} else {
 		ret = cpu_suspend(state, psci_suspend_finisher);
+	}
 
 	return ret;
 }
Signature · 7 lines
2.30.2


_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Reviewed-by on v2 3/4
#4 lore raw

Re: [PATCH v2 3/4] PSCI: Use cpuidle context helpers in psci_cpu_suspend_enter()

Quoted 11 lines · Marc Zyngier

The PSCI CPU suspend code isn't aware of the PMR vs DAIF game, resulting in a system that locks up if entering CPU suspend with GICv3 pNMI enabled.

To save the day, teach the suspend code about our new cpuidle context helpers, which will do everything that's required just like the usual WFI cpuidle code.

This fixes my Altra system, which would otherwise lock-up at boot time when booted with irqchip.gicv3_pseudo_nmi=1.

--
Regards,
Sudeep

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

#5 lore raw

PATCH v2 4/4 arm64: suspend: Use cpuidle context helpers in cpu_suspend()

In reply to Marc Zyngier

Use cpuidle context helpers to switch to using DAIF.IF instead of PMR to mask interrupts, ensuring that we suspend with interrupts being able to reach the CPU interface.

---
 arch/arm64/kernel/suspend.c | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)
arch/arm64/kernel/suspend.c +11 -1
index e3f72df9509d..938ce6fbee8a 100644
@@ -7,6 +7,7 @@
 #include <asm/alternative.h>
 #include <asm/cacheflush.h>
 #include <asm/cpufeature.h>
+#include <asm/cpuidle.h>
 #include <asm/daifflags.h>
 #include <asm/debug-monitors.h>
 #include <asm/exec.h>
@@ -91,6 +92,7 @@ int cpu_suspend(unsigned long arg, int (*fn)(unsigned long))
 	int ret = 0;
 	unsigned long flags;
 	struct sleep_stack_data state;
+	struct arm_cpuidle_irq_context context;
 
 	/* Report any MTE async fault before going to suspend */
 	mte_suspend_enter();
@@ -103,12 +105,18 @@ int cpu_suspend(unsigned long arg, int (*fn)(unsigned long))
 	flags = local_daif_save();
 
 	/*
-	 * Function graph tracer state gets incosistent when the kernel
+	 * Function graph tracer state gets inconsistent when the kernel
 	 * calls functions that never return (aka suspend finishers) hence
 	 * disable graph tracing during their execution.
 	 */
 	pause_graph_tracing();
 
+	/*
+	 * Switch to using DAIF.IF instead of PMR in order to reliably
+	 * resume if we're using pseudo-NMIs.
+	 */
+	arm_cpuidle_save_irq_context(&context);
+
 	if (__cpu_suspend_enter(&state)) {
 		/* Call the suspend finisher */
 		ret = fn(arg);
@@ -126,6 +134,8 @@ int cpu_suspend(unsigned long arg, int (*fn)(unsigned long))
 		RCU_NONIDLE(__cpu_suspend_exit());
 	}
 
+	arm_cpuidle_restore_irq_context(&context);
+
 	unpause_graph_tracing();
 
 	/*
Signature · 7 lines
2.30.2


_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Reviewed-by on v2 4/4
#6 lore raw

Re: [PATCH v2 4/4] arm64: suspend: Use cpuidle context helpers in cpu_suspend()

Quoted 4 lines · Marc Zyngier

Use cpuidle context helpers to switch to using DAIF.IF instead of PMR to mask interrupts, ensuring that we suspend with interrupts being able to reach the CPU interface.

--
Regards,
Sudeep

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

maintainer Applied
#7 lore raw
In reply to Marc Zyngier
Quoted 8 lines · Marc Zyngier

This is a follow-up on the series posted at [1], with some added fixes. The main difference is the covering of system suspend, although it is untested (the GICv3 boxes I have don't support system suspend...).

Tested on an Ampere Altra system.

[...]

Applied to arm64 (for-next/cpuidle), thanks!

[1/4] arm64: Add cpuidle context save/restore helpers

      https://git.kernel.org/arm64/c/8848f0665b3c
[2/4] arm64: Convert cpu_do_idle() to using cpuidle context helpers
      https://git.kernel.org/arm64/c/d4dc10277255
[3/4] PSCI: Use cpuidle context helpers in psci_cpu_suspend_enter()
      https://git.kernel.org/arm64/c/c9223b616298
[4/4] arm64: suspend: Use cpuidle context helpers in cpu_suspend()
      https://git.kernel.org/arm64/c/77345ef70445

Cheers,

Signature · 10 lines
Will

https://fixes.arm64.dev
https://next.arm64.dev
https://will.arm64.dev

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel