Skip to contentSkip to conversation

arm64/irq: use NMI to send stop IPI

PATCH In review Pingfan Liu 2 participants 10 messages Dec 8, 2020 – Dec 15, 2020

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 arm64/irq: use NMI to send stop IPI

Like other arches, both 'kexec -l' and 'kexec -p' expect to use NMI to interrupt other cpus in order to capture cpus tightly to a known state.

So for both of the cases, try to use NMI IPI if irqchip supports NMI.

Cc Catalin Marinas <[email protected]>
Cc Will Deacon <[email protected]>
Cc Mark Rutland <[email protected]>
Cc Marc Zyngier <[email protected]>
Cc Julien Thierry <[email protected]>

To: [email protected]

---
 arch/arm64/kernel/smp.c | 37 +++++++++++++++++++++++++++++++++++++
 1 file changed, 37 insertions(+)
arch/arm64/kernel/smp.c +37 -0
index 18e9727..ff61e2c 100644
@@ -884,6 +884,31 @@ static void ipi_cpu_crash_stop(unsigned int cpu, struct pt_regs *regs)
 }
 
 /*
+ * Overlapped with the counterpart in do_handle_IPI(), only just for nmi purpose
+ */
+static irqreturn_t nmi_ipi_handler(int irq, void *data)
+{
+	unsigned int cpu = smp_processor_id();
+	int ipinr = irq - ipi_irq_base;
+
+	switch (ipinr) {
+	case IPI_CPU_STOP:
+		local_cpu_stop();
+		unreachable();
+		break;
+
+	case IPI_CPU_CRASH_STOP:
+		if (IS_ENABLED(CONFIG_KEXEC_CORE)) {
+			ipi_cpu_crash_stop(cpu, get_irq_regs());
+
+			unreachable();
+		}
+		break;
+	}
+	return IRQ_NONE;
+}
+
+/*
  * Main handler for inter-processor interrupts
  */
 static void do_handle_IPI(int ipinr)
@@ -902,10 +927,12 @@ static void do_handle_IPI(int ipinr)
 		generic_smp_call_function_interrupt();
 		break;
 
+	/* If NMI IPI is not supported */
 	case IPI_CPU_STOP:
 		local_cpu_stop();
 		break;
 
+	/* If NMI IPI is not supported */
 	case IPI_CPU_CRASH_STOP:
 		if (IS_ENABLED(CONFIG_KEXEC_CORE)) {
 			ipi_cpu_crash_stop(cpu, get_irq_regs());
@@ -989,6 +1016,14 @@ void __init set_smp_ipi_range(int ipi_base, int n)
 	for (i = 0; i < nr_ipi; i++) {
 		int err;
 
+		if (i == IPI_CPU_CRASH_STOP || i == IPI_CPU_STOP) {
+			err = request_percpu_nmi(ipi_base + i, nmi_ipi_handler,
+				"NMI-IPI", &cpu_number);
+			if (!err)
+				continue;
+
+			pr_warn("Fail to request nmi for NMI-IPI\n");
+		}
 		err = request_percpu_irq(ipi_base + i, ipi_handler,
 					 "IPI", &cpu_number);
 		WARN_ON(err);
@@ -1038,6 +1073,7 @@ void smp_send_stop(void)
 
 		if (system_state <= SYSTEM_RUNNING)
 			pr_crit("SMP: stopping secondary CPUs\n");
+		/* NMI is expected, but it is subject to the irq line's ability */
 		smp_cross_call(&mask, IPI_CPU_STOP);
 	}
 
@@ -1084,6 +1120,7 @@ void crash_smp_send_stop(void)
 	atomic_set(&waiting_for_crash_ipi, num_other_online_cpus());
 
 	pr_crit("SMP: stopping secondary CPUs\n");
+	/* NMI is expected, but it is subject to the irq line's ability */
 	smp_cross_call(&mask, IPI_CPU_CRASH_STOP);
 
 	/* Wait up to one second for other CPUs to stop */
Signature · 7 lines
2.7.5


_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
maintainer
#1 lore raw
Quoted 4 lines · Pingfan Liu

Like other arches, both 'kexec -l' and 'kexec -p' expect to use NMI to interrupt other cpus in order to capture cpus tightly to a known state.

So for both of the cases, try to use NMI IPI if irqchip supports NMI.

There are at least 2 other competing series doing the same thing, and (ab)using NMIs for dubious purposes (kexec, kgdb...)

(1) Please work with others instead of reinventing things from scratch, with similar problems

(2) No additional NMI patches will make it until we have resolved the issues described in [1], which is likely to take some time

Thanks,

        M.

[1] https://lore.kernel.org/linux-arm-kernel/[email protected]/

Signature · 6 lines
Jazz is not dead. It just smells funny...

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
#2 lore raw
Quoted 10 lines · Marc Zyngier
Quoted 4 lines · Pingfan Liu

Like other arches, both 'kexec -l' and 'kexec -p' expect to use NMI to interrupt other cpus in order to capture cpus tightly to a known state.

So for both of the cases, try to use NMI IPI if irqchip supports NMI.

There are at least 2 other competing series doing the same thing, and (ab)using NMIs for dubious purposes (kexec, kgdb...)

I think kexec has a good justification, but start with (2) at present.

Quoted 3 lines

(1) Please work with others instead of reinventing things from scratch, with similar problems

I will.

Quoted 4 lines

(2) No additional NMI patches will make it until we have resolved the issues
described in [1], which is likely to take some time

After serval days of chasing this issue, I have a raw idea about it. What about re-structure the code like DEFINE_IDTENTRY_SYSVEC_SIMPLE(sysvec_reschedule_ipi) ?

I will send a series soon afterward.

Thanks,
Pingfan

Quoted 8 lines

Thanks,

         M.

[1]
https://lore.kernel.org/linux-arm-kernel/[email protected]/
--
Jazz is not dead. It just smells funny...

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

#3 lore raw

PATCH 1/3 kernel/irq: __handle_domain_irq() makes irq_enter/exit arch optional

Arches may implement their own counterpart of irq_enter/exit in the frontend. Hence make the pair in __handle_domain_irq() arch optional

Cc Catalin Marinas <[email protected]>
Cc Will Deacon <[email protected]>
Cc Mark Rutland <[email protected]>
Cc Marc Zyngier <[email protected]>
Cc Julien Thierry <[email protected]>

To: [email protected]

---
 kernel/irq/Kconfig   | 4 ++++
 kernel/irq/irqdesc.c | 4 ++++
 2 files changed, 8 insertions(+)
kernel/irq/Kconfig +4 -0
index 164a031cfdb6..f17a9c8771c1 100644
@@ -100,6 +100,10 @@ config IRQ_MSI_IOMMU
 config HANDLE_DOMAIN_IRQ
 	bool
 
+config ARCH_HAS_IRQENTRY_ENTER
+	bool
+	select HANDLE_DOMAIN_IRQ
+
 config IRQ_TIMINGS
 	bool
 
kernel/irq/irqdesc.c +4 -0
index 1a7723604399..77819e489f9b 100644
@@ -669,7 +669,9 @@ int __handle_domain_irq(struct irq_domain *domain, unsigned int hwirq,
 	unsigned int irq = hwirq;
 	int ret = 0;
 
+#ifndef CONFIG_ARCH_HAS_IRQENTRY_ENTER
 	irq_enter();
+#endif
 
 #ifdef CONFIG_IRQ_DOMAIN
 	if (lookup)
@@ -687,7 +689,9 @@ int __handle_domain_irq(struct irq_domain *domain, unsigned int hwirq,
 		generic_handle_irq(irq);
 	}
 
+#ifndef CONFIG_ARCH_HAS_IRQENTRY_ENTER
 	irq_exit();
+#endif
 	set_irq_regs(old_regs);
 	return ret;
 }
Signature · 7 lines
2.29.2


_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
#4 lore raw

PATCH 2/3 arm64/irq-gic-v3: make gic_handle_irq() cope with enter_from_kernel_mode()

The call to irq_enter() originated from gic_handle_irq() has already redundant, since arm64 has enter_from_kernel_mode() akin to irqenter_entry().

So eliminate the call in __handle_domain_irq(). And accordingly supplement irq_enter_rcu().

Cc Catalin Marinas <[email protected]>
Cc Will Deacon <[email protected]>
Cc Mark Rutland <[email protected]>
Cc Marc Zyngier <[email protected]>
Cc Julien Thierry <[email protected]>

To: [email protected]

---
 arch/arm64/Kconfig           | 1 +
 drivers/irqchip/irq-gic-v3.c | 6 +++++-
 2 files changed, 6 insertions(+), 1 deletion(-)
arch/arm64/Kconfig +1 -0
index a6b5b7ef40ae..84ffed84158f 100644
@@ -201,6 +201,7 @@ config ARM64
 	select SWIOTLB
 	select SYSCTL_EXCEPTION_TRACE
 	select THREAD_INFO_IN_TASK
+	select ARCH_HAS_IRQENTRY_ENTER
 	help
 	  ARM 64-bit (AArch64) Linux support.
 
drivers/irqchip/irq-gic-v3.c +5 -1
index 16fecc0febe8..a66e0b04e42b 100644
@@ -654,14 +654,17 @@ static asmlinkage void __exception_irq_entry gic_handle_irq(struct pt_regs *regs
 		return;
 	}
 
+	irq_enter_rcu();
 	if (gic_prio_masking_enabled()) {
 		gic_pmr_mask_irqs();
 		gic_arch_enable_irqs();
 	}
 
 	/* Check for special IDs first */
-	if ((irqnr >= 1020 && irqnr <= 1023))
+	if ((irqnr >= 1020 && irqnr <= 1023)) {
+		irq_exit_rcu();
 		return;
+	}
 
 	if (static_branch_likely(&supports_deactivate_key))
 		gic_write_eoir(irqnr);
@@ -672,6 +675,7 @@ static asmlinkage void __exception_irq_entry gic_handle_irq(struct pt_regs *regs
 		WARN_ONCE(true, "Unexpected interrupt received!\n");
 		gic_deactivate_unhandled(irqnr);
 	}
+	irq_exit_rcu();
 }
 
 static u32 gic_get_pribits(void)
Signature · 7 lines
2.29.2


_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
#5 lore raw

PATCH 3/3 arm64/irq-gic-v3: make reschedule-ipi light weight

In reply to Pingfan Liu

To achieve the light weight as DEFINE_IDTENTRY_SYSVEC_SIMPLE(sysvec_reschedule_ipi) on x86, it had better treat irqnr differently at the frontend. And let IPI_RESCHEDULE call __irq_enter_raw() instead of irq_enter_rcu().

Cc Catalin Marinas <[email protected]>
Cc Will Deacon <[email protected]>
Cc Mark Rutland <[email protected]>
Cc Marc Zyngier <[email protected]>
Cc Julien Thierry <[email protected]>

To: [email protected]

---
 drivers/irqchip/irq-gic-v3.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)
drivers/irqchip/irq-gic-v3.c +11 -2
index a66e0b04e42b..777f5ace6862 100644
@@ -642,6 +642,9 @@ static inline void gic_handle_nmi(u32 irqnr, struct pt_regs *regs)
 		nmi_exit();
 }
 
+/* RESCHEDULE IPI hwirq nr is 0, and the only raw one */
+static unsigned long raw_interrupt_mask = 1;
+
 static asmlinkage void __exception_irq_entry gic_handle_irq(struct pt_regs *regs)
 {
 	u32 irqnr;
@@ -654,7 +657,10 @@ static asmlinkage void __exception_irq_entry gic_handle_irq(struct pt_regs *regs
 		return;
 	}
 
-	irq_enter_rcu();
+	if (raw_interrupt_mask & 1 << irqnr)
+		__irq_enter_raw();
+	else
+		irq_enter_rcu();
 	if (gic_prio_masking_enabled()) {
 		gic_pmr_mask_irqs();
 		gic_arch_enable_irqs();
@@ -675,7 +681,10 @@ static asmlinkage void __exception_irq_entry gic_handle_irq(struct pt_regs *regs
 		WARN_ONCE(true, "Unexpected interrupt received!\n");
 		gic_deactivate_unhandled(irqnr);
 	}
-	irq_exit_rcu();
+	if (raw_interrupt_mask & 1 << irqnr)
+		__irq_exit_raw();
+	else
+		irq_exit_rcu();
 }
 
 static u32 gic_get_pribits(void)
Signature · 7 lines
2.29.2


_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
maintainer
#6 lore raw
In reply to Pingfan Liu

On 2020-12-14 02:20, Pingfan Liu wrote:

[...]

Quoted 3 lines

After serval days of chasing this issue, I have a raw idea about it. What about re-structure the code like DEFINE_IDTENTRY_SYSVEC_SIMPLE(sysvec_reschedule_ipi) ?

I'm not sure how you want to mimic what x86 does, given that we have at least three different root interrupt controllers on arm64 (plus a fourth coming up), all of which have access methods that are not part of the CPU architecture, and that are shared with the 32bit part.

Quoted 1 line

I will send a series soon afterward.

Please wait until the merge window is over.

Thanks,

        M.
Signature · 6 lines
Jazz is not dead. It just smells funny...

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
#7 lore raw
Quoted 9 lines · Marc Zyngier

On 2020-12-14 02:20, Pingfan Liu wrote:

[...]

Quoted 3 lines

After serval days of chasing this issue, I have a raw idea about it. What about re-structure the code like DEFINE_IDTENTRY_SYSVEC_SIMPLE(sysvec_reschedule_ipi) ?

Quoted 6 lines

I'm not sure how you want to mimic what x86 does, given that we have at least three different root interrupt controllers on arm64 (plus a fourth coming up), all of which have access methods that are not part of the CPU architecture, and that are shared with the 32bit part.

I just found that Mark had implemented enter_el1_irq_or_nmi() for arm64, and think it is another way to handle the problem.

I had thought if aligned to x86, then linux/kernel/entry/common.c can be used. It may bring benefits, e.g. shift off the rcu/lockdep/dyntick to core kernel.

Anyway, I am a new beginner at this. If my opinion is not good enough, please just ignore it.

Quoted 4 lines
Quoted 1 line

I will send a series soon afterward.

Please wait until the merge window is over.

Sorry that it has been sent out before your reply. If there is V2, I will do it after the merge window.

Thanks,
Pingfan

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

maintainer
#8 lore raw
Quoted 16 lines · Pingfan Liu
Quoted 9 lines · Marc Zyngier

On 2020-12-14 02:20, Pingfan Liu wrote:

[...]

Quoted 3 lines

After serval days of chasing this issue, I have a raw idea about it. What about re-structure the code like DEFINE_IDTENTRY_SYSVEC_SIMPLE(sysvec_reschedule_ipi) ?

Quoted 6 lines

I'm not sure how you want to mimic what x86 does, given that we have at least three different root interrupt controllers on arm64 (plus a fourth coming up), all of which have access methods that are not part of the CPU architecture, and that are shared with the 32bit part.

Quoted 6 lines

I just found that Mark had implemented enter_el1_irq_or_nmi() for arm64, and think it is another way to handle the problem.

I had thought if aligned to x86, then linux/kernel/entry/common.c can be used. It may bring benefits, e.g. shift off the rcu/lockdep/dyntick to core kernel.

Using the entry/common.c stuff is indeed what we are planning to do, but the current arm64 code doesn't lend itself to that just yet,

Quoted 9 lines

Anyway, I am a new beginner at this. If my opinion is not good enough, please just ignore it.

Quoted 4 lines
Quoted 1 line

I will send a series soon afterward.

Please wait until the merge window is over.

Sorry that it has been sent out before your reply. If there is V2, I will do it after the merge window.

Please look at the series I pointed you too (and its follow-up), which has most of what you are trying to achieve already.

Thanks,

        M.
Signature · 6 lines
Jazz is not dead. It just smells funny...

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
#9 lore raw
Quoted 33 lines · Marc Zyngier
Quoted 16 lines · Pingfan Liu
Quoted 9 lines · Marc Zyngier

On 2020-12-14 02:20, Pingfan Liu wrote:

[...]

Quoted 3 lines

After serval days of chasing this issue, I have a raw idea about it. What about re-structure the code like DEFINE_IDTENTRY_SYSVEC_SIMPLE(sysvec_reschedule_ipi) ?

Quoted 6 lines

I'm not sure how you want to mimic what x86 does, given that we have at least three different root interrupt controllers on arm64 (plus a fourth coming up), all of which have access methods that are not part of the CPU architecture, and that are shared with the 32bit part.

Quoted 6 lines

I just found that Mark had implemented enter_el1_irq_or_nmi() for arm64, and think it is another way to handle the problem.

I had thought if aligned to x86, then linux/kernel/entry/common.c can be used. It may bring benefits, e.g. shift off the rcu/lockdep/dyntick to core kernel.

Using the entry/common.c stuff is indeed what we are planning to do, but the current arm64 code doesn't lend itself to that just yet,

Quoted 5 lines

Anyway, I am a new beginner at this. If my opinion is not good enough, please just ignore it.

Quoted 2 lines
Quoted 1 line

I will send a series soon afterward.

Quoted 8 lines
Quoted 4 lines
Quoted 2 lines

Please wait until the merge window is over.

Sorry that it has been sent out before your reply. If there is V2, I will do it after the merge window.

Please look at the series I pointed you too (and its follow-up), which has most of what you are trying to achieve already.

OK, and glad to know this issue is close to the end.

Thanks,
Pingfan

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel