Skip to contentSkip to conversation

powerpc/crash: Remove the test for cpu_online in the IPI callback

PATCH Superseded Balbir Singh 2 participants 9 messages Dec 15, 2017 PowerPC

v3 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

PATCH v3 1/3 powerpc/crash: Remove the test for cpu_online in the IPI callback

Our check was extra cautious, we've audited crash_send_ipi and it sends an IPI only to online CPU's. Removal of this check should have not functional impact on crash kdump.

---
 arch/powerpc/kernel/crash.c | 3 ---
 1 file changed, 3 deletions(-)
arch/powerpc/kernel/crash.c +0 -3
index cbabb5adccd9..29c56ca2ddfd 100644
@@ -69,9 +69,6 @@ static void crash_ipi_callback(struct pt_regs *regs)
 
 	int cpu = smp_processor_id();
 
-	if (!cpu_online(cpu))
-		return;
-
 	hard_irq_disable();
 	if (!cpumask_test_cpu(cpu, &cpus_state_saved)) {
 		crash_save_cpu(regs, cpu);
Signature · 1 line
2.13.6
#1 lore raw

PATCH v3 2/3 powerpc/powernv: Identify scom driven system reset

In irq_set_pending_from_srr1() we were missing 0x2 as system reset identified from SRR1 caused by back to back system resets or when interrupts are caused by SCOM when the thread is not in power saving mode.

This helps us get to NMI handling in both the case where NMI is caused when in power-saving and not in power-saving mode. The actual exploitation is expected when we are doing a kdump and an offline CPU might not be in power-saving mode due to an already spurious IPI or any other reason.

---
 arch/powerpc/kernel/irq.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/powerpc/kernel/irq.c +1 -1
index b7a84522e652..ec89104e8ab9 100644
@@ -413,7 +413,7 @@ bool prep_irq_for_idle_irqsoff(void)
 #define IRQ_SYSTEM_RESET	0xff
 
 static const u8 srr1_to_lazyirq[0x10] = {
-	0, 0, 0,
+	0, 0, IRQ_SYSTEM_RESET,
 	PACA_IRQ_DBELL,
 	IRQ_SYSTEM_RESET,
 	PACA_IRQ_DBELL,
Signature · 1 line
2.13.6
#2 lore raw

Re: [PATCH v3 2/3] powerpc/powernv: Identify scom driven system reset

Quoted 5 lines · On Fri, 15 Dec 2017 12:27:39 +1100 Balbir Singh <[email protected]> wrote:

In irq_set_pending_from_srr1() we were missing 0x2 as system reset identified from SRR1 caused by back to back system resets or when interrupts are caused by SCOM when the thread is not in power saving mode.

Quoted 7 lines

This helps us get to NMI handling in both the case where NMI is caused when in power-saving and not in power-saving mode. The actual exploitation is expected when we are doing a kdump and an offline CPU might not be in power-saving mode due to an already spurious IPI or any other reason.

When not in power saving mode, we don't look at SRR1 at all, so we don't need this. You should never be getting it returned as the result of your idle instruction (except on DD1 which has a bug, but firmware doesn't implement the NMI IPI).

It's possible we could pay more attention to the reason, for example in the powernv system reset handle we might only call the NMI IPI handler if it was a scom sreset... but that would be a regs->msr test.

Quoted 17 lines
---
 arch/powerpc/kernel/irq.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/powerpc/kernel/irq.c +1 -1
index b7a84522e652..ec89104e8ab9 100644
@@ -413,7 +413,7 @@ bool prep_irq_for_idle_irqsoff(void)
 #define IRQ_SYSTEM_RESET	0xff
 
 static const u8 srr1_to_lazyirq[0x10] = {
-	0, 0, 0,
+	0, 0, IRQ_SYSTEM_RESET,
 	PACA_IRQ_DBELL,
 	IRQ_SYSTEM_RESET,
 	PACA_IRQ_DBELL,
#3 lore raw

Re: [PATCH v3 2/3] powerpc/powernv: Identify scom driven system reset

Quoted 14 lines · Nicholas Piggin
Quoted 11 lines · On Fri, 15 Dec 2017 12:27:39 +1100 Balbir Singh <[email protected]> wrote:

In irq_set_pending_from_srr1() we were missing 0x2 as system reset identified from SRR1 caused by back to back system resets or when interrupts are caused by SCOM when the thread is not in power saving mode.

This helps us get to NMI handling in both the case where NMI is caused when in power-saving and not in power-saving mode. The actual exploitation is expected when we are doing a kdump and an offline CPU might not be in power-saving mode due to an already spurious IPI or any other reason.

Quoted 7 lines
Quoted 1 line

When not in power saving mode, we don't look at SRR1 at all, so we don't need this. You should never be getting it returned as the result of your idle instruction (except on DD1 which has a bug, but firmware doesn't implement the NMI IPI).

I added this for the next patch. We call irq_set_pending_from_srr1 while coming out of CPU idle, but if for any reason the IPI was spurious and then we see an NMI during kdump, I did want to detect that and callback into the NMI handler.

Quoted 4 lines

It's possible we could pay more attention to the reason, for example in the powernv system reset handle we might only call the NMI IPI handler if it was a scom sreset... but that would be a regs->msr test.

regs->msr is the same as SRR1 in my kdump case, but your right in general we want to look at regs->msr

Balbir Singh

#4 lore raw

Re: [PATCH v3 2/3] powerpc/powernv: Identify scom driven system reset

Quoted 23 lines · On Fri, 15 Dec 2017 13:54:18 +1100 Balbir Singh <[email protected]> wrote:
Quoted 14 lines · Nicholas Piggin
Quoted 11 lines · On Fri, 15 Dec 2017 12:27:39 +1100 Balbir Singh <[email protected]> wrote:

In irq_set_pending_from_srr1() we were missing 0x2 as system reset identified from SRR1 caused by back to back system resets or when interrupts are caused by SCOM when the thread is not in power saving mode.

This helps us get to NMI handling in both the case where NMI is caused when in power-saving and not in power-saving mode. The actual exploitation is expected when we are doing a kdump and an offline CPU might not be in power-saving mode due to an already spurious IPI or any other reason.

Quoted 7 lines
Quoted 1 line

When not in power saving mode, we don't look at SRR1 at all, so we don't need this. You should never be getting it returned as the result of your idle instruction (except on DD1 which has a bug, but firmware doesn't implement the NMI IPI).

Quoted 4 lines

I added this for the next patch. We call irq_set_pending_from_srr1 while coming out of CPU idle, but if for any reason the IPI was spurious and then we see an NMI during kdump, I did want to detect that and callback into the NMI handler.

You'll never see it. The SRR1 value you get is the idle wakeup code. So any scom-when-not-idle bit should never be set.

Thanks,
Nick

#5 lore raw

PATCH v3 3/3 powernv/kdump: Fix cases where the kdump kernel can get HMI's

In reply to Balbir Singh

Certain HMI's such as malfunction error propagate through all threads/core on the system. If a thread was offline prior to us crashing the system and jumping to the kdump kernel, bad things happen when it wakes up due to an HMI in the kdump kernel.

There are several possible ways to solve this problem

1. Put the offline cores in a state such that they are not woken up for machine check and HMI errors. This does not work, since we might need to wake up offline threads to handle TB errors
2. Ignore HMI errors, setup HMEER to mask HMI errors, but this still leads the window open for any MCEs and masking them for the duration of the dump might be a concern
3. Wake up offline CPUs, as in send them to crash_ipi_callback (not wake them up as in mark them online as seen by the hotplug). kexec does a wake_online_cpus() call, this patch does something similar, but instead sends an IPI and forces them to crash_ipi_callback()

This patch takes approach #3.

Care is taken to enable this only for powenv platforms via crash_wake_offline (a global value set at setup time). The crash code sends out IPI's to all CPU's which then move to crash_ipi_callback and kexec_smp_wait().

---

Changelog v3
- Use SRR1's reason to wake up to drive replay_system_reset() as a means of getting to kdump() as opposed to calling crash_ipi_callback based on comments from Nick Piggin.

 arch/powerpc/include/asm/kexec.h     |  2 ++
 arch/powerpc/kernel/crash.c          | 13 ++++++++++++-
 arch/powerpc/kernel/smp.c            | 18 ++++++++++++++++++
 arch/powerpc/platforms/powernv/smp.c | 12 ++++++++++++
 4 files changed, 44 insertions(+), 1 deletion(-)
arch/powerpc/include/asm/kexec.h +2 -0
index 4419d435639a..9dcbfa6bbb91 100644
@@ -73,6 +73,8 @@ extern void kexec_smp_wait(void);	/* get and clear naca physid, wait for
 					  master to copy new code to 0 */
 extern int crashing_cpu;
 extern void crash_send_ipi(void (*crash_ipi_callback)(struct pt_regs *));
+extern void crash_ipi_callback(struct pt_regs *);
+extern int crash_wake_offline;
 
 struct kimage;
 struct pt_regs;
arch/powerpc/kernel/crash.c +12 -1
index 29c56ca2ddfd..00b215125d3e 100644
@@ -44,6 +44,14 @@
 #define REAL_MODE_TIMEOUT	10000
 
 static int time_to_dump;
+/*
+ * crash_wake_offline should be set to 1 by platforms that intend to wake
+ * up offline cpus prior to jumping to a kdump kernel. Currently powernv
+ * sets it to 1, since we want to avoid things from happening when an
+ * offline CPU wakes up due to something like an HMI (malfunction error),
+ * which propagates to all threads.
+ */
+int crash_wake_offline;
 
 #define CRASH_HANDLER_MAX 3
 /* List of shutdown handles */
@@ -63,7 +71,7 @@ static int handle_fault(struct pt_regs *regs)
 #ifdef CONFIG_SMP
 
 static atomic_t cpus_in_crash;
-static void crash_ipi_callback(struct pt_regs *regs)
+void crash_ipi_callback(struct pt_regs *regs)
 {
 	static cpumask_t cpus_state_saved = CPU_MASK_NONE;
 
@@ -106,6 +114,9 @@ static void crash_kexec_prepare_cpus(int cpu)
 
 	printk(KERN_EMERG "Sending IPI to other CPUs\n");
 
+	if (crash_wake_offline)
+		ncpus = num_present_cpus() - 1;
+
 	crash_send_ipi(crash_ipi_callback);
 	smp_wmb();
 
arch/powerpc/kernel/smp.c +18 -0
index e0a4c1f82e25..bbe7634b3a43 100644
@@ -543,7 +543,25 @@ void smp_send_debugger_break(void)
 #ifdef CONFIG_KEXEC_CORE
 void crash_send_ipi(void (*crash_ipi_callback)(struct pt_regs *))
 {
+	int cpu;
+
 	smp_send_nmi_ipi(NMI_IPI_ALL_OTHERS, crash_ipi_callback, 1000000);
+	if (kdump_in_progress() && crash_wake_offline) {
+		for_each_present_cpu(cpu) {
+			if (cpu_online(cpu))
+				continue;
+			/*
+			 * crash_ipi_callback will wait for
+			 * all cpus, including offline CPUs.
+			 * We don't care about nmi_ipi_function.
+			 * Offline cpus will jump straight into
+			 * crash_ipi_callback, we can skip the
+			 * entire NMI dance and waiting for
+			 * cpus to clear pending mask, etc.
+			 */
+			do_smp_send_nmi_ipi(cpu);
+		}
+	}
 }
 #endif
 
arch/powerpc/platforms/powernv/smp.c +12 -0
index ba030669eca1..1594bbff3dec 100644
@@ -37,6 +37,8 @@
 #include <asm/kvm_ppc.h>
 #include <asm/ppc-opcode.h>
 #include <asm/cpuidle.h>
+#include <asm/kexec.h>
+#include <asm/reg.h>
 
 #include "powernv.h"
 
@@ -212,6 +214,13 @@ static void pnv_smp_cpu_kill_self(void)
 		}
 		smp_mb();
 
+		/*
+		 * For kdump kernels, we process the ipi and jump to
+		 * handling the system reset exception.
+		 */
+		if (kdump_in_progress())
+			irq_set_pending_from_srr1(srr1);
+
 		if (cpu_core_split_required())
 			continue;
 
@@ -371,5 +380,8 @@ void __init pnv_smp_init(void)
 
 #ifdef CONFIG_HOTPLUG_CPU
 	ppc_md.cpu_die	= pnv_smp_cpu_kill_self;
+#ifdef CONFIG_KEXEC_CORE
+	crash_wake_offline = 1;
+#endif
 #endif
 }
Signature · 1 line
2.13.6
#6 lore raw

Re: [PATCH v3 3/3] powernv/kdump: Fix cases where the kdump kernel can get HMI's

Quoted 142 lines · On Fri, 15 Dec 2017 12:27:40 +1100 Balbir Singh <[email protected]> wrote:

Certain HMI's such as malfunction error propagate through all threads/core on the system. If a thread was offline prior to us crashing the system and jumping to the kdump kernel, bad things happen when it wakes up due to an HMI in the kdump kernel.

There are several possible ways to solve this problem

1. Put the offline cores in a state such that they are not woken up for machine check and HMI errors. This does not work, since we might need to wake up offline threads to handle TB errors
2. Ignore HMI errors, setup HMEER to mask HMI errors, but this still leads the window open for any MCEs and masking them for the duration of the dump might be a concern
3. Wake up offline CPUs, as in send them to crash_ipi_callback (not wake them up as in mark them online as seen by the hotplug). kexec does a wake_online_cpus() call, this patch does something similar, but instead sends an IPI and forces them to crash_ipi_callback()

This patch takes approach #3.

Care is taken to enable this only for powenv platforms via crash_wake_offline (a global value set at setup time). The crash code sends out IPI's to all CPU's which then move to crash_ipi_callback and kexec_smp_wait().

---

Changelog v3
- Use SRR1's reason to wake up to drive replay_system_reset() as a means of getting to kdump() as opposed to calling crash_ipi_callback based on comments from Nick Piggin.

 arch/powerpc/include/asm/kexec.h     |  2 ++
 arch/powerpc/kernel/crash.c          | 13 ++++++++++++-
 arch/powerpc/kernel/smp.c            | 18 ++++++++++++++++++
 arch/powerpc/platforms/powernv/smp.c | 12 ++++++++++++
 4 files changed, 44 insertions(+), 1 deletion(-)
arch/powerpc/include/asm/kexec.h +2 -0
index 4419d435639a..9dcbfa6bbb91 100644
@@ -73,6 +73,8 @@ extern void kexec_smp_wait(void);	/* get and clear naca physid, wait for
 					  master to copy new code to 0 */
 extern int crashing_cpu;
 extern void crash_send_ipi(void (*crash_ipi_callback)(struct pt_regs *));
+extern void crash_ipi_callback(struct pt_regs *);
+extern int crash_wake_offline;
 
 struct kimage;
 struct pt_regs;
arch/powerpc/kernel/crash.c +12 -1
index 29c56ca2ddfd..00b215125d3e 100644
@@ -44,6 +44,14 @@
 #define REAL_MODE_TIMEOUT	10000
 
 static int time_to_dump;
+/*
+ * crash_wake_offline should be set to 1 by platforms that intend to wake
+ * up offline cpus prior to jumping to a kdump kernel. Currently powernv
+ * sets it to 1, since we want to avoid things from happening when an
+ * offline CPU wakes up due to something like an HMI (malfunction error),
+ * which propagates to all threads.
+ */
+int crash_wake_offline;
 
 #define CRASH_HANDLER_MAX 3
 /* List of shutdown handles */
@@ -63,7 +71,7 @@ static int handle_fault(struct pt_regs *regs)
 #ifdef CONFIG_SMP
 
 static atomic_t cpus_in_crash;
-static void crash_ipi_callback(struct pt_regs *regs)
+void crash_ipi_callback(struct pt_regs *regs)
 {
 	static cpumask_t cpus_state_saved = CPU_MASK_NONE;
 
@@ -106,6 +114,9 @@ static void crash_kexec_prepare_cpus(int cpu)
 
 	printk(KERN_EMERG "Sending IPI to other CPUs\n");
 
+	if (crash_wake_offline)
+		ncpus = num_present_cpus() - 1;
+
 	crash_send_ipi(crash_ipi_callback);
 	smp_wmb();
 
arch/powerpc/kernel/smp.c +18 -0
index e0a4c1f82e25..bbe7634b3a43 100644
@@ -543,7 +543,25 @@ void smp_send_debugger_break(void)
 #ifdef CONFIG_KEXEC_CORE
 void crash_send_ipi(void (*crash_ipi_callback)(struct pt_regs *))
 {
+	int cpu;
+
 	smp_send_nmi_ipi(NMI_IPI_ALL_OTHERS, crash_ipi_callback, 1000000);
+	if (kdump_in_progress() && crash_wake_offline) {
+		for_each_present_cpu(cpu) {
+			if (cpu_online(cpu))
+				continue;
+			/*
+			 * crash_ipi_callback will wait for
+			 * all cpus, including offline CPUs.
+			 * We don't care about nmi_ipi_function.
+			 * Offline cpus will jump straight into
+			 * crash_ipi_callback, we can skip the
+			 * entire NMI dance and waiting for
+			 * cpus to clear pending mask, etc.
+			 */
+			do_smp_send_nmi_ipi(cpu);
+		}
+	}
 }
 #endif
 
arch/powerpc/platforms/powernv/smp.c +2 -0
index ba030669eca1..1594bbff3dec 100644
@@ -37,6 +37,8 @@
 #include <asm/kvm_ppc.h>
 #include <asm/ppc-opcode.h>
 #include <asm/cpuidle.h>
+#include <asm/kexec.h>
+#include <asm/reg.h>
 
 #include "powernv.h"
 
@@ -212,6 +214,13 @@ static void pnv_smp_cpu_kill_self(void)
 		}
 		smp_mb();
 

arch/powerpc/platforms/powernv/smp.c static void pnv_smp_cpu_kill_self(void) · line 226

Quoted 10 lines
arch/powerpc/platforms/powernv/smp.c +7 -0
+		/*
+		 * For kdump kernels, we process the ipi and jump to
+		 * handling the system reset exception.
+		 */
+		if (kdump_in_progress())
+			irq_set_pending_from_srr1(srr1);
+
 		if (cpu_core_split_required())
 			continue;
 

I wonder if we want to special-case it for system reset only? Everything is all going down for kdump anyway I guess, but theoretically we don't want to put other interrupts in our pending mask.

We might as well just do it unconditionally as well, so we can easily test and make sure we do something sane if we happen to take a stray sreset here for some reason (e.g., pdbg).

I would do this:

 } else if ((srr1 & wmask) == SRR1_WAKERESET) {
     /* kdump or debug tools can sreset offline CPUs */
     irq_set_pending_from_srr1(srr1);

}

But then you still need an explicit kdump check for non-sreset wakeups because the platform may not implement sreset, or it may fall back to normal IPI if the sreset scom fails. So you then still need your

if (kdump) crash_ipi_callback

Thanks,
Nick

#7 lore raw

Re: [PATCH v3 3/3] powernv/kdump: Fix cases where the kdump kernel can get HMI's

Quoted 169 lines · Nicholas Piggin
Quoted 142 lines · On Fri, 15 Dec 2017 12:27:40 +1100 Balbir Singh <[email protected]> wrote:

Certain HMI's such as malfunction error propagate through all threads/core on the system. If a thread was offline prior to us crashing the system and jumping to the kdump kernel, bad things happen when it wakes up due to an HMI in the kdump kernel.

There are several possible ways to solve this problem

1. Put the offline cores in a state such that they are not woken up for machine check and HMI errors. This does not work, since we might need to wake up offline threads to handle TB errors
2. Ignore HMI errors, setup HMEER to mask HMI errors, but this still leads the window open for any MCEs and masking them for the duration of the dump might be a concern
3. Wake up offline CPUs, as in send them to crash_ipi_callback (not wake them up as in mark them online as seen by the hotplug). kexec does a wake_online_cpus() call, this patch does something similar, but instead sends an IPI and forces them to crash_ipi_callback()

This patch takes approach #3.

Care is taken to enable this only for powenv platforms via crash_wake_offline (a global value set at setup time). The crash code sends out IPI's to all CPU's which then move to crash_ipi_callback and kexec_smp_wait().

---

Changelog v3
- Use SRR1's reason to wake up to drive replay_system_reset() as a means of getting to kdump() as opposed to calling crash_ipi_callback based on comments from Nick Piggin.

 arch/powerpc/include/asm/kexec.h     |  2 ++
 arch/powerpc/kernel/crash.c          | 13 ++++++++++++-
 arch/powerpc/kernel/smp.c            | 18 ++++++++++++++++++
 arch/powerpc/platforms/powernv/smp.c | 12 ++++++++++++
 4 files changed, 44 insertions(+), 1 deletion(-)
arch/powerpc/include/asm/kexec.h +2 -0
index 4419d435639a..9dcbfa6bbb91 100644
@@ -73,6 +73,8 @@ extern void kexec_smp_wait(void);   /* get and clear naca physid, wait for
                                        master to copy new code to 0 */
 extern int crashing_cpu;
 extern void crash_send_ipi(void (*crash_ipi_callback)(struct pt_regs *));
+extern void crash_ipi_callback(struct pt_regs *);
+extern int crash_wake_offline;
 
 struct kimage;
 struct pt_regs;
arch/powerpc/kernel/crash.c +12 -1
index 29c56ca2ddfd..00b215125d3e 100644
@@ -44,6 +44,14 @@
 #define REAL_MODE_TIMEOUT    10000
 
 static int time_to_dump;
+/*
+ * crash_wake_offline should be set to 1 by platforms that intend to wake
+ * up offline cpus prior to jumping to a kdump kernel. Currently powernv
+ * sets it to 1, since we want to avoid things from happening when an
+ * offline CPU wakes up due to something like an HMI (malfunction error),
+ * which propagates to all threads.
+ */
+int crash_wake_offline;
 
 #define CRASH_HANDLER_MAX 3
 /* List of shutdown handles */
@@ -63,7 +71,7 @@ static int handle_fault(struct pt_regs *regs)
 #ifdef CONFIG_SMP
 
 static atomic_t cpus_in_crash;
-static void crash_ipi_callback(struct pt_regs *regs)
+void crash_ipi_callback(struct pt_regs *regs)
 {
      static cpumask_t cpus_state_saved = CPU_MASK_NONE;
 
@@ -106,6 +114,9 @@ static void crash_kexec_prepare_cpus(int cpu)
 
      printk(KERN_EMERG "Sending IPI to other CPUs\n");
 
+     if (crash_wake_offline)
+             ncpus = num_present_cpus() - 1;
+
      crash_send_ipi(crash_ipi_callback);
      smp_wmb();
 
arch/powerpc/kernel/smp.c +18 -0
index e0a4c1f82e25..bbe7634b3a43 100644
@@ -543,7 +543,25 @@ void smp_send_debugger_break(void)
 #ifdef CONFIG_KEXEC_CORE
 void crash_send_ipi(void (*crash_ipi_callback)(struct pt_regs *))
 {
+     int cpu;
+
      smp_send_nmi_ipi(NMI_IPI_ALL_OTHERS, crash_ipi_callback, 1000000);
+     if (kdump_in_progress() && crash_wake_offline) {
+             for_each_present_cpu(cpu) {
+                     if (cpu_online(cpu))
+                             continue;
+                     /*
+                      * crash_ipi_callback will wait for
+                      * all cpus, including offline CPUs.
+                      * We don't care about nmi_ipi_function.
+                      * Offline cpus will jump straight into
+                      * crash_ipi_callback, we can skip the
+                      * entire NMI dance and waiting for
+                      * cpus to clear pending mask, etc.
+                      */
+                     do_smp_send_nmi_ipi(cpu);
+             }
+     }
 }
 #endif
 
arch/powerpc/platforms/powernv/smp.c +2 -0
index ba030669eca1..1594bbff3dec 100644
@@ -37,6 +37,8 @@
 #include <asm/kvm_ppc.h>
 #include <asm/ppc-opcode.h>
 #include <asm/cpuidle.h>
+#include <asm/kexec.h>
+#include <asm/reg.h>
 
 #include "powernv.h"
 
@@ -212,6 +214,13 @@ static void pnv_smp_cpu_kill_self(void)
              }
              smp_mb();
 

arch/powerpc/platforms/powernv/smp.c static void pnv_smp_cpu_kill_self(void) · line 226

Quoted 10 lines
arch/powerpc/platforms/powernv/smp.c +7 -0
+             /*
+              * For kdump kernels, we process the ipi and jump to
+              * handling the system reset exception.
+              */
+             if (kdump_in_progress())
+                     irq_set_pending_from_srr1(srr1);
+
              if (cpu_core_split_required())
                      continue;
 

I wonder if we want to special-case it for system reset only? Everything is all going down for kdump anyway I guess, but theoretically we don't want to put other interrupts in our pending mask.

We might as well just do it unconditionally as well, so we can easily test and make sure we do something sane if we happen to take a stray sreset here for some reason (e.g., pdbg).

I would do this:

 } else if ((srr1 & wmask) == SRR1_WAKERESET) {
     /* kdump or debug tools can sreset offline CPUs */
     irq_set_pending_from_srr1(srr1);
Quoted 7 lines

}

But then you still need an explicit kdump check for non-sreset wakeups because the platform may not implement sreset, or it may fall back to normal IPI if the sreset scom fails. So you then still need your

if (kdump) crash_ipi_callback

Yep, its required for the the non NMI case.

How does this look -- based on your comment

+ } else if ((srr1 & wmask) == SRR1_WAKESRESET) {
+ irq_set_pending_from_srr1(srr1);
+ /* Does not return */

                }
+
                smp_mb();

                /*
                 * For kdump kernels, we process the ipi and jump to
-                * handling the system reset exception.
+                * crash_ipi_callback
                 */

- if (kdump_in_progress())
- irq_set_pending_from_srr1(srr1);
+ if (kdump_in_progress()) {
+ /*

+                        * If we got to this point, we've not used
+                        * NMI's, otherwise we would have gone
+                        * via the SRR1_WAKESRESET path. We are
+                        * using regular IPI's for waking up offline
+                        * threads.
+                        */
+                       struct pt_regs regs;
+
+                       ppc_save_regs(&regs);
+                       crash_ipi_callback(regs);
+                       /* Does not return */

+ }

Balbir Singh

Quoted 3 lines

Thanks,
Nick

#8 lore raw

Re: [PATCH v3 3/3] powernv/kdump: Fix cases where the kdump kernel can get HMI's

Quoted 33 lines · On Fri, 15 Dec 2017 14:34:03 +1100 Balbir Singh <[email protected]> wrote:
Quoted 8 lines · Nicholas Piggin

On Fri, 15 Dec 2017 12:27:40 +1100
Balbir Singh <[email protected]> wrote:

But then you still need an explicit kdump check for non-sreset wakeups because the platform may not implement sreset, or it may fall back to normal IPI if the sreset scom fails. So you then still need your

if (kdump) crash_ipi_callback

Yep, its required for the the non NMI case.

How does this look -- based on your comment

(quoted diff) +9 -3
+               } else if ((srr1 & wmask) == SRR1_WAKESRESET) {
+                       irq_set_pending_from_srr1(srr1);
+                       /* Does not return */
                }
+
                smp_mb();
 
                /*
                 * For kdump kernels, we process the ipi and jump to
-                * handling the system reset exception.
+                * crash_ipi_callback
                 */
-               if (kdump_in_progress())
-                       irq_set_pending_from_srr1(srr1);
+               if (kdump_in_progress()) {
+                       /*
+                        * If we got to this point, we've not used
+                        * NMI's, otherwise we would have gone

Review

Quoted 10 lines
(quoted diff) +10 -0
+                        * via the SRR1_WAKESRESET path. We are
+                        * using regular IPI's for waking up offline
+                        * threads.
+                        */
+                       struct pt_regs regs;
+
+                       ppc_save_regs(&regs);
+                       crash_ipi_callback(regs);
+                       /* Does not return */
+               }

That looks like it should do the trick.

Thanks,
Nick