Skip to contentSkip to conversation

tip-queue 2015-12-14

PATCH New Borislav Petkov 1 participant 8 messages Dec 14, 2015

Subsystems (3)
lore
Share & tools
lore

Borislav Petkov

author maintainer
#0 lore raw

cover tip-queue 2015-12-14

From: Borislav Petkov <[email protected]>

Hi Ingo,

here's a pile which addresses issues which arise when using an NMI button on certain platforms which provide it, to cause an external NMI and thus cause a concurrent kdump to fail. For that, we're getting a new cmdline option called "apic_extnmi" which can control which CPUs (or none) to receive the external NMI. The default behavior is what the kernel does currently - only the BSP receives it.

I've made sure that each commit message explains in great detail what the problem is as this whole area is not trivial.

Please apply,
thanks.

Borislav Petkov (1):
x86/mm: Align macro defines

Hidehiro Kawai (6):
panic, x86: Fix re-entrance problem due to panic on NMI panic, x86: Allow CPUs to save registers even if looping in NMI context
kexec: Fix race between panic() and crash_kexec()
x86/apic: Introduce apic_extnmi command line parameter x86/nmi: Save regs in crash dump on external NMI
Documentation: Document kernel.panic_on_io_nmi sysctl

author maintainer
#1 lore raw

PATCH 1/7 panic, x86: Fix re-entrance problem due to panic on NMI

From: Hidehiro Kawai <[email protected]>

If panic on NMI happens just after panic() on the same CPU, panic() is recursively called. Kernel stalls, as a result, after failing to acquire panic_lock.

To avoid this problem, don't call panic() in NMI context if we've already entered panic().

For that, introduce nmi_panic() macro to reduce code duplication. In the case of panic on NMI, don't return from NMI handlers if another CPU already panicked.

Cc Aaron Tomlin <[email protected]>
Cc Andrew Morton <[email protected]>
Cc Andy Lutomirski <[email protected]>
Cc Baoquan He <[email protected]>
Cc Chris Metcalf <[email protected]>
Cc David Hildenbrand <[email protected]>
Cc Don Zickus <[email protected]>
Cc "Eric W. Biederman" <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Gobinda Charan Maji <[email protected]>
Cc HATAYAMA Daisuke <[email protected]>
Cc "H. Peter Anvin" <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Javi Merino <[email protected]>
Cc Jonathan Corbet <[email protected]>
Cc Masami Hiramatsu <[email protected]>
Cc Michal Nazarewicz <[email protected]>
Cc Nicolas Iooss <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Prarit Bhargava <[email protected]>
Cc Rasmus Villemoes <[email protected]>
Cc Rusty Russell <[email protected]>
Cc Seth Jennings <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Ulrich Obergfell <[email protected]>
Cc Vitaly Kuznetsov <[email protected]>
Cc Vivek Goyal <[email protected]>
[ Cleanup comments, fixup formatting. ]
---
 arch/x86/kernel/nmi.c  | 16 ++++++++++++----
 include/linux/kernel.h | 20 ++++++++++++++++++++
 kernel/panic.c         | 16 +++++++++++++---
 kernel/watchdog.c      |  2 +-
 4 files changed, 46 insertions(+), 8 deletions(-)
arch/x86/kernel/nmi.c +12 -4
index 697f90db0e37..fca87938d739 100644
@@ -231,7 +231,7 @@ pci_serr_error(unsigned char reason, struct pt_regs *regs)
 #endif
 
 	if (panic_on_unrecovered_nmi)
-		panic("NMI: Not continuing");
+		nmi_panic("NMI: Not continuing");
 
 	pr_emerg("Dazed and confused, but trying to continue\n");
 
@@ -255,8 +255,16 @@ io_check_error(unsigned char reason, struct pt_regs *regs)
 		 reason, smp_processor_id());
 	show_regs(regs);
 
-	if (panic_on_io_nmi)
-		panic("NMI IOCK error: Not continuing");
+	if (panic_on_io_nmi) {
+		nmi_panic("NMI IOCK error: Not continuing");
+
+		/*
+		 * If we end up here, it means we have received an NMI while
+		 * processing panic(). Simply return without delaying and
+		 * re-enabling NMIs.
+		 */
+		return;
+	}
 
 	/* Re-enable the IOCK line, wait for a few seconds */
 	reason = (reason & NMI_REASON_CLEAR_MASK) | NMI_REASON_CLEAR_IOCHK;
@@ -297,7 +305,7 @@ unknown_nmi_error(unsigned char reason, struct pt_regs *regs)
 
 	pr_emerg("Do you have a strange power saving mode enabled?\n");
 	if (unknown_nmi_panic || panic_on_unrecovered_nmi)
-		panic("NMI: Not continuing");
+		nmi_panic("NMI: Not continuing");
 
 	pr_emerg("Dazed and confused, but trying to continue\n");
 }
include/linux/kernel.h +20 -0
index 350dfb08aee3..750cc5c7c999 100644
@@ -446,6 +446,26 @@ extern int sysctl_panic_on_stackoverflow;
 extern bool crash_kexec_post_notifiers;
 
 /*
+ * panic_cpu is used for synchronizing panic() and crash_kexec() execution. It
+ * holds a CPU number which is executing panic() currently. A value of
+ * PANIC_CPU_INVALID means no CPU has entered panic() or crash_kexec().
+ */
+extern atomic_t panic_cpu;
+#define PANIC_CPU_INVALID	-1
+
+/*
+ * A variant of panic() called from NMI context. We return if we've already
+ * panicked on this CPU.
+ */
+#define nmi_panic(fmt, ...)						\
+do {									\
+	int cpu = raw_smp_processor_id();				\
+									\
+	if (atomic_cmpxchg(&panic_cpu, PANIC_CPU_INVALID, cpu) != cpu)	\
+		panic(fmt, ##__VA_ARGS__);				\
+} while (0)
+
+/*
  * Only to be used by arch init code. If the user over-wrote the default
  * CONFIG_PANIC_TIMEOUT, honor it.
  */
kernel/panic.c +13 -3
index 4b150bc0c6c1..3344524cf6ff 100644
@@ -61,6 +61,8 @@ void __weak panic_smp_self_stop(void)
 		cpu_relax();
 }
 
+atomic_t panic_cpu = ATOMIC_INIT(PANIC_CPU_INVALID);
+
 /**
  *	panic - halt the system
  *	@fmt: The text string to print
@@ -71,17 +73,17 @@ void __weak panic_smp_self_stop(void)
  */
 void panic(const char *fmt, ...)
 {
-	static DEFINE_SPINLOCK(panic_lock);
 	static char buf[1024];
 	va_list args;
 	long i, i_next = 0;
 	int state = 0;
+	int old_cpu, this_cpu;
 
 	/*
 	 * Disable local interrupts. This will prevent panic_smp_self_stop
 	 * from deadlocking the first cpu that invokes the panic, since
 	 * there is nothing to prevent an interrupt handler (that runs
-	 * after the panic_lock is acquired) from invoking panic again.
+	 * after setting panic_cpu) from invoking panic() again.
 	 */
 	local_irq_disable();
 
@@ -94,8 +96,16 @@ void panic(const char *fmt, ...)
 	 * multiple parallel invocations of panic, all other CPUs either
 	 * stop themself or will wait until they are stopped by the 1st CPU
 	 * with smp_send_stop().
+	 *
+	 * `old_cpu == PANIC_CPU_INVALID' means this is the 1st CPU which
+	 * comes here, so go ahead.
+	 * `old_cpu == this_cpu' means we came from nmi_panic() which sets
+	 * panic_cpu to this CPU.  In this case, this is also the 1st CPU.
 	 */
-	if (!spin_trylock(&panic_lock))
+	this_cpu = raw_smp_processor_id();
+	old_cpu  = atomic_cmpxchg(&panic_cpu, PANIC_CPU_INVALID, this_cpu);
+
+	if (old_cpu != PANIC_CPU_INVALID && old_cpu != this_cpu)
 		panic_smp_self_stop();
 
 	console_verbose();
kernel/watchdog.c +1 -1
index 18f34cf75f74..b9be18fae154 100644
@@ -351,7 +351,7 @@ static void watchdog_overflow_callback(struct perf_event *event,
 			trigger_allbutself_cpu_backtrace();
 
 		if (hardlockup_panic)
-			panic("Hard LOCKUP");
+			nmi_panic("Hard LOCKUP");
 
 		__this_cpu_write(hard_watchdog_warn, true);
 		return;
Signature · 1 line
2.3.5
author maintainer
#2 lore raw

PATCH 2/7 panic, x86: Allow CPUs to save registers even if looping in NMI context

In reply to Borislav Petkov

From: Hidehiro Kawai <[email protected]>

Currently, kdump_nmi_shootdown_cpus(), a subroutine of crash_kexec(), sends an NMI IPI to CPUs which haven't called panic() to stop them, save their register information and do some cleanups for crash dumping. However, if such a CPU is infinitely looping in NMI context, we fail to save its register information into the crash dump.

For example, this can happen when unknown NMIs are broadcast to all CPUs as follows:

CPU 0 CPU 1
=========================== ==========================
receive an unknown NMI
unknown_nmi_error()
panic() receive an unknown NMI
spin_trylock(&panic_lock) unknown_nmi_error()
crash_kexec() panic()
spin_trylock(&panic_lock) panic_smp_self_stop()
infinite loop
kdump_nmi_shootdown_cpus()
issue NMI IPI -----------> blocked until IRET
infinite loop...

Here, since CPU 1 is in NMI context, the second NMI from CPU 0 is blocked until CPU 1 executes IRET. However, CPU 1 never executes IRET, so the NMI is not handled and the callback function to save registers is never called.

In practice, this can happen on some servers which broadcast NMIs to all CPUs when the NMI button is pushed.

To save registers in this case, we need to:

a) Return from NMI handler instead of looping infinitely or
b) Call the callback function directly from the infinite loop

Inherently, a) is risky because NMI is also used to prevent corrupted data from being propagated to devices. So, we chose b).

This patch does the following:

1. Move the infinite looping of CPUs which haven't called panic() in NMI context (actually done by panic_smp_self_stop()) outside of panic() to enable us to refer pt_regs. Please note that panic_smp_self_stop() is still used for normal context.

2. Call a callback of kdump_nmi_shootdown_cpus() directly to save registers and do some cleanups after setting waiting_for_crash_ipi which is used for counting down the number of CPUs which handled the callback

Cc Aaron Tomlin <[email protected]>
Cc Andrew Morton <[email protected]>
Cc Andy Lutomirski <[email protected]>
Cc Baoquan He <[email protected]>
Cc Chris Metcalf <[email protected]>
Cc Dave Young <[email protected]>
Cc David Hildenbrand <[email protected]>
Cc Don Zickus <[email protected]>
Cc Eric Biederman <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Gobinda Charan Maji <[email protected]>
Cc HATAYAMA Daisuke <[email protected]>
Cc Hidehiro Kawai <[email protected]>
Cc "H. Peter Anvin" <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Javi Merino <[email protected]>
Cc Jiang Liu <[email protected]>
Cc Jonathan Corbet <[email protected]>
Cc Masami Hiramatsu <[email protected]>
Cc Michal Nazarewicz <[email protected]>
Cc Nicolas Iooss <[email protected]>
Cc Oleg Nesterov <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Prarit Bhargava <[email protected]>
Cc Rasmus Villemoes <[email protected]>
Cc Seth Jennings <[email protected]>
Cc Stefan Lippers-Hollmann <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Ulrich Obergfell <[email protected]>
Cc Vitaly Kuznetsov <[email protected]>
Cc Vivek Goyal <[email protected]>
Cc Yasuaki Ishimatsu <[email protected]>
[ Cleanup comments, fixup formatting. ]
---
 arch/x86/kernel/nmi.c    |  6 +++---
 arch/x86/kernel/reboot.c | 20 ++++++++++++++++++++
 include/linux/kernel.h   | 16 ++++++++++++----
 kernel/panic.c           |  9 +++++++++
 kernel/watchdog.c        |  2 +-
 5 files changed, 45 insertions(+), 8 deletions(-)
arch/x86/kernel/nmi.c +3 -3
index fca87938d739..424aec4a4c71 100644
@@ -231,7 +231,7 @@ pci_serr_error(unsigned char reason, struct pt_regs *regs)
 #endif
 
 	if (panic_on_unrecovered_nmi)
-		nmi_panic("NMI: Not continuing");
+		nmi_panic(regs, "NMI: Not continuing");
 
 	pr_emerg("Dazed and confused, but trying to continue\n");
 
@@ -256,7 +256,7 @@ io_check_error(unsigned char reason, struct pt_regs *regs)
 	show_regs(regs);
 
 	if (panic_on_io_nmi) {
-		nmi_panic("NMI IOCK error: Not continuing");
+		nmi_panic(regs, "NMI IOCK error: Not continuing");
 
 		/*
 		 * If we end up here, it means we have received an NMI while
@@ -305,7 +305,7 @@ unknown_nmi_error(unsigned char reason, struct pt_regs *regs)
 
 	pr_emerg("Do you have a strange power saving mode enabled?\n");
 	if (unknown_nmi_panic || panic_on_unrecovered_nmi)
-		nmi_panic("NMI: Not continuing");
+		nmi_panic(regs, "NMI: Not continuing");
 
 	pr_emerg("Dazed and confused, but trying to continue\n");
 }
arch/x86/kernel/reboot.c +20 -0
index 02693dd9a079..1da13022d544 100644
@@ -718,6 +718,7 @@ static int crashing_cpu;
 static nmi_shootdown_cb shootdown_callback;
 
 static atomic_t waiting_for_crash_ipi;
+static int crash_ipi_issued;
 
 static int crash_nmi_callback(unsigned int val, struct pt_regs *regs)
 {
@@ -780,6 +781,9 @@ void nmi_shootdown_cpus(nmi_shootdown_cb callback)
 
 	smp_send_nmi_allbutself();
 
+	/* Kick CPUs looping in NMI context. */
+	WRITE_ONCE(crash_ipi_issued, 1);
+
 	msecs = 1000; /* Wait at most a second for the other cpus to stop */
 	while ((atomic_read(&waiting_for_crash_ipi) > 0) && msecs) {
 		mdelay(1);
@@ -788,6 +792,22 @@ void nmi_shootdown_cpus(nmi_shootdown_cb callback)
 
 	/* Leave the nmi callback set */
 }
+
+/* Override the weak function in kernel/panic.c */
+void nmi_panic_self_stop(struct pt_regs *regs)
+{
+	while (1) {
+		/*
+		 * Wait for the crash dumping IPI to be issued, and then
+		 * call its callback directly.
+		 */
+		if (READ_ONCE(crash_ipi_issued))
+			crash_nmi_callback(0, regs); /* Don't return */
+
+		cpu_relax();
+	}
+}
+
 #else /* !CONFIG_SMP */
 void nmi_shootdown_cpus(nmi_shootdown_cb callback)
 {
include/linux/kernel.h +12 -4
index 750cc5c7c999..7311c3294e25 100644
@@ -255,6 +255,7 @@ extern long (*panic_blink)(int state);
 __printf(1, 2)
 void panic(const char *fmt, ...)
 	__noreturn __cold;
+void nmi_panic_self_stop(struct pt_regs *);
 extern void oops_enter(void);
 extern void oops_exit(void);
 void print_oops_end_marker(void);
@@ -455,14 +456,21 @@ extern atomic_t panic_cpu;
 
 /*
  * A variant of panic() called from NMI context. We return if we've already
- * panicked on this CPU.
+ * panicked on this CPU. If another CPU already panicked, loop in
+ * nmi_panic_self_stop() which can provide architecture dependent code such
+ * as saving register state for crash dump.
  */
-#define nmi_panic(fmt, ...)						\
+#define nmi_panic(regs, fmt, ...)					\
 do {									\
-	int cpu = raw_smp_processor_id();				\
+	int old_cpu, cpu;						\
 									\
-	if (atomic_cmpxchg(&panic_cpu, PANIC_CPU_INVALID, cpu) != cpu)	\
+	cpu = raw_smp_processor_id();					\
+	old_cpu = atomic_cmpxchg(&panic_cpu, PANIC_CPU_INVALID, cpu);	\
+									\
+	if (old_cpu == PANIC_CPU_INVALID)				\
 		panic(fmt, ##__VA_ARGS__);				\
+	else if (old_cpu != cpu)					\
+		nmi_panic_self_stop(regs);				\
 } while (0)
 
 /*
kernel/panic.c +9 -0
index 3344524cf6ff..06f31b49b3b4 100644
@@ -61,6 +61,15 @@ void __weak panic_smp_self_stop(void)
 		cpu_relax();
 }
 
+/*
+ * Stop ourselves in NMI context if another CPU has already panicked. Arch code
+ * may override this to prepare for crash dumping, e.g. save regs info.
+ */
+void __weak nmi_panic_self_stop(struct pt_regs *regs)
+{
+	panic_smp_self_stop();
+}
+
 atomic_t panic_cpu = ATOMIC_INIT(PANIC_CPU_INVALID);
 
 /**
kernel/watchdog.c +1 -1
index b9be18fae154..84b5035cb6a5 100644
@@ -351,7 +351,7 @@ static void watchdog_overflow_callback(struct perf_event *event,
 			trigger_allbutself_cpu_backtrace();
 
 		if (hardlockup_panic)
-			nmi_panic("Hard LOCKUP");
+			nmi_panic(regs, "Hard LOCKUP");
 
 		__this_cpu_write(hard_watchdog_warn, true);
 		return;
Signature · 1 line
2.3.5
author maintainer
#3 lore raw

PATCH 3/7 kexec: Fix race between panic() and crash_kexec()

In reply to Borislav Petkov

From: Hidehiro Kawai <[email protected]>

Currently, panic() and crash_kexec() can be called at the same time. For example (x86 case):

CPU 0:
oops_end()
crash_kexec()
mutex_trylock() // acquired
nmi_shootdown_cpus() // stop other CPUs

CPU 1:
panic()
crash_kexec()
mutex_trylock() // failed to acquire
smp_send_stop() // stop other CPUs
infinite loop

If CPU 1 calls smp_send_stop() before nmi_shootdown_cpus(), kdump fails.

In another case:

CPU 0:
oops_end()
crash_kexec()
mutex_trylock() // acquired
<NMI>
io_check_error()
panic()
crash_kexec()
mutex_trylock() // failed to acquire
infinite loop

Clearly, this is an undesirable result.

To fix this problem, this patch changes crash_kexec() to exclude others by using the panic_cpu atomic.

Cc Andrew Morton <[email protected]>
Cc Baoquan He <[email protected]>
Cc Dave Young <[email protected]>
Cc "Eric W. Biederman" <[email protected]>
Cc HATAYAMA Daisuke <[email protected]>
Cc "H. Peter Anvin" <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Jonathan Corbet <[email protected]>
Cc Martin Schwidefsky <[email protected]>
Cc Masami Hiramatsu <[email protected]>
Cc Minfei Huang <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Seth Jennings <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Vitaly Kuznetsov <[email protected]>
Cc Vivek Goyal <[email protected]>
Cc x86-ml <[email protected]>
---
 include/linux/kexec.h |  2 ++
 kernel/kexec_core.c   | 30 +++++++++++++++++++++++++++++-
 kernel/panic.c        |  8 ++++++--
 3 files changed, 37 insertions(+), 3 deletions(-)
include/linux/kexec.h +2 -0
index d140b1e9faa7..7b68d2788a56 100644
@@ -237,6 +237,7 @@ extern int kexec_purgatory_get_set_symbol(struct kimage *image,
 					  unsigned int size, bool get_value);
 extern void *kexec_purgatory_get_symbol_addr(struct kimage *image,
 					     const char *name);
+extern void __crash_kexec(struct pt_regs *);
 extern void crash_kexec(struct pt_regs *);
 int kexec_should_crash(struct task_struct *);
 void crash_save_cpu(struct pt_regs *regs, int cpu);
@@ -332,6 +333,7 @@ int __weak arch_kexec_apply_relocations(const Elf_Ehdr *ehdr, Elf_Shdr *sechdrs,
 #else /* !CONFIG_KEXEC_CORE */
 struct pt_regs;
 struct task_struct;
+static inline void __crash_kexec(struct pt_regs *regs) { }
 static inline void crash_kexec(struct pt_regs *regs) { }
 static inline int kexec_should_crash(struct task_struct *p) { return 0; }
 #define kexec_in_progress false
kernel/kexec_core.c +29 -1
index 11b64a63c0f8..c823f3001e12 100644
@@ -853,7 +853,12 @@ struct kimage *kexec_image;
 struct kimage *kexec_crash_image;
 int kexec_load_disabled;
 
-void crash_kexec(struct pt_regs *regs)
+/*
+ * No panic_cpu check version of crash_kexec().  This function is called
+ * only when panic_cpu holds the current CPU number; this is the only CPU
+ * which processes crash_kexec routines.
+ */
+void __crash_kexec(struct pt_regs *regs)
 {
 	/* Take the kexec_mutex here to prevent sys_kexec_load
 	 * running on one cpu from replacing the crash kernel
@@ -876,6 +881,29 @@ void crash_kexec(struct pt_regs *regs)
 	}
 }
 
+void crash_kexec(struct pt_regs *regs)
+{
+	int old_cpu, this_cpu;
+
+	/*
+	 * Only one CPU is allowed to execute the crash_kexec() code as with
+	 * panic().  Otherwise parallel calls of panic() and crash_kexec()
+	 * may stop each other.  To exclude them, we use panic_cpu here too.
+	 */
+	this_cpu = raw_smp_processor_id();
+	old_cpu = atomic_cmpxchg(&panic_cpu, PANIC_CPU_INVALID, this_cpu);
+	if (old_cpu == PANIC_CPU_INVALID) {
+		/* This is the 1st CPU which comes here, so go ahead. */
+		__crash_kexec(regs);
+
+		/*
+		 * Reset panic_cpu to allow another panic()/crash_kexec()
+		 * call.
+		 */
+		atomic_set(&panic_cpu, PANIC_CPU_INVALID);
+	}
+}
+
 size_t crash_get_memory_size(void)
 {
 	size_t size = 0;
kernel/panic.c +6 -2
index 06f31b49b3b4..b333380c6bb2 100644
@@ -136,9 +136,11 @@ void panic(const char *fmt, ...)
 	 * everything else.
 	 * If we want to run this after calling panic_notifiers, pass
 	 * the "crash_kexec_post_notifiers" option to the kernel.
+	 *
+	 * Bypass the panic_cpu check and call __crash_kexec directly.
 	 */
 	if (!crash_kexec_post_notifiers)
-		crash_kexec(NULL);
+		__crash_kexec(NULL);
 
 	/*
 	 * Note smp_send_stop is the usual smp shutdown function, which
@@ -161,9 +163,11 @@ void panic(const char *fmt, ...)
 	 * panic_notifiers and dumping kmsg before kdump.
 	 * Note: since some panic_notifiers can make crashed kernel
 	 * more unstable, it can increase risks of the kdump failure too.
+	 *
+	 * Bypass the panic_cpu check and call __crash_kexec directly.
 	 */
 	if (crash_kexec_post_notifiers)
-		crash_kexec(NULL);
+		__crash_kexec(NULL);
 
 	bust_spinlocks(0);
 
Signature · 1 line
2.3.5
author maintainer
#4 lore raw

PATCH 4/7 x86/apic: Introduce apic_extnmi command line parameter

In reply to Borislav Petkov

From: Hidehiro Kawai <[email protected]>

This patch introduces a command line parameter apic_extnmi:

apic_extnmi=( bsp|all|none )

The default value is "bsp" and this is the current behavior: only the Boot-Strapping Processor receives an external NMI.

"all" allows external NMIs to be broadcast to all CPUs. This would raise the success rate of panic on NMI when BSP hangs in NMI context or the external NMI is swallowed by other NMI handlers on the BSP.

If you specify "none", no CPUs receive external NMIs. This is useful for the dump capture kernel so that it cannot be shot down by accidentally pressing the external NMI button (on platforms which have it) while saving a crash dump.

Cc Andrew Morton <[email protected]>
Cc Andy Lutomirski <[email protected]>
Cc Bandan Das <[email protected]>
Cc Baoquan He <[email protected]>
Cc "Eric W. Biederman" <[email protected]>
Cc "H. Peter Anvin" <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Jiang Liu <[email protected]>
Cc Joerg Roedel <[email protected]>
Cc Jonathan Corbet <[email protected]>
Cc "Maciej W. Rozycki" <[email protected]>
Cc Masami Hiramatsu <[email protected]>
Cc Paolo Bonzini <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Ricardo Ribalda Delgado <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Viresh Kumar <[email protected]>
Cc Vivek Goyal <[email protected]>
Cc x86-ml <[email protected]>
---
 Documentation/kernel-parameters.txt |  9 +++++++++
 arch/x86/include/asm/apic.h         |  5 +++++
 arch/x86/kernel/apic/apic.c         | 34 ++++++++++++++++++++++++++++++++--
 3 files changed, 46 insertions(+), 2 deletions(-)
Documentation/kernel-parameters.txt +9 -0
index 742f69d18fc8..74acea53e9e3 100644
@@ -472,6 +472,15 @@ bytes respectively. Such letter suffixes can also be entirely omitted.
 			Change the amount of debugging information output
 			when initialising the APIC and IO-APIC components.
 
+	apic_extnmi=	[APIC,X86] External NMI delivery setting
+			Format: { bsp (default) | all | none }
+			bsp:  External NMI is delivered only to CPU 0
+			all:  External NMIs are broadcast to all CPUs as a
+			      backup of CPU 0
+			none: External NMI is masked for all CPUs. This is
+			      useful so that a dump capture kernel won't be
+			      shot down by NMI
+
 	autoconf=	[IPV6]
 			See Documentation/networking/ipv6.txt.
 
arch/x86/include/asm/apic.h +5 -0
index a30316bf801a..357171342f53 100644
@@ -23,6 +23,11 @@
 #define APIC_VERBOSE 1
 #define APIC_DEBUG   2
 
+/* Macros for apic_extnmi which controls external NMI masking */
+#define APIC_EXTNMI_BSP		0 /* Default */
+#define APIC_EXTNMI_ALL		1
+#define APIC_EXTNMI_NONE	2
+
 /*
  * Define the default level of output to be very little
  * This can be turned up by using apic=verbose for more
arch/x86/kernel/apic/apic.c +32 -2
index 2f69e3b184f6..8584319d0117 100644
@@ -82,6 +82,12 @@ physid_mask_t phys_cpu_present_map;
 static unsigned int disabled_cpu_apicid __read_mostly = BAD_APICID;
 
 /*
+ * This variable controls which CPUs receive external NMIs.  By default,
+ * external NMIs are delivered only to the BSP.
+ */
+static int apic_extnmi = APIC_EXTNMI_BSP;
+
+/*
  * Map cpu index to physical APIC ID
  */
 DEFINE_EARLY_PER_CPU_READ_MOSTLY(u16, x86_cpu_to_apicid, BAD_APICID);
@@ -1161,6 +1167,8 @@ void __init init_bsp_APIC(void)
 	value = APIC_DM_NMI;
 	if (!lapic_is_integrated())		/* 82489DX */
 		value |= APIC_LVT_LEVEL_TRIGGER;
+	if (apic_extnmi == APIC_EXTNMI_NONE)
+		value |= APIC_LVT_MASKED;
 	apic_write(APIC_LVT1, value);
 }
 
@@ -1378,9 +1386,11 @@ void setup_local_APIC(void)
 	apic_write(APIC_LVT0, value);
 
 	/*
-	 * only the BP should see the LINT1 NMI signal, obviously.
+	 * Only the BSP sees the LINT1 NMI signal by default. This can be
+	 * modified by apic_extnmi= boot option.
 	 */
-	if (!cpu)
+	if ((!cpu && apic_extnmi != APIC_EXTNMI_NONE) ||
+	    apic_extnmi == APIC_EXTNMI_ALL)
 		value = APIC_DM_NMI;
 	else
 		value = APIC_DM_NMI | APIC_LVT_MASKED;
@@ -2548,3 +2558,23 @@ static int __init apic_set_disabled_cpu_apicid(char *arg)
 	return 0;
 }
 early_param("disable_cpu_apicid", apic_set_disabled_cpu_apicid);
+
+static int __init apic_set_extnmi(char *arg)
+{
+	if (!arg)
+		return -EINVAL;
+
+	if (!strncmp("all", arg, 3))
+		apic_extnmi = APIC_EXTNMI_ALL;
+	else if (!strncmp("none", arg, 4))
+		apic_extnmi = APIC_EXTNMI_NONE;
+	else if (!strncmp("bsp", arg, 3))
+		apic_extnmi = APIC_EXTNMI_BSP;
+	else {
+		pr_warn("Unknown external NMI delivery mode `%s' ignored\n", arg);
+		return -EINVAL;
+	}
+
+	return 0;
+}
+early_param("apic_extnmi", apic_set_extnmi);
Signature · 1 line
2.3.5
author maintainer
#5 lore raw

PATCH 5/7 x86/nmi: Save regs in crash dump on external NMI

In reply to Borislav Petkov

From: Hidehiro Kawai <[email protected]>

Now, multiple CPUs can receive an external NMI simultaneously by specifying the "apic_extnmi=all" command line parameter. When we take a crash dump by using external NMI with this option, we fail to save registers into the crash dump. This happens as follows:

CPU 0 CPU 1
================================ =============================
receive an external NMI
default_do_nmi() receive an external NMI

    spin_lock(&nmi_reason_lock)      default_do_nmi()
    io_check_error()                   spin_lock(&nmi_reason_lock)
      panic()                            busy loop
      ...
        kdump_nmi_shootdown_cpus()
          issue NMI IPI -----------> blocked until IRET
                                         busy loop...

Here, since CPU 1 is in NMI context, an additional NMI from CPU 0 remains unhandled until CPU 1 IRETs. However, CPU 1 will never execute IRET so the NMI is not handled and the callback function to save registers is never called.

To solve this issue, we check if the IPI for crash dumping was issued while waiting for nmi_reason_lock to be released, and if so, call its callback function directly. If the IPI is not issued (e.g. kdump is disabled), the actual behavior doesn't change.

Cc Andrew Morton <[email protected]>
Cc Andy Lutomirski <[email protected]>
Cc Baoquan He <[email protected]>
Cc Dave Young <[email protected]>
Cc "Eric W. Biederman" <[email protected]>
Cc "H. Peter Anvin" <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Jiang Liu <[email protected]>
Cc Jonathan Corbet <[email protected]>
Cc Masami Hiramatsu <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Stefan Lippers-Hollmann <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Vivek Goyal <[email protected]>
Cc x86-ml <[email protected]>
---
 arch/x86/include/asm/reboot.h |  1 +
 arch/x86/kernel/nmi.c         | 16 ++++++++++++++--
 arch/x86/kernel/reboot.c      | 24 +++++++++++++++++-------
 3 files changed, 32 insertions(+), 9 deletions(-)
arch/x86/include/asm/reboot.h +1 -0
index a82c4f1b4d83..2cb1cc253d51 100644
@@ -25,5 +25,6 @@ void __noreturn machine_real_restart(unsigned int type);
 
 typedef void (*nmi_shootdown_cb)(int, struct pt_regs*);
 void nmi_shootdown_cpus(nmi_shootdown_cb callback);
+void run_crash_ipi_callback(struct pt_regs *regs);
 
 #endif /* _ASM_X86_REBOOT_H */
arch/x86/kernel/nmi.c +14 -2
index 424aec4a4c71..8a2cdd736fa4 100644
@@ -29,6 +29,7 @@
 #include <asm/mach_traps.h>
 #include <asm/nmi.h>
 #include <asm/x86_init.h>
+#include <asm/reboot.h>
 
 #define CREATE_TRACE_POINTS
 #include <trace/events/nmi.h>
@@ -356,8 +357,19 @@ static void default_do_nmi(struct pt_regs *regs)
 		return;
 	}
 
-	/* Non-CPU-specific NMI: NMI sources can be processed on any CPU */
-	raw_spin_lock(&nmi_reason_lock);
+	/*
+	 * Non-CPU-specific NMI: NMI sources can be processed on any CPU.
+	 *
+	 * Another CPU may be processing panic routines while holding
+	 * nmi_reason_lock. Check if the CPU issued the IPI for crash dumping,
+	 * and if so, call its callback directly.  If there is no CPU preparing
+	 * crash dump, we simply loop here.
+	 */
+	while (!raw_spin_trylock(&nmi_reason_lock)) {
+		run_crash_ipi_callback(regs);
+		cpu_relax();
+	}
+
 	reason = x86_platform.get_nmi_reason();
 
 	if (reason & NMI_REASON_MASK) {
arch/x86/kernel/reboot.c +17 -7
index 1da13022d544..d64889aa2d46 100644
@@ -793,17 +793,23 @@ void nmi_shootdown_cpus(nmi_shootdown_cb callback)
 	/* Leave the nmi callback set */
 }
 
+/*
+ * Check if the crash dumping IPI got issued and if so, call its callback
+ * directly. This function is used when we have already been in NMI handler.
+ * It doesn't return.
+ */
+void run_crash_ipi_callback(struct pt_regs *regs)
+{
+	if (crash_ipi_issued)
+		crash_nmi_callback(0, regs);
+}
+
 /* Override the weak function in kernel/panic.c */
 void nmi_panic_self_stop(struct pt_regs *regs)
 {
 	while (1) {
-		/*
-		 * Wait for the crash dumping IPI to be issued, and then
-		 * call its callback directly.
-		 */
-		if (READ_ONCE(crash_ipi_issued))
-			crash_nmi_callback(0, regs); /* Don't return */
-
+		/* If no CPU is preparing crash dump, we simply loop here. */
+		run_crash_ipi_callback(regs);
 		cpu_relax();
 	}
 }
@@ -813,4 +819,8 @@ void nmi_shootdown_cpus(nmi_shootdown_cb callback)
 {
 	/* No other CPUs to shoot down */
 }
+
+void run_crash_ipi_callback(struct pt_regs *regs)
+{
+}
 #endif
Signature · 1 line
2.3.5
author maintainer
#6 lore raw

PATCH 6/7 Documentation: Document kernel.panic_on_io_nmi sysctl

In reply to Borislav Petkov

From: Hidehiro Kawai <[email protected]>

kernel.panic_on_io_nmi sysctl was introduced by commit

5211a242d0cb ("x86: Add sysctl to allow panic on IOCK NMI error")

but its documentation is missing. So, add it.

Cc Andrew Morton <[email protected]>
Cc Baoquan He <[email protected]>
Cc Chris Metcalf <[email protected]>
Cc Don Zickus <[email protected]>
Cc "Eric W. Biederman" <[email protected]>
Cc Heinrich Schuchardt <[email protected]>
Cc "H. Peter Anvin" <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Jiri Kosina <[email protected]>
Cc Jonathan Corbet <[email protected]>
Cc Manfred Spraul <[email protected]>
Cc Masami Hiramatsu <[email protected]>
Cc Michal Hocko <[email protected]>
Cc Nicolas Iooss <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Seth Jennings <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Ulrich Obergfell <[email protected]>
Cc Vivek Goyal <[email protected]>
Cc x86-ml <[email protected]>
---
 Documentation/sysctl/kernel.txt | 15 +++++++++++++++
 1 file changed, 15 insertions(+)
Documentation/sysctl/kernel.txt +15 -0
index af70d1541d3a..73c6b1ef0e84 100644
@@ -551,6 +551,21 @@ the recommended setting is 60.
 
 ==============================================================
 
+panic_on_io_nmi:
+
+Controls the kernel's behavior when a CPU receives an NMI caused by
+an IO error.
+
+0: try to continue operation (default)
+
+1: panic immediately. The IO error triggered an NMI. This indicates a
+   serious system condition which could result in IO data corruption.
+   Rather than continuing, panicking might be a better choice. Some
+   servers issue this sort of NMI when the dump button is pushed,
+   and you can use this option to take a crash dump.
+
+==============================================================
+
 panic_on_oops:
 
 Controls the kernel's behaviour when an oops or BUG is encountered.
Signature · 1 line
2.3.5
author maintainer
#7 lore raw

PATCH 7/7 x86/mm: Align macro defines

In reply to Borislav Petkov

From: Borislav Petkov <[email protected]>

Bring PAGE_{SHIFT,SIZE,MASK} to the same indentation level as the rest of the header.

---
 arch/x86/include/asm/page_types.h | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
arch/x86/include/asm/page_types.h +3 -3
index cc071c6f7d4d..7bd0099384ca 100644
@@ -5,9 +5,9 @@
 #include <linux/types.h>
 
 /* PAGE_SHIFT determines the page size */
-#define PAGE_SHIFT	12
-#define PAGE_SIZE	(_AC(1,UL) << PAGE_SHIFT)
-#define PAGE_MASK	(~(PAGE_SIZE-1))
+#define PAGE_SHIFT		12
+#define PAGE_SIZE		(_AC(1,UL) << PAGE_SHIFT)
+#define PAGE_MASK		(~(PAGE_SIZE-1))
 
 #define PMD_PAGE_SIZE		(_AC(1, UL) << PMD_SHIFT)
 #define PMD_PAGE_MASK		(~(PMD_PAGE_SIZE-1))
Signature · 1 line
2.3.5