Skip to contentSkip to conversation

Avoid broadcasting NMI IPIs

PATCH In review Keith Owens 2 participants 4 messages Jun 23, 2006 – Jun 24, 2006 Generic include/asm header files

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 Avoid broadcasting NMI IPIs

On some i386/x86_64 systems, sending an NMI IPI as a broadcast will reset the system. This seems to be a BIOS bug which affects machines where one or more cpus are not under OS control. It occurs on HT systems with a version of the OS that is not compiled without HT support. It also occurs when a system is booted with max_cpus=n where 2 <= n < cpus known to the BIOS. The fix is to always send NMI IPI as a mask instead of as a broadcast.

---

This patch needs my earlier patch that defines NMI_VECTOR for i386.
http://marc.theaimsgroup.com/?l=linux-arch&m=115096692430934&w=2

I was going to do a bigger patch that removed all the broadcast mode IPI code, so all systems would use cpu masks for IPI. Broadcast mode is currently restricted to small machines with CONFIG_HOTPLUG_CPU=n, everything else already uses mask mode. Removing broadcast mode would simplify the maze of per-platform IPI handling, everything would be using masks. But I decided to do the minimal fix and leave the IPI clean up (and removing the trailing white space in those files) for another time.

 arch/x86_64/kernel/genapic_flat.c        |   25 ++++++++++++++++---------
 include/asm-i386/mach-default/mach_ipi.h |    4 ++--
 2 files changed, 18 insertions(+), 11 deletions(-)
arch/x86_64/kernel/genapic_flat.c +16 -9
Index: linux/arch/x86_64/kernel/genapic_flat.c
===================================================================
@@ -78,22 +78,29 @@ static void flat_send_IPI_mask(cpumask_t
 
 static void flat_send_IPI_allbutself(int vector)
 {
-#ifndef CONFIG_HOTPLUG_CPU
-	if (((num_online_cpus()) - 1) >= 1)
-		__send_IPI_shortcut(APIC_DEST_ALLBUT, vector,APIC_DEST_LOGICAL);
+#ifdef	CONFIG_HOTPLUG_CPU
+	int hotplug = 1;
 #else
-	cpumask_t allbutme = cpu_online_map;
+	int hotplug = 0;
+#endif
+	if (hotplug || vector == NMI_VECTOR) {
+		cpumask_t allbutme = cpu_online_map;
 
-	cpu_clear(smp_processor_id(), allbutme);
+		cpu_clear(smp_processor_id(), allbutme);
 
-	if (!cpus_empty(allbutme))
-		flat_send_IPI_mask(allbutme, vector);
-#endif
+		if (!cpus_empty(allbutme))
+			flat_send_IPI_mask(allbutme, vector);
+	} else if (num_online_cpus() > 1) {
+		__send_IPI_shortcut(APIC_DEST_ALLBUT, vector,APIC_DEST_LOGICAL);
+	}
 }
 
 static void flat_send_IPI_all(int vector)
 {
-	__send_IPI_shortcut(APIC_DEST_ALLINC, vector, APIC_DEST_LOGICAL);
+	if (vector == NMI_VECTOR)
+		flat_send_IPI_mask(cpu_online_map, vector);
+	else
+		__send_IPI_shortcut(APIC_DEST_ALLINC, vector, APIC_DEST_LOGICAL);
 }
 
 static int flat_apic_id_registered(void)
include/asm-i386/mach-default/mach_ipi.h +2 -2
Index: linux/include/asm-i386/mach-default/mach_ipi.h
===================================================================
@@ -13,7 +13,7 @@ static inline void send_IPI_mask(cpumask
 
 static inline void __local_send_IPI_allbutself(int vector)
 {
-	if (no_broadcast) {
+	if (no_broadcast || vector == NMI_VECTOR) {
 		cpumask_t mask = cpu_online_map;
 
 		cpu_clear(smp_processor_id(), mask);
@@ -24,7 +24,7 @@ static inline void __local_send_IPI_allb
 
 static inline void __local_send_IPI_all(int vector)
 {
-	if (no_broadcast)
+	if (no_broadcast || vector == NMI_VECTOR)
 		send_IPI_mask(cpu_online_map, vector);
 	else
 		__send_IPI_shortcut(APIC_DEST_ALLINC, vector);
#1 lore raw
Quoted 7 lines · Keith Owens

On some i386/x86_64 systems, sending an NMI IPI as a broadcast will reset the system. This seems to be a BIOS bug which affects machines where one or more cpus are not under OS control. It occurs on HT systems with a version of the OS that is not compiled without HT support. It also occurs when a system is booted with max_cpus=n where 2 <= n < cpus known to the BIOS. The fix is to always send NMI IPI as a mask instead of as a broadcast.

Merged thanks.

P.S.: Linux-arch isn't really the right list to cc x86 patches too. I'm sure the other arch maintainers couldn't care less about such x86isms.

-Andi

#2 lore raw
Quoted 9 lines · Andi Kleen (on Fri, 23 Jun 2006 14:32:56 +0200) wrote:
Quoted 7 lines · Keith Owens

On some i386/x86_64 systems, sending an NMI IPI as a broadcast will reset the system. This seems to be a BIOS bug which affects machines where one or more cpus are not under OS control. It occurs on HT systems with a version of the OS that is not compiled without HT support. It also occurs when a system is booted with max_cpus=n where 2 <= n < cpus known to the BIOS. The fix is to always send NMI IPI as a mask instead of as a broadcast.

Quoted 4 lines

Merged thanks.

P.S.: Linux-arch isn't really the right list to cc x86 patches too. I'm sure the other arch maintainers couldn't care less about such x86isms.

These two patches were for both i386 and x86_64, not just for x86_64.

#3 lore raw
Quoted 10 lines · Keith Owens
Quoted 9 lines · Andi Kleen (on Fri, 23 Jun 2006 14:32:56 +0200) wrote:
Quoted 7 lines · Keith Owens

On some i386/x86_64 systems, sending an NMI IPI as a broadcast will reset the system. This seems to be a BIOS bug which affects machines where one or more cpus are not under OS control. It occurs on HT systems with a version of the OS that is not compiled without HT support. It also occurs when a system is booted with max_cpus=n where 2 <= n < cpus known to the BIOS. The fix is to always send NMI IPI as a mask instead of as a broadcast.

Quoted 6 lines
Quoted 4 lines

Merged thanks.

P.S.: Linux-arch isn't really the right list to cc x86 patches too. I'm sure the other arch maintainers couldn't care less about such x86isms.

These two patches were for both i386 and x86_64, not just for x86_64.

It's still not the right list for that. linux-arch is just to broadcast stuff interesting to all architecture maintainers, not some kind of x86 list.

BTW they didn't compile without fixes on x86-64-UP nor i386-SMP. Please compile test patches better next time.

-Andi