x86: Don't send RESCHEDULE_VECTOR to offlined cpus

x86: Don't send RESCHEDULE_VECTOR to offlined cpus.
From: Gautham R Shenoy <[email protected]>

In the x86 native_smp_send_reschedule_function(), don't send the IPI if the cpu has gone offline already. Warn nevertheless!!

---

 arch/x86/kernel/smp_32.c |    5 ++++-
 1 files changed, 4 insertions(+), 1 deletions(-)

index dc0cde9..4df9042 100644
@@ -472,7 +472,10 @@ void flush_tlb_all(void)
  */
 static void native_smp_send_reschedule(int cpu)
 {
-	WARN_ON(cpu_is_offline(cpu));
+	if (unlikely(cpu_is_offline(cpu))) {
+		WARN_ON(1);
+		return;
+	}
 	send_IPI_mask(cpumask_of_cpu(cpu), RESCHEDULE_VECTOR);
 }
 

Thanks and Regards
gautham

have you seen this happen?

Ingo

Yup, this afternoon while running
cpu-hotplug stress tests (http://lkml.org/lkml/2008/3/3/11)
in parallel with kern-bench, I got this on the serial console.

------------[ cut here ]------------
WARNING: at arch/x86/kernel/smp_32.c:475 native_smp_send_reschedule+0x22/0x3f()
Modules linked in: dock
Pid: 4722, comm: kstopmachine Not tainted 2.6.25-rc3 #46
 [<c011f424>] warn_on_slowpath+0x41/0x51
 [<c013ac74>] ? __lock_acquire+0xaae/0xaf6
 [<c011a446>] ? resched_cpu+0x2c/0x6f
 [<c0111691>] native_smp_send_reschedule+0x22/0x3f
 [<c01186c7>] __resched_task+0x5f/0x63
 [<c011a479>] resched_cpu+0x5f/0x6f
 [<c011bb6d>] scheduler_tick+0x214/0x28f
 [<c012737b>] update_process_times+0x3d/0x49
 [<c013751d>] tick_sched_timer+0x6e/0xa6
 [<c01374af>] ? tick_sched_timer+0x0/0xa6
 [<c0131e8c>] __run_hrtimer+0x39/0x70
 [<c0132668>] hrtimer_interrupt+0xeb/0x154
 [<c01127bc>] smp_apic_timer_interrupt+0x6c/0x80
 [<c0143670>] ? stopmachine+0x0/0x98
 [<c0105553>] apic_timer_interrupt+0x33/0x38
 [<c0143670>] ? stopmachine+0x0/0x98
 [<c013007b>] ? sample_to_timespec+0x16/0x35
 [<c0143701>] ? stopmachine+0x91/0x98
 [<c01056eb>] kernel_thread_helper+0x7/0x10
 =======================
---[ end trace 1016b68a0b60b50b ]---
------------[ cut here ]------------

Thanks and Regards
gautham

from arch/m32r/kernel/smp.c:134

void smp_send_reschedule(int cpu_id)

{
        WARN_ON(cpu_is_offline(cpu_id));
        send_IPI_mask(cpumask_of_cpu(cpu_id), RESCHEDULE_IPI, 1);
}

should this be changed as well?

Note that this may be a side-effect of the bug we are chasing with the rd->online problem.

E.g. if the rd->online issue is causing us to route tasks to dead CPUs, this phenomenon would probably go away once rd->online is fixed (which I believe it is with the patches that were submitted).

Regards,
-Greg

yeah, that makes sense. I've applied your patch nevertheless because it is the better failure mode for that code.

Ingo