Skip to contentSkip to conversation

smp_call_function_single with wait=0 considered harmful

DISCUSSION Christoph Hellwig 6 participants 8 messages Dec 4, 2013 – Mar 11, 2014

lore
Share & tools
lore
#0 lore raw

While doing my recent work on the generic smp function calls I noticed that smp_call_function_single without the wait flag can't work, as it allocates struct call_single_data on stack, and without the wait flag will happily return before the IPI has been executed.

This affects the following callers:

arch/ia64/kernel/mca.c:mca_cpu_callback() arch/ia64/kernel/smpboot.c:ia64_sync_itc() arch/x86/kernel/kvm.c:kvm_cpu_notify() arch/x86/oprofile/nmi_int.c:oprofile_cpu_notifier() arch/x86/pci/amd_bus.c:amd_cpu_notify() drivers/staging/octeon/ethernet-rx.c:cvm_oct_enable_one_cpu() kernel/stop_machine.c:stop_two_cpus()

It would be good to get these fixed so that we could remove the parameter. Either convert them to wait, or use a preallocated call_single_data and __smp_call_function_single.

After that I'd like to remove the wait argument to prevent further abuses.

#1 lore raw
Quoted 4 lines · Christoph Hellwig

While doing my recent work on the generic smp function calls I noticed that smp_call_function_single without the wait flag can't work, as it allocates struct call_single_data on stack, and without the wait flag will happily return before the IPI has been executed.

I don't understand the problem yet. With wait==0, smp_call_function_single() sets "csd = &__get_cpu_var(csd_data)", so it's not using a struct on the stack. We'll queue up "func" and likely will return before it is executed, but that should be fine because nobody will overwrite csd_data until it *is* executed and csd_unlock() has been called.

Quoted 7 lines

This affects the following callers:

arch/ia64/kernel/mca.c:mca_cpu_callback() arch/ia64/kernel/smpboot.c:ia64_sync_itc() arch/x86/kernel/kvm.c:kvm_cpu_notify() arch/x86/oprofile/nmi_int.c:oprofile_cpu_notifier() arch/x86/pci/amd_bus.c:amd_cpu_notify()

I don't see any reason why amd_cpu_notify() needs to use wait==0.

Quoted 9 lines

drivers/staging/octeon/ethernet-rx.c:cvm_oct_enable_one_cpu() kernel/stop_machine.c:stop_two_cpus()

It would be good to get these fixed so that we could remove the parameter. Either convert them to wait, or use a preallocated call_single_data and __smp_call_function_single.

After that I'd like to remove the wait argument to prevent further abuses.

#2 lore raw
Quoted 5 lines · Bjorn Helgaas

smp_call_function_single() sets "csd = &__get_cpu_var(csd_data)", so it's not using a struct on the stack. We'll queue up "func" and likely will return before it is executed, but that should be fine because nobody will overwrite csd_data until it *is* executed and csd_unlock() has been called.

You're right, I missed the usage of the per-cpu data later in the function.

#3 lore raw
In reply to Christoph Hellwig
Quoted 4 lines · Christoph Hellwig

While doing my recent work on the generic smp function calls I noticed that smp_call_function_single without the wait flag can't work, as it allocates struct call_single_data on stack, and without the wait flag will happily return before the IPI has been executed.

It doesn't actually; it uses a per-cpu one in the !wait case.

The subsequent csd_lock() ensures it will wait for any prior user to complete, so only if you're doing multiple smp_call_function_single() invocations back-to-back will they queue up.

Quoted 1 line

This affects the following callers:

<snip>

Quoted 1 line

kernel/stop_machine.c:stop_two_cpus()

That site should work with .wait=1 just fine, but given the above, the .wait=0 doesn't appear problematic at all.

#4 lore raw
Quoted 5 lines · Peter Zijlstra
Quoted 1 line · Christoph Hellwig

kernel/stop_machine.c:stop_two_cpus()

That site should work with .wait=1 just fine, but given the above, the .wait=0 doesn't appear problematic at all.

Scratch that; its broken, but not because of smp_call_function_single().

---
Subject: stop_machine: Fix^2 race between stop_two_cpus() and stop_cpus()

We must use smp_call_function_single(.wait=1) for the irq_cpu_stop_queue_work() to ensure the queueing is actually done under stop_cpus_lock. Without this we could have dropped the lock by the time we do the queueing and get the race we tried to fix.

Fixes 7053ea1a34fa ("stop_machine: Fix race between stop_two_cpus() and stop_cpus()")
Cc Prarit Bhargava <[email protected]>
Cc Rik van Riel <[email protected]>
Cc Mel Gorman <[email protected]>
---
 kernel/stop_machine.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/stop_machine.c +1 -1
index 84571e09c907..01fbae5b97b7 100644
@@ -293,7 +293,7 @@ int stop_two_cpus(unsigned int cpu1, unsigned int cpu2, cpu_stop_fn_t fn, void *
 	 */
 	smp_call_function_single(min(cpu1, cpu2),
 				 &irq_cpu_stop_queue_work,
-				 &call_args, 0);
+				 &call_args, 1);
 	lg_local_unlock(&stop_cpus_lock);
 	preempt_enable();
 
Reviewed-by
#5 lore raw
Quoted 7 lines · Peter Zijlstra

Subject: stop_machine: Fix^2 race between stop_two_cpus() and stop_cpus()

We must use smp_call_function_single(.wait=1) for the irq_cpu_stop_queue_work() to ensure the queueing is actually done under stop_cpus_lock. Without this we could have dropped the lock by the time we do the queueing and get the race we tried to fix.

Quoted 5 lines
Fixes 7053ea1a34fa ("stop_machine: Fix race between stop_two_cpus() and stop_cpus()")
Cc Prarit Bhargava <[email protected]>
Cc Rik van Riel <[email protected]>
Cc Mel Gorman <[email protected]>
Signature · 1 line
All rights reversed
Reviewed-by
#6 lore raw
In reply to Peter Zijlstra
Quoted 17 lines · Peter Zijlstra
Quoted 5 lines · Peter Zijlstra
Quoted 1 line · Christoph Hellwig

kernel/stop_machine.c:stop_two_cpus()

That site should work with .wait=1 just fine, but given the above, the .wait=0 doesn't appear problematic at all.

Scratch that; its broken, but not because of smp_call_function_single().

---
Subject: stop_machine: Fix^2 race between stop_two_cpus() and stop_cpus()

We must use smp_call_function_single(.wait=1) for the irq_cpu_stop_queue_work() to ensure the queueing is actually done under stop_cpus_lock. Without this we could have dropped the lock by the time we do the queueing and get the race we tried to fix.

Quoted 5 lines
Fixes 7053ea1a34fa ("stop_machine: Fix race between stop_two_cpus() and stop_cpus()")
Cc Prarit Bhargava <[email protected]>
Cc Rik van Riel <[email protected]>
Cc Mel Gorman <[email protected]>

P.

#7 lore raw

[tip:sched/core] stop_machine: Fix^2 race between stop_two_cpus() and stop_cpus()

In reply to Peter Zijlstra
stop_machine: Fix^2 race between stop_two_cpus() and stop_cpus()
Commit-ID:  177c53d943368fc97644ebc0a250dc8e2d124250
Gitweb:     http://git.kernel.org/tip/177c53d943368fc97644ebc0a250dc8e2d124250
Author:     Peter Zijlstra <[email protected]>
AuthorDate: Fri, 28 Feb 2014 13:39:05 +0100
Committer:  Ingo Molnar <[email protected]>
CommitDate: Tue, 11 Mar 2014 11:33:47 +0100

stop_machine: Fix^2 race between stop_two_cpus() and stop_cpus()

We must use smp_call_function_single(.wait=1) for the irq_cpu_stop_queue_work() to ensure the queueing is actually done under stop_cpus_lock. Without this we could have dropped the lock by the time we do the queueing and get the race we tried to fix.

Fixes 7053ea1a34fa ("stop_machine: Fix race between stop_two_cpus() and stop_cpus()")
---
 kernel/stop_machine.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/stop_machine.c +1 -1
index 84571e0..01fbae5 100644
@@ -293,7 +293,7 @@ int stop_two_cpus(unsigned int cpu1, unsigned int cpu2, cpu_stop_fn_t fn, void *
 	 */
 	smp_call_function_single(min(cpu1, cpu2),
 				 &irq_cpu_stop_queue_work,
-				 &call_args, 0);
+				 &call_args, 1);
 	lg_local_unlock(&stop_cpus_lock);
 	preempt_enable();