smp_call_function_single with wait=0 considered harmful
While doing my recent work on the generic smp function calls I noticed that smp_call_function_single without the wait flag can't work, as it allocates struct call_single_data on stack, and without the wait flag will happily return before the IPI has been executed.
This affects the following callers:
arch/ia64/kernel/mca.c:mca_cpu_callback() arch/ia64/kernel/smpboot.c:ia64_sync_itc() arch/x86/kernel/kvm.c:kvm_cpu_notify() arch/x86/oprofile/nmi_int.c:oprofile_cpu_notifier() arch/x86/pci/amd_bus.c:amd_cpu_notify() drivers/staging/octeon/ethernet-rx.c:cvm_oct_enable_one_cpu() kernel/stop_machine.c:stop_two_cpus()
It would be good to get these fixed so that we could remove the parameter. Either convert them to wait, or use a preallocated call_single_data and __smp_call_function_single.
After that I'd like to remove the wait argument to prevent further abuses.
Quoted 4 lines · Christoph Hellwig
While doing my recent work on the generic smp function calls I noticed that
smp_call_function_singlewithout the wait flag can't work, as it allocates structcall_single_dataon stack, and without the wait flag will happily return before the IPI has been executed.
I don't understand the problem yet. With wait==0, smp_call_function_single() sets "csd = &__get_cpu_var(csd_data)", so it's not using a struct on the stack. We'll queue up "func" and likely will return before it is executed, but that should be fine because nobody will overwrite csd_data until it *is* executed and csd_unlock() has been called.
Quoted 7 lines
This affects the following callers:
arch/ia64/kernel/mca.c:mca_cpu_callback() arch/ia64/kernel/smpboot.c:ia64_sync_itc() arch/x86/kernel/kvm.c:kvm_cpu_notify() arch/x86/oprofile/nmi_int.c:oprofile_cpu_notifier() arch/x86/pci/amd_bus.c:amd_cpu_notify()
I don't see any reason why amd_cpu_notify() needs to use wait==0.
Quoted 9 lines
drivers/staging/octeon/ethernet-rx.c:cvm_oct_enable_one_cpu() kernel/stop_machine.c:stop_two_cpus()
It would be good to get these fixed so that we could remove the parameter. Either convert them to wait, or use a preallocated
call_single_dataand__smp_call_function_single.After that I'd like to remove the wait argument to prevent further abuses.
Quoted 5 lines · Bjorn Helgaas
smp_call_function_single()sets "csd =&__get_cpu_var(csd_data)", so it's not using a struct on the stack. We'll queue up "func" and likely will return before it is executed, but that should be fine because nobody will overwritecsd_datauntil it *is* executed andcsd_unlock()has been called.
You're right, I missed the usage of the per-cpu data later in the function.
Quoted 4 lines · Christoph Hellwig
While doing my recent work on the generic smp function calls I noticed that
smp_call_function_singlewithout the wait flag can't work, as it allocates structcall_single_dataon stack, and without the wait flag will happily return before the IPI has been executed.
It doesn't actually; it uses a per-cpu one in the !wait case.
The subsequent csd_lock() ensures it will wait for any prior user to complete, so only if you're doing multiple smp_call_function_single() invocations back-to-back will they queue up.
Quoted 1 line
This affects the following callers:
<snip>
Quoted 1 line
kernel/stop_machine.c:stop_two_cpus()
That site should work with .wait=1 just fine, but given the above, the .wait=0 doesn't appear problematic at all.
Quoted 5 lines · Peter Zijlstra
Quoted 1 line · Christoph Hellwig
kernel/stop_machine.c:stop_two_cpus()
That site should work with .wait=1 just fine, but given the above, the .wait=0 doesn't appear problematic at all.
Scratch that; its broken, but not because of smp_call_function_single().
---
Subject: stop_machine: Fix^2 race between stop_two_cpus() and stop_cpus()
We must use smp_call_function_single(.wait=1) for the irq_cpu_stop_queue_work() to ensure the queueing is actually done under stop_cpus_lock. Without this we could have dropped the lock by the time we do the queueing and get the race we tried to fix.
--- kernel/stop_machine.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/stop_machine.c +1 -1
@@ -293,7 +293,7 @@ int stop_two_cpus(unsigned int cpu1, unsigned int cpu2, cpu_stop_fn_t fn, void **/ smp_call_function_single (min (cpu1, cpu2), &irq_cpu_stop_queue_work, - &call_args,0 ); + &call_args,1 );lg_local_unlock (&stop_cpus_lock);preempt_enable ();
Rik van Riel
Reviewed-byQuoted 7 lines · Peter Zijlstra
Subject:
stop_machine: Fix^2 race betweenstop_two_cpus()andstop_cpus()We must use
smp_call_function_single(.wait=1)for theirq_cpu_stop_queue_work()to ensure the queueing is actually done understop_cpus_lock. Without this we could have dropped the lock by the time we do the queueing and get the race we tried to fix.
Quoted 5 lines
Fixes 7053ea1a34fa ("stop_machine: Fix race between stop_two_cpus() and stop_cpus()")Cc Prarit Bhargava <[email protected]>Cc Rik van Riel <[email protected]>Cc Mel Gorman <[email protected]>Signed-off-by Peter Zijlstra <[email protected]>
Signature · 1 line
All rights reversed
Prarit Bhargava
Reviewed-byQuoted 17 lines · Peter Zijlstra
Quoted 5 lines · Peter Zijlstra
Quoted 1 line · Christoph Hellwig
kernel/stop_machine.c:stop_two_cpus()
That site should work with .wait=1 just fine, but given the above, the .wait=0 doesn't appear problematic at all.
Scratch that; its broken, but not because of
smp_call_function_single().---
Subject:stop_machine: Fix^2 race betweenstop_two_cpus()andstop_cpus()We must use
smp_call_function_single(.wait=1)for theirq_cpu_stop_queue_work()to ensure the queueing is actually done understop_cpus_lock. Without this we could have dropped the lock by the time we do the queueing and get the race we tried to fix.
Quoted 5 lines
Fixes 7053ea1a34fa ("stop_machine: Fix race between stop_two_cpus() and stop_cpus()")Cc Prarit Bhargava <[email protected]>Cc Rik van Riel <[email protected]>Cc Mel Gorman <[email protected]>Signed-off-by Peter Zijlstra <[email protected]>
P.
tip-bot for Peter Zijlstra
merge bot[tip:sched/core] stop_machine: Fix^2 race between stop_two_cpus() and stop_cpus()
In reply to Peter Zijlstrastop_machine: Fix^2 race between stop_two_cpus() and stop_cpus()
Commit-ID: 177c53d943368fc97644ebc0a250dc8e2d124250 Gitweb: http://git.kernel.org/tip/177c53d943368fc97644ebc0a250dc8e2d124250 Author: Peter Zijlstra <[email protected]> AuthorDate: Fri, 28 Feb 2014 13:39:05 +0100 Committer: Ingo Molnar <[email protected]> CommitDate: Tue, 11 Mar 2014 11:33:47 +0100
stop_machine: Fix^2 race between stop_two_cpus() and stop_cpus()
We must use smp_call_function_single(.wait=1) for the irq_cpu_stop_queue_work() to ensure the queueing is actually done under stop_cpus_lock. Without this we could have dropped the lock by the time we do the queueing and get the race we tried to fix.
--- kernel/stop_machine.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/stop_machine.c +1 -1
@@ -293,7 +293,7 @@ int stop_two_cpus(unsigned int cpu1, unsigned int cpu2, cpu_stop_fn_t fn, void **/ smp_call_function_single (min (cpu1, cpu2), &irq_cpu_stop_queue_work, - &call_args,0 ); + &call_args,1 );lg_local_unlock (&stop_cpus_lock);preempt_enable ();