wait_task_inactive: don't use the dummy version when !SMP && PREEMPT
PATCH 3/3 wait_task_inactive: don't use the dummy version when !SMP && PREEMPT
wait_task_inactive() must return success only when/if the task leaves the runqueue, it doesn't work correctly when !SMP && PREEMPT because in that case we use the "dummy" version which doesn't check .on_rq.
Change the "#if" around the full-blown version from SMP to SMP || PREEMPT.
The patch looks monstrous because it moves the (unchanged) definition of wait_task_inactive() outside of "#ifdef CONFIG_SMP", but it is quite trivial.
include/linux/sched.h | 2 kernel/sched.c | 210 +++++++++++++++++++++++++------------------------- 2 files changed, 107 insertions(+), 105 deletions(-)
include/linux/sched.h renamed from include/linux/sched.h~3_WTI_PREEMPT +1 -1
@@ -1874,7 +1874,7 @@ struct task_struct *fork_idle(int);extern void set_task_comm (struct task_struct *tsk,char *from);extern char *get_task_comm (char *to,struct task_struct *tsk); -#ifdef CONFIG_SMP +#if defined(CONFIG_SMP ) || defined(CONFIG_PREEMPT )extern unsigned long wait_task_inactive (struct task_struct *,long match_state);#else static inline unsigned long wait_task_inactive(struct task_struct *p,
kernel/sched.c renamed from kernel/sched.c~3_WTI_PREEMPT +106 -104
222 lines not on this page · raw message
Quoted 4 lines · Oleg Nesterov
The patch looks monstrous because it moves the (unchanged) definition of
wait_task_inactive()outside of "#ifdefCONFIG_SMP", but it is quite trivial.
Hmm. Doesn't this just deadlock in UP (PREEMPT) if wait_task_interactive() is ever called from a no-preempt context?
And if that's never the case, the comment should be updated to reflect that (right now it says that it's only invalid to call it with interrupts disabled to avoid cross-IPI deadlocks).
Oh, and shouldn't it do a "yield()" instead of a cpu_relax() on UP?
Inquiring minds want to know. That function was very much expressly designed for SMP, not for preemption, and I want to understand why it's ok (_if_ it's ok).
Linus
Quoted 9 lines · Linus Torvalds
Quoted 4 lines · Oleg Nesterov
The patch looks monstrous because it moves the (unchanged) definition of
wait_task_inactive()outside of "#ifdefCONFIG_SMP", but it is quite trivial.Hmm. Doesn't this just deadlock in UP (PREEMPT) if
wait_task_interactive()is ever called from a no-preempt context?
Given that it calls schedule_timeout_uninterruptible(), it can't be used from the no-preempt context,
Quoted 3 lines
And if that's never the case, the comment should be updated to reflect that (right now it says that it's only invalid to call it with interrupts disabled to avoid cross-IPI deadlocks).
Yes, I think this function is might_sleep(),
Quoted 1 line
Oh, and shouldn't it do a "
yield()" instead of acpu_relax()on UP?
I _think_ that rq->curr must be == current without CONFIG_SMP, but
Quoted 2 lines
and I want to understand why it's
ok (_if_ it's ok).
me too.
Hopefully Ingo can ack/nack.
Oleg.
2008/7/30 Linus Torvalds <[email protected]>:
Quoted 8 lines · Oleg Nesterov
Quoted 4 lines
The patch looks monstrous because it moves the (unchanged) definition of
wait_task_inactive()outside of "#ifdefCONFIG_SMP", but it is quite trivial.
Quoted 8 lines
Hmm. Doesn't this just deadlock in UP (PREEMPT) if
wait_task_interactive()is ever called from a no-preempt context?And if that's never the case, the comment should be updated to reflect that (right now it says that it's only invalid to call it with interrupts disabled to avoid cross-IPI deadlocks).
Oh, and shouldn't it do a "
yield()" instead of acpu_relax()on UP?
This part could have been skipped for UP. task_running(rq, p) just can't give 'true' for UP (otherwise it's a bug). The only relevant part is "on_rq = p->se.on_rq".
Quoted 3 lines
Inquiring minds want to know. That function was very much expressly designed for SMP,
It looks so. Otherwise it's behavior is not symmetric and I think, either [1] it shouldn't be a "nop" for !SMP or [2] there shouldn't be a version for !SMP at all --> so no one can make false assumptions.
(if [1], then I think a separate function for PREEMPT would look better, i.e. without parts with task_running())
e.g. consider this code from kthread_bind():
/* Must have done schedule() in kthread() before we set_task_cpu */
wait_task_inactive(k, 0);
set_task_cpu(k, cpu);
k->cpus_allowed = cpumask_of_cpu(cpu);
k->rt.nr_cpus_allowed = 1;
k->flags |= PF_THREAD_BOUND;set_task_cpu(k, cpu) is safe _only_ if 'k' is not on the run-queue (and can't be placed onto it behind our back -- heh, a bit subtle).
Now, for !SMP + PREEMPT it's not a case. set_task_cpu() may be called while 'k' is still on the run-queue (more precisely, preempted in kthread() between complete(&create->started); and schedule();).
Yes, set_task_cpu() is a "nop" for UP so that's ok in this particular case. But let's suppose, another use-case would be introduced with 'false' assumptions causing troubles for !SMP.
Quoted 4 lines
not for preemption, and I want to understand why it's ok (_if_ it's ok).
Linus
Signature · 2 lines
Best regards, Dmitry Adamushko
Quoted 8 lines · Dmitry Adamushko
2008/7/30 Linus Torvalds <[email protected]>:
Quoted 2 lines
Oh, and shouldn't it do a "
yield()" instead of acpu_relax()on UP?This part could have been skipped for UP.
task_running(rq, p)just can't give 'true' for UP (otherwise it's a bug). The only relevant part is "on_rq=p->se.on_rq".
That was my understanding, thanks for your confirmation.
Quoted 2 lines
(if [1], then I think a separate function for PREEMPT would look better, i.e. without parts with
task_running())
in that case we can also eliminate task_rq_lock() afaics.
Quoted 13 lines
e.g. consider this code from
kthread_bind():/* Must have done schedule() in kthread() before we set_task_cpu */ wait_task_inactive(k, 0); set_task_cpu(k, cpu); k->cpus_allowed = cpumask_of_cpu(cpu); k->rt.nr_cpus_allowed = 1; k->flags |= PF_THREAD_BOUND;
set_task_cpu(k, cpu)is safe _only_ if 'k' is not on the run-queue (and can't be placed onto it behind our back -- heh, a bit subtle).
Quoted 7 lines
Now, for !SMP + PREEMPT it's not a case.
set_task_cpu()may be called while 'k' is still on the run-queue (more precisely, preempted inkthread()betweencomplete(&create->started); andschedule();).Yes,
set_task_cpu()is a "nop" for UP so that's ok in this particular case. But let's suppose, another use-case would be introduced with 'false' assumptions causing troubles for !SMP.
Completely agreed.
Currently the only user which can suffer on UP && PREEMPT is task_current_syscall(). As Roland suggested we can fix it if we change the !SMP version to return ->nivcsw + ->nvcsw. But personally I don't like the fact that we will have the subltle difference in behaviour depending on CONFIG_SMP. We have to wait for .on_rq == 0 on SMP, imho it is better to do the same on UP even if none of the current callers needs this.
That said, I think this problem is minor and I don't (and can't) have the strong opinion on how to fix it. I'd better wait for the authoritative verdict.
Oleg.
I'm not convinced we need this at all (as per prior thread "Q: wait_task_inactive() and !CONFIG_SMP && CONFIG_PREEMPT").
Let's not get hung up on what the explanation of the function's semantics supposedly is. We can change the comments to match the reality. We can rename the function to better reflect its true guarantee, if you like.
There are two uses of wait_task_inactive. One is kthread_bind. We have established that a nop is fine for !SMP there.
The other uses are for tracing. Presently that is the traditional use in ptrace, and task_current_syscall.
I've described that the requirement of the traditional ptrace call is quite weak. A nop is fine for !SMP. Preemption is not an issue because all that matters is that no access to pt_regs/thread_struct can race with the actual context switch (by definition has to be on another CPU to be simultaneous).
For task_current_syscall, it also doesn't matter that it's left the run queue. It just matters that the call before looking at pt_regs and the call after indicate whether it stayed switched out for the duration. As per the prior thread on this, I think that's covered by using nivcsw+nvcsw as I did originally rather than nvcsw alone. So a short inline is still fine.
The other uses I anticipate (e.g. utrace) follow the same model as task_current_syscall (check before and after) and have only the same requirements.
Thanks,
Roland