Skip to contentSkip to conversation

Q: down_killable() is racy? or schedule() is not right?

DISCUSSION Oleg Nesterov 4 participants 5 messages Jun 3, 2008 – Jun 9, 2008

lore
Share & tools
lore
#0 lore raw

I just noticed we have generic semaphores, a couple of questions.

	down():

		spin_lock_irqsave(&sem->lock, flags);
		...
		__down(sem);

Why _irqsave ? we must not do down() with irqs disabled, and of course __down() restores/clears irqs unconditionally.

Another question,

	__down_common(TASK_KILLABLE):

			if (state == TASK_KILLABLE && fatal_signal_pending(task))
				goto interrupted;

			/* --- WINDOW --- */

			__set_task_state(task, TASK_KILLABLE);
			schedule_timeout(timeout);

This looks racy. If SIGKILL comes in the WINDOW above, the event is lost. The task will wait for up() or timeout with the fatal signal pending, and it is not possible to wakeup it via kill() again.

This is easy to fix, but I wonder if we should change schedule() instead. Note that __down_common() does 2 checks,

		if (state == TASK_INTERRUPTIBLE && signal_pending(task))
			goto interrupted;
		if (state == TASK_KILLABLE && fatal_signal_pending(task))
			goto interrupted;

they look very symmetrical, but the first one is OK, and the second is racy. Also, I think we have the similar issues with lock_page_killable().

How about something like

	int signal_pending_state(struct task_struct *tsk)
	{
		if (!(state & (TASK_INTERRUPTIBLE | TASK_WAKEKILL)))
			return 0;
		if (signal_pending(tsk))
			return 0;

		return (state & TASK_INTERRUPTIBLE) ||
			__fatal_signal_pending(tsk);
	}

now,

	--- kernel/sched.c
	+++ kernel/sched.c
	@@ -4510,8 +4510,7 @@ need_resched_nonpreemptible:
		clear_tsk_need_resched(prev);
	 
		if (prev->state && !(preempt_count() & PREEMPT_ACTIVE)) {
	-		if (unlikely((prev->state & TASK_INTERRUPTIBLE) &&
	-				signal_pending(prev))) {
	+		if (unlikely(signal_pending_state(prev))) {
				prev->state = TASK_RUNNING;
			} else {
				deactivate_task(rq, prev, 1);

Thoughts?

Oleg.

#1 lore raw
Quoted 10 lines · Oleg Nesterov

I just noticed we have generic semaphores, a couple of questions.

	down():

		spin_lock_irqsave(&sem->lock, flags);
		...
		__down(sem);

Why _irqsave ? we must not do down() with irqs disabled, and of course __down() restores/clears irqs unconditionally.

How about reading the fine comments?

I would paste it, but Debian has fucked up my X copy and paste. Line 13 of kernel/semaphore.c.

Quoted 10 lines
	__down_common(TASK_KILLABLE):

			if (state == TASK_KILLABLE && fatal_signal_pending(task))
				goto interrupted;

			/* --- WINDOW --- */

			__set_task_state(task, TASK_KILLABLE);
			schedule_timeout(timeout);
Quoted 3 lines

This looks racy. If SIGKILL comes in the WINDOW above, the event is lost. The task will wait for up() or timeout with the fatal signal pending, and it is not possible to wakeup it via kill() again.

Hmmm. I think you're right. But mutex.c has the same problem, then. The wait_event_* macros get this right -- they set the task state before they check for a signal.

Quoted 9 lines

This is easy to fix, but I wonder if we should change schedule() instead. Note that __down_common() does 2 checks,

		if (state == TASK_INTERRUPTIBLE && signal_pending(task))
			goto interrupted;
		if (state == TASK_KILLABLE && fatal_signal_pending(task))
			goto interrupted;

they look very symmetrical, but the first one is OK, and the second is racy.

Oh, because of the special casing in sched.c. Why not just move the __set_task_state before the checks for signals pending? We'd have to reset to TASK_RUNNING at the 'interrupted:' label, but that's OK.

Quoted 1 line

Also, I think we have the similar issues with lock_page_killable().

I don't think so because __wait_on_bit_lock sets the state before checking the 'action' (sync_page_killable).

Quoted 28 lines

How about something like

	int signal_pending_state(struct task_struct *tsk)
	{
		if (!(state & (TASK_INTERRUPTIBLE | TASK_WAKEKILL)))
			return 0;
		if (signal_pending(tsk))
			return 0;

		return (state & TASK_INTERRUPTIBLE) ||
			__fatal_signal_pending(tsk);
	}

now,

	--- kernel/sched.c
	+++ kernel/sched.c
	@@ -4510,8 +4510,7 @@ need_resched_nonpreemptible:
		clear_tsk_need_resched(prev);
	 
		if (prev->state && !(preempt_count() & PREEMPT_ACTIVE)) {
	-		if (unlikely((prev->state & TASK_INTERRUPTIBLE) &&
	-				signal_pending(prev))) {
	+		if (unlikely(signal_pending_state(prev))) {
				prev->state = TASK_RUNNING;
			} else {
				deactivate_task(rq, prev, 1);
Quoted 1 line

Thoughts?

That might be worth doing anyway, but I'd leave that up to Ingo.

Signature · 4 lines
Intel are signing my paycheques ... these opinions are still mine
"Bill, look, we understand that you're interested in selling us this
operating system, but compare it to ours.  We can't possibly take such
a retrograde step."
#2 lore raw
Quoted 7 lines · Matthew Wilcox
Quoted 3 lines · Oleg Nesterov

Why _irqsave ? we must not do down() with irqs disabled, and of course __down() restores/clears irqs unconditionally.

How about reading the fine comments?

Thanks,

Quoted 10 lines

I would paste it, but Debian has fucked up my X copy and paste. Line 13 of kernel/semaphore.c.

Quoted 7 lines
	__down_common(TASK_KILLABLE):

			if (state == TASK_KILLABLE && fatal_signal_pending(task))
				goto interrupted;

			/* --- WINDOW --- */
Quoted 8 lines
Quoted 6 lines
			__set_task_state(task, TASK_KILLABLE);
			schedule_timeout(timeout);

This looks racy. If SIGKILL comes in the WINDOW above, the event is lost. The task will wait for up() or timeout with the fatal signal pending, and it is not possible to wakeup it via kill() again.

Hmmm. I think you're right. But mutex.c has the same problem, then.

and do_wait_for_common()

Quoted 8 lines
Quoted 8 lines

This is easy to fix, but I wonder if we should change schedule() instead. Note that __down_common() does 2 checks,

		if (state == TASK_INTERRUPTIBLE && signal_pending(task))
			goto interrupted;
		if (state == TASK_KILLABLE && fatal_signal_pending(task))
			goto interrupted;
Quoted 4 lines
Quoted 1 line

they look very symmetrical, but the first one is OK, and the second is racy.

Oh, because of the special casing in sched.c. Why not just move the __set_task_state before the checks for signals pending?

Yes sure, this all is fixeable (we need set_task_state() of course).

But please compare

	current->state = TASK_INTERRUPTIBLE;
	schedule();
and
	current->state = TASK_KILLABLE;
	schedule();

it seems to me that it is not nice they behave "differently".

Quoted 4 lines
Quoted 1 line

Also, I think we have the similar issues with lock_page_killable().

I don't think so because __wait_on_bit_lock sets the state before checking the 'action' (sync_page_killable).

You are right.

Quoted 26 lines
Quoted 26 lines
	int signal_pending_state(struct task_struct *tsk)
	{
		if (!(state & (TASK_INTERRUPTIBLE | TASK_WAKEKILL)))
			return 0;
		if (signal_pending(tsk))
			return 0;

		return (state & TASK_INTERRUPTIBLE) ||
			__fatal_signal_pending(tsk);
	}

now,

	--- kernel/sched.c
	+++ kernel/sched.c
	@@ -4510,8 +4510,7 @@ need_resched_nonpreemptible:
		clear_tsk_need_resched(prev);
	 
		if (prev->state && !(preempt_count() & PREEMPT_ACTIVE)) {
	-		if (unlikely((prev->state & TASK_INTERRUPTIBLE) &&
	-				signal_pending(prev))) {
	+		if (unlikely(signal_pending_state(prev))) {
				prev->state = TASK_RUNNING;
			} else {
				deactivate_task(rq, prev, 1);
Quoted 3 lines
Quoted 1 line

Thoughts?

That might be worth doing anyway, but I'd leave that up to Ingo.

Yes, we need Ingo's opinion ;)

Oleg.

#3 lore raw
In reply to Oleg Nesterov

2008/6/3 Oleg Nesterov <[email protected]>:

Quoted 24 lines

I just noticed we have generic semaphores, a couple of questions.

       down():

               spin_lock_irqsave(&sem->lock, flags);
               ...
               __down(sem);

Why _irqsave ? we must not do down() with irqs disabled, and of course __down() restores/clears irqs unconditionally.

Another question,

       __down_common(TASK_KILLABLE):

                       if (state == TASK_KILLABLE && fatal_signal_pending(task))
                               goto interrupted;

                       /* --- WINDOW --- */

                       __set_task_state(task, TASK_KILLABLE);
                       schedule_timeout(timeout);
Quoted 5 lines

This looks racy. If SIGKILL comes in the WINDOW above, the event is lost. The task will wait for up() or timeout with the fatal signal pending, and it is not possible to wakeup it via kill() again.

This is easy to fix, but I wonder if we should change schedule() instead.

[ for what it's worth ] I think, you are definitely right here.

The schedule() would be the right place to fix it. At the very least, because otherwise callers are obliged to always check for fatal_signal_pending(task) before scheduling with state == TASK_KILLABLE. e.g. schedule_timeout_killable().

Not very nice, IMHO.

Quoted 6 lines
       int signal_pending_state(struct task_struct *tsk)
       {
               if (!(state & (TASK_INTERRUPTIBLE | TASK_WAKEKILL)))
                       return 0;
               if (signal_pending(tsk))
                       return 0;

I guess, it should be ! signal_pending(tsk).

Quoted 13 lines
               return (state & TASK_INTERRUPTIBLE) ||
                       __fatal_signal_pending(tsk);
       }

               if (state == TASK_INTERRUPTIBLE && signal_pending(task))
                       goto interrupted;
               if (state == TASK_KILLABLE && fatal_signal_pending(task))

Oleg.

Signature · 2 lines
Best regards,
Dmitry Adamushko
#4 lore raw
Quoted 7 lines · Dmitry Adamushko
Quoted 6 lines

This looks racy. If SIGKILL comes in the WINDOW above, the event is lost. The task will wait for up() or timeout with the fatal signal pending, and it is not possible to wakeup it via kill() again.

This is easy to fix, but I wonder if we should change schedule() instead.

Quoted 8 lines

[ for what it's worth ] I think, you are definitely right here.

The schedule() would be the right place to fix it. At the very least, because otherwise callers are obliged to always check for fatal_signal_pending(task) before scheduling with state == TASK_KILLABLE. e.g. schedule_timeout_killable().

Not very nice, IMHO.

i guess we should fix this in schedule() - is there a patch i could try?

Ingo