Skip to contentSkip to conversation

pi-futex: set PF_EXITING without taking ->pi_lock

PATCH Superseded Oleg Nesterov 4 participants 12 messages Jul 21, 2007 – Aug 6, 2007 Networking

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 pi-futex: set PF_EXITING without taking ->pi_lock

It is a bit annoying that do_exit() takes ->pi_lock to set PF_EXITING. All we need is to synchronize with lookup_pi_state() which saw this task without PF_EXITING under ->pi_lock.

Change do_exit() to use spin_unlock_wait().

kernel/exit.c renamed from kernel/exit.c~PF_EXIT +3 -3
@@ -908,13 +908,13 @@ fastcall NORET_TYPE void do_exit(long co
 		schedule();
 	}
 
+	tsk->flags |= PF_EXITING;
 	/*
 	 * tsk->flags are checked in the futex code to protect against
 	 * an exiting task cleaning up the robust pi futexes.
 	 */
-	spin_lock_irq(&tsk->pi_lock);
-	tsk->flags |= PF_EXITING;
-	spin_unlock_irq(&tsk->pi_lock);
+	smp_mb();
+	spin_unlock_wait(&tsk->pi_lock);
 
 	if (unlikely(in_atomic()))
 		printk(KERN_INFO "note: %s[%d] exited with preempt_count %d\n",
Acked-by on v1 1/1
#1 lore raw
Quoted 7 lines · Oleg Nesterov

It is a bit annoying that do_exit() takes ->pi_lock to set PF_EXITING. All we need is to synchronize with lookup_pi_state() which saw this task without PF_EXITING under ->pi_lock.

Change do_exit() to use spin_unlock_wait().

kernel/exit.c fastcall NORET_TYPE void do_exit(long co · line 917

Quoted 5 lines
kernel/exit.c renamed from kernel/exit.c~PF_EXIT +2 -3
-	spin_lock_irq(&tsk->pi_lock);
-	tsk->flags |= PF_EXITING;
-	spin_unlock_irq(&tsk->pi_lock);
+	smp_mb();
+	spin_unlock_wait(&tsk->pi_lock);

hm, isnt spin_unlock_wait() an SMP barrier in itself? (if not then it should be.)

Ingo

#2 lore raw
Quoted 7 lines · Ingo Molnar
Quoted 4 lines · Oleg Nesterov

It is a bit annoying that do_exit() takes ->pi_lock to set PF_EXITING. All we need is to synchronize with lookup_pi_state() which saw this task without PF_EXITING under ->pi_lock.

Quoted 5 lines
Quoted 3 lines

Change do_exit() to use spin_unlock_wait().

Thanks!

Quoted 7 lines

kernel/exit.c fastcall NORET_TYPE void do_exit(long co · line 917

Quoted 5 lines
kernel/exit.c renamed from kernel/exit.c~PF_EXIT +2 -3
-	spin_lock_irq(&tsk->pi_lock);
-	tsk->flags |= PF_EXITING;
-	spin_unlock_irq(&tsk->pi_lock);
+	smp_mb();
+	spin_unlock_wait(&tsk->pi_lock);

hm, isnt spin_unlock_wait() an SMP barrier in itself?

no, only barrier() due to cpu_relax()

Quoted 1 line

(if not then it should be.)

I think you are right, I can't imagine a valid usage of spin_unlock_wait() without a barrier.

For example, from net/dccp/ccid.c

	static void ccids_write_lock(void)
	{
		spin_lock(&ccids_lock);
		while (atomic_read(&ccids_lockct) != 0) {
			spin_unlock(&ccids_lock);
			yield();
			spin_lock(&ccids_lock);
		}
	}

	static inline void ccids_read_lock(void)
	{
		atomic_inc(&ccids_lockct);
		spin_unlock_wait(&ccids_lock);
	}

This looks racy, in theory atomic_inc() and spin_unlock_wait() could be re-ordered. However, in this particular case we have an "optimized" smp_mb_after_atomic_inc(), perhaps it is good that the caller can choose the "right" barrier by hand.

Oleg.

#3 lore raw

PATCH 1/1 fix theoretical ccids_{read,write}_lock() race

Make sure that spin_unlock_wait() is properly ordered wrt atomic_inc().

net/dccp/ccid.c renamed from net/dccp/ccid.c~ccid +1 -0
@@ -40,6 +40,7 @@ static inline void ccids_write_unlock(vo
 static inline void ccids_read_lock(void)
 {
 	atomic_inc(&ccids_lockct);
+	smp_mb__after_atomic_inc();
 	spin_unlock_wait(&ccids_lock);
 }
 
#4 lore raw

Re: [PATCH] fix theoretical ccids_{read,write}_lock() race

Quoted 7 lines · On Sat, 21 Jul 2007 19:02:06 +0400 Oleg Nesterov <[email protected]> wrote:

Make sure that spin_unlock_wait() is properly ordered wrt atomic_inc().

net/dccp/ccid.c static inline void ccids_write_unlock(vo · line 46

Quoted 7 lines
net/dccp/ccid.c renamed from net/dccp/ccid.c~ccid +1 -0
@@ -40,6 +40,7 @@ static inline void ccids_write_unlock(vo
 static inline void ccids_read_lock(void)
 {
 	atomic_inc(&ccids_lockct);
+	smp_mb__after_atomic_inc();
 	spin_unlock_wait(&ccids_lock);
 }
 

Why not just use standard rwlocks in there?

(This is probably an FAQ, but it should be).

#5 lore raw

Re: [PATCH] fix theoretical ccids_{read,write}_lock() race

Quoted 20 lines · Andrew Morton
Quoted 7 lines · On Sat, 21 Jul 2007 19:02:06 +0400 Oleg Nesterov <[email protected]> wrote:

Make sure that spin_unlock_wait() is properly ordered wrt atomic_inc().

net/dccp/ccid.c static inline void ccids_write_unlock(vo · line 46

Quoted 7 lines
net/dccp/ccid.c renamed from net/dccp/ccid.c~ccid +1 -0
@@ -40,6 +40,7 @@ static inline void ccids_write_unlock(vo
 static inline void ccids_read_lock(void)
 {
 	atomic_inc(&ccids_lockct);
+	smp_mb__after_atomic_inc();
 	spin_unlock_wait(&ccids_lock);
 }
 

Why not just use standard rwlocks in there?

Quoted 1 line

(This is probably an FAQ, but it should be).

Perhaps because read_lock() doesn't allow to sleep?

Oleg.

#6 lore raw

Re: [PATCH] fix theoretical ccids_{read,write}_lock() race

Quoted 21 lines · On Sat, 21 Jul 2007 23:11:04 +0400 Oleg Nesterov <[email protected]> wrote:
Quoted 20 lines · Andrew Morton
Quoted 7 lines · On Sat, 21 Jul 2007 19:02:06 +0400 Oleg Nesterov <[email protected]> wrote:

Make sure that spin_unlock_wait() is properly ordered wrt atomic_inc().

net/dccp/ccid.c static inline void ccids_write_unlock(vo · line 46

Quoted 7 lines
net/dccp/ccid.c renamed from net/dccp/ccid.c~ccid +1 -0
@@ -40,6 +40,7 @@ static inline void ccids_write_unlock(vo
 static inline void ccids_read_lock(void)
 {
 	atomic_inc(&ccids_lockct);
+	smp_mb__after_atomic_inc();
 	spin_unlock_wait(&ccids_lock);
 }
 

Why not just use standard rwlocks in there?

Quoted 4 lines
Quoted 1 line

(This is probably an FAQ, but it should be).

Perhaps because read_lock() doesn't allow to sleep?

down_read() does.

afaict the code doesn't sleep while holding that lock anwyay.

#7 lore raw

Re: [PATCH] fix theoretical ccids_{read,write}_lock() race

Quoted 24 lines · Andrew Morton
Quoted 21 lines · On Sat, 21 Jul 2007 23:11:04 +0400 Oleg Nesterov <[email protected]> wrote:
Quoted 20 lines · Andrew Morton
Quoted 7 lines · On Sat, 21 Jul 2007 19:02:06 +0400 Oleg Nesterov <[email protected]> wrote:

Make sure that spin_unlock_wait() is properly ordered wrt atomic_inc().

net/dccp/ccid.c static inline void ccids_write_unlock(vo · line 46

Quoted 7 lines
net/dccp/ccid.c renamed from net/dccp/ccid.c~ccid +1 -0
@@ -40,6 +40,7 @@ static inline void ccids_write_unlock(vo
 static inline void ccids_read_lock(void)
 {
 	atomic_inc(&ccids_lockct);
+	smp_mb__after_atomic_inc();
 	spin_unlock_wait(&ccids_lock);
 }
 

Why not just use standard rwlocks in there?

Quoted 8 lines
Quoted 4 lines
Quoted 1 line

(This is probably an FAQ, but it should be).

Perhaps because read_lock() doesn't allow to sleep?

down_read() does.

afaict the code doesn't sleep while holding that lock anwyay.

Ah yes, it doesn't call kmem_cache_alloc() under ccids_read_lock().

So, really, why not read_lock/write_lock ?

Oleg.

#8 lore raw
In reply to Oleg Nesterov
Quoted 10 lines · Oleg Nesterov
	static inline void ccids_read_lock(void)
	{
		atomic_inc(&ccids_lockct);
		spin_unlock_wait(&ccids_lock);
	}

This looks racy, in theory atomic_inc() and spin_unlock_wait() could be re-ordered. However, in this particular case we have an "optimized" smp_mb_after_atomic_inc(), perhaps it is good that the caller can choose the "right" barrier by hand.

_all_ default locking and atomic APIs should be barrier-safe i believe. (and that includes atomic_inc() too) Most people dont have barriers on their mind when their code. _If_ someone is barrier-conscious then we should have barrier-less APIs too for that purpose of squeezing the last half cycle out of the code, but it should be a non-default choice. The
reason: nobody notices an unnecessary barrier, but a missing barrier can be nasty.

Ingo

#9 lore raw
Quoted 14 lines · Ingo Molnar
Quoted 10 lines · Oleg Nesterov
	static inline void ccids_read_lock(void)
	{
		atomic_inc(&ccids_lockct);
		spin_unlock_wait(&ccids_lock);
	}

This looks racy, in theory atomic_inc() and spin_unlock_wait() could be re-ordered. However, in this particular case we have an "optimized" smp_mb_after_atomic_inc(), perhaps it is good that the caller can choose the "right" barrier by hand.

Quoted 7 lines

_all_ default locking and atomic APIs should be barrier-safe i believe. (and that includes atomic_inc() too) Most people dont have barriers on their mind when their code. _If_ someone is barrier-conscious then we should have barrier-less APIs too for that purpose of squeezing the last half cycle out of the code, but it should be a non-default choice. The
reason: nobody notices an unnecessary barrier, but a missing barrier can be nasty.

Personally, I agree (but I am not sure the idea to make atomic_inc() barrier-safe would be very popular).

Question: should we make spinlock_t barrier-safe?

Suppose that the task "p" does

	current->state = TASK_INTERRUPIBLE;
	mb();

	if (CONDITION)
		break;

	schedule();

and another CPU does

	CONDITION = 1;
	try_to_wake_up(p);

This is commonly used, but not correct _in theory_. If wake_up() happens when p->array != NULL, we have

	CONDITION = 1;			// [1]
	spin_lock(rq->lock);
	task->state = TASK_RUNNING;	// [2]

and we can miss an event. Because in theory [1] may leak into the critical section, and could be re-ordered with [2].

Another problem is that try_to_wake_up() first checks task->state and does nothing if it is TASK_RUNNING, so we need a full mb(), not just wmb().

Should we change spin_lock(), or introduce smp_mb_before_spinlock(), or I missed something?

NOTE: I do not pretend to know what kind of barrier spin_lock() provides in practice, but according to the documentation lock() is only a one-way barrier.

(I am glad I have an opportunity to raise this issue again :)

Oleg.

#10 lore raw
Quoted 31 lines · Oleg Nesterov

Question: should we make spinlock_t barrier-safe?

Suppose that the task "p" does

	current->state = TASK_INTERRUPIBLE;
	mb();

	if (CONDITION)
		break;

	schedule();

and another CPU does

	CONDITION = 1;
	try_to_wake_up(p);

This is commonly used, but not correct _in theory_. If wake_up() happens when p->array != NULL, we have

	CONDITION = 1;			// [1]
	spin_lock(rq->lock);
	task->state = TASK_RUNNING;	// [2]

and we can miss an event. Because in theory [1] may leak into the critical section, and could be re-ordered with [2].

Another problem is that try_to_wake_up() first checks task->state and does nothing if it is TASK_RUNNING, so we need a full mb(), not just wmb().

Quoted 6 lines

Should we change spin_lock(), or introduce smp_mb_before_spinlock(), or I missed something?

NOTE: I do not pretend to know what kind of barrier spin_lock() provides in practice, but according to the documentation lock() is only a one-way barrier.

i think your worry is legitimate.

spin_lock() provides a full barrier on most platforms (certainly so on x86). But ... ia64 might have it as a one-way barrier?

Ingo

#11 lore raw
In reply to Ingo Molnar
Quoted 14 lines · Ingo Molnar
Quoted 10 lines · Oleg Nesterov
	static inline void ccids_read_lock(void)
	{
		atomic_inc(&ccids_lockct);
		spin_unlock_wait(&ccids_lock);
	}

This looks racy, in theory atomic_inc() and spin_unlock_wait() could be re-ordered. However, in this particular case we have an "optimized" smp_mb_after_atomic_inc(), perhaps it is good that the caller can choose the "right" barrier by hand.

Quoted 7 lines

_all_ default locking and atomic APIs should be barrier-safe i believe. (and that includes atomic_inc() too) Most people dont have barriers on their mind when their code. _If_ someone is barrier-conscious then we should have barrier-less APIs too for that purpose of squeezing the last half cycle out of the code, but it should be a non-default choice. The
reason: nobody notices an unnecessary barrier, but a missing barrier can be nasty.

The approach we have taken on powerpc is that the atomic_*_test and atomic_*_return functions have a barrier, but the straight atomic_inc etc. don't.

As for putting barriers in, it's not a half cycle, it's more like 50 to 100 on some processors. Added to that, what I think you are actually advocating is *two* full barriers - one before the increment and one after. That seems like an enormous penalty to pay just because some people want to roll their own lock primitives instead of using the standard ones.

Why is ccids_read_lock trying to implement a rwlock without using an rwlock? Could it be converted to an ordinary rwlock? Or an rwsem?

Paul.