pi-futex: set PF_EXITING without taking ->pi_lock
PATCH 1/1 pi-futex: set PF_EXITING without taking ->pi_lock
It is a bit annoying that do_exit() takes ->pi_lock to set PF_EXITING. All we need is to synchronize with lookup_pi_state() which saw this task without PF_EXITING under ->pi_lock.
Change do_exit() to use spin_unlock_wait().
kernel/exit.c renamed from kernel/exit.c~PF_EXIT +3 -3
@@ -908,13 +908,13 @@ fastcall NORET_TYPE void do_exit(long coschedule (); } + tsk->flags |=PF_EXITING ;/* * tsk->flags are checked in the futex code to protect against * an exiting task cleaning up the robust pi futexes. */ -spin_lock_irq (&tsk->pi_lock); - tsk->flags |=PF_EXITING ; -spin_unlock_irq (&tsk->pi_lock); +smp_mb (); +spin_unlock_wait (&tsk->pi_lock);if (unlikely (in_atomic ())) printk(KERN_INFO "note: %s[%d] exited with preempt_count %d\n",
Quoted 7 lines · Oleg Nesterov
It is a bit annoying that
do_exit()takes ->pi_lock to setPF_EXITING. All we need is to synchronize withlookup_pi_state()which saw this task withoutPF_EXITINGunder ->pi_lock.Change
do_exit()to usespin_unlock_wait().Signed-off-by Oleg Nesterov <[email protected]>
kernel/exit.c fastcall NORET_TYPE void do_exit(long co · line 917
Quoted 5 lines
kernel/exit.c renamed from kernel/exit.c~PF_EXIT +2
-3-spin_lock_irq (&tsk->pi_lock); - tsk->flags |=PF_EXITING ; -spin_unlock_irq (&tsk->pi_lock); +smp_mb (); +spin_unlock_wait (&tsk->pi_lock);
hm, isnt spin_unlock_wait() an SMP barrier in itself? (if not then it should be.)
Ingo
Quoted 7 lines · Ingo Molnar
Quoted 4 lines · Oleg Nesterov
It is a bit annoying that
do_exit()takes ->pi_lock to setPF_EXITING. All we need is to synchronize withlookup_pi_state()which saw this task withoutPF_EXITINGunder ->pi_lock.
Quoted 5 lines
Quoted 3 lines
Change
do_exit()to usespin_unlock_wait().Signed-off-by Oleg Nesterov <[email protected]>Acked-by Ingo Molnar <[email protected]>
Thanks!
Quoted 7 lines
kernel/exit.c fastcall NORET_TYPE void do_exit(long co · line 917
Quoted 5 lines
kernel/exit.c renamed from kernel/exit.c~PF_EXIT +2
-3-spin_lock_irq (&tsk->pi_lock); - tsk->flags |=PF_EXITING ; -spin_unlock_irq (&tsk->pi_lock); +smp_mb (); +spin_unlock_wait (&tsk->pi_lock);hm, isnt
spin_unlock_wait()an SMP barrier in itself?
no, only barrier() due to cpu_relax()
Quoted 1 line
(if not then it should be.)
I think you are right, I can't imagine a valid usage of spin_unlock_wait() without a barrier.
For example, from net/dccp/ccid.c
static void ccids_write_lock(void)
{
spin_lock(&ccids_lock);
while (atomic_read(&ccids_lockct) != 0) {
spin_unlock(&ccids_lock);
yield();
spin_lock(&ccids_lock);
}
}
static inline void ccids_read_lock(void)
{
atomic_inc(&ccids_lockct);
spin_unlock_wait(&ccids_lock);
}This looks racy, in theory atomic_inc() and spin_unlock_wait() could be re-ordered. However, in this particular case we have an "optimized" smp_mb_after_atomic_inc(), perhaps it is good that the caller can choose the "right" barrier by hand.
Oleg.
PATCH 1/1 fix theoretical ccids_{read,write}_lock() race
Make sure that spin_unlock_wait() is properly ordered wrt atomic_inc().
net/dccp/ccid.c renamed from net/dccp/ccid.c~ccid +1 -0
@@ -40,6 +40,7 @@ static inline void ccids_write_unlock(vostatic inline void ccids_read_lock (void ) {atomic_inc (&ccids_lockct); +smp_mb__after_atomic_inc ();spin_unlock_wait (&ccids_lock); }
Re: [PATCH] fix theoretical ccids_{read,write}_lock() race
Quoted 7 lines · On Sat, 21 Jul 2007 19:02:06 +0400 Oleg Nesterov <[email protected]> wrote:
Make sure that
spin_unlock_wait()is properly ordered wrtatomic_inc().Signed-off-by Oleg Nesterov <[email protected]>
net/dccp/ccid.c static inline void ccids_write_unlock(vo · line 46
Quoted 7 lines
net/dccp/ccid.c renamed from net/dccp/ccid.c~ccid +1
-0@@ -40,6 +40,7 @@ static inline void ccids_write_unlock(vostatic inline void ccids_read_lock (void ) {atomic_inc (&ccids_lockct); +smp_mb__after_atomic_inc ();spin_unlock_wait (&ccids_lock); }
Why not just use standard rwlocks in there?
(This is probably an FAQ, but it should be).
Re: [PATCH] fix theoretical ccids_{read,write}_lock() race
Quoted 20 lines · Andrew Morton
Quoted 7 lines · On Sat, 21 Jul 2007 19:02:06 +0400 Oleg Nesterov <[email protected]> wrote:
Make sure that
spin_unlock_wait()is properly ordered wrtatomic_inc().Signed-off-by Oleg Nesterov <[email protected]>net/dccp/ccid.c static inline void ccids_write_unlock(vo · line 46
Quoted 7 lines
net/dccp/ccid.c renamed from net/dccp/ccid.c~ccid +1
-0@@ -40,6 +40,7 @@ static inline void ccids_write_unlock(vostatic inline void ccids_read_lock (void ) {atomic_inc (&ccids_lockct); +smp_mb__after_atomic_inc ();spin_unlock_wait (&ccids_lock); }Why not just use standard rwlocks in there?
Quoted 1 line
(This is probably an FAQ, but it should be).
Perhaps because read_lock() doesn't allow to sleep?
Oleg.
Re: [PATCH] fix theoretical ccids_{read,write}_lock() race
Quoted 21 lines · On Sat, 21 Jul 2007 23:11:04 +0400 Oleg Nesterov <[email protected]> wrote:
Quoted 20 lines · Andrew Morton
Quoted 7 lines · On Sat, 21 Jul 2007 19:02:06 +0400 Oleg Nesterov <[email protected]> wrote:
Make sure that
spin_unlock_wait()is properly ordered wrtatomic_inc().Signed-off-by Oleg Nesterov <[email protected]>net/dccp/ccid.c static inline void ccids_write_unlock(vo · line 46
Quoted 7 lines
net/dccp/ccid.c renamed from net/dccp/ccid.c~ccid +1
-0@@ -40,6 +40,7 @@ static inline void ccids_write_unlock(vostatic inline void ccids_read_lock (void ) {atomic_inc (&ccids_lockct); +smp_mb__after_atomic_inc ();spin_unlock_wait (&ccids_lock); }Why not just use standard rwlocks in there?
Quoted 4 lines
Quoted 1 line
(This is probably an FAQ, but it should be).
Perhaps because
read_lock()doesn't allow to sleep?
down_read() does.
afaict the code doesn't sleep while holding that lock anwyay.
Re: [PATCH] fix theoretical ccids_{read,write}_lock() race
Quoted 24 lines · Andrew Morton
Quoted 21 lines · On Sat, 21 Jul 2007 23:11:04 +0400 Oleg Nesterov <[email protected]> wrote:
Quoted 20 lines · Andrew Morton
Quoted 7 lines · On Sat, 21 Jul 2007 19:02:06 +0400 Oleg Nesterov <[email protected]> wrote:
Make sure that
spin_unlock_wait()is properly ordered wrtatomic_inc().Signed-off-by Oleg Nesterov <[email protected]>net/dccp/ccid.c static inline void ccids_write_unlock(vo · line 46
Quoted 7 lines
net/dccp/ccid.c renamed from net/dccp/ccid.c~ccid +1
-0@@ -40,6 +40,7 @@ static inline void ccids_write_unlock(vostatic inline void ccids_read_lock (void ) {atomic_inc (&ccids_lockct); +smp_mb__after_atomic_inc ();spin_unlock_wait (&ccids_lock); }Why not just use standard rwlocks in there?
Quoted 8 lines
Quoted 4 lines
Quoted 1 line
(This is probably an FAQ, but it should be).
Perhaps because
read_lock()doesn't allow to sleep?
down_read()does.afaict the code doesn't sleep while holding that lock anwyay.
Ah yes, it doesn't call kmem_cache_alloc() under ccids_read_lock().
So, really, why not read_lock/write_lock ?
Oleg.
Quoted 10 lines · Oleg Nesterov
static inline void ccids_read_lock(void) { atomic_inc(&ccids_lockct); spin_unlock_wait(&ccids_lock); }This looks racy, in theory
atomic_inc()andspin_unlock_wait()could be re-ordered. However, in this particular case we have an "optimized"smp_mb_after_atomic_inc(), perhaps it is good that the caller can choose the "right" barrier by hand.
_all_ default locking and atomic APIs should be barrier-safe i believe. (and that includes atomic_inc() too) Most people dont have barriers on their mind when their code. _If_ someone is barrier-conscious then we should have barrier-less APIs too for that purpose of squeezing the last half cycle out of the code, but it should be a non-default choice. The
reason: nobody notices an unnecessary barrier, but a missing barrier can be nasty.
Ingo
Quoted 14 lines · Ingo Molnar
Quoted 10 lines · Oleg Nesterov
static inline void ccids_read_lock(void) { atomic_inc(&ccids_lockct); spin_unlock_wait(&ccids_lock); }This looks racy, in theory
atomic_inc()andspin_unlock_wait()could be re-ordered. However, in this particular case we have an "optimized"smp_mb_after_atomic_inc(), perhaps it is good that the caller can choose the "right" barrier by hand.
Quoted 7 lines
_all_ default locking and atomic APIs should be barrier-safe i believe. (and that includes
atomic_inc()too) Most people dont have barriers on their mind when their code. _If_ someone is barrier-conscious then we should have barrier-less APIs too for that purpose of squeezing the last half cycle out of the code, but it should be a non-default choice. The
reason: nobody notices an unnecessary barrier, but a missing barrier can be nasty.
Personally, I agree (but I am not sure the idea to make atomic_inc() barrier-safe would be very popular).
Question: should we make spinlock_t barrier-safe?
Suppose that the task "p" does
current->state = TASK_INTERRUPIBLE; mb(); if (CONDITION) break; schedule();
and another CPU does
CONDITION = 1; try_to_wake_up(p);
This is commonly used, but not correct _in theory_. If wake_up() happens when p->array != NULL, we have
CONDITION = 1; // [1] spin_lock(rq->lock); task->state = TASK_RUNNING; // [2]
and we can miss an event. Because in theory [1] may leak into the critical section, and could be re-ordered with [2].
Another problem is that try_to_wake_up() first checks task->state and does nothing if it is TASK_RUNNING, so we need a full mb(), not just wmb().
Should we change spin_lock(), or introduce smp_mb_before_spinlock(), or I missed something?
NOTE: I do not pretend to know what kind of barrier spin_lock() provides in practice, but according to the documentation lock() is only a one-way barrier.
(I am glad I have an opportunity to raise this issue again :)
Oleg.
Quoted 31 lines · Oleg Nesterov
Question: should we make
spinlock_tbarrier-safe?Suppose that the task "p" does
current->state = TASK_INTERRUPIBLE; mb(); if (CONDITION) break; schedule();and another CPU does
CONDITION = 1; try_to_wake_up(p);This is commonly used, but not correct
_intheory_. Ifwake_up()happens whenp->array!= NULL, we haveCONDITION = 1; // [1] spin_lock(rq->lock); task->state = TASK_RUNNING; // [2]and we can miss an event. Because in theory [1] may leak into the critical section, and could be re-ordered with [2].
Another problem is that
try_to_wake_up()first checkstask->stateand does nothing if it isTASK_RUNNING, so we need a fullmb(), not justwmb().
Quoted 6 lines
Should we change
spin_lock(), or introducesmp_mb_before_spinlock(), or I missed something?NOTE: I do not pretend to know what kind of barrier
spin_lock()provides in practice, but according to the documentationlock()is only a one-way barrier.
i think your worry is legitimate.
spin_lock() provides a full barrier on most platforms (certainly so on x86). But ... ia64 might have it as a one-way barrier?
Ingo
Quoted 14 lines · Ingo Molnar
Quoted 10 lines · Oleg Nesterov
static inline void ccids_read_lock(void) { atomic_inc(&ccids_lockct); spin_unlock_wait(&ccids_lock); }This looks racy, in theory
atomic_inc()andspin_unlock_wait()could be re-ordered. However, in this particular case we have an "optimized"smp_mb_after_atomic_inc(), perhaps it is good that the caller can choose the "right" barrier by hand.
Quoted 7 lines
_all_ default locking and atomic APIs should be barrier-safe i believe. (and that includes
atomic_inc()too) Most people dont have barriers on their mind when their code. _If_ someone is barrier-conscious then we should have barrier-less APIs too for that purpose of squeezing the last half cycle out of the code, but it should be a non-default choice. The
reason: nobody notices an unnecessary barrier, but a missing barrier can be nasty.
The approach we have taken on powerpc is that the atomic_*_test and atomic_*_return functions have a barrier, but the straight atomic_inc etc. don't.
As for putting barriers in, it's not a half cycle, it's more like 50 to 100 on some processors. Added to that, what I think you are actually advocating is *two* full barriers - one before the increment and one after. That seems like an enormous penalty to pay just because some people want to roll their own lock primitives instead of using the standard ones.
Why is ccids_read_lock trying to implement a rwlock without using an rwlock? Could it be converted to an ordinary rwlock? Or an rwsem?
Paul.