Skip to contentSkip to conversation

atomic: Only take lock when the counter drops to zero on UP as well

PATCH In review Jan Blunck 4 participants 10 messages Apr 11, 2009 – Apr 23, 2009 Library code

lore
Share & tools
lore

Jan Blunck

author
#0 lore raw

PATCH 1/1 atomic: Only take lock when the counter drops to zero on UP as well

I think it is wrong to unconditionally take the lock before calling atomic_dec_and_test() in _atomic_dec_and_lock(). This will deadlock in situation where it is known that the counter will not reach zero (e.g. holding another reference to the same object) but the lock is already taken.

---
 lib/dec_and_lock.c |    3 +--
 1 files changed, 1 insertions(+), 2 deletions(-)
lib/dec_and_lock.c +1 -2
index a65c314..e73822a 100644
@@ -19,11 +19,10 @@
  */
 int _atomic_dec_and_lock(atomic_t *atomic, spinlock_t *lock)
 {
-#ifdef CONFIG_SMP
 	/* Subtract 1 from counter unless that drops it to 0 (ie. it was 1) */
 	if (atomic_add_unless(atomic, -1, 1))
 		return 0;
-#endif
+
 	/* Otherwise do it the slow way */
 	spin_lock(lock);
 	if (atomic_dec_and_test(atomic))
Signature · 1 line
1.6.0.2
#1 lore raw
Quoted 4 lines · Jan Blunck

I think it is wrong to unconditionally take the lock before calling atomic_dec_and_test() in _atomic_dec_and_lock(). This will deadlock in situation where it is known that the counter will not reach zero (e.g. holding another reference to the same object) but the lock is already taken.

The thought of calling _atomic_dec_and_lock() when you already hold the lock really really scares me.

Could you please give an example where you need to do this?

Thanx, Paul

Quoted 25 lines
---
 lib/dec_and_lock.c |    3 +--
 1 files changed, 1 insertions(+), 2 deletions(-)
lib/dec_and_lock.c +1 -2
index a65c314..e73822a 100644
@@ -19,11 +19,10 @@
  */
 int _atomic_dec_and_lock(atomic_t *atomic, spinlock_t *lock)
 {
-#ifdef CONFIG_SMP
 	/* Subtract 1 from counter unless that drops it to 0 (ie. it was 1) */
 	if (atomic_add_unless(atomic, -1, 1))
 		return 0;
-#endif
+
 	/* Otherwise do it the slow way */
 	spin_lock(lock);
 	if (atomic_dec_and_test(atomic))
Signature · 1 line
1.6.0.2
author
#2 lore raw

Am 11.04.2009 um 19:49 schrieb "Paul E. McKenney" <[email protected]>:

Quoted 11 lines · Jan Blunck
Quoted 4 lines

I think it is wrong to unconditionally take the lock before calling atomic_dec_and_test() in _atomic_dec_and_lock(). This will deadlock in situation where it is known that the counter will not reach zero (e.g. holding another reference to the same object) but the lock is already taken.

The thought of calling _atomic_dec_and_lock() when you already hold the lock really really scares me.

Could you please give an example where you need to do this?

There is a part of the union mount patches that needs to do a union_put() (which itself includes a path_put() that uses atomic_dec_and_lock() in mntput() ). Since it is changing the namespace I need to hold the vfsmount lock. I know that the mnt's count > 1 since it is a parent of the mnt I'm changing in the mount tree. I could possibly delay the union_put().

In general this let's atomic_dec_and_lock() behave similar on SMP and UP. Remember that this already works with CONFIG_SMP as before Nick's patch.

Quoted 28 lines

Thanx, Paul

Quoted 26 lines

---
lib/dec_and_lock.c | 3 +--

1 files changed, 1 insertions(+), 2 deletions(-)
lib/dec_and_lock.c +0 -1
index a65c314..e73822a 100644
@@ -19,11 +19,10 @@
 */
 int _atomic_dec_and_lock(atomic_t *atomic, spinlock_t *lock)
 {
-#ifdef CONFIG_SMP
   /* Subtract 1 from counter unless that drops it to 0 (ie. it was  

1) */
if (atomic_add_unless(atomic, -1, 1))

       return 0;
lib/dec_and_lock.c +1 -1
-#endif
+
   /* Otherwise do it the slow way */
   spin_lock(lock);
   if (atomic_dec_and_test(atomic))
Signature · 1 line
1.6.0.2
#3 lore raw
Quoted 16 lines · Jan Blunck

Am 11.04.2009 um 19:49 schrieb "Paul E. McKenney"
<[email protected]>:

Quoted 7 lines · Jan Blunck
Quoted 5 lines

I think it is wrong to unconditionally take the lock before calling atomic_dec_and_test() in _atomic_dec_and_lock(). This will deadlock in situation where it is known that the counter will not reach zero (e.g. holding
another reference to the same object) but the lock is already taken.

Quoted 5 lines

The thought of calling _atomic_dec_and_lock() when you already hold the lock really really scares me.

Could you please give an example where you need to do this?

Quoted 8 lines

There is a part of the union mount patches that needs to do a union_put() (which itself includes a path_put() that uses atomic_dec_and_lock() in mntput() ). Since it is changing the namespace I need to hold the vfsmount lock. I know that the mnt's count > 1 since it is a parent of the mnt I'm changing in the mount tree. I could possibly delay the union_put().

In general this let's atomic_dec_and_lock() behave similar on SMP and UP. Remember that this already works with CONFIG_SMP as before Nick's patch.

I asked, I guess. ;-)

There is some sort of common code path, so that you cannot simply call atomic_dec() when holding the lock?

Thanx, Paul

Quoted 26 lines
Quoted 26 lines
Quoted 26 lines

---
lib/dec_and_lock.c | 3 +--

1 files changed, 1 insertions(+), 2 deletions(-)
lib/dec_and_lock.c +1 -2
index a65c314..e73822a 100644
@@ -19,11 +19,10 @@
 */
 int _atomic_dec_and_lock(atomic_t *atomic, spinlock_t *lock)
 {
-#ifdef CONFIG_SMP
   /* Subtract 1 from counter unless that drops it to 0 (ie. it was 1) */
   if (atomic_add_unless(atomic, -1, 1))
       return 0;
-#endif
+
   /* Otherwise do it the slow way */
   spin_lock(lock);
   if (atomic_dec_and_test(atomic))
Signature · 1 line
1.6.0.2
author
#4 lore raw
Quoted 23 lines · Paul E. McKenney
Quoted 22 lines · Jan Blunck

Am 11.04.2009 um 19:49 schrieb "Paul E. McKenney"
<[email protected]>:

Quoted 7 lines · Jan Blunck
Quoted 5 lines

I think it is wrong to unconditionally take the lock before calling atomic_dec_and_test() in _atomic_dec_and_lock(). This will deadlock in situation where it is known that the counter will not reach zero (e.g. holding
another reference to the same object) but the lock is already taken.

Quoted 5 lines

The thought of calling _atomic_dec_and_lock() when you already hold the lock really really scares me.

Could you please give an example where you need to do this?

There is a part of the union mount patches that needs to do a union_put() (which itself includes a path_put() that uses atomic_dec_and_lock() in mntput() ). Since it is changing the namespace I need to hold the vfsmount lock. I know that the mnt's count > 1 since it is a parent of the mnt I'm changing in the mount tree. I could possibly delay the union_put().

Quoted 7 lines
Quoted 2 lines

In general this let's atomic_dec_and_lock() behave similar on SMP and UP. Remember that this already works with CONFIG_SMP as before Nick's patch.

I asked, I guess. ;-)

There is some sort of common code path, so that you cannot simply call atomic_dec() when holding the lock?

If it is possible I don't want to introduce another special mntput() variant just for that code path.

Thanks,
Jan

#5 lore raw
Quoted 28 lines · Jan Blunck
Quoted 26 lines · Paul E. McKenney
Quoted 24 lines · Jan Blunck

Am 11.04.2009 um 19:49 schrieb "Paul E. McKenney"
<[email protected]>:

Quoted 7 lines · Jan Blunck
Quoted 5 lines

I think it is wrong to unconditionally take the lock before calling atomic_dec_and_test() in _atomic_dec_and_lock(). This will deadlock in situation where it is known that the counter will not reach zero (e.g. holding
another reference to the same object) but the lock is already taken.

Quoted 5 lines

The thought of calling _atomic_dec_and_lock() when you already hold the lock really really scares me.

Could you please give an example where you need to do this?

There is a part of the union mount patches that needs to do a union_put() (which itself includes a path_put() that uses atomic_dec_and_lock() in mntput() ). Since it is changing the namespace I need to hold the vfsmount lock. I know that the mnt's count > 1 since it is a parent of the mnt I'm changing in the mount tree. I could possibly delay the union_put().

In general this let's atomic_dec_and_lock() behave similar on SMP and UP. Remember that this already works with CONFIG_SMP as before Nick's patch.

Quoted 7 lines
Quoted 4 lines

I asked, I guess. ;-)

There is some sort of common code path, so that you cannot simply call atomic_dec() when holding the lock?

If it is possible I don't want to introduce another special mntput() variant just for that code path.

Fair enough!!!

Thanx, Paul

Acked-by on v1 1/1
#6 lore raw
In reply to Jan Blunck
Quoted 6 lines · Jan Blunck

I think it is wrong to unconditionally take the lock before calling atomic_dec_and_test() in _atomic_dec_and_lock(). This will deadlock in situation where it is known that the counter will not reach zero (e.g. holding another reference to the same object) but the lock is already taken.

Paul's worry about callers aside, I think it is probably a good idea to reduce ifdefs and share more code.

So for this patch,

Quoted 23 lines
---
 lib/dec_and_lock.c |    3 +--
 1 files changed, 1 insertions(+), 2 deletions(-)
lib/dec_and_lock.c +1 -2
index a65c314..e73822a 100644
@@ -19,11 +19,10 @@
  */
 int _atomic_dec_and_lock(atomic_t *atomic, spinlock_t *lock)
 {
-#ifdef CONFIG_SMP
 	/* Subtract 1 from counter unless that drops it to 0 (ie. it was 1) */
 	if (atomic_add_unless(atomic, -1, 1))
 		return 0;
-#endif
+
 	/* Otherwise do it the slow way */
 	spin_lock(lock);
 	if (atomic_dec_and_test(atomic))
Signature · 1 line
1.6.0.2
#7 lore raw
Quoted 10 lines · Nick Piggin
Quoted 6 lines · Jan Blunck

I think it is wrong to unconditionally take the lock before calling atomic_dec_and_test() in _atomic_dec_and_lock(). This will deadlock in situation where it is known that the counter will not reach zero (e.g. holding another reference to the same object) but the lock is already taken.

Paul's worry about callers aside, I think it is probably a good idea to reduce ifdefs and share more code.

I am also OK with this patch.

Quoted 27 lines

So for this patch,

Quoted 23 lines
---
 lib/dec_and_lock.c |    3 +--
 1 files changed, 1 insertions(+), 2 deletions(-)
lib/dec_and_lock.c +1 -2
index a65c314..e73822a 100644
@@ -19,11 +19,10 @@
  */
 int _atomic_dec_and_lock(atomic_t *atomic, spinlock_t *lock)
 {
-#ifdef CONFIG_SMP
 	/* Subtract 1 from counter unless that drops it to 0 (ie. it was 1) */
 	if (atomic_add_unless(atomic, -1, 1))
 		return 0;
-#endif
+
 	/* Otherwise do it the slow way */
 	spin_lock(lock);
 	if (atomic_dec_and_test(atomic))
Signature · 1 line
1.6.0.2
maintainer
#8 lore raw
In reply to Jan Blunck
Quoted 5 lines · On Fri, 10 Apr 2009 18:13:57 +0200 Jan Blunck <[email protected]> wrote:

I think it is wrong to unconditionally take the lock before calling atomic_dec_and_test() in _atomic_dec_and_lock(). This will deadlock in situation where it is known that the counter will not reach zero (e.g. holding another reference to the same object) but the lock is already taken.

It can't deadlock, because spin_lock() doesn't do anything on CONFIG_SMP=n.

You might get lockdep whines on CONFIG_SMP=n, but they'd be false positives because lockdep doesn't know that we generate additional code for SMP builds.

Quoted 11 lines
---
 lib/dec_and_lock.c |    3 +--
 1 files changed, 1 insertions(+), 2 deletions(-)
lib/dec_and_lock.c +0 -0
index a65c314..e73822a 100644
@@ -19,11 +19,10 @@
  */
 int _atomic_dec_and_lock(atomic_t *atomic, spinlock_t *lock)

lib/dec_and_lock.c line 28

Quoted 10 lines
lib/dec_and_lock.c +1 -2
 {
-#ifdef CONFIG_SMP
 	/* Subtract 1 from counter unless that drops it to 0 (ie. it was 1) */
 	if (atomic_add_unless(atomic, -1, 1))
 		return 0;
-#endif
+
 	/* Otherwise do it the slow way */
 	spin_lock(lock);
 	if (atomic_dec_and_test(atomic))

The patch looks reasonable from a cleanup/consistency POV, but the analysis and changelog need a bit of help, methinks.

author
#9 lore raw
Quoted 9 lines · Andrew Morton
Quoted 5 lines · On Fri, 10 Apr 2009 18:13:57 +0200 Jan Blunck <[email protected]> wrote:

I think it is wrong to unconditionally take the lock before calling atomic_dec_and_test() in _atomic_dec_and_lock(). This will deadlock in situation where it is known that the counter will not reach zero (e.g. holding another reference to the same object) but the lock is already taken.

Quoted 6 lines

It can't deadlock, because spin_lock() doesn't do anything on CONFIG_SMP=n.

You might get lockdep whines on CONFIG_SMP=n, but they'd be false positives because lockdep doesn't know that we generate additional code for SMP builds.

Sorry, you are right. spin_lock() isn't the problem here. _raw_spin_lock() calls into __spin_lock_debug():

static void __spin_lock_debug(spinlock_t *lock)

{
        u64 i;
        u64 loops = loops_per_jiffy * HZ;
        int print_once = 1;

        for (;;) {
                for (i = 0; i < loops; i++) {
                        if (__raw_spin_trylock(&lock->raw_lock))
                                return;
                        __delay(1);
                }
                /* lockup suspected: */
                if (print_once) {
                        print_once = 0;
                        printk(KERN_EMERG "BUG: spinlock lockup on CPU#%d, "
                                        "%s/%d, %p\n",
                                raw_smp_processor_id(), current->comm,
                                task_pid_nr(current), lock);
                        dump_stack();
#ifdef CONFIG_SMP
                        trigger_all_cpu_backtrace();
#endif
                }
        }
}

This is an endless loop in this cases since the lock is already held and therefore __raw_spin_trylock() never succeeds.

Quoted 16 lines
Quoted 16 lines
---
 lib/dec_and_lock.c |    3 +--
 1 files changed, 1 insertions(+), 2 deletions(-)
lib/dec_and_lock.c +0 -1
index a65c314..e73822a 100644
@@ -19,11 +19,10 @@
  */
 int _atomic_dec_and_lock(atomic_t *atomic, spinlock_t *lock)
 {
-#ifdef CONFIG_SMP
 	/* Subtract 1 from counter unless that drops it to 0 (ie. it was 1) */
 	if (atomic_add_unless(atomic, -1, 1))
 		return 0;
Quoted 9 lines

lib/dec_and_lock.c line 28

Quoted 5 lines
lib/dec_and_lock.c +1 -1
-#endif
+
 	/* Otherwise do it the slow way */
 	spin_lock(lock);
 	if (atomic_dec_and_test(atomic))

The patch looks reasonable from a cleanup/consistency POV, but the analysis and changelog need a bit of help, methinks.

Sorry, I'll come up with a more verbose description of the root cause of how this locks up.

Cheers,
Jan