locking/rwsem: Fix DEBUG_RWSEM warning from thaw_sup

There are use cases where a rwsem can be acquired by one task, but released by another task. In thess cases, it may not be appropriate for the lock waiters to spin on the task that acquires the lock. One example will be the filesystem freeze/thaw code.

To handle such use cases, a new RWSEM_WRITER_OWNED_NOSPIN flag can now be set in the owner field of the rwsem by the new rwsem_set_writer_owned_nospin() function to indicate that the rwsem is writer owned, but optimistic spinning on the rwsem should be disabled.

Later on, the new rwsem_set_writer_owned() function can be called to set the new owner, if it is known. This function should not be called without a prior rwsem_set_writer_owned_nospin() call.

Signed-off-by Waiman Long <[email protected]>

There are use cases where a rwsem can be acquired by one task, but released by another task. In thess cases, it may not be appropriate for the lock waiters to spin on the task that acquires the lock. One example will be the filesystem freeze/thaw code.

To handle such use cases, a new RWSEM_WRITER_OWNED_NOSPIN flag can now be set in the owner field of the rwsem by the new rwsem_set_writer_owned_nospin() function to indicate that the rwsem is writer owned, but optimistic spinning on the rwsem should be disabled.

Later on, the new rwsem_set_writer_owned() function can be called to set the new owner, if it is known. This function should not be called without a prior rwsem_set_writer_owned_nospin() call.

The percpu_rwsem_release() is called when the ownership of the embedded rwsem is to be transferred to another task. The new owner, however, may take a while to get the ownership of the lock via percpu_rwsem_acquire(). During that period, the rwsem is now marked as writer-owned with no optimistic spinning.

Signed-off-by Waiman Long <[email protected]>

---
 include/linux/percpu-rwsem.h | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

include/linux/percpu-rwsem.h +3 -4

index b1f37a8..dd37102 100644
@@ -131,16 +131,16 @@ static inline void percpu_rwsem_release(struct percpu_rw_semaphore *sem,
                                        bool read, unsigned long ip)
 {
        lock_release(&sem->rw_sem.dep_map, 1, ip);
-#ifdef CONFIG_RWSEM_SPIN_ON_OWNER
        if (!read)
-               sem->rw_sem.owner = NULL;
-#endif
+               rwsem_set_writer_owned_nospin(&sem->rw_sem);
 }
 
 static inline void percpu_rwsem_acquire(struct percpu_rw_semaphore *sem,
                                        bool read, unsigned long ip)
 {
        lock_acquire(&sem->rw_sem.dep_map, 0, 1, read, 1, NULL, ip);
+       if (!read)
+               rwsem_set_writer_owned(&sem->rw_sem, current);
 }
 
 #endif
--

1.8.3.1

Quoted 6 lines · Waiman Long

The percpu_rwsem_release() is called when the ownership of the embedded rwsem is to be transferred to another task. The new owner, however, may take a while to get the ownership of the lock via percpu_rwsem_acquire(). During that period, the rwsem is now marked as writer-owned with no optimistic spinning.

Waiman,

Thanks for the fix. I will test it soon.

For this commit message I suggest that you add parts of the reproducer found here:
https://marc.info/?l=linux-fsdevel&m=152622016219975&w=2

Quoted 6 lines · Waiman Long

The percpu_rwsem_release() is called when the ownership of the embedded rwsem is to be transferred to another task. The new owner, however, may take a while to get the ownership of the lock via percpu_rwsem_acquire(). During that period, the rwsem is now marked as writer-owned with no optimistic spinning.

Waiman,

Thanks for the fix. I will test it soon.

For this commit message I suggest that you add parts of the reproducer found here:
https://marc.info/?l=linux-fsdevel&m=152622016219975&w=2

Thanks,
Amir.

The percpu_rwsem_release() is called when the ownership of the embedded rwsem is to be transferred to another task. The new owner, however, may take a while to get the ownership of the lock via percpu_rwsem_acquire(). During that period, the rwsem is now marked as writer-owned with no optimistic spinning.

Quoted 5 lines · Waiman Long

The percpu_rwsem_release() is called when the ownership of the embedded rwsem is to be transferred to another task. The new owner, however, may take a while to get the ownership of the lock via percpu_rwsem_acquire(). During that period, the rwsem is now marked as writer-owned with no optimistic spinning.

This does not explain the problem sufficiently to even begin considering if the proposed solution is sensible.

Now this behavior upsets lockdep and that's why we fool it by telling the semaphore got released before returning to userspace (through percpu_rwsem_release() helper) and similarly we tell lockdep we've got the semaphore when an unfreeze syscall is called by percpu_rwsem_acquire(). Now Amir has discovered that also rwsem debugging code gets confused by this behavior

and previously also someone noticed that rwsem spinning does not make sense and can be broken by this behavior.

The percpu_rwsem_release() is called when the ownership of the embedded rwsem is to be transferred to another task. The new owner, however, may take a while to get the ownership of the lock via percpu_rwsem_acquire(). During that period, the rwsem is now marked as writer-owned with no optimistic spinning.

Signed-off-by Waiman Long <[email protected]>

---
 include/linux/percpu-rwsem.h | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

include/linux/percpu-rwsem.h +0 -3

index b1f37a8..dd37102 100644
@@ -131,16 +131,16 @@ static inline void percpu_rwsem_release(struct percpu_rw_semaphore *sem,
 					bool read, unsigned long ip)
 {
 	lock_release(&sem->rw_sem.dep_map, 1, ip);
-#ifdef CONFIG_RWSEM_SPIN_ON_OWNER
 	if (!read)
-		sem->rw_sem.owner = NULL;
-#endif

So what's wrong with adding:

	if (!read)
		sem->rw_sem.owner = current;

Afaict the whole .owner=NULL thing in release already stops the spinners

and the above 'fixes' the debug splat.

Quoted 5 lines · Peter Zijlstra

So what's wrong with adding:

	if (!read)
		sem->rw_sem.owner = current;

Agreed, I have already suggested this change twice. Except we obviously need to check CONFIG_RWSEM_SPIN_ON_OWNER (->owner doesn't exists otherwise) or even CONFIG_DEBUG_RWSEMS to make the purpose more clear.

Quoted 1 line

Afaict the whole .owner=NULL thing in release already stops the spinners

Not really, the new writer will spin in this case, afaics.

But this is another problem and probably we do not care. The new writer is almost impossible in this particular case, another freeze_super() should notice frozen != SB_UNFROZEN and return EBUSY.

Quoted 1 line

and the above 'fixes' the debug splat.

Yes.

Waiman, can't we trivially fix the problem first? Then we can add the helpers and think about other improvements.

Quoted 8 lines

Quoted 2 lines

Afaict the whole .owner=NULL thing in release already stops the spinners

Not really, the new writer will spin in this case, afaics.

But this is another problem and probably we do not care. The new writer is almost impossible in this particular case, another freeze_super() should notice frozen != SB_UNFROZEN and return EBUSY.

rwsem_spin_on_owner() checks rwsem_owner_is_writer(), which does owner && owner != RWSEM_READER_OWNED, which will fail for !owner.

Or am I completely confused again?

Quoted 11 lines · Peter Zijlstra

Quoted 8 lines

Quoted 2 lines

Afaict the whole .owner=NULL thing in release already stops the spinners

Not really, the new writer will spin in this case, afaics.

But this is another problem and probably we do not care. The new writer is almost impossible in this particular case, another freeze_super() should notice frozen != SB_UNFROZEN and return EBUSY.

rwsem_spin_on_owner() checks rwsem_owner_is_writer(), which does owner && owner != RWSEM_READER_OWNED, which will fail for !owner.

Yep. So rwsem_spin_on_owner() goes to "out:" and returns !rwsem_owner_is_reader() == T.

IOW, afaics owner == NULL means "spin unconditionally", I guess this is for the case when the new writer is going to do rwsem_set_owner() or up_write() has already called rwsem_clear_owner() but didn't do up_write() yet.

Probably makes sense, but the code is not very clean,

Quoted 23 lines · Waiman Long

The percpu_rwsem_release() is called when the ownership of the embedded rwsem is to be transferred to another task. The new owner, however, may take a while to get the ownership of the lock via percpu_rwsem_acquire(). During that period, the rwsem is now marked as writer-owned with no optimistic spinning.

Signed-off-by Waiman Long <[email protected]>

---
 include/linux/percpu-rwsem.h | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

include/linux/percpu-rwsem.h +0 -3

index b1f37a8..dd37102 100644
@@ -131,16 +131,16 @@ static inline void percpu_rwsem_release(struct percpu_rw_semaphore *sem,
 					bool read, unsigned long ip)
 {
 	lock_release(&sem->rw_sem.dep_map, 1, ip);
-#ifdef CONFIG_RWSEM_SPIN_ON_OWNER
 	if (!read)
-		sem->rw_sem.owner = NULL;
-#endif

include/linux/percpu-rwsem.h static inline void percpu_rwsem_release(struct percpu_rw_semaphore *sem, · line 144

Quoted 10 lines

include/linux/percpu-rwsem.h +3 -0

+		rwsem_set_writer_owned_nospin(&sem->rw_sem);
 }
 
 static inline void percpu_rwsem_acquire(struct percpu_rw_semaphore *sem,
 					bool read, unsigned long ip)
 {
 	lock_acquire(&sem->rw_sem.dep_map, 0, 1, read, 1, NULL, ip);
+	if (!read)
+		rwsem_set_writer_owned(&sem->rw_sem, current);
 }

So what's wrong with adding:

	if (!read)
		sem->rw_sem.owner = current;

?

Afaict the whole .owner=NULL thing in release already stops the spinners dead, and the above 'fixes' the debug splat. And this avoids exposing that horrible interface and keeps the mucking private to rwsem/percpu_rwsem.

Quoted 4 lines

Afaict the whole .owner=NULL thing in release already stops the spinners dead, and the above 'fixes' the debug splat. And this avoids exposing that horrible interface and keeps the mucking private to rwsem/percpu_rwsem.

Actually setting owner to NULL does not stop spinning. The code just assume that the lock is going to be freed and spin in the outer loop. We need some special value to indicate that spinning should be stopped. How about just exposing a special value for that in linux/rwsem.h? Any suggestion for a good name?