locking/rwsem: Add up_write_non_owner() for percpu_up_write()
Subsystems (2)
Waiman Long
reviewerPATCH 1/1 locking/rwsem: Add up_write_non_owner() for percpu_up_write()
The commit 8c5db92a705d ("locking/rwsem: Add DEBUG_RWSEMS to look for lock/unlock mismatches") causes a warning in ext4 fstests due to the fact that the freeze and thaw ioctls, by design, are run in different processes. This is not a bug in the commit itself. Rather, we need a up_write() variant that annotates the fact that it can be called from a task different from the one that took the lock.
The new API is up_write_non_owner() and the percpu_up_write() function is modified to use it instead of the plain up_write().
--- include/linux/rwsem.h | 6 ++++++ kernel/locking/percpu-rwsem.c | 4 +++- kernel/locking/rwsem.c | 13 +++++++++++++ 3 files changed, 22 insertions(+), 1 deletion(-)
include/linux/rwsem.h +6 -0
@@ -187,4 +187,10 @@ static inline int rwsem_is_contended(struct rw_semaphore *sem)# define up_read_non_owner (sem) up_read(sem)#endif +#ifdef CONFIG_DEBUG_RWSEMS +extern void up_write_non_owner (struct rw_semaphore *sem); +#else +# define up_write_non_owner (sem) up_write(sem) +#endif +#endif /* _LINUX_RWSEM_H */
kernel/locking/percpu-rwsem.c +3 -1
@@ -179,8 +179,10 @@ void percpu_up_write(struct percpu_rw_semaphore *sem)/* * Release the write lock, this will allow readers back in the game. +* percpu_up_write() may be called from a task different from the one +* taking the lock. */ -up_write (&sem->rw_sem); +up_write_non_owner (&sem->rw_sem);/* * Once this completes (at least one RCU-sched grace period hence) the
kernel/locking/rwsem.c +13 -0
@@ -222,4 +222,17 @@ void up_read_non_owner(struct rw_semaphore *sem)#endif +#ifdef CONFIG_DEBUG_RWSEMS +/* +* release a write lock from a different task +*/ +void up_write_non_owner (struct rw_semaphore *sem) +{ +rwsem_release (&sem->dep_map,1 , _RET_IP_); +DEBUG_RWSEMS_WARN_ON (!sem->owner || (sem->owner ==RWSEM_READER_OWNED )); +rwsem_clear_owner (sem); +__up_write (sem); +} +EXPORT_SYMBOL (up_write_non_owner); +#endif
Signature · 1 line
1.8.3.1
Waiman Long
reviewerQuoted 60 lines · Waiman Long
The commit
8c5db92a705d("locking/rwsem: AddDEBUG_RWSEMSto look for lock/unlock mismatches") causes a warning in ext4 fstests due to the fact that the freeze and thaw ioctls, by design, are run in different processes. This is not a bug in the commit itself. Rather, we need aup_write()variant that annotates the fact that it can be called from a task different from the one that took the lock.The new API is
up_write_non_owner()and thepercpu_up_write()function is modified to use it instead of the plainup_write().Signed-off-by Waiman Long <[email protected]>--- include/linux/rwsem.h | 6 ++++++ kernel/locking/percpu-rwsem.c | 4 +++- kernel/locking/rwsem.c | 13 +++++++++++++ 3 files changed, 22 insertions(+), 1 deletion(-)include/linux/rwsem.h +6
-0@@ -187,4 +187,10 @@ static inline int rwsem_is_contended(struct rw_semaphore *sem)# define up_read_non_owner (sem) up_read(sem)#endif +#ifdef CONFIG_DEBUG_RWSEMS +extern void up_write_non_owner (struct rw_semaphore *sem); +#else +# define up_write_non_owner (sem) up_write(sem) +#endif +#endif /* _LINUX_RWSEM_H */ kernel/locking/percpu-rwsem.c +3
-1@@ -179,8 +179,10 @@ void percpu_up_write(struct percpu_rw_semaphore *sem)/* * Release the write lock, this will allow readers back in the game. +* percpu_up_write() may be called from a task different from the one +* taking the lock. */ -up_write (&sem->rw_sem); +up_write_non_owner (&sem->rw_sem);/* * Once this completes (at least one RCU-sched grace period hence) the kernel/locking/rwsem.c +4
-0@@ -222,4 +222,17 @@ void up_read_non_owner(struct rw_semaphore *sem)#endif +#ifdef CONFIG_DEBUG_RWSEMS +/* +* release a write lock from a different task +*/
kernel/locking/rwsem.c void up_read_non_owner(struct rw_semaphore *sem) · line 238
Quoted 10 lines
kernel/locking/rwsem.c +9
-0+void up_write_non_owner (struct rw_semaphore *sem) +{ +rwsem_release (&sem->dep_map,1 , _RET_IP_); +DEBUG_RWSEMS_WARN_ON (!sem->owner || (sem->owner ==RWSEM_READER_OWNED )); +rwsem_clear_owner (sem); +__up_write (sem); +} +EXPORT_SYMBOL (up_write_non_owner); +#endif
Theodore,
Could you try this patch to see if it can fix your fstests warning?
Thanks,
Longman
Quoted 4 lines · Waiman Long
The commit
8c5db92a705d("locking/rwsem: AddDEBUG_RWSEMSto look for lock/unlock mismatches") causes a warning in ext4 fstests due to the fact that the freeze and thaw ioctls, by design, are run in different processes.
While true, it's not just ext4 fstests --- it's any file system which supports freeze/thaw, since it's happening in the vfs layer. I have just as easily replicated the problem using "kvm-xfstests -c xfs generic/068". Or "kvm-xfstests -c btrfs generic/068".
Cheers,
- Ted
Quoted 25 lines · Waiman Long
kernel/locking/percpu-rwsem.c +3
-1@@ -179,8 +179,10 @@ void percpu_up_write(struct percpu_rw_semaphore *sem)/* * Release the write lock, this will allow readers back in the game. +* percpu_up_write() may be called from a task different from the one +* taking the lock. */ -up_write (&sem->rw_sem); +up_write_non_owner (&sem->rw_sem);/* * Once this completes (at least one RCU-sched grace period hence) the kernel/locking/rwsem.c +2
-0@@ -222,4 +222,17 @@ void up_read_non_owner(struct rw_semaphore *sem)#endif +#ifdef CONFIG_DEBUG_RWSEMS +/*
kernel/locking/rwsem.c void up_read_non_owner(struct rw_semaphore *sem) · line 236
Quoted 10 lines
kernel/locking/rwsem.c +9
-0+* release a write lock from a different task +*/ +void up_write_non_owner (struct rw_semaphore *sem) +{ +rwsem_release (&sem->dep_map,1 , _RET_IP_); +DEBUG_RWSEMS_WARN_ON (!sem->owner || (sem->owner ==RWSEM_READER_OWNED )); +rwsem_clear_owner (sem); +__up_write (sem); +}
Hmm. Can you look at lockdep_sb_freeze_release() and lockdep_sb_freeze_acquire()?
At first glance, it would be much better to set sem->owner = current in percpu_rwsem_acquire(), no?
Oleg.
Waiman Long
reviewerQuoted 32 lines · Oleg Nesterov
Quoted 31 lines · Waiman Long
kernel/locking/percpu-rwsem.c +3
-1@@ -179,8 +179,10 @@ void percpu_up_write(struct percpu_rw_semaphore *sem)/* * Release the write lock, this will allow readers back in the game. +* percpu_up_write() may be called from a task different from the one +* taking the lock. */ -up_write (&sem->rw_sem); +up_write_non_owner (&sem->rw_sem);/* * Once this completes (at least one RCU-sched grace period hence) the kernel/locking/rwsem.c +8
-0@@ -222,4 +222,17 @@ void up_read_non_owner(struct rw_semaphore *sem)#endif +#ifdef CONFIG_DEBUG_RWSEMS +/* +* release a write lock from a different task +*/ +void up_write_non_owner (struct rw_semaphore *sem) +{ +rwsem_release (&sem->dep_map,1 , _RET_IP_); +DEBUG_RWSEMS_WARN_ON (!sem->owner || (sem->owner ==RWSEM_READER_OWNED ));
Quoted 4 lines
kernel/locking/rwsem.c void up_read_non_owner(struct rw_semaphore *sem) · line 236
Quoted 3 lines
kernel/locking/rwsem.c +3
-0+rwsem_clear_owner (sem); +__up_write (sem); +}Hmm. Can you look at
lockdep_sb_freeze_release()andlockdep_sb_freeze_acquire()?
These 2 functions are there to deal with the lockdep code.
Quoted 2 lines
At first glance, it would be much better to set
sem->owner= current inpercpu_rwsem_acquire(), no?
The primary purpose of the owner field is to enable optimistic spinning to improve locking performance. So it needs to be set during an up_write() call.
My rwsem debug patch does use it also to check for consistency in the use of lock/unlock call. Anyway, I don't think it is right to set it again in percpu_rwsem_acquire() if there is no guarantee that the task that call percpu_rwsem_acquire will be the one that will do the unlock.
I am wondering if it makes sense to do optimistic spinning in the case of percpu_rwsem where the unlocker may be a different task. We could set a special code for writer owned lock, but don't do optimistic spinning in this case.
Cheers,
Longman
Quoted 5 lines · Waiman Long
Quoted 1 line · Oleg Nesterov
Hmm. Can you look at
lockdep_sb_freeze_release()andlockdep_sb_freeze_acquire()?These 2 functions are there to deal with the lockdep code.
Plus they clearly document why sem->owner check is not right when it comes to super_block->s_writers[]. Not only freeze and thaw can be called by different processes, we need to return to user-space with rwsem held for writing.
Quoted 6 lines
Quoted 2 lines
At first glance, it would be much better to set
sem->owner= current inpercpu_rwsem_acquire(), no?The primary purpose of the owner field is to enable optimistic spinning to improve locking performance. So it needs to be set during an
up_write()call.
Unless, again, the "owner" has to do lockdep_sb_freeze_release() for any reason.
And please note that percpu_rwsem_release() already clears rw_sem.owner. It checks CONFIG_RWSEM_SPIN_ON_OWNER, but this is simply because rw_semaphore->owner doesn't exist otherwise.
Quoted 4 lines
My rwsem debug patch does use it also to check for consistency in the use of lock/unlock call. Anyway, I don't think it is right to set it again in
percpu_rwsem_acquire()if there is no guarantee that the task that callpercpu_rwsem_acquirewill be the one that will do the unlock.
Hmm. Perhaps I missed something, but I think this should be true.
Of course, you need to check "if (!read)", but again, this is what percpu_rwsem_release() already does.
Quoted 2 lines
I am wondering if it makes sense to do optimistic spinning in the case of
percpu_rwsemwhere the unlocker may be a different task.
Again, perhaps I missed something, but see above. percpu_rwsem does not really differ from the regular rwsem, however its usage in sb->s_writers[] differs.
Oleg.
Waiman Long
reviewerQuoted 8 lines · Oleg Nesterov
Quoted 3 lines · Waiman Long
Quoted 1 line · Oleg Nesterov
Hmm. Can you look at
lockdep_sb_freeze_release()andlockdep_sb_freeze_acquire()?These 2 functions are there to deal with the lockdep code.
Plus they clearly document why
sem->ownercheck is not right when it comes to super_block->s_writers[]. Not only freeze and thaw can be called by different processes, we need to return to user-space with rwsem held for writing.
Sorry for the late reply as I was busy on other work.
I have just sent out a v2 patch to hopefully address your concern. Please let me know your thought on that.
Thanks,
Longman