locking/mutex: Modifications to mutex
cover locking/mutex: Modifications to mutex
This patchset contains modifications to mutex.
Patch 1 further reduces unnecessary atomic xchg() operations upon entering the mutex slowpath in mutex_lock().
Patch 2 corrects/updates the documentation on mutex optimistic spinning.
Patch 3 optimizes mutex trylock slowpath.
Jason Low (3):
mutex: Try to acquire mutex only if it is unlocked
mutex: Correct documentation on mutex optimistic spinning
mutex: Optimize mutex trylock slowpath
kernel/locking/mutex.c | 47 ++++++++++++++++++++++++++--------------------- 1 files changed, 26 insertions(+), 21 deletions(-)
RFC PATCH 1/3 locking/mutex: Try to acquire mutex only if it is unlocked
Upon entering the slowpath in __mutex_lock_common(), we try once more to acquire the mutex. We only try to acquire it if MUTEX_SHOW_NO_WAITER (lock->count >= 0) is true in order to avoid using the atomic xchg() operation whenever it is not necessary. However, we really only need to try to acquire if the mutex is free (lock->count == 1).
This patch changes it so that we only try-acquire the mutex upon entering the slowpath if it is unlocked, rather than if there are no waiters. This helps further reduce unncessary atomic xchg() operations. Furthermore, this patch introduces and uses a new MUTEX_IS_UNLOCKED() macro to improve readbability.
--- kernel/locking/mutex.c | 10 ++++++---- 1 files changed, 6 insertions(+), 4 deletions(-)
kernel/locking/mutex.c +6 -4
@@ -48,9 +48,10 @@/* * A negative mutex count indicates that waiters are sleeping waiting for the -* mutex. +* mutex, and a count of one indicates the mutex is unlocked. */ #define MUTEX_SHOW_NO_WAITER (mutex) (atomic_read(&(mutex)->count) >= 0) +#define MUTEX_IS_UNLOCKED (mutex) (atomic_read(&(mutex)->count) == 1)void __mutex_init (struct mutex *lock,const char *name,struct lock_class_key *key)
@@ -440,7 +441,8 @@ __mutex_lock_common(struct mutex *lock, long state, unsigned int subclass,if (owner && !mutex_spin_on_owner (lock, owner))break ; -if ((atomic_read (&lock->count) ==1 ) && +/* Try to acquire the lock if it is free. */ +if (MUTEX_IS_UNLOCKED (lock) && (atomic_cmpxchg (&lock->count,1 ,0 ) ==1 )) {lock_acquired (&lock->dep_map, ip);if (use_ww_ctx) {
@@ -485,8 +487,8 @@ slowpath:#endif spin_lock_mutex (&lock->wait_lock, flags); -/* once more, can we acquire the lock? */ -if (MUTEX_SHOW_NO_WAITER (lock) && (atomic_xchg (&lock->count,0 ) ==1 )) +/* once more, try to acquire the lock if it is free: */ +if (MUTEX_IS_UNLOCKED (lock) && (atomic_xchg (&lock->count,0 ) ==1 )) goto skip_wait;debug_mutex_lock_common (lock, &waiter);
Signature · 1 line
1.7.1
Peter Zijlstra
maintainerRe: [RFC PATCH 1/3] locking/mutex: Try to acquire mutex only if it is unlocked
Quoted 22 lines · Jason Low
Upon entering the slowpath in
__mutex_lock_common(), we try once more to acquire the mutex. We only try to acquire it ifMUTEX_SHOW_NO_WAITER(lock->count>= 0) is true in order to avoid using the atomicxchg()operation whenever it is not necessary. However, we really only need to try to acquire if the mutex is free (lock->count== 1).This patch changes it so that we only try-acquire the mutex upon entering the slowpath if it is unlocked, rather than if there are no waiters. This helps further reduce unncessary atomic
xchg()operations. Furthermore, this patch introduces and uses a newMUTEX_IS_UNLOCKED()macro to improve readbability.Signed-off-by Jason Low <[email protected]>--- kernel/locking/mutex.c | 10 ++++++---- 1 files changed, 6 insertions(+), 4 deletions(-)kernel/locking/mutex.c +0
-0
kernel/locking/mutex.c line 54
Quoted 8 lines
kernel/locking/mutex.c +2
-1@@ -48,9 +48,10 @@/* * A negative mutex count indicates that waiters are sleeping waiting for the -* mutex. +* mutex, and a count of one indicates the mutex is unlocked. */ #define MUTEX_SHOW_NO_WAITER (mutex) (atomic_read(&(mutex)->count) >= 0) +#define MUTEX_IS_UNLOCKED (mutex) (atomic_read(&(mutex)->count) == 1)
So I recently saw that MUTEX_SHOW_NO_WAITER thing and cried a little; and now you're adding more of that same nonsense.
Please make them inline functions, also can we rename the SHOW_NO_WAITER thing, because its not at all clear to me wtf it does; should it be
called: mutex_no_waiters() or somesuch?
Re: [RFC PATCH 1/3] locking/mutex: Try to acquire mutex only if it is unlocked
Quoted 35 lines · Peter Zijlstra
Quoted 22 lines · Jason Low
Upon entering the slowpath in
__mutex_lock_common(), we try once more to acquire the mutex. We only try to acquire it ifMUTEX_SHOW_NO_WAITER(lock->count>= 0) is true in order to avoid using the atomicxchg()operation whenever it is not necessary. However, we really only need to try to acquire if the mutex is free (lock->count== 1).This patch changes it so that we only try-acquire the mutex upon entering the slowpath if it is unlocked, rather than if there are no waiters. This helps further reduce unncessary atomic
xchg()operations. Furthermore, this patch introduces and uses a newMUTEX_IS_UNLOCKED()macro to improve readbability.Signed-off-by Jason Low <[email protected]>--- kernel/locking/mutex.c | 10 ++++++---- 1 files changed, 6 insertions(+), 4 deletions(-)kernel/locking/mutex.c +0
-0kernel/locking/mutex.c line 54
Quoted 8 lines
kernel/locking/mutex.c +2
-1@@ -48,9 +48,10 @@/* * A negative mutex count indicates that waiters are sleeping waiting for the -* mutex. +* mutex, and a count of one indicates the mutex is unlocked. */ #define MUTEX_SHOW_NO_WAITER (mutex) (atomic_read(&(mutex)->count) >= 0) +#define MUTEX_IS_UNLOCKED (mutex) (atomic_read(&(mutex)->count) == 1)So I recently saw that
MUTEX_SHOW_NO_WAITERthing and cried a little; and now you're adding more of that same nonsense.
Quoted 3 lines
Please make them inline functions, also can we rename the
SHOW_NO_WAITERthing, because its not at all clear to me wtf it does; should it be
called:mutex_no_waiters()or somesuch?
Agreed.
In addition, how about the following helpers instead:
- mutex_is_unlocked() : count > 0
- mutex_has_waiters() : count < 0, or list_empty(->wait_list)
If combined with the negation, I think that pretty much covers the whole life-cycle of a mutex. The mutex.c file is full of places where we can replace crude atomic_reads with these.
Thanks,
Davidlohr
Re: [RFC PATCH 1/3] locking/mutex: Try to acquire mutex only if it is unlocked
Quoted 40 lines · Davidlohr Bueso
Quoted 35 lines · Peter Zijlstra
Quoted 22 lines · Jason Low
Upon entering the slowpath in
__mutex_lock_common(), we try once more to acquire the mutex. We only try to acquire it ifMUTEX_SHOW_NO_WAITER(lock->count>= 0) is true in order to avoid using the atomicxchg()operation whenever it is not necessary. However, we really only need to try to acquire if the mutex is free (lock->count== 1).This patch changes it so that we only try-acquire the mutex upon entering the slowpath if it is unlocked, rather than if there are no waiters. This helps further reduce unncessary atomic
xchg()operations. Furthermore, this patch introduces and uses a newMUTEX_IS_UNLOCKED()macro to improve readbability.Signed-off-by Jason Low <[email protected]>--- kernel/locking/mutex.c | 10 ++++++---- 1 files changed, 6 insertions(+), 4 deletions(-)kernel/locking/mutex.c +0
-0kernel/locking/mutex.c line 54
Quoted 8 lines
kernel/locking/mutex.c +2
-1@@ -48,9 +48,10 @@/* * A negative mutex count indicates that waiters are sleeping waiting for the -* mutex. +* mutex, and a count of one indicates the mutex is unlocked. */ #define MUTEX_SHOW_NO_WAITER (mutex) (atomic_read(&(mutex)->count) >= 0) +#define MUTEX_IS_UNLOCKED (mutex) (atomic_read(&(mutex)->count) == 1)So I recently saw that
MUTEX_SHOW_NO_WAITERthing and cried a little; and now you're adding more of that same nonsense.Quoted 3 lines
Please make them inline functions, also can we rename the
SHOW_NO_WAITERthing, because its not at all clear to me wtf it does; should it be
called:mutex_no_waiters()or somesuch?
Quoted 5 lines
Agreed.
In addition, how about the following helpers instead:
-mutex_is_unlocked(): count > 0
-mutex_has_waiters(): count < 0, orlist_empty(->wait_list)
^ err, that's !list_empty()
Re: [RFC PATCH 1/3] locking/mutex: Try to acquire mutex only if it is unlocked
Quoted 6 lines · Davidlohr Bueso
Quoted 3 lines · Davidlohr Bueso
In addition, how about the following helpers instead:
-mutex_is_unlocked(): count > 0
-mutex_has_waiters(): count < 0, orlist_empty(->wait_list)^ err, that's !list_empty()
Between checking for (count < 0) or checking for !list_empty(wait_list) for waiters:
Now that I think about it, I would expect a mutex_has_waiters() function to return !list_empty(wait_list) as that really tells whether or not there are waiters. For example, in highly contended cases, there can still be waiters on the mutex if count is 1.
Likewise, in places where we currently use "MUTEX_SHOW_NO_WAITER", we need to check for (count < 0) to ensure lock->count is a negative value before the thread sleeps on the mutex.
One option would be to still remove MUTEX_SHOW_NO_WAITER(), directly use atomic_read() in place of the macro, and just comment on why we have an extra atomic_read() that may "appear redundant". Another option could be to provide a function that checks for "potential waiters" on the mutex.
Any thoughts?
Re: [RFC PATCH 1/3] locking/mutex: Try to acquire mutex only if it is unlocked
Quoted 18 lines · Jason Low
Quoted 5 lines · Davidlohr Bueso
Quoted 3 lines · Davidlohr Bueso
In addition, how about the following helpers instead:
-mutex_is_unlocked(): count > 0
-mutex_has_waiters(): count < 0, orlist_empty(->wait_list)^ err, that's !list_empty()Between checking for (count < 0) or checking for !list_empty(wait_list) for waiters:
Now that I think about it, I would expect a
mutex_has_waiters()function to return !list_empty(wait_list) as that really tells whether or not there are waiters. For example, in highly contended cases, there can still be waiters on the mutex if count is 1.Likewise, in places where we currently use "
MUTEX_SHOW_NO_WAITER", we need to check for (count < 0) to ensurelock->countis a negative value before the thread sleeps on the mutex.
Quoted 7 lines
One option would be to still remove
MUTEX_SHOW_NO_WAITER(), directly useatomic_read()in place of the macro, and just comment on why we have an extraatomic_read()that may "appear redundant". Another option could be to provide a function that checks for "potential waiters" on the mutex.Any thoughts?
For the first MUTEX_SHOW_NO_WAITER() call site, you can replace it with a check for (count > 0). The second call site within the for loop, however, is a bit more tricky. It has to serve 2 purposes:
1. Opportunistically get the lock
2. Set the count value to -1 to indicate someone is waiting on the lock, that is why an xchg() operation has to be done even if its value is 0.
I do agree that the naming isn't that good. Maybe it can be changed to something like
static inline int mutex_value_has_waiters(mutex *lock) { return lock->count < 0; }
-Longman
Re: [RFC PATCH 1/3] locking/mutex: Try to acquire mutex only if it is unlocked
Quoted 24 lines · Wai Man
Quoted 23 lines · Jason Low
Quoted 5 lines · Davidlohr Bueso
Quoted 3 lines · Davidlohr Bueso
In addition, how about the following helpers instead:
-mutex_is_unlocked(): count > 0
-mutex_has_waiters(): count < 0, orlist_empty(->wait_list)^ err, that's !list_empty()Between checking for (count < 0) or checking for !list_empty(wait_list) for waiters:
Now that I think about it, I would expect a
mutex_has_waiters()function to return !list_empty(wait_list) as that really tells whether or not there are waiters. For example, in highly contended cases, there can still be waiters on the mutex if count is 1.Likewise, in places where we currently use "
MUTEX_SHOW_NO_WAITER", we need to check for (count < 0) to ensurelock->countis a negative value before the thread sleeps on the mutex.One option would be to still remove
MUTEX_SHOW_NO_WAITER(), directly useatomic_read()in place of the macro, and just comment on why we have an extraatomic_read()that may "appear redundant". Another option could be to provide a function that checks for "potential waiters" on the mutex.
Quoted 5 lines
Quoted 2 lines
Any thoughts?
For the first
MUTEX_SHOW_NO_WAITER()call site, you can replace it with a check for (count > 0).
Yup, in my v2 patch, the first call site becomes !mutex_is_locked(lock) which is really a check for (count == 1).
Quoted 7 lines
The second call site within the for loop, however, is a bit more tricky. It has to serve 2 purposes:
1. Opportunistically get the lock
2. Set the count value to -1 to indicate someone is waiting on the lock, that is why anxchg()operation has to be done even if its value is 0.
Quoted 5 lines
I do agree that the naming isn't that good. Maybe it can be changed to something like
static inline int
mutex_value_has_waiters(mutex *lock){ returnlock->count< 0; }
So I can imagine that a mutex_value_has_waiters() function might still not be a great name, since the mutex can have waiters in the case that the value lock->count >= 0.
In the second call site, do you think we should just do a direct atomic_read(lock->count) >= 0 and comment that we only do the xchg if the count is non-negative to avoid unnecessary xchg? That what I did in my v2 patch.
Thanks,
Jason
Re: [RFC PATCH 1/3] locking/mutex: Try to acquire mutex only if it is unlocked
Quoted 25 lines · Jason Low
Quoted 24 lines · Wai Man
Quoted 23 lines · Jason Low
Quoted 5 lines · Davidlohr Bueso
Quoted 3 lines · Davidlohr Bueso
In addition, how about the following helpers instead:
-mutex_is_unlocked(): count > 0
-mutex_has_waiters(): count < 0, orlist_empty(->wait_list)^ err, that's !list_empty()Between checking for (count < 0) or checking for !list_empty(wait_list) for waiters:
Now that I think about it, I would expect a
mutex_has_waiters()function to return !list_empty(wait_list) as that really tells whether or not there are waiters. For example, in highly contended cases, there can still be waiters on the mutex if count is 1.Likewise, in places where we currently use "
MUTEX_SHOW_NO_WAITER", we need to check for (count < 0) to ensurelock->countis a negative value before the thread sleeps on the mutex.One option would be to still remove
MUTEX_SHOW_NO_WAITER(), directly useatomic_read()in place of the macro, and just comment on why we have an extraatomic_read()that may "appear redundant". Another option could be to provide a function that checks for "potential waiters" on the mutex.
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines
Any thoughts?
For the first
MUTEX_SHOW_NO_WAITER()call site, you can replace it with a check for (count > 0).Yup, in my v2 patch, the first call site becomes !mutex_is_locked(lock) which is really a check for (count == 1).
Yes, your v2 patch looks fine to me.
Quoted 12 lines
Quoted 7 lines
The second call site within the for loop, however, is a bit more tricky. It has to serve 2 purposes:
1. Opportunistically get the lock
2. Set the count value to -1 to indicate someone is waiting on the lock, that is why anxchg()operation has to be done even if its value is 0.Quoted 5 lines
I do agree that the naming isn't that good. Maybe it can be changed to something like
static inline int
mutex_value_has_waiters(mutex *lock){ returnlock->count< 0; }
Quoted 8 lines
So I can imagine that a
mutex_value_has_waiters()function might still not be a great name, since the mutex can have waiters in the case that the valuelock->count>= 0.In the second call site, do you think we should just do a direct
atomic_read(lock->count)>= 0 and comment that we only do the xchg if the count is non-negative to avoid unnecessary xchg? That what I did in my v2 patch.
I think that is a good idea to avoid any controversy in naming.
-Longman
Re: [RFC PATCH 1/3] locking/mutex: Try to acquire mutex only if it is unlocked
In reply to Davidlohr BuesoQuoted 40 lines · Davidlohr Bueso
Quoted 35 lines · Peter Zijlstra
Quoted 22 lines · Jason Low
Upon entering the slowpath in
__mutex_lock_common(), we try once more to acquire the mutex. We only try to acquire it ifMUTEX_SHOW_NO_WAITER(lock->count>= 0) is true in order to avoid using the atomicxchg()operation whenever it is not necessary. However, we really only need to try to acquire if the mutex is free (lock->count== 1).This patch changes it so that we only try-acquire the mutex upon entering the slowpath if it is unlocked, rather than if there are no waiters. This helps further reduce unncessary atomic
xchg()operations. Furthermore, this patch introduces and uses a newMUTEX_IS_UNLOCKED()macro to improve readbability.Signed-off-by Jason Low <[email protected]>--- kernel/locking/mutex.c | 10 ++++++---- 1 files changed, 6 insertions(+), 4 deletions(-)kernel/locking/mutex.c +0
-0kernel/locking/mutex.c line 54
Quoted 8 lines
kernel/locking/mutex.c +2
-1@@ -48,9 +48,10 @@/* * A negative mutex count indicates that waiters are sleeping waiting for the -* mutex. +* mutex, and a count of one indicates the mutex is unlocked. */ #define MUTEX_SHOW_NO_WAITER (mutex) (atomic_read(&(mutex)->count) >= 0) +#define MUTEX_IS_UNLOCKED (mutex) (atomic_read(&(mutex)->count) == 1)So I recently saw that
MUTEX_SHOW_NO_WAITERthing and cried a little; and now you're adding more of that same nonsense.Quoted 3 lines
Please make them inline functions, also can we rename the
SHOW_NO_WAITERthing, because its not at all clear to me wtf it does; should it be
called:mutex_no_waiters()or somesuch?
Quoted 5 lines
Agreed.
In addition, how about the following helpers instead:
-mutex_is_unlocked(): count > 0
-mutex_has_waiters(): count < 0, orlist_empty(->wait_list)
Sounds good. Likewise, for "mutex_is_unlocked()" I've noticed that there is a mutex_is_locked() function provided in the linux/mutex.h file. Perhaps we can just reuse that function and use !mutex_is_locked() for places where we want to check if unlocked?
Re: [RFC PATCH 1/3] locking/mutex: Try to acquire mutex only if it is unlocked
In reply to Peter ZijlstraQuoted 35 lines · Peter Zijlstra
Quoted 22 lines · Jason Low
Upon entering the slowpath in
__mutex_lock_common(), we try once more to acquire the mutex. We only try to acquire it ifMUTEX_SHOW_NO_WAITER(lock->count>= 0) is true in order to avoid using the atomicxchg()operation whenever it is not necessary. However, we really only need to try to acquire if the mutex is free (lock->count== 1).This patch changes it so that we only try-acquire the mutex upon entering the slowpath if it is unlocked, rather than if there are no waiters. This helps further reduce unncessary atomic
xchg()operations. Furthermore, this patch introduces and uses a newMUTEX_IS_UNLOCKED()macro to improve readbability.Signed-off-by Jason Low <[email protected]>--- kernel/locking/mutex.c | 10 ++++++---- 1 files changed, 6 insertions(+), 4 deletions(-)kernel/locking/mutex.c +0
-0kernel/locking/mutex.c line 54
Quoted 8 lines
kernel/locking/mutex.c +2
-1@@ -48,9 +48,10 @@/* * A negative mutex count indicates that waiters are sleeping waiting for the -* mutex. +* mutex, and a count of one indicates the mutex is unlocked. */ #define MUTEX_SHOW_NO_WAITER (mutex) (atomic_read(&(mutex)->count) >= 0) +#define MUTEX_IS_UNLOCKED (mutex) (atomic_read(&(mutex)->count) == 1)So I recently saw that
MUTEX_SHOW_NO_WAITERthing and cried a little; and now you're adding more of that same nonsense.
Quoted 3 lines
Please make them inline functions, also can we rename the
SHOW_NO_WAITERthing, because its not at all clear to me wtf it does; should it be
called:mutex_no_waiters()or somesuch?
Okay, I can make them inline functions. I mainly added the macro to keep it consistent with the MUTEX_SHOW_NO_WAITER() check, but we can surely make this more clear. mutex_no_waiters() sounds fine, or perhaps something like mutex_has_no_waiters()?
Re: [RFC PATCH 1/3] locking/mutex: Try to acquire mutex only if it is unlocked
Quoted 12 lines · Jason Low
Quoted 11 lines · Peter Zijlstra
Quoted 7 lines
/*
* A negative mutex count indicates that waiters are sleeping waiting for the
- * mutex.
+ * mutex, and a count of one indicates the mutex is unlocked.
*/
#defineMUTEX_SHOW_NO_WAITER(mutex)(atomic_read(&(mutex)->count) >= 0)
+#defineMUTEX_IS_UNLOCKED(mutex)(atomic_read(&(mutex)->count) == 1)So I recently saw that
MUTEX_SHOW_NO_WAITERthing and cried a little; and now you're adding more of that same nonsense.
Quoted 6 lines
Quoted 3 lines
Please make them inline functions, also can we rename the
SHOW_NO_WAITERthing, because its not at all clear to me wtf it does; should it be
called:mutex_no_waiters()or somesuch?Okay, I can make them inline functions. I mainly added the macro to keep it consistent with the
MUTEX_SHOW_NO_WAITER()check, but we can surely
Consistency with a digusting and nonsensical macro is not really a good argument.
Quoted 2 lines
make this more clear.
mutex_no_waiters()sounds fine, or perhaps something likemutex_has_no_waiters()?
Uuurg. So we end up with
if (!mutex_has_no_waiters(m))
if we check for waiters?
Can we please go with the most intuitive thing:
mutex_has_waiters()
and have the callsites prepend the '!' in case they want to check there is no waiter?
For heavens sake, we do not name macros/inlines in a way which fits the intended use case. We name them so they make sense.
Your change log blurbs about readability. I have no idea what your understandig of readability is, but neither MUTEX_SHOW_NO_WAITERS nor mutex_has_no_waiters qualify for me. Ditto for MUTEX_IS_UNLOCKED.
Care to look at the other lock implementations:
rt_mutex_has_waiters()
spin_is_locked()
....Why would it make sense to come up with reverse conventions for mutex?
Thanks,
tglx
Re: [RFC PATCH 1/3] locking/mutex: Try to acquire mutex only if it is unlocked
Quoted 17 lines · Thomas Gleixner
Quoted 16 lines · Jason Low
Quoted 14 lines · Peter Zijlstra
Quoted 7 lines
/*
* A negative mutex count indicates that waiters are sleeping waiting for the
- * mutex.
+ * mutex, and a count of one indicates the mutex is unlocked.
*/
#defineMUTEX_SHOW_NO_WAITER(mutex)(atomic_read(&(mutex)->count) >= 0)
+#defineMUTEX_IS_UNLOCKED(mutex)(atomic_read(&(mutex)->count) == 1)So I recently saw that
MUTEX_SHOW_NO_WAITERthing and cried a little; and now you're adding more of that same nonsense.Please make them inline functions, also can we rename the
SHOW_NO_WAITERthing, because its not at all clear to me wtf it does; should it be
called:mutex_no_waiters()or somesuch?
Quoted 5 lines
Quoted 2 lines
Okay, I can make them inline functions. I mainly added the macro to keep it consistent with the
MUTEX_SHOW_NO_WAITER()check, but we can surelyConsistency with a digusting and nonsensical macro is not really a good argument.
I agree :)
Quoted 7 lines
Quoted 2 lines
make this more clear.
mutex_no_waiters()sounds fine, or perhaps something likemutex_has_no_waiters()?Uuurg. So we end up with
if (!mutex_has_no_waiters(m))
Quoted 8 lines
if we check for waiters?
Can we please go with the most intuitive thing:
mutex_has_waiters()and have the callsites prepend the '!' in case they want to check there is no waiter?
Yes, !mutex_has_waiters() sounds like the better option to check for no waiters. Same with using the already provided mutex_is_locked() function.
Quoted 13 lines
For heavens sake, we do not name macros/inlines in a way which fits the intended use case. We name them so they make sense.
Your change log blurbs about readability. I have no idea what your understandig of readability is, but neither
MUTEX_SHOW_NO_WAITERSnormutex_has_no_waitersqualify for me. Ditto forMUTEX_IS_UNLOCKED.Care to look at the other lock implementations:
rt_mutex_has_waiters() spin_is_locked() ....
Quoted 5 lines
Why would it make sense to come up with reverse conventions for mutex?
Thanks,
tglx
Thanks,
Jason
Re: [RFC PATCH 1/3] locking/mutex: Try to acquire mutex only if it is unlocked
In reply to Jason LowQuoted 35 lines · Jason Low
Quoted 34 lines · Peter Zijlstra
Quoted 22 lines · Jason Low
Upon entering the slowpath in
__mutex_lock_common(), we try once more to acquire the mutex. We only try to acquire it ifMUTEX_SHOW_NO_WAITER(lock->count>= 0) is true in order to avoid using the atomicxchg()operation whenever it is not necessary. However, we really only need to try to acquire if the mutex is free (lock->count== 1).This patch changes it so that we only try-acquire the mutex upon entering the slowpath if it is unlocked, rather than if there are no waiters. This helps further reduce unncessary atomic
xchg()operations. Furthermore, this patch introduces and uses a newMUTEX_IS_UNLOCKED()macro to improve readbability.Signed-off-by Jason Low<[email protected]>--- kernel/locking/mutex.c | 10 ++++++---- 1 files changed, 6 insertions(+), 4 deletions(-)kernel/locking/mutex.c +0
-0kernel/locking/mutex.c line 54
Quoted 8 lines
kernel/locking/mutex.c +2
-1@@ -48,9 +48,10 @@/* * A negative mutex count indicates that waiters are sleeping waiting for the -* mutex. +* mutex, and a count of one indicates the mutex is unlocked. */ #define MUTEX_SHOW_NO_WAITER (mutex) (atomic_read(&(mutex)->count)>= 0) +#define MUTEX_IS_UNLOCKED (mutex) (atomic_read(&(mutex)->count) == 1)So I recently saw that
MUTEX_SHOW_NO_WAITERthing and cried a little; and now you're adding more of that same nonsense.
Quoted 8 lines
Quoted 3 lines
Please make them inline functions, also can we rename the
SHOW_NO_WAITERthing, because its not at all clear to me wtf it does; should it be
called:mutex_no_waiters()or somesuch?Okay, I can make them inline functions. I mainly added the macro to keep it consistent with the
MUTEX_SHOW_NO_WAITER()check, but we can surely make this more clear.mutex_no_waiters()sounds fine, or perhaps something likemutex_has_no_waiters()?
You can remove the MUTEX_SHOW_NO_WAITER macro as all the call sites are to be replaced. I didn't check directly for unlocked count because of fairness concern in my original patch, but I think checking directly for unlocked count should be fine too.
-Longman
Re: [RFC PATCH 1/3] locking/mutex: Try to acquire mutex only if it is unlocked
Quoted 5 lines · Waiman Long
Quoted 4 lines · Jason Low
Quoted 3 lines · Peter Zijlstra
Please make them inline functions, also can we rename the
SHOW_NO_WAITERthing, because its not at all clear to me wtf it does; should it be
called:mutex_no_waiters()or somesuch?
Quoted 8 lines
Quoted 5 lines
Okay, I can make them inline functions. I mainly added the macro to keep it consistent with the
MUTEX_SHOW_NO_WAITER()check, but we can surely make this more clear.mutex_no_waiters()sounds fine, or perhaps something likemutex_has_no_waiters()?You can remove the
MUTEX_SHOW_NO_WAITERmacro as all the call sites are to be replaced.
Sure.
Quoted 3 lines
I didn't check directly for unlocked count because of fairness concern in my original patch, but I think checking directly for unlocked count should be fine too.
Can you elaborate on the "fairness concern"? In the current code, we're already directly checking for unlocked count in atomic_read(&lock->count) == 1 if that's what you're referring to.
Thanks,
Jason
RFC PATCH 2/3 locking/mutex: Correct documentation on mutex optimistic spinning
In reply to Jason LowThe mutex optimistic spinning documentation states that we spin for acquisition when we find that there are no pending waiters. However, in actuality, whether or not there are waiters for the mutex doesn't determine if we will spin for it.
This patch removes that statement and also adds a comment which mentions that we spin for the mutex while we don't need to reschedule.
--- kernel/locking/mutex.c | 10 ++++------ 1 files changed, 4 insertions(+), 6 deletions(-)
kernel/locking/mutex.c +4 -6
@@ -389,12 +389,10 @@ __mutex_lock_common(struct mutex *lock, long state, unsigned int subclass,/* * Optimistic spinning. * -* We try to spin for acquisition when we find that there are no -* pending waiters and the lock owner is currently running on a -* (different) CPU. -* -* The rationale is that if the lock owner is running, it is likely to -* release the lock soon. +* We try to spin for acquisition when we find that the lock owner +* is currently running on a (different) CPU and while we don't +* need to reschedule. The rationale is that if the lock owner is +* running, it is likely to release the lock soon. * * Since this needs the lock owner, and this mutex implementation * doesn't track the owner atomically in the lock field, we need to
Signature · 1 line
1.7.1
Re: [RFC PATCH 2/3] locking/mutex: Correct documentation on mutex optimistic spinning
Quoted 9 lines · Jason Low
The mutex optimistic spinning documentation states that we spin for acquisition when we find that there are no pending waiters. However, in actuality, whether or not there are waiters for the mutex doesn't determine if we will spin for it.
This patch removes that statement and also adds a comment which mentions that we spin for the mutex while we don't need to reschedule.
Signed-off-by Jason Low <[email protected]>
I think this could go in with v4 of the mutex doc rewrite patch... guessing for 3.17 now.
Thanks,
Davidlohr
Re: [RFC PATCH 2/3] locking/mutex: Correct documentation on mutex optimistic spinning
Quoted 9 lines · Davidlohr Bueso
Quoted 8 lines · Jason Low
The mutex optimistic spinning documentation states that we spin for acquisition when we find that there are no pending waiters. However, in actuality, whether or not there are waiters for the mutex doesn't determine if we will spin for it.
This patch removes that statement and also adds a comment which mentions that we spin for the mutex while we don't need to reschedule.
Quoted 6 lines
Quoted 1 line
Signed-off-by Jason Low <[email protected]>Acked-by Davidlohr Bueso <[email protected]>I think this could go in with v4 of the mutex doc rewrite patch... guessing for 3.17 now.
Yeah, this patch could be for 3.17.
Thanks,
Jason
RFC PATCH 3/3 locking/mutex: Optimize mutex trylock slowpath
In reply to Jason LowIn __mutex_trylock_slowpath(), we acquire the wait_lock spinlock, xchg() lock->count with -1, then set lock->count back to 0 if there are no waiters, and return true if the prev lock count was 1.
However, if we the mutex is already locked, then there may not be much point in attempting the above operations. In this patch, we only attempt the above operations if the mutex is unlocked.
The new MUTEX_IS_UNLOCKED() macro is also used for this.
--- kernel/locking/mutex.c | 27 ++++++++++++++++----------- 1 files changed, 16 insertions(+), 11 deletions(-)
kernel/locking/mutex.c +16 -11
@@ -821,21 +821,26 @@ static inline int __mutex_trylock_slowpath(atomic_t *lock_count) { struct mutex *lock = container_of(lock_count, struct mutex, count);unsigned long flags; -int prev; +int prev =0 ; -spin_lock_mutex (&lock->wait_lock, flags); +/* +* Only need to trylock the mutex if it is unlocked. +*/ +if (MUTEX_IS_UNLOCKED (lock)) { +spin_lock_mutex (&lock->wait_lock, flags); - prev =atomic_xchg (&lock->count,-1 ); -if (likely (prev ==1 )) { -mutex_set_owner (lock); -mutex_acquire (&lock->dep_map,0 ,1 , _RET_IP_); - } + prev =atomic_xchg (&lock->count,-1 ); +if (likely (prev ==1 )) { +mutex_set_owner (lock); +mutex_acquire (&lock->dep_map,0 ,1 , _RET_IP_); + } -/* Set it back to 0 if there are no waiters: */ -if (likely (list_empty (&lock->wait_list))) -atomic_set (&lock->count,0 ); +/* Set it back to 0 if there are no waiters: */ +if (likely (list_empty (&lock->wait_list))) +atomic_set (&lock->count,0 ); -spin_unlock_mutex (&lock->wait_lock, flags); +spin_unlock_mutex (&lock->wait_lock, flags); + }return prev ==1 ; }
Signature · 1 line
1.7.1
Re: [RFC PATCH 3/3] locking/mutex: Optimize mutex trylock slowpath
Quoted 5 lines · Jason Low
In
__mutex_trylock_slowpath(), we acquire thewait_lockspinlock,xchg()lock->countwith -1, then setlock->countback to 0 if there are no waiters, and return true if the prev lock count was 1.However, if we the mutex is already locked, then there may not be
^^ leave that out.
Quoted 1 line
much point in attempting the above operations.
Isn't this redundant? I mean, if we enter the slowpath its because __mutex_fastpath_trylock() already failed so we already know that the lock is taken.
What kind of testing has this change been put through? Any advantages? (ie: how many cycles are we saving here?), the trylock mechanism is already pretty darn fast.
Thanks,
Davidlohr
Re: [RFC PATCH 3/3] locking/mutex: Optimize mutex trylock slowpath
Quoted 5 lines · Davidlohr Bueso
Quoted 4 lines · Jason Low
In
__mutex_trylock_slowpath(), we acquire thewait_lockspinlock,xchg()lock->countwith -1, then setlock->countback to 0 if there are no waiters, and return true if the prev lock count was 1.
Quoted 8 lines
Quoted 1 line
However, if we the mutex is already locked, then there may not be
^^ leave that out.
Quoted 1 line
much point in attempting the above operations.
Isn't this redundant? I mean, if we enter the slowpath its because
__mutex_fastpath_trylock()already failed so we already know that the lock is taken.
This function is really just used as an alternative method of trylock for !__HAVE_ARCH_CMPXCHG. In that case, the fastpath can call directly into the slowpath function, without checking for if the lock is taken.
Quoted 3 lines
What kind of testing has this change been put through? Any advantages? (ie: how many cycles are we saving here?), the trylock mechanism is already pretty darn fast.
While I did run tests with this patch, this particular patch shouldn't show benefits on my machine as it should be using the more efficient atomic_cmpxchg. The advantage is in !__HAVE_ARCH_CMPXCHG, where we would avoid taking a spinlock and 2 atomic operations when the mutex is already taken.
Thanks.
Re: [RFC PATCH 3/3] locking/mutex: Optimize mutex trylock slowpath
Quoted 11 lines · Jason Low
Quoted 10 lines · Davidlohr Bueso
Quoted 5 lines · Jason Low
In
__mutex_trylock_slowpath(), we acquire thewait_lockspinlock,xchg()lock->countwith -1, then setlock->countback to 0 if there are no waiters, and return true if the prev lock count was 1.However, if we the mutex is already locked, then there may not be
^^ leave that out.
Quoted 1 line
much point in attempting the above operations.
Quoted 7 lines
Quoted 3 lines
Isn't this redundant? I mean, if we enter the slowpath its because
__mutex_fastpath_trylock()already failed so we already know that the lock is taken.This function is really just used as an alternative method of trylock for !__HAVE_ARCH_CMPXCHG. In that case, the fastpath can call directly into the slowpath function, without checking for if the lock is taken.
Ah, ok I hadn't seen that we do this in 32bit x86 and was wondering why the heck we fallback on a slowpath for something like trylock, which should return right away no matter what. I'd suggest explicitly mentioning this in the changelog. Otherwise makes sense now.
Re: [RFC PATCH 3/3] locking/mutex: Optimize mutex trylock slowpath
Quoted 16 lines · Davidlohr Bueso
Quoted 15 lines · Jason Low
Quoted 13 lines · Davidlohr Bueso
Quoted 5 lines · Jason Low
In
__mutex_trylock_slowpath(), we acquire thewait_lockspinlock,xchg()lock->countwith -1, then setlock->countback to 0 if there are no waiters, and return true if the prev lock count was 1.However, if we the mutex is already locked, then there may not be
^^ leave that out.
Quoted 1 line
much point in attempting the above operations.
Isn't this redundant? I mean, if we enter the slowpath its because
__mutex_fastpath_trylock()already failed so we already know that the lock is taken.
Quoted 8 lines
Quoted 3 lines
This function is really just used as an alternative method of trylock for !__HAVE_ARCH_CMPXCHG. In that case, the fastpath can call directly into the slowpath function, without checking for if the lock is taken.
Ah, ok I hadn't seen that we do this in 32bit x86 and was wondering why the heck we fallback on a slowpath for something like trylock, which should return right away no matter what. I'd suggest explicitly mentioning this in the changelog. Otherwise makes sense now.
Yup, I was also initially wondering why we have a slowpath for trylock, which prompted me to take a look into this function. I'll add some more information about this to the changelog.
Thanks,
Jason
Quoted 1 line · Jason Low
This patchset contains modifications to mutex.
Wondering why the RFC...