POSIX: Fix race when setting claimed bit
PATCH 1/1 POSIX: Fix race when setting claimed bit
Patch 2 of my fast lock series actually also contained an attempt to fix a race I spotted in the code that atomically sets the claimed bit. I forgot about this fact and even, worse, I only replace the original race with another one.
So here comes a new attempt to fix the issue that the lock owner and/or the claimed bit can change between trylock and the cmpxchg under nklock. Please have a look and cross-check the logic.
The patch applies on top of vanilla SVN, so my series has to be rebased and the fix has to be ported to native as well - where we just found it in the field.
Jan
--- ksrc/skins/posix/mutex.h | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-)
ksrc/skins/posix/mutex.h +6 -5
@@ -134,17 +134,18 @@ static inline int pse51_mutex_timedlock_/* Set bit 0, so that mutex_unlock will know that the mutex is claimed. Hold the nklock, for mutual exclusion with slow mutex_unlock. */ xnlock_get_irqsave (&nklock, s); + owner =xnarch_atomic_intptr_get (mutex->owner);while (!test_claimed (owner)) { - old =xnarch_atomic_intptr_cmpxchg (mutex->owner, - owner,set_claimed (owner,1 )); -if (likely (old == owner)) -break ; -if (old ==NULL ) { +if (owner ==NULL ) {/* Owner called fast mutex_unlock (on another cpu) */ xnlock_put_irqrestore (&nklock, s); goto retry_lock; } + old =xnarch_atomic_intptr_cmpxchg (mutex->owner, + owner,set_claimed (owner,1 )); +if (likely (old == owner)) +break ; owner = old; }
Quoted 5 lines · Jan Kiszka
Patch 2 of my fast lock series actually also contained an attempt to fix a race I spotted in the code that atomically sets the claimed bit. I forgot about this fact and even, worse, I only replace the original race with another one.
Quoted 7 lines
So here comes a new attempt to fix the issue that the lock owner and/or the claimed bit can change between trylock and the cmpxchg under nklock. Please have a look and cross-check the logic.
The patch applies on top of vanilla SVN, so my series has to be rebased and the fix has to be ported to native as well - where we just found it in the field.
Could you explain the race ?
Signature · 1 line
Gilles.
Quoted 10 lines · Gilles Chanteperdrix
Quoted 9 lines · Jan Kiszka
Patch 2 of my fast lock series actually also contained an attempt to fix a race I spotted in the code that atomically sets the claimed bit. I forgot about this fact and even, worse, I only replace the original race with another one.
So here comes a new attempt to fix the issue that the lock owner and/or the claimed bit can change between trylock and the cmpxchg under nklock. Please have a look and cross-check the logic.
Quoted 5 lines
Quoted 3 lines
The patch applies on top of vanilla SVN, so my series has to be rebased and the fix has to be ported to native as well - where we just found it in the field.
Could you explain the race ?
We read the owner in pse51_mutex_trylock_internal. In pse51_mutex_timedlock_internal under nklock, we check that old value for the claimed bit. If the old value happened to have it set, we continue happily without worries. But it may have been cleared meanwhile _before_ we entered the nklock.
I previously tried to fix this by re-reading the lock value, but then I failed to account for the case that the value may have become NULL meanwhile. The new version should cover both cases.
Jan
Signature · 2 lines
Siemens AG, Corporate Technology, CT SE 2 Corporate Competence Center Embedded Linux
Quoted 5 lines · Jan Kiszka
Patch 2 of my fast lock series actually also contained an attempt to fix a race I spotted in the code that atomically sets the claimed bit. I forgot about this fact and even, worse, I only replace the original race with another one.
Quoted 7 lines
So here comes a new attempt to fix the issue that the lock owner and/or the claimed bit can change between trylock and the cmpxchg under nklock. Please have a look and cross-check the logic.
The patch applies on top of vanilla SVN, so my series has to be rebased and the fix has to be ported to native as well - where we just found it in the field.
Ok. Got it.
Quoted 12 lines
Jan
--- ksrc/skins/posix/mutex.h | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-)Index:
b/ksrc/skins/posix/mutex.h
===================================================================
ksrc/skins/posix/mutex.h static inline int pse51_mutex_timedlock_ · line 137
Quoted 4 lines
ksrc/skins/posix/mutex.h +1
-0@@ -134,17 +134,18 @@ static inline int pse51_mutex_timedlock_/* Set bit 0, so that mutex_unlock will know that the mutex is claimed. Hold the nklock, for mutual exclusion with slow mutex_unlock. */ xnlock_get_irqsave (&nklock, s); + owner =xnarch_atomic_intptr_get (mutex->owner);
Bad, this makes two atomic ops. So, I would rather propose:
ksrc/skins/posix/mutex.h static inline int pse51_mutex_timedlock_ · line 138
Quoted 1 line
ksrc/skins/posix/mutex.h +0
-0while (!test_claimed (owner)) {
- while(!test_claimed(owner)) {
+ do {
Quoted 6 lines
ksrc/skins/posix/mutex.h +1
-5- old =xnarch_atomic_intptr_cmpxchg (mutex->owner, - owner,set_claimed (owner,1 )); -if (likely (old == owner)) -break ; -if (old ==NULL ) { +if (owner ==NULL ) {
ksrc/skins/posix/mutex.h static inline int pse51_mutex_timedlock_ · line 149
Quoted 10 lines
ksrc/skins/posix/mutex.h +4
-0/* Owner called fast mutex_unlock (on another cpu) */ xnlock_put_irqrestore (&nklock, s); goto retry_lock; } + old =xnarch_atomic_intptr_cmpxchg (mutex->owner, + owner,set_claimed (owner,1 )); +if (likely (old == owner)) +break ; owner = old;
- }
+ } while (!test_claimed(owner))
Quoted 6 lines
_______________________________________________
Xenomai-core mailing list
[email protected]
https://mail.gna.org/listinfo/xenomai-core
Signature · 1 line
Gilles.
Quoted 22 lines · Gilles Chanteperdrix
Quoted 5 lines · Jan Kiszka
Patch 2 of my fast lock series actually also contained an attempt to fix a race I spotted in the code that atomically sets the claimed bit. I forgot about this fact and even, worse, I only replace the original race with another one.
Quoted 7 lines
So here comes a new attempt to fix the issue that the lock owner and/or the claimed bit can change between trylock and the cmpxchg under nklock. Please have a look and cross-check the logic.
The patch applies on top of vanilla SVN, so my series has to be rebased and the fix has to be ported to native as well - where we just found it in the field.
Ok. Got it.
Quoted 6 lines
Jan
--- ksrc/skins/posix/mutex.h | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-)
Quoted 11 lines
ksrc/skins/posix/mutex.h static inline int pse51_mutex_timedlock_ · line 137
Quoted 9 lines
Index:
b/ksrc/skins/posix/mutex.h
===================================================================ksrc/skins/posix/mutex.h +1
-0@@ -134,17 +134,18 @@ static inline int pse51_mutex_timedlock_/* Set bit 0, so that mutex_unlock will know that the mutex is claimed. Hold the nklock, for mutual exclusion with slow mutex_unlock. */ xnlock_get_irqsave (&nklock, s); + owner =xnarch_atomic_intptr_get (mutex->owner);Bad, this makes two atomic ops. So, I would rather propose:
Err, how many archs perform special dances for atomic_read?
Quoted 20 lines
ksrc/skins/posix/mutex.h static inline int pse51_mutex_timedlock_ · line 138
Quoted 2 lines
ksrc/skins/posix/mutex.h +0
-0while (!test_claimed (owner)) {- while(!test_claimed(owner)) {
+ do {Quoted 16 lines
ksrc/skins/posix/mutex.h +5
-5- old =xnarch_atomic_intptr_cmpxchg (mutex->owner, - owner,set_claimed (owner,1 )); -if (likely (old == owner)) -break ; -if (old ==NULL ) { +if (owner ==NULL ) {/* Owner called fast mutex_unlock (on another cpu) */ xnlock_put_irqrestore (&nklock, s); goto retry_lock; } + old =xnarch_atomic_intptr_cmpxchg (mutex->owner, + owner,set_claimed (owner,1 )); +if (likely (old == owner)) +break ; owner = old;
Review
Quoted 2 lines
(quoted diff) +1
-1- } + } while (!test_claimed(owner))
Your version slows down the contended case by performing redundant atomic cmpxchgs. My version puts minimal additional overhead in the !test_claimed case, but introduces no further atomic ops and avoids to much traffic on mutex->owner's cacheline when there are >1 waiters.
Jan
Signature · 2 lines
Siemens AG, Corporate Technology, CT SE 2 Corporate Competence Center Embedded Linux
Quoted 33 lines · Jan Kiszka
Quoted 21 lines · Gilles Chanteperdrix
Quoted 5 lines · Jan Kiszka
Patch 2 of my fast lock series actually also contained an attempt to fix a race I spotted in the code that atomically sets the claimed bit. I forgot about this fact and even, worse, I only replace the original race with another one.
Quoted 7 lines
So here comes a new attempt to fix the issue that the lock owner and/or the claimed bit can change between trylock and the cmpxchg under nklock. Please have a look and cross-check the logic.
The patch applies on top of vanilla SVN, so my series has to be rebased and the fix has to be ported to native as well - where we just found it in the field.
Ok. Got it.
Quoted 6 lines
Jan
--- ksrc/skins/posix/mutex.h | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-)Quoted 10 lines
ksrc/skins/posix/mutex.h static inline int pse51_mutex_timedlock_ · line 137
Quoted 9 lines
Index:
b/ksrc/skins/posix/mutex.h
===================================================================ksrc/skins/posix/mutex.h +1
-0@@ -134,17 +134,18 @@ static inline int pse51_mutex_timedlock_/* Set bit 0, so that mutex_unlock will know that the mutex is claimed. Hold the nklock, for mutual exclusion with slow mutex_unlock. */ xnlock_get_irqsave (&nklock, s); + owner =xnarch_atomic_intptr_get (mutex->owner);Bad, this makes two atomic ops. So, I would rather propose:
Quoted 1 line
Err, how many archs perform special dances for
atomic_read?
powerpc, arm >= 6
Quoted 23 lines
Quoted 20 lines
ksrc/skins/posix/mutex.h static inline int pse51_mutex_timedlock_ · line 138
Quoted 2 lines
ksrc/skins/posix/mutex.h +0
-0while (!test_claimed (owner)) {- while(!test_claimed(owner)) {
+ do {Quoted 16 lines
ksrc/skins/posix/mutex.h +5
-5- old =xnarch_atomic_intptr_cmpxchg (mutex->owner, - owner,set_claimed (owner,1 )); -if (likely (old == owner)) -break ; -if (old ==NULL ) { +if (owner ==NULL ) {/* Owner called fast mutex_unlock (on another cpu) */ xnlock_put_irqrestore (&nklock, s); goto retry_lock; } + old =xnarch_atomic_intptr_cmpxchg (mutex->owner, + owner,set_claimed (owner,1 )); +if (likely (old == owner)) +break ; owner = old;Review
Quoted 2 lines
(quoted diff) +1
-1- } + } while (!test_claimed(owner))
Quoted 4 lines
Your version slows down the contended case by performing redundant atomic cmpxchgs. My version puts minimal additional overhead in the !test_claimed case, but introduces no further atomic ops and avoids to much traffic on mutex->owner's cacheline when there are >1 waiters.
But your version slows down the common case where the owner will not have changed... I do not care for making the uncommon case slower as long as the common case is fast.
Signature · 1 line
Gilles.
Quoted 33 lines · Gilles Chanteperdrix
Quoted 32 lines · Jan Kiszka
Quoted 31 lines · Gilles Chanteperdrix
Quoted 5 lines · Jan Kiszka
Patch 2 of my fast lock series actually also contained an attempt to fix a race I spotted in the code that atomically sets the claimed bit. I forgot about this fact and even, worse, I only replace the original race with another one.
Quoted 7 lines
So here comes a new attempt to fix the issue that the lock owner and/or the claimed bit can change between trylock and the cmpxchg under nklock. Please have a look and cross-check the logic.
The patch applies on top of vanilla SVN, so my series has to be rebased and the fix has to be ported to native as well - where we just found it in the field.
Ok. Got it.
Quoted 11 lines
Jan
--- ksrc/skins/posix/mutex.h | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-)Index:
b/ksrc/skins/posix/mutex.h
===================================================================ksrc/skins/posix/mutex.h static inline int pse51_mutex_timedlock_ · line 137
Quoted 4 lines
ksrc/skins/posix/mutex.h +1
-0@@ -134,17 +134,18 @@ static inline int pse51_mutex_timedlock_/* Set bit 0, so that mutex_unlock will know that the mutex is claimed. Hold the nklock, for mutual exclusion with slow mutex_unlock. */ xnlock_get_irqsave (&nklock, s); + owner =xnarch_atomic_intptr_get (mutex->owner);Bad, this makes two atomic ops. So, I would rather propose:
Quoted 3 lines
Quoted 1 line
Err, how many archs perform special dances for
atomic_read?powerpc, arm >= 6
Actually arm >= 6 has a special atomic_set but no special atomic_read, the problem comes rather from arm <= 5 where the owner is stored on an uncached area.
Signature · 1 line
Gilles.
Quoted 33 lines · Gilles Chanteperdrix
Quoted 32 lines · Jan Kiszka
Quoted 31 lines · Gilles Chanteperdrix
Quoted 5 lines · Jan Kiszka
Patch 2 of my fast lock series actually also contained an attempt to fix a race I spotted in the code that atomically sets the claimed bit. I forgot about this fact and even, worse, I only replace the original race with another one.
Quoted 7 lines
So here comes a new attempt to fix the issue that the lock owner and/or the claimed bit can change between trylock and the cmpxchg under nklock. Please have a look and cross-check the logic.
The patch applies on top of vanilla SVN, so my series has to be rebased and the fix has to be ported to native as well - where we just found it in the field.
Ok. Got it.
Quoted 11 lines
Jan
--- ksrc/skins/posix/mutex.h | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-)Index:
b/ksrc/skins/posix/mutex.h
===================================================================ksrc/skins/posix/mutex.h static inline int pse51_mutex_timedlock_ · line 137
Quoted 4 lines
ksrc/skins/posix/mutex.h +1
-0@@ -134,17 +134,18 @@ static inline int pse51_mutex_timedlock_/* Set bit 0, so that mutex_unlock will know that the mutex is claimed. Hold the nklock, for mutual exclusion with slow mutex_unlock. */ xnlock_get_irqsave (&nklock, s); + owner =xnarch_atomic_intptr_get (mutex->owner);Bad, this makes two atomic ops. So, I would rather propose:
Quoted 3 lines
Quoted 1 line
Err, how many archs perform special dances for
atomic_read?powerpc, arm >= 6
None of both. All atomic_read for arm use plain ((v)->counter), powerpc simply uses assembly to avoid volatile atomic_t types. Do you happen to confuse atomic_read and set here?
Quoted 22 lines
Quoted 22 lines
Quoted 22 lines
ksrc/skins/posix/mutex.h static inline int pse51_mutex_timedlock_ · line 138
Quoted 2 lines
ksrc/skins/posix/mutex.h +0
-0while (!test_claimed (owner)) {- while(!test_claimed(owner)) {
+ do {Quoted 16 lines
ksrc/skins/posix/mutex.h +5
-5- old =xnarch_atomic_intptr_cmpxchg (mutex->owner, - owner,set_claimed (owner,1 )); -if (likely (old == owner)) -break ; -if (old ==NULL ) { +if (owner ==NULL ) {/* Owner called fast mutex_unlock (on another cpu) */ xnlock_put_irqrestore (&nklock, s); goto retry_lock; } + old =xnarch_atomic_intptr_cmpxchg (mutex->owner, + owner,set_claimed (owner,1 )); +if (likely (old == owner)) +break ; owner = old;(quoted diff) +1
-1- } + } while (!test_claimed(owner))
Quoted 8 lines
Quoted 4 lines
Your version slows down the contended case by performing redundant atomic cmpxchgs. My version puts minimal additional overhead in the !test_claimed case, but introduces no further atomic ops and avoids to much traffic on mutex->owner's cacheline when there are >1 waiters.
But your version slows down the common case where the owner will not have changed... I do not care for making the uncommon case slower as long as the common case is fast.
The tiny slowdown (a simple, typically cached read and test) is negligible compared to the full slow path, even if there is only one waiter. Atomic ops are heavier, sleeping is much heavier.
Jan
Signature · 2 lines
Siemens AG, Corporate Technology, CT SE 2 Corporate Competence Center Embedded Linux
Quoted 66 lines · Jan Kiszka
Quoted 33 lines · Gilles Chanteperdrix
Quoted 32 lines · Jan Kiszka
Quoted 31 lines · Gilles Chanteperdrix
Quoted 5 lines · Jan Kiszka
Patch 2 of my fast lock series actually also contained an attempt to fix a race I spotted in the code that atomically sets the claimed bit. I forgot about this fact and even, worse, I only replace the original race with another one.
Quoted 7 lines
So here comes a new attempt to fix the issue that the lock owner and/or the claimed bit can change between trylock and the cmpxchg under nklock. Please have a look and cross-check the logic.
The patch applies on top of vanilla SVN, so my series has to be rebased and the fix has to be ported to native as well - where we just found it in the field.
Ok. Got it.
Quoted 11 lines
Jan
--- ksrc/skins/posix/mutex.h | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-)Index:
b/ksrc/skins/posix/mutex.h
===================================================================ksrc/skins/posix/mutex.h static inline int pse51_mutex_timedlock_ · line 137
Quoted 4 lines
ksrc/skins/posix/mutex.h +1
-0@@ -134,17 +134,18 @@ static inline int pse51_mutex_timedlock_/* Set bit 0, so that mutex_unlock will know that the mutex is claimed. Hold the nklock, for mutual exclusion with slow mutex_unlock. */ xnlock_get_irqsave (&nklock, s); + owner =xnarch_atomic_intptr_get (mutex->owner);Bad, this makes two atomic ops. So, I would rather propose:
Quoted 2 lines
Quoted 1 line
Err, how many archs perform special dances for
atomic_read?powerpc, arm >= 6
None of both. All
atomic_readfor arm use plain ((v)->counter), powerpc simply uses assembly to avoid volatileatomic_ttypes. Do you happen to confuseatomic_readand set here?Quoted 25 lines
Quoted 25 lines
Quoted 19 lines
ksrc/skins/posix/mutex.h static inline int pse51_mutex_timedlock_ · line 138
Quoted 1 line
ksrc/skins/posix/mutex.h +0
-0while (!test_claimed (owner)) {- while(!test_claimed(owner)) {
+ do {Quoted 16 lines
ksrc/skins/posix/mutex.h +5
-5- old =xnarch_atomic_intptr_cmpxchg (mutex->owner, - owner,set_claimed (owner,1 )); -if (likely (old == owner)) -break ; -if (old ==NULL ) { +if (owner ==NULL ) {/* Owner called fast mutex_unlock (on another cpu) */ xnlock_put_irqrestore (&nklock, s); goto retry_lock; } + old =xnarch_atomic_intptr_cmpxchg (mutex->owner, + owner,set_claimed (owner,1 )); +if (likely (old == owner)) +break ; owner = old;Review
Quoted 2 lines
(quoted diff) +1
-1- } + } while (!test_claimed(owner))Your version slows down the contended case by performing redundant atomic cmpxchgs. My version puts minimal additional overhead in the !test_claimed case, but introduces no further atomic ops and avoids to much traffic on mutex->owner's cacheline when there are >1 waiters.
Quoted 5 lines
Quoted 3 lines
But your version slows down the common case where the owner will not have changed... I do not care for making the uncommon case slower as long as the common case is fast.
The tiny slowdown (a simple, typically cached read and test) is
Two tests, in fact, when reordering the alternative code a bit.
Quoted 2 lines
negligible compared to the full slow path, even if there is only one waiter. Atomic ops are heavier, sleeping is much heavier.
OTOH, we can safe some text size which is precious as well. So I'm convinced to go your way (with a modification):
--- ksrc/skins/posix/mutex.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
ksrc/skins/posix/mutex.h +2 -2
@@ -134,7 +134,7 @@ static inline int pse51_mutex_timedlock_/* Set bit 0, so that mutex_unlock will know that the mutex is claimed. Hold the nklock, for mutual exclusion with slow mutex_unlock. */ xnlock_get_irqsave (&nklock, s); -while (!test_claimed (owner)) { + do { old = xnarch_atomic_intptr_cmpxchg(mutex->owner, owner, set_claimed(owner, 1)); if (likely(old == owner))
@@ -146,7 +146,7 @@ static inline int pse51_mutex_timedlock_ goto retry_lock; } owner = old; - } + }while (!test_claimed (owner));xnsynch_set_owner (&mutex->synchbase,clear_claimed (owner)); ++mutex->sleepers;
Quoted 2 lines · Jan Kiszka
OTOH, we can safe some text size which is precious as well. So I'm convinced to go your way (with a modification):
My approach sucks: we get a silly atomic_cmpxchg if the mutex is already claimed, which is as least as much a common case as a currently unclaimed mutex. Need to think a bit. But I think a good solution is to re-read only if the mutex has been seen as already claimed.
Signature · 1 line
Gilles.
Quoted 8 lines · Gilles Chanteperdrix
Quoted 2 lines · Jan Kiszka
OTOH, we can safe some text size which is precious as well. So I'm convinced to go your way (with a modification):
My approach sucks: we get a silly
atomic_cmpxchgif the mutex is already claimed, which is as least as much a common case as a currently unclaimed mutex. Need to think a bit. But I think a good solution is to re-read only if the mutex has been seen as already claimed.
That makes no difference as then we will go through the cmpxchg path anyway.
There is _no_ way around re-reading under nklock, all we can avoid is atomic cmpxchg in the case of >1 waiters. But that would come at the price of more complexity for all waiter.
However, let's find some solution. I bet things will look different again when we start fiddling with a generic lock + the additional bit to replace XNWAKEN.
Jan
Signature · 2 lines
Siemens AG, Corporate Technology, CT SE 2 Corporate Competence Center Embedded Linux
Quoted 11 lines · Jan Kiszka
Quoted 7 lines · Gilles Chanteperdrix
Quoted 2 lines · Jan Kiszka
OTOH, we can safe some text size which is precious as well. So I'm convinced to go your way (with a modification):
My approach sucks: we get a silly
atomic_cmpxchgif the mutex is already claimed, which is as least as much a common case as a currently unclaimed mutex. Need to think a bit. But I think a good solution is to re-read only if the mutex has been seen as already claimed.That makes no difference as then we will go through the cmpxchg path anyway.
Quoted 7 lines
There is _no_ way around re-reading under nklock, all we can avoid is atomic cmpxchg in the case of >1 waiters. But that would come at the price of more complexity for all waiter.
However, let's find some solution. I bet things will look different again when we start fiddling with a generic lock + the additional bit to replace XNWAKEN.
What I meant is that if the claimed bit is already set, we can avoid the cmpxchg altogether, which was the intent of the original code. So I propose the following version:
if(test_claimed(owner))
owner = xnarch_atomic_intptr_get(mutex->owner);
while(!test_claimed(owner)) {
old = xnarch_atomic_intptr_cmpxchg(mutex->owner,
owner, set_claimed(owner, 1));
if (likely(old == owner))
break;
if (old == NULL) {
/* Owner called fast mutex_unlock
(on another cpu) */
xnlock_put_irqrestore(&nklock, s);
goto retry_lock;
}
owner = old;
}The compiler rearranges things correctly (at least on ARM), and avoids the redundant test.
Signature · 1 line
Gilles.
Quoted 19 lines · Gilles Chanteperdrix
Quoted 10 lines · Jan Kiszka
Quoted 7 lines · Gilles Chanteperdrix
Quoted 2 lines · Jan Kiszka
OTOH, we can safe some text size which is precious as well. So I'm convinced to go your way (with a modification):
My approach sucks: we get a silly
atomic_cmpxchgif the mutex is already claimed, which is as least as much a common case as a currently unclaimed mutex. Need to think a bit. But I think a good solution is to re-read only if the mutex has been seen as already claimed.That makes no difference as then we will go through the cmpxchg path anyway.
Quoted 7 lines
There is _no_ way around re-reading under nklock, all we can avoid is atomic cmpxchg in the case of >1 waiters. But that would come at the price of more complexity for all waiter.
However, let's find some solution. I bet things will look different again when we start fiddling with a generic lock + the additional bit to replace XNWAKEN.
Quoted 6 lines
What I meant is that if the claimed bit is already set, we can avoid the cmpxchg altogether, which was the intent of the original code. So I propose the following version:
if(test_claimed(owner)) owner = xnarch_atomic_intptr_get(mutex->owner);
This version lacks a test for owner == 0 here, otherwise you re-create my old bug that bite me today.
Quoted 14 lines
while(!test_claimed(owner)) {
old = xnarch_atomic_intptr_cmpxchg(mutex->owner,
owner, set_claimed(owner, 1));
if (likely(old == owner))
break;
if (old == NULL) {
/* Owner called fast mutex_unlock
(on another cpu) */
xnlock_put_irqrestore(&nklock, s);
goto retry_lock;
}
owner = old;
}Quoted 3 lines
The compiler rearranges things correctly (at least on ARM), and avoids the redundant test.
My latest concern remains: Is all this additional complexity, are all these conditional branches and the text size increase worth the effort? How much cycles or micoseconds would be gain on the most suffering architecture?
Jan
Quoted 23 lines · Jan Kiszka
Quoted 22 lines · Gilles Chanteperdrix
Quoted 10 lines · Jan Kiszka
Quoted 7 lines · Gilles Chanteperdrix
Quoted 2 lines · Jan Kiszka
OTOH, we can safe some text size which is precious as well. So I'm convinced to go your way (with a modification):
My approach sucks: we get a silly
atomic_cmpxchgif the mutex is already claimed, which is as least as much a common case as a currently unclaimed mutex. Need to think a bit. But I think a good solution is to re-read only if the mutex has been seen as already claimed.That makes no difference as then we will go through the cmpxchg path anyway.
Quoted 7 lines
There is _no_ way around re-reading under nklock, all we can avoid is atomic cmpxchg in the case of >1 waiters. But that would come at the price of more complexity for all waiter.
However, let's find some solution. I bet things will look different again when we start fiddling with a generic lock + the additional bit to replace XNWAKEN.
What I meant is that if the claimed bit is already set, we can avoid the cmpxchg altogether, which was the intent of the original code. So I propose the following version:
Quoted 5 lines
Quoted 2 lines
if(test_claimed(owner)) owner = xnarch_atomic_intptr_get(mutex->owner);This version lacks a test for owner == 0 here, otherwise you re-create my old bug that bite me today.
Ok, so jumping in the middle of the loop is the best thing to do then.
Quoted 15 lines
Quoted 15 lines
while(!test_claimed(owner)) { old = xnarch_atomic_intptr_cmpxchg(mutex->owner, owner, set_claimed(owner, 1)); if (likely(old == owner)) break; if (old == NULL) { /* Owner called fast mutex_unlock (on another cpu) */ xnlock_put_irqrestore(&nklock, s); goto retry_lock; } owner = old; }
Quoted 8 lines
Quoted 3 lines
The compiler rearranges things correctly (at least on ARM), and avoids the redundant test.
My latest concern remains: Is all this additional complexity, are all these conditional branches and the text size increase worth the effort? How much cycles or micoseconds would be gain on the most suffering architecture?
Since the else and the while are folded together and the loop becomes post-tested, and with the help of conditional instructions, I do not think there is much more conditional branch (we need to to jump over the while loop anyway). However avoiding the cmpxchg is a real gain, since it is implemented as irq save/irq restore on an old arm.
if(test_claimed(owner)) {
old = xnarch_atomic_intptr_get(mutex->owner);
goto test_old;
}
while(!test_claimed(owner)) {
old = xnarch_atomic_intptr_cmpxchg(mutex->owner,
owner, set_claimed(owner, 1));
if (likely(old == owner))
break;
test_old:
if (old == NULL) {
/* Owner called fast mutex_unlock
(on another cpu) */
xnlock_put_irqrestore(&nklock, s);
goto retry_lock;
}
owner = old;
}Signature · 1 line
Gilles.
Quoted 69 lines · Gilles Chanteperdrix
Quoted 19 lines · Jan Kiszka
Quoted 18 lines · Gilles Chanteperdrix
Quoted 17 lines · Jan Kiszka
Quoted 7 lines · Gilles Chanteperdrix
Quoted 2 lines · Jan Kiszka
OTOH, we can safe some text size which is precious as well. So I'm convinced to go your way (with a modification):
My approach sucks: we get a silly
atomic_cmpxchgif the mutex is already claimed, which is as least as much a common case as a currently unclaimed mutex. Need to think a bit. But I think a good solution is to re-read only if the mutex has been seen as already claimed.That makes no difference as then we will go through the cmpxchg path anyway.
There is _no_ way around re-reading under nklock, all we can avoid is atomic cmpxchg in the case of >1 waiters. But that would come at the price of more complexity for all waiter.
However, let's find some solution. I bet things will look different again when we start fiddling with a generic lock + the additional bit to replace XNWAKEN.
Quoted 8 lines
Quoted 6 lines
What I meant is that if the claimed bit is already set, we can avoid the cmpxchg altogether, which was the intent of the original code. So I propose the following version:
if(test_claimed(owner)) owner = xnarch_atomic_intptr_get(mutex->owner);This version lacks a test for owner == 0 here, otherwise you re-create my old bug that bite me today.
Ok, so jumping in the middle of the loop is the best thing to do then.
Quoted 14 lines
Quoted 14 lines
while(!test_claimed(owner)) { old = xnarch_atomic_intptr_cmpxchg(mutex->owner, owner, set_claimed(owner, 1)); if (likely(old == owner)) break; if (old == NULL) { /* Owner called fast mutex_unlock (on another cpu) */ xnlock_put_irqrestore(&nklock, s); goto retry_lock; } owner = old; }Quoted 7 lines
Quoted 3 lines
The compiler rearranges things correctly (at least on ARM), and avoids the redundant test.
My latest concern remains: Is all this additional complexity, are all these conditional branches and the text size increase worth the effort? How much cycles or micoseconds would be gain on the most suffering architecture?
Since the else and the while are folded together and the loop becomes post-tested, and with the help of conditional instructions, I do not think there is much more conditional branch (we need to to jump over the while loop anyway). However avoiding the cmpxchg is a real gain, since it is implemented as irq save/irq restore on an old arm.
if(test_claimed(owner)) { old = xnarch_atomic_intptr_get(mutex->owner); goto test_old; } while(!test_claimed(owner)) { old = xnarch_atomic_intptr_cmpxchg(mutex->owner, owner, set_claimed(owner, 1)); if (likely(old == owner)) break; test_old:
Quoted 8 lines
if (old == NULL) {
/* Owner called fast mutex_unlock
(on another cpu) */
xnlock_put_irqrestore(&nklock, s);
goto retry_lock;
}
owner = old;
}Should work now, but the beautifulness of the code suffers a bit...
What about making the expected compiler optimizations explicit?
if (test_claimed(owner)) {
old = xnarch_atomic_intptr_get(mutex->owner);
goto test_no_owner;
}
do {
old = xnarch_atomic_intptr_cmpxchg(mutex->owner, owner,
set_claimed(owner, 1));
if (likely(old == owner))
break;
test_no_owner:
if (old == NULL) {
/* Owner called fast mutex_unlock
(on another cpu) */
xnlock_put_irqrestore(&nklock, s);
goto retry_lock;
}
owner = old;
} while (!test_claimed(owner));Jan
Quoted 77 lines · Jan Kiszka
Quoted 67 lines · Gilles Chanteperdrix
Quoted 19 lines · Jan Kiszka
Quoted 18 lines · Gilles Chanteperdrix
Quoted 17 lines · Jan Kiszka
Quoted 7 lines · Gilles Chanteperdrix
Quoted 2 lines · Jan Kiszka
OTOH, we can safe some text size which is precious as well. So I'm convinced to go your way (with a modification):
My approach sucks: we get a silly
atomic_cmpxchgif the mutex is already claimed, which is as least as much a common case as a currently unclaimed mutex. Need to think a bit. But I think a good solution is to re-read only if the mutex has been seen as already claimed.That makes no difference as then we will go through the cmpxchg path anyway.
There is _no_ way around re-reading under nklock, all we can avoid is atomic cmpxchg in the case of >1 waiters. But that would come at the price of more complexity for all waiter.
However, let's find some solution. I bet things will look different again when we start fiddling with a generic lock + the additional bit to replace XNWAKEN.
Quoted 8 lines
Quoted 6 lines
What I meant is that if the claimed bit is already set, we can avoid the cmpxchg altogether, which was the intent of the original code. So I propose the following version:
if(test_claimed(owner)) owner = xnarch_atomic_intptr_get(mutex->owner);This version lacks a test for owner == 0 here, otherwise you re-create my old bug that bite me today.
Ok, so jumping in the middle of the loop is the best thing to do then.
Quoted 14 lines
Quoted 14 lines
while(!test_claimed(owner)) { old = xnarch_atomic_intptr_cmpxchg(mutex->owner, owner, set_claimed(owner, 1)); if (likely(old == owner)) break; if (old == NULL) { /* Owner called fast mutex_unlock (on another cpu) */ xnlock_put_irqrestore(&nklock, s); goto retry_lock; } owner = old; }Quoted 7 lines
Quoted 3 lines
The compiler rearranges things correctly (at least on ARM), and avoids the redundant test.
My latest concern remains: Is all this additional complexity, are all these conditional branches and the text size increase worth the effort? How much cycles or micoseconds would be gain on the most suffering architecture?
Since the else and the while are folded together and the loop becomes post-tested, and with the help of conditional instructions, I do not think there is much more conditional branch (we need to to jump over the while loop anyway). However avoiding the cmpxchg is a real gain, since it is implemented as irq save/irq restore on an old arm.
if(test_claimed(owner)) { old = xnarch_atomic_intptr_get(mutex->owner); goto test_old; } while(!test_claimed(owner)) { old = xnarch_atomic_intptr_cmpxchg(mutex->owner, owner, set_claimed(owner, 1)); if (likely(old == owner)) break; test_old:Quoted 8 lines
if (old == NULL) { /* Owner called fast mutex_unlock (on another cpu) */ xnlock_put_irqrestore(&nklock, s); goto retry_lock; } owner = old; }
Quoted 3 lines
Should work now, but the beautifulness of the code suffers a bit...
What about making the expected compiler optimizations explicit?
Yes, of course.
Signature · 1 line
Gilles.
PATCH 1/1 POSIX: Fix race when setting claimed bit - v2
In reply to Jan Kiszka[ The refactored version according to our discussion. ]
This patch fixes the race when the claimed bit changes between the lock less check an the entry of the nklock-protected section.
--- ksrc/skins/posix/mutex.h | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)
ksrc/skins/posix/mutex.h +7 -2
@@ -134,11 +134,16 @@ static inline int pse51_mutex_timedlock_/* Set bit 0, so that mutex_unlock will know that the mutex is claimed. Hold the nklock, for mutual exclusion with slow mutex_unlock. */ xnlock_get_irqsave (&nklock, s); -while (!test_claimed (owner)) { +if (test_claimed (owner)) { + old =xnarch_atomic_intptr_get (mutex->owner); + goto test_no_owner; + } + do { old = xnarch_atomic_intptr_cmpxchg(mutex->owner, owner, set_claimed(owner, 1)); if (likely(old == owner)) break; + test_no_owner: if (old == NULL) { /* Owner called fast mutex_unlock (on another cpu) */
@@ -146,7 +151,7 @@ static inline int pse51_mutex_timedlock_ goto retry_lock; } owner = old; - } + }while (!test_claimed (owner));xnsynch_set_owner (&mutex->synchbase,clear_claimed (owner)); ++mutex->sleepers;
Gilles Chanteperdrix
AppliedRe: [Xenomai-core] [PATCH] POSIX: Fix race when setting claimed bit - v2
Quoted 4 lines · Jan Kiszka
[ The refactored version according to our discussion. ]
This patch fixes the race when the claimed bit changes between the lock less check an the entry of the nklock-protected section.
Applied, thanks.
Signature · 1 line
Gilles.