asm-generic: cmpxchg: implement dummy cmpxchg64_relaxed operation
Subsystems (3)
RFC PATCH 1/4 asm-generic: cmpxchg: implement dummy cmpxchg64_relaxed operation
cmpxchg64_relaxed can be used to provide barrier-less semantics for a 64-bit cmpxchg operation in cases where the strong memory ordering is not required.
This patch implements a dummy implementation for asm-generic, falling back to the usual cmpxchg64 code.
--- include/asm-generic/cmpxchg.h | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
include/asm-generic/cmpxchg.h +3 -2
@@ -102,7 +102,8 @@ unsigned long __xchg(unsigned long x, volatile void *ptr, int size)#define cmpxchg64_local (ptr, o, n) __cmpxchg64_local_generic((ptr), (o), (n))#endif -#define cmpxchg (ptr, o, n) cmpxchg_local((ptr), (o), (n)) -#define cmpxchg64 (ptr, o, n) cmpxchg64_local((ptr), (o), (n)) +#define cmpxchg (ptr, o, n) cmpxchg_local((ptr), (o), (n)) +#define cmpxchg64 (ptr, o, n) cmpxchg64_local((ptr), (o), (n)) +#define cmpxchg64_relaxed (ptr, o, n) cmpxchg64((ptr), (o), (n))#endif /* __ASM_GENERIC_CMPXCHG_H */
Signature · 1 line
1.8.2.2
RFC PATCH 2/4 x86: cmpxchg: implement dummy cmpxchg64_relaxed operation
cmpxchg64_relaxed can be used to provide barrier-less semantics for a 64-bit cmpxchg operation in cases where the strong memory ordering is not required. A useful use-case for this is in the recently merged lockless lockref code.
This patch implements a dummy implementation for x86, since the memory ordering issues aren't a concern for this architecture.
--- arch/x86/include/asm/cmpxchg.h | 3 +++ 1 file changed, 3 insertions(+)
arch/x86/include/asm/cmpxchg.h +3 -0
@@ -152,6 +152,9 @@ extern void __add_wrong_size(void)#define cmpxchg_local (ptr, old, new) \ __cmpxchg_local(ptr, old, new, sizeof(*(ptr))) + +#define cmpxchg64_relaxed (ptr, old, new) \ + cmpxchg64(ptr, old, new)#endif /*
Signature · 1 line
1.8.2.2
RFC PATCH 3/4 ia64: cmpxchg: implement dummy cmpxchg64_relaxed operation
In reply to Will Deaconcmpxchg64_relaxed can be used to provide barrier-less semantics for a 64-bit cmpxchg operation in cases where the strong memory ordering is not required. A useful use-case for this is in the recently merged lockless lockref code.
This patch implements a dummy implementation for ia64, which could probably be improved by removing the half barrier associated with the default cmpxchg64 macro.
--- arch/ia64/include/uapi/asm/cmpxchg.h | 1 + 1 file changed, 1 insertion(+)
arch/ia64/include/uapi/asm/cmpxchg.h +1 -0
@@ -124,6 +124,7 @@ extern long ia64_cmpxchg_called_with_bad_pointer(void);#define cmpxchg_local cmpxchg#define cmpxchg64_local cmpxchg64 +#define cmpxchg64_relaxed cmpxchg64#ifdef CONFIG_IA64_DEBUG_CMPXCHG # define CMPXCHG_BUGCHECK_DECL int _cmpxchg_bugcheck_count = 128;
Signature · 1 line
1.8.2.2
RFC PATCH 4/4 lib: lockref: use relaxed cmpxchg64 variant for lockless updates
In reply to Will DeaconThe 64-bit cmpxchg operation on the lockref is ordered by virtue of hazarding between the cmpxchg operation and the reference count manipulation. On weakly ordered memory architectures (such as ARM), it can be of great benefit to omit the barrier instructions where they are not needed.
This patch moves the lockless lockref code over to the new cmpxchg64_relaxed operation, which doesn't provide barrier semantics.
---
So here's a quick stab at allowing the memory barrier semantics to be avoided on weakly ordered architectures. This helps ARM, but it would be interesting to see if ia64 gets a boost too (although I've not relaxed their cmpxchg because there is uapi stuff involved that I wasn't comfortable refactoring).
lib/lockref.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
lib/lockref.c +3 -2
@@ -14,8 +14,9 @@while (likely (arch_spin_value_unlocked (old.lock.rlock.raw_lock))) { \struct lockref new = old, prev = old; \CODE \ - old.lock_count = cmpxchg64(&lockref->lock_count, \ - old.lock_count, new.lock_count); \ + old.lock_count = cmpxchg64_relaxed(&lockref->lock_count, \ + old.lock_count, \ + new.lock_count); \if (likely (old.lock_count == prev.lock_count)) { \SUCCESS ; \ } \
Signature · 1 line
1.8.2.2
Quoted 3 lines · Will Deacon
This patch implements a dummy implementation for asm-generic, falling back to the usual cmpxchg64 code.
I don't like the "let's add dummy operations for everybody who doesn't care" when it is this specialized.
I'd much rather just add a single
#ifndef cmpxchg64_relaxed # define cmpxchg64_relaxed cmpxchg64 #endif
to the LOCKREF code, and then ARM (and others) can define it as they wish.
And *if* anybody else ever realizes that they want this outside of the lockref code, let's look at doing that then. Right now I don't know of any users, and I'd be leery of people using this willy-nilly, because very few people really understand memory ordering.
Linus
Quoted 8 lines · Linus Torvalds
Quoted 3 lines · Will Deacon
This patch implements a dummy implementation for asm-generic, falling back to the usual cmpxchg64 code.
I don't like the "let's add dummy operations for everybody who doesn't care" when it is this specialized.
Quoted 7 lines
I'd much rather just add a single
#ifndef cmpxchg64_relaxed # define cmpxchg64_relaxed cmpxchg64 #endifto the LOCKREF code, and then ARM (and others) can define it as they wish.
Okey doke.
Quoted 4 lines
And *if* anybody else ever realizes that they want this outside of the lockref code, let's look at doing that then. Right now I don't know of any users, and I'd be leery of people using this willy-nilly, because very few people really understand memory ordering.
Agreed, and I really doubt there are many cases where the cmpxchg hazarding works out nicely so as to obviate the need for memory barriers.
I'll send a revised patch.
Cheers,
Will