Skip to contentSkip to conversation

asm-generic: cmpxchg: implement dummy cmpxchg64_relaxed operation

RFC PATCH In review Will Deacon 2 participants 6 messages Sep 26, 2013

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

RFC PATCH 1/4 asm-generic: cmpxchg: implement dummy cmpxchg64_relaxed operation

cmpxchg64_relaxed can be used to provide barrier-less semantics for a 64-bit cmpxchg operation in cases where the strong memory ordering is not required.

This patch implements a dummy implementation for asm-generic, falling back to the usual cmpxchg64 code.

---
 include/asm-generic/cmpxchg.h | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
include/asm-generic/cmpxchg.h +3 -2
index 811fb1e..298b9d4 100644
@@ -102,7 +102,8 @@ unsigned long __xchg(unsigned long x, volatile void *ptr, int size)
 #define cmpxchg64_local(ptr, o, n) __cmpxchg64_local_generic((ptr), (o), (n))
 #endif
 
-#define cmpxchg(ptr, o, n)	cmpxchg_local((ptr), (o), (n))
-#define cmpxchg64(ptr, o, n)	cmpxchg64_local((ptr), (o), (n))
+#define cmpxchg(ptr, o, n)		cmpxchg_local((ptr), (o), (n))
+#define cmpxchg64(ptr, o, n)		cmpxchg64_local((ptr), (o), (n))
+#define cmpxchg64_relaxed(ptr, o, n)	cmpxchg64((ptr), (o), (n))
 
 #endif /* __ASM_GENERIC_CMPXCHG_H */
Signature · 1 line
1.8.2.2
#1 lore raw

RFC PATCH 2/4 x86: cmpxchg: implement dummy cmpxchg64_relaxed operation

cmpxchg64_relaxed can be used to provide barrier-less semantics for a 64-bit cmpxchg operation in cases where the strong memory ordering is not required. A useful use-case for this is in the recently merged lockless lockref code.

This patch implements a dummy implementation for x86, since the memory ordering issues aren't a concern for this architecture.

---
 arch/x86/include/asm/cmpxchg.h | 3 +++
 1 file changed, 3 insertions(+)
arch/x86/include/asm/cmpxchg.h +3 -0
index d47786a..aacb99a0 100644
@@ -152,6 +152,9 @@ extern void __add_wrong_size(void)
 
 #define cmpxchg_local(ptr, old, new)					\
 	__cmpxchg_local(ptr, old, new, sizeof(*(ptr)))
+
+#define cmpxchg64_relaxed(ptr, old, new)				\
+	cmpxchg64(ptr, old, new)
 #endif
 
 /*
Signature · 1 line
1.8.2.2
#2 lore raw

RFC PATCH 3/4 ia64: cmpxchg: implement dummy cmpxchg64_relaxed operation

In reply to Will Deacon

cmpxchg64_relaxed can be used to provide barrier-less semantics for a 64-bit cmpxchg operation in cases where the strong memory ordering is not required. A useful use-case for this is in the recently merged lockless lockref code.

This patch implements a dummy implementation for ia64, which could probably be improved by removing the half barrier associated with the default cmpxchg64 macro.

---
 arch/ia64/include/uapi/asm/cmpxchg.h | 1 +
 1 file changed, 1 insertion(+)
arch/ia64/include/uapi/asm/cmpxchg.h +1 -0
index 4f37dbb..8984b6e 100644
@@ -124,6 +124,7 @@ extern long ia64_cmpxchg_called_with_bad_pointer(void);
 
 #define cmpxchg_local		cmpxchg
 #define cmpxchg64_local		cmpxchg64
+#define cmpxchg64_relaxed	cmpxchg64
 
 #ifdef CONFIG_IA64_DEBUG_CMPXCHG
 # define CMPXCHG_BUGCHECK_DECL	int _cmpxchg_bugcheck_count = 128;
Signature · 1 line
1.8.2.2
#3 lore raw

RFC PATCH 4/4 lib: lockref: use relaxed cmpxchg64 variant for lockless updates

In reply to Will Deacon

The 64-bit cmpxchg operation on the lockref is ordered by virtue of hazarding between the cmpxchg operation and the reference count manipulation. On weakly ordered memory architectures (such as ARM), it can be of great benefit to omit the barrier instructions where they are not needed.

This patch moves the lockless lockref code over to the new cmpxchg64_relaxed operation, which doesn't provide barrier semantics.

---

So here's a quick stab at allowing the memory barrier semantics to be avoided on weakly ordered architectures. This helps ARM, but it would be interesting to see if ia64 gets a boost too (although I've not relaxed their cmpxchg because there is uapi stuff involved that I wasn't comfortable refactoring).

 lib/lockref.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
lib/lockref.c +3 -2
index 677d036..6d896ab 100644
@@ -14,8 +14,9 @@
 	while (likely(arch_spin_value_unlocked(old.lock.rlock.raw_lock))) {  	\
 		struct lockref new = old, prev = old;				\
 		CODE								\
-		old.lock_count = cmpxchg64(&lockref->lock_count,		\
-					   old.lock_count, new.lock_count);	\
+		old.lock_count = cmpxchg64_relaxed(&lockref->lock_count,	\
+						   old.lock_count,		\
+						   new.lock_count);		\
 		if (likely(old.lock_count == prev.lock_count)) {		\
 			SUCCESS;						\
 		}								\
Signature · 1 line
1.8.2.2
#4 lore raw
In reply to Will Deacon
Quoted 3 lines · Will Deacon

This patch implements a dummy implementation for asm-generic, falling back to the usual cmpxchg64 code.

I don't like the "let's add dummy operations for everybody who doesn't care" when it is this specialized.

I'd much rather just add a single

   #ifndef cmpxchg64_relaxed
   # define cmpxchg64_relaxed cmpxchg64
   #endif

to the LOCKREF code, and then ARM (and others) can define it as they wish.

And *if* anybody else ever realizes that they want this outside of the lockref code, let's look at doing that then. Right now I don't know of any users, and I'd be leery of people using this willy-nilly, because very few people really understand memory ordering.

Linus

#5 lore raw
Quoted 8 lines · Linus Torvalds
Quoted 3 lines · Will Deacon

This patch implements a dummy implementation for asm-generic, falling back to the usual cmpxchg64 code.

I don't like the "let's add dummy operations for everybody who doesn't care" when it is this specialized.

Quoted 7 lines

I'd much rather just add a single

   #ifndef cmpxchg64_relaxed
   # define cmpxchg64_relaxed cmpxchg64
   #endif

to the LOCKREF code, and then ARM (and others) can define it as they wish.

Okey doke.

Quoted 4 lines

And *if* anybody else ever realizes that they want this outside of the lockref code, let's look at doing that then. Right now I don't know of any users, and I'd be leery of people using this willy-nilly, because very few people really understand memory ordering.

Agreed, and I really doubt there are many cases where the cmpxchg hazarding works out nicely so as to obviate the need for memory barriers.

I'll send a revised patch.

Cheers,

Will