Skip to contentSkip to conversation

Rework & improve riscv cmpxchg.h and atomic.h

RFC PATCH In review Leonardo Bras 8 participants 26 messages Aug 10, 2023 – Jan 3, 2024

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

v5 cover Rework & improve riscv cmpxchg.h and atomic.h

While studying riscv's cmpxchg.h file, I got really interested in understanding how RISCV asm implemented the different versions of {cmp,}xchg.

When I understood the pattern, it made sense for me to remove the duplications and create macros to make it easier to understand what exactly changes between the versions: Instruction sufixes & barriers.

Also, did the same kind of work on atomic.c.

After that, I noted both cmpxchg and xchg only accept variables of size 4 and 8, compared to x86 and arm64 which do 1,2,4,8.

Now that deduplication is done, it is quite direct to implement them for variable sizes 1 and 2, so I did it. Then Guo Ren already presented me some possible users :)

I did compare the generated asm on a test.c that contained usage for every changed function, and could not detect any change on patches 1 + 2 + 3 compared with upstream.

Pathes 4 & 5 were compiled-tested, merged with guoren/qspinlock_v11 and booted just fine with qemu -machine virt -append "qspinlock".

(tree: https://gitlab.com/LeoBras/linux/-/commits/guo_qspinlock_v11)

Thanks!
Leo

Changes since squashed cmpxchg RFCv4:
- Added (__typeof__(*(p))) before returning from {cmp,}xchg, as done in current upstream, (possibly) fixing the bug from kernel test robot
https://lore.kernel.org/all/[email protected]/

Changes since squashed cmpxchg RFCv3:
- Fixed bug on cmpxchg macro for var size 1 & 2: now working
- Macros for var size 1 & 2's lr.w and sc.w now are guaranteed to receive input of a 32-bit aligned address
- Renamed internal macros from _mask to _masked for patches 4 & 5
- __rc variable on macros for var size 1 & 2 changed from register to ulong
https://lore.kernel.org/all/[email protected]/

Changes since squashed cmpxchg RFCv2:
- Removed rc parameter from the new macro: it can be internal to the macro
- 2 new patches: cmpxchg size 1 and 2, xchg size 1 and 2
https://lore.kernel.org/all/[email protected]/

Changes since squashed cmpxchg RFCv1:
- Unified with atomic.c patchset
- Rebased on top of torvalds/master (thanks Andrea Parri!)
- Removed helper macros that were not being used elsewhere in the kernel.
https://lore.kernel.org/all/[email protected]/
https://lore.kernel.org/all/[email protected]/

Changes since (cmpxchg) RFCv3:
- Squashed the 6 original patches in 2: one for cmpxchg and one for xchg
https://lore.kernel.org/all/[email protected]/

Changes since (cmpxchg) RFCv2:
- Fixed macros that depend on having a local variable with a magic name
- Previous cast to (long) is now only applied on 4-bytes cmpxchg
https://lore.kernel.org/all/[email protected]/

Changes since (cmpxchg) RFCv1:
- Fixed patch 4/6 suffix from 'w.aqrl' to '.w.aqrl', to avoid build error
https://lore.kernel.org/all/[email protected]/

Leonardo Bras (5):
riscv/cmpxchg: Deduplicate xchg() asm functions riscv/cmpxchg: Deduplicate cmpxchg() asm and macros riscv/atomic.h : Deduplicate arch_atomic.* riscv/cmpxchg: Implement cmpxchg for variables of size 1 and 2 riscv/cmpxchg: Implement xchg for variables of size 1 and 2

 arch/riscv/include/asm/atomic.h  | 164 ++++++-------
 arch/riscv/include/asm/cmpxchg.h | 404 ++++++++++---------------------
 2 files changed, 200 insertions(+), 368 deletions(-)

base-commit: cacc6e22932f373a91d7be55a9b992dc77f4c59b

Signature · 7 lines
2.41.0


_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
#1 lore raw

RFC PATCH v5 1/5 riscv/cmpxchg: Deduplicate xchg() asm functions

In this header every xchg define (_relaxed, _acquire, _release, vanilla) contain it's own asm file, both for 4-byte variables an 8-byte variables, on a total of 8 versions of mostly the same asm.

This is usually bad, as it means any change may be done in up to 8 different places.

Unify those versions by creating a new define with enough parameters to generate any version of the previous 8.

Then unify the result under a more general define, and simplify arch_xchg* generation.

(This did not cause any change in generated asm)

---
 arch/riscv/include/asm/cmpxchg.h | 138 ++++++-------------------------
 1 file changed, 23 insertions(+), 115 deletions(-)
arch/riscv/include/asm/cmpxchg.h +23 -115
index 2f4726d3cfcc..48478a8eecee 100644
@@ -11,140 +11,48 @@
 #include <asm/barrier.h>
 #include <asm/fence.h>
 
-#define __xchg_relaxed(ptr, new, size)					\
+#define __arch_xchg(sfx, prepend, append, r, p, n)			\
 ({									\
-	__typeof__(ptr) __ptr = (ptr);					\
-	__typeof__(new) __new = (new);					\
-	__typeof__(*(ptr)) __ret;					\
-	switch (size) {							\
-	case 4:								\
-		__asm__ __volatile__ (					\
-			"	amoswap.w %0, %2, %1\n"			\
-			: "=r" (__ret), "+A" (*__ptr)			\
-			: "r" (__new)					\
-			: "memory");					\
-		break;							\
-	case 8:								\
-		__asm__ __volatile__ (					\
-			"	amoswap.d %0, %2, %1\n"			\
-			: "=r" (__ret), "+A" (*__ptr)			\
-			: "r" (__new)					\
-			: "memory");					\
-		break;							\
-	default:							\
-		BUILD_BUG();						\
-	}								\
-	__ret;								\
-})
-
-#define arch_xchg_relaxed(ptr, x)					\
-({									\
-	__typeof__(*(ptr)) _x_ = (x);					\
-	(__typeof__(*(ptr))) __xchg_relaxed((ptr),			\
-					    _x_, sizeof(*(ptr)));	\
+	__asm__ __volatile__ (						\
+		prepend							\
+		"	amoswap" sfx " %0, %2, %1\n"			\
+		append							\
+		: "=r" (r), "+A" (*(p))					\
+		: "r" (n)						\
+		: "memory");						\
 })
 
-#define __xchg_acquire(ptr, new, size)					\
+#define _arch_xchg(ptr, new, sfx, prepend, append)			\
 ({									\
 	__typeof__(ptr) __ptr = (ptr);					\
-	__typeof__(new) __new = (new);					\
-	__typeof__(*(ptr)) __ret;					\
-	switch (size) {							\
+	__typeof__(*(__ptr)) __new = (new);				\
+	__typeof__(*(__ptr)) __ret;					\
+	switch (sizeof(*__ptr)) {					\
 	case 4:								\
-		__asm__ __volatile__ (					\
-			"	amoswap.w %0, %2, %1\n"			\
-			RISCV_ACQUIRE_BARRIER				\
-			: "=r" (__ret), "+A" (*__ptr)			\
-			: "r" (__new)					\
-			: "memory");					\
+		__arch_xchg(".w" sfx, prepend, append,			\
+			      __ret, __ptr, __new);			\
 		break;							\
 	case 8:								\
-		__asm__ __volatile__ (					\
-			"	amoswap.d %0, %2, %1\n"			\
-			RISCV_ACQUIRE_BARRIER				\
-			: "=r" (__ret), "+A" (*__ptr)			\
-			: "r" (__new)					\
-			: "memory");					\
+		__arch_xchg(".d" sfx, prepend, append,			\
+			      __ret, __ptr, __new);			\
 		break;							\
 	default:							\
 		BUILD_BUG();						\
 	}								\
-	__ret;								\
+	(__typeof__(*(__ptr)))__ret;					\
 })
 
-#define arch_xchg_acquire(ptr, x)					\
-({									\
-	__typeof__(*(ptr)) _x_ = (x);					\
-	(__typeof__(*(ptr))) __xchg_acquire((ptr),			\
-					    _x_, sizeof(*(ptr)));	\
-})
+#define arch_xchg_relaxed(ptr, x)					\
+	_arch_xchg(ptr, x, "", "", "")
 
-#define __xchg_release(ptr, new, size)					\
-({									\
-	__typeof__(ptr) __ptr = (ptr);					\
-	__typeof__(new) __new = (new);					\
-	__typeof__(*(ptr)) __ret;					\
-	switch (size) {							\
-	case 4:								\
-		__asm__ __volatile__ (					\
-			RISCV_RELEASE_BARRIER				\
-			"	amoswap.w %0, %2, %1\n"			\
-			: "=r" (__ret), "+A" (*__ptr)			\
-			: "r" (__new)					\
-			: "memory");					\
-		break;							\
-	case 8:								\
-		__asm__ __volatile__ (					\
-			RISCV_RELEASE_BARRIER				\
-			"	amoswap.d %0, %2, %1\n"			\
-			: "=r" (__ret), "+A" (*__ptr)			\
-			: "r" (__new)					\
-			: "memory");					\
-		break;							\
-	default:							\
-		BUILD_BUG();						\
-	}								\
-	__ret;								\
-})
+#define arch_xchg_acquire(ptr, x)					\
+	_arch_xchg(ptr, x, "", "", RISCV_ACQUIRE_BARRIER)
 
 #define arch_xchg_release(ptr, x)					\
-({									\
-	__typeof__(*(ptr)) _x_ = (x);					\
-	(__typeof__(*(ptr))) __xchg_release((ptr),			\
-					    _x_, sizeof(*(ptr)));	\
-})
-
-#define __arch_xchg(ptr, new, size)					\
-({									\
-	__typeof__(ptr) __ptr = (ptr);					\
-	__typeof__(new) __new = (new);					\
-	__typeof__(*(ptr)) __ret;					\
-	switch (size) {							\
-	case 4:								\
-		__asm__ __volatile__ (					\
-			"	amoswap.w.aqrl %0, %2, %1\n"		\
-			: "=r" (__ret), "+A" (*__ptr)			\
-			: "r" (__new)					\
-			: "memory");					\
-		break;							\
-	case 8:								\
-		__asm__ __volatile__ (					\
-			"	amoswap.d.aqrl %0, %2, %1\n"		\
-			: "=r" (__ret), "+A" (*__ptr)			\
-			: "r" (__new)					\
-			: "memory");					\
-		break;							\
-	default:							\
-		BUILD_BUG();						\
-	}								\
-	__ret;								\
-})
+	_arch_xchg(ptr, x, "", RISCV_RELEASE_BARRIER, "")
 
 #define arch_xchg(ptr, x)						\
-({									\
-	__typeof__(*(ptr)) _x_ = (x);					\
-	(__typeof__(*(ptr))) __arch_xchg((ptr), _x_, sizeof(*(ptr)));	\
-})
+	_arch_xchg(ptr, x, ".aqrl", "", "")
 
 #define xchg32(ptr, x)							\
 ({									\
Signature · 7 lines
2.41.0


_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
#2 lore raw

RFC PATCH v5 2/5 riscv/cmpxchg: Deduplicate cmpxchg() asm and macros

In reply to Leonardo Bras

In this header every cmpxchg define (_relaxed, _acquire, _release, vanilla) contain it's own asm file, both for 4-byte variables an 8-byte variables, on a total of 8 versions of mostly the same asm.

This is usually bad, as it means any change may be done in up to 8 different places.

Unify those versions by creating a new define with enough parameters to generate any version of the previous 8.

Then unify the result under a more general define, and simplify arch_cmpxchg* generation

(This did not cause any change in generated asm)

---
 arch/riscv/include/asm/cmpxchg.h | 195 ++++++-------------------------
 1 file changed, 33 insertions(+), 162 deletions(-)
arch/riscv/include/asm/cmpxchg.h +33 -162
index 48478a8eecee..e3e0ac7ba061 100644
@@ -71,190 +71,61 @@
  * store NEW in MEM.  Return the initial value in MEM.  Success is
  * indicated by comparing RETURN with OLD.
  */
-#define __cmpxchg_relaxed(ptr, old, new, size)				\
-({									\
-	__typeof__(ptr) __ptr = (ptr);					\
-	__typeof__(*(ptr)) __old = (old);				\
-	__typeof__(*(ptr)) __new = (new);				\
-	__typeof__(*(ptr)) __ret;					\
-	register unsigned int __rc;					\
-	switch (size) {							\
-	case 4:								\
-		__asm__ __volatile__ (					\
-			"0:	lr.w %0, %2\n"				\
-			"	bne  %0, %z3, 1f\n"			\
-			"	sc.w %1, %z4, %2\n"			\
-			"	bnez %1, 0b\n"				\
-			"1:\n"						\
-			: "=&r" (__ret), "=&r" (__rc), "+A" (*__ptr)	\
-			: "rJ" ((long)__old), "rJ" (__new)		\
-			: "memory");					\
-		break;							\
-	case 8:								\
-		__asm__ __volatile__ (					\
-			"0:	lr.d %0, %2\n"				\
-			"	bne %0, %z3, 1f\n"			\
-			"	sc.d %1, %z4, %2\n"			\
-			"	bnez %1, 0b\n"				\
-			"1:\n"						\
-			: "=&r" (__ret), "=&r" (__rc), "+A" (*__ptr)	\
-			: "rJ" (__old), "rJ" (__new)			\
-			: "memory");					\
-		break;							\
-	default:							\
-		BUILD_BUG();						\
-	}								\
-	__ret;								\
-})
 
-#define arch_cmpxchg_relaxed(ptr, o, n)					\
-({									\
-	__typeof__(*(ptr)) _o_ = (o);					\
-	__typeof__(*(ptr)) _n_ = (n);					\
-	(__typeof__(*(ptr))) __cmpxchg_relaxed((ptr),			\
-					_o_, _n_, sizeof(*(ptr)));	\
-})
 
-#define __cmpxchg_acquire(ptr, old, new, size)				\
+#define __arch_cmpxchg(lr_sfx, sc_sfx, prepend, append, r, p, co, o, n)	\
 ({									\
-	__typeof__(ptr) __ptr = (ptr);					\
-	__typeof__(*(ptr)) __old = (old);				\
-	__typeof__(*(ptr)) __new = (new);				\
-	__typeof__(*(ptr)) __ret;					\
 	register unsigned int __rc;					\
-	switch (size) {							\
-	case 4:								\
-		__asm__ __volatile__ (					\
-			"0:	lr.w %0, %2\n"				\
-			"	bne  %0, %z3, 1f\n"			\
-			"	sc.w %1, %z4, %2\n"			\
-			"	bnez %1, 0b\n"				\
-			RISCV_ACQUIRE_BARRIER				\
-			"1:\n"						\
-			: "=&r" (__ret), "=&r" (__rc), "+A" (*__ptr)	\
-			: "rJ" ((long)__old), "rJ" (__new)		\
-			: "memory");					\
-		break;							\
-	case 8:								\
-		__asm__ __volatile__ (					\
-			"0:	lr.d %0, %2\n"				\
-			"	bne %0, %z3, 1f\n"			\
-			"	sc.d %1, %z4, %2\n"			\
-			"	bnez %1, 0b\n"				\
-			RISCV_ACQUIRE_BARRIER				\
-			"1:\n"						\
-			: "=&r" (__ret), "=&r" (__rc), "+A" (*__ptr)	\
-			: "rJ" (__old), "rJ" (__new)			\
-			: "memory");					\
-		break;							\
-	default:							\
-		BUILD_BUG();						\
-	}								\
-	__ret;								\
-})
-
-#define arch_cmpxchg_acquire(ptr, o, n)					\
-({									\
-	__typeof__(*(ptr)) _o_ = (o);					\
-	__typeof__(*(ptr)) _n_ = (n);					\
-	(__typeof__(*(ptr))) __cmpxchg_acquire((ptr),			\
-					_o_, _n_, sizeof(*(ptr)));	\
+									\
+	__asm__ __volatile__ (						\
+		prepend							\
+		"0:	lr" lr_sfx " %0, %2\n"				\
+		"	bne  %0, %z3, 1f\n"				\
+		"	sc" sc_sfx " %1, %z4, %2\n"			\
+		"	bnez %1, 0b\n"					\
+		append							\
+		"1:\n"							\
+		: "=&r" (r), "=&r" (__rc), "+A" (*(p))			\
+		: "rJ" (co o), "rJ" (n)					\
+		: "memory");						\
 })
 
-#define __cmpxchg_release(ptr, old, new, size)				\
+#define _arch_cmpxchg(ptr, old, new, sc_sfx, prepend, append)		\
 ({									\
 	__typeof__(ptr) __ptr = (ptr);					\
-	__typeof__(*(ptr)) __old = (old);				\
-	__typeof__(*(ptr)) __new = (new);				\
-	__typeof__(*(ptr)) __ret;					\
-	register unsigned int __rc;					\
-	switch (size) {							\
+	__typeof__(*(__ptr)) __old = (old);				\
+	__typeof__(*(__ptr)) __new = (new);				\
+	__typeof__(*(__ptr)) __ret;					\
+									\
+	switch (sizeof(*__ptr)) {					\
 	case 4:								\
-		__asm__ __volatile__ (					\
-			RISCV_RELEASE_BARRIER				\
-			"0:	lr.w %0, %2\n"				\
-			"	bne  %0, %z3, 1f\n"			\
-			"	sc.w %1, %z4, %2\n"			\
-			"	bnez %1, 0b\n"				\
-			"1:\n"						\
-			: "=&r" (__ret), "=&r" (__rc), "+A" (*__ptr)	\
-			: "rJ" ((long)__old), "rJ" (__new)		\
-			: "memory");					\
+		__arch_cmpxchg(".w", ".w" sc_sfx, prepend, append,	\
+				__ret, __ptr, (long), __old, __new);	\
 		break;							\
 	case 8:								\
-		__asm__ __volatile__ (					\
-			RISCV_RELEASE_BARRIER				\
-			"0:	lr.d %0, %2\n"				\
-			"	bne %0, %z3, 1f\n"			\
-			"	sc.d %1, %z4, %2\n"			\
-			"	bnez %1, 0b\n"				\
-			"1:\n"						\
-			: "=&r" (__ret), "=&r" (__rc), "+A" (*__ptr)	\
-			: "rJ" (__old), "rJ" (__new)			\
-			: "memory");					\
+		__arch_cmpxchg(".d", ".d" sc_sfx, prepend, append,	\
+				__ret, __ptr, /**/, __old, __new);	\
 		break;							\
 	default:							\
 		BUILD_BUG();						\
 	}								\
-	__ret;								\
+	(__typeof__(*(__ptr)))__ret;					\
 })
 
-#define arch_cmpxchg_release(ptr, o, n)					\
-({									\
-	__typeof__(*(ptr)) _o_ = (o);					\
-	__typeof__(*(ptr)) _n_ = (n);					\
-	(__typeof__(*(ptr))) __cmpxchg_release((ptr),			\
-					_o_, _n_, sizeof(*(ptr)));	\
-})
+#define arch_cmpxchg_relaxed(ptr, o, n)					\
+	_arch_cmpxchg((ptr), (o), (n), "", "", "")
 
-#define __cmpxchg(ptr, old, new, size)					\
-({									\
-	__typeof__(ptr) __ptr = (ptr);					\
-	__typeof__(*(ptr)) __old = (old);				\
-	__typeof__(*(ptr)) __new = (new);				\
-	__typeof__(*(ptr)) __ret;					\
-	register unsigned int __rc;					\
-	switch (size) {							\
-	case 4:								\
-		__asm__ __volatile__ (					\
-			"0:	lr.w %0, %2\n"				\
-			"	bne  %0, %z3, 1f\n"			\
-			"	sc.w.rl %1, %z4, %2\n"			\
-			"	bnez %1, 0b\n"				\
-			"	fence rw, rw\n"				\
-			"1:\n"						\
-			: "=&r" (__ret), "=&r" (__rc), "+A" (*__ptr)	\
-			: "rJ" ((long)__old), "rJ" (__new)		\
-			: "memory");					\
-		break;							\
-	case 8:								\
-		__asm__ __volatile__ (					\
-			"0:	lr.d %0, %2\n"				\
-			"	bne %0, %z3, 1f\n"			\
-			"	sc.d.rl %1, %z4, %2\n"			\
-			"	bnez %1, 0b\n"				\
-			"	fence rw, rw\n"				\
-			"1:\n"						\
-			: "=&r" (__ret), "=&r" (__rc), "+A" (*__ptr)	\
-			: "rJ" (__old), "rJ" (__new)			\
-			: "memory");					\
-		break;							\
-	default:							\
-		BUILD_BUG();						\
-	}								\
-	__ret;								\
-})
+#define arch_cmpxchg_acquire(ptr, o, n)					\
+	_arch_cmpxchg((ptr), (o), (n), "", "", RISCV_ACQUIRE_BARRIER)
+
+#define arch_cmpxchg_release(ptr, o, n)					\
+	_arch_cmpxchg((ptr), (o), (n), "", RISCV_RELEASE_BARRIER, "")
 
 #define arch_cmpxchg(ptr, o, n)						\
-({									\
-	__typeof__(*(ptr)) _o_ = (o);					\
-	__typeof__(*(ptr)) _n_ = (n);					\
-	(__typeof__(*(ptr))) __cmpxchg((ptr),				\
-				       _o_, _n_, sizeof(*(ptr)));	\
-})
+	_arch_cmpxchg((ptr), (o), (n), ".rl", "", "	fence rw, rw\n")
 
 #define arch_cmpxchg_local(ptr, o, n)					\
-	(__cmpxchg_relaxed((ptr), (o), (n), sizeof(*(ptr))))
+	arch_cmpxchg_relaxed((ptr), (o), (n))
 
 #define arch_cmpxchg64(ptr, o, n)					\
 ({									\
Signature · 7 lines
2.41.0


_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
#3 lore raw

RFC PATCH v5 3/5 riscv/atomic.h : Deduplicate arch_atomic.*

In reply to Leonardo Bras

Some functions use mostly the same asm for 32-bit and 64-bit versions.

Make a macro that is generic enough and avoid code duplication.

(This did not cause any change in generated asm)

---
 arch/riscv/include/asm/atomic.h | 164 +++++++++++++++-----------------
 1 file changed, 76 insertions(+), 88 deletions(-)
arch/riscv/include/asm/atomic.h +76 -88
index f5dfef6c2153..80cca7ac16fd 100644
@@ -196,22 +196,28 @@ ATOMIC_OPS(xor, xor, i)
 #undef ATOMIC_FETCH_OP
 #undef ATOMIC_OP_RETURN
 
+#define _arch_atomic_fetch_add_unless(_prev, _rc, counter, _a, _u, sfx)	\
+({									\
+	__asm__ __volatile__ (						\
+		"0:	lr." sfx "     %[p],  %[c]\n"			\
+		"	beq	       %[p],  %[u], 1f\n"		\
+		"	add            %[rc], %[p], %[a]\n"		\
+		"	sc." sfx ".rl  %[rc], %[rc], %[c]\n"		\
+		"	bnez           %[rc], 0b\n"			\
+		"	fence          rw, rw\n"			\
+		"1:\n"							\
+		: [p]"=&r" (_prev), [rc]"=&r" (_rc), [c]"+A" (counter)	\
+		: [a]"r" (_a), [u]"r" (_u)				\
+		: "memory");						\
+})
+
 /* This is required to provide a full barrier on success. */
 static __always_inline int arch_atomic_fetch_add_unless(atomic_t *v, int a, int u)
 {
        int prev, rc;
 
-	__asm__ __volatile__ (
-		"0:	lr.w     %[p],  %[c]\n"
-		"	beq      %[p],  %[u], 1f\n"
-		"	add      %[rc], %[p], %[a]\n"
-		"	sc.w.rl  %[rc], %[rc], %[c]\n"
-		"	bnez     %[rc], 0b\n"
-		"	fence    rw, rw\n"
-		"1:\n"
-		: [p]"=&r" (prev), [rc]"=&r" (rc), [c]"+A" (v->counter)
-		: [a]"r" (a), [u]"r" (u)
-		: "memory");
+	_arch_atomic_fetch_add_unless(prev, rc, v->counter, a, u, "w");
+
 	return prev;
 }
 #define arch_atomic_fetch_add_unless arch_atomic_fetch_add_unless
@@ -222,77 +228,86 @@ static __always_inline s64 arch_atomic64_fetch_add_unless(atomic64_t *v, s64 a,
        s64 prev;
        long rc;
 
-	__asm__ __volatile__ (
-		"0:	lr.d     %[p],  %[c]\n"
-		"	beq      %[p],  %[u], 1f\n"
-		"	add      %[rc], %[p], %[a]\n"
-		"	sc.d.rl  %[rc], %[rc], %[c]\n"
-		"	bnez     %[rc], 0b\n"
-		"	fence    rw, rw\n"
-		"1:\n"
-		: [p]"=&r" (prev), [rc]"=&r" (rc), [c]"+A" (v->counter)
-		: [a]"r" (a), [u]"r" (u)
-		: "memory");
+	_arch_atomic_fetch_add_unless(prev, rc, v->counter, a, u, "d");
+
 	return prev;
 }
 #define arch_atomic64_fetch_add_unless arch_atomic64_fetch_add_unless
 #endif
 
+#define _arch_atomic_inc_unless_negative(_prev, _rc, counter, sfx)	\
+({									\
+	__asm__ __volatile__ (						\
+		"0:	lr." sfx "      %[p],  %[c]\n"			\
+		"	bltz            %[p],  1f\n"			\
+		"	addi            %[rc], %[p], 1\n"		\
+		"	sc." sfx ".rl   %[rc], %[rc], %[c]\n"		\
+		"	bnez            %[rc], 0b\n"			\
+		"	fence           rw, rw\n"			\
+		"1:\n"							\
+		: [p]"=&r" (_prev), [rc]"=&r" (_rc), [c]"+A" (counter)	\
+		:							\
+		: "memory");						\
+})
+
 static __always_inline bool arch_atomic_inc_unless_negative(atomic_t *v)
 {
 	int prev, rc;
 
-	__asm__ __volatile__ (
-		"0:	lr.w      %[p],  %[c]\n"
-		"	bltz      %[p],  1f\n"
-		"	addi      %[rc], %[p], 1\n"
-		"	sc.w.rl   %[rc], %[rc], %[c]\n"
-		"	bnez      %[rc], 0b\n"
-		"	fence     rw, rw\n"
-		"1:\n"
-		: [p]"=&r" (prev), [rc]"=&r" (rc), [c]"+A" (v->counter)
-		:
-		: "memory");
+	_arch_atomic_inc_unless_negative(prev, rc, v->counter, "w");
+
 	return !(prev < 0);
 }
 
 #define arch_atomic_inc_unless_negative arch_atomic_inc_unless_negative
 
+#define _arch_atomic_dec_unless_positive(_prev, _rc, counter, sfx)	\
+({									\
+	__asm__ __volatile__ (						\
+		"0:	lr." sfx "      %[p],  %[c]\n"			\
+		"	bgtz            %[p],  1f\n"			\
+		"	addi            %[rc], %[p], -1\n"		\
+		"	sc." sfx ".rl   %[rc], %[rc], %[c]\n"		\
+		"	bnez            %[rc], 0b\n"			\
+		"	fence           rw, rw\n"			\
+		"1:\n"							\
+		: [p]"=&r" (_prev), [rc]"=&r" (_rc), [c]"+A" (counter)	\
+		:							\
+		: "memory");						\
+})
+
 static __always_inline bool arch_atomic_dec_unless_positive(atomic_t *v)
 {
 	int prev, rc;
 
-	__asm__ __volatile__ (
-		"0:	lr.w      %[p],  %[c]\n"
-		"	bgtz      %[p],  1f\n"
-		"	addi      %[rc], %[p], -1\n"
-		"	sc.w.rl   %[rc], %[rc], %[c]\n"
-		"	bnez      %[rc], 0b\n"
-		"	fence     rw, rw\n"
-		"1:\n"
-		: [p]"=&r" (prev), [rc]"=&r" (rc), [c]"+A" (v->counter)
-		:
-		: "memory");
+	_arch_atomic_dec_unless_positive(prev, rc, v->counter, "w");
+
 	return !(prev > 0);
 }
 
 #define arch_atomic_dec_unless_positive arch_atomic_dec_unless_positive
 
+#define _arch_atomic_dec_if_positive(_prev, _rc, counter, sfx)		\
+({									\
+	__asm__ __volatile__ (						\
+		"0:	lr." sfx "     %[p],  %[c]\n"			\
+		"	addi           %[rc], %[p], -1\n"		\
+		"	bltz           %[rc], 1f\n"			\
+		"	sc." sfx ".rl  %[rc], %[rc], %[c]\n"		\
+		"	bnez           %[rc], 0b\n"			\
+		"	fence          rw, rw\n"			\
+		"1:\n"							\
+		: [p]"=&r" (_prev), [rc]"=&r" (_rc), [c]"+A" (counter)	\
+		:							\
+		: "memory");						\
+})
+
 static __always_inline int arch_atomic_dec_if_positive(atomic_t *v)
 {
        int prev, rc;
 
-	__asm__ __volatile__ (
-		"0:	lr.w     %[p],  %[c]\n"
-		"	addi     %[rc], %[p], -1\n"
-		"	bltz     %[rc], 1f\n"
-		"	sc.w.rl  %[rc], %[rc], %[c]\n"
-		"	bnez     %[rc], 0b\n"
-		"	fence    rw, rw\n"
-		"1:\n"
-		: [p]"=&r" (prev), [rc]"=&r" (rc), [c]"+A" (v->counter)
-		:
-		: "memory");
+	_arch_atomic_dec_if_positive(prev, rc, v->counter, "w");
+
 	return prev - 1;
 }
 
@@ -304,17 +319,8 @@ static __always_inline bool arch_atomic64_inc_unless_negative(atomic64_t *v)
 	s64 prev;
 	long rc;
 
-	__asm__ __volatile__ (
-		"0:	lr.d      %[p],  %[c]\n"
-		"	bltz      %[p],  1f\n"
-		"	addi      %[rc], %[p], 1\n"
-		"	sc.d.rl   %[rc], %[rc], %[c]\n"
-		"	bnez      %[rc], 0b\n"
-		"	fence     rw, rw\n"
-		"1:\n"
-		: [p]"=&r" (prev), [rc]"=&r" (rc), [c]"+A" (v->counter)
-		:
-		: "memory");
+	_arch_atomic_inc_unless_negative(prev, rc, v->counter, "d");
+
 	return !(prev < 0);
 }
 
@@ -325,17 +331,8 @@ static __always_inline bool arch_atomic64_dec_unless_positive(atomic64_t *v)
 	s64 prev;
 	long rc;
 
-	__asm__ __volatile__ (
-		"0:	lr.d      %[p],  %[c]\n"
-		"	bgtz      %[p],  1f\n"
-		"	addi      %[rc], %[p], -1\n"
-		"	sc.d.rl   %[rc], %[rc], %[c]\n"
-		"	bnez      %[rc], 0b\n"
-		"	fence     rw, rw\n"
-		"1:\n"
-		: [p]"=&r" (prev), [rc]"=&r" (rc), [c]"+A" (v->counter)
-		:
-		: "memory");
+	_arch_atomic_dec_unless_positive(prev, rc, v->counter, "d");
+
 	return !(prev > 0);
 }
 
@@ -346,17 +343,8 @@ static __always_inline s64 arch_atomic64_dec_if_positive(atomic64_t *v)
        s64 prev;
        long rc;
 
-	__asm__ __volatile__ (
-		"0:	lr.d     %[p],  %[c]\n"
-		"	addi      %[rc], %[p], -1\n"
-		"	bltz     %[rc], 1f\n"
-		"	sc.d.rl  %[rc], %[rc], %[c]\n"
-		"	bnez     %[rc], 0b\n"
-		"	fence    rw, rw\n"
-		"1:\n"
-		: [p]"=&r" (prev), [rc]"=&r" (rc), [c]"+A" (v->counter)
-		:
-		: "memory");
+	_arch_atomic_dec_if_positive(prev, rc, v->counter, "d");
+
 	return prev - 1;
 }
 
Signature · 7 lines
2.41.0


_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
#4 lore raw

RFC PATCH v5 4/5 riscv/cmpxchg: Implement cmpxchg for variables of size 1 and 2

In reply to Leonardo Bras

cmpxchg for variables of size 1-byte and 2-bytes is not yet available for riscv, even though its present in other architectures such as arm64 and x86. This could lead to not being able to implement some locking mechanisms or requiring some rework to make it work properly.

Implement 1-byte and 2-bytes cmpxchg in order to achieve parity with other architectures.

---
 arch/riscv/include/asm/cmpxchg.h | 34 ++++++++++++++++++++++++++++++++
 1 file changed, 34 insertions(+)
arch/riscv/include/asm/cmpxchg.h +34 -0
index e3e0ac7ba061..ac9d0eeb74e6 100644
@@ -72,6 +72,35 @@
  * indicated by comparing RETURN with OLD.
  */
 
+#define __arch_cmpxchg_masked(sc_sfx, prepend, append, r, p, o, n)	\
+({									\
+	u32 *__ptr32b = (u32 *)((ulong)(p) & ~0x3);			\
+	ulong __s = ((ulong)(p) & (0x4 - sizeof(*p))) * BITS_PER_BYTE;	\
+	ulong __mask = GENMASK(((sizeof(*p)) * BITS_PER_BYTE) - 1, 0)	\
+			<< __s;						\
+	ulong __newx = (ulong)(n) << __s;				\
+	ulong __oldx = (ulong)(o) << __s;				\
+	ulong __retx;							\
+	ulong __rc;							\
+									\
+	__asm__ __volatile__ (						\
+		prepend							\
+		"0:	lr.w %0, %2\n"					\
+		"	and  %1, %0, %z5\n"				\
+		"	bne  %1, %z3, 1f\n"				\
+		"	and  %1, %0, %z6\n"				\
+		"	or   %1, %1, %z4\n"				\
+		"	sc.w" sc_sfx " %1, %1, %2\n"			\
+		"	bnez %1, 0b\n"					\
+		append							\
+		"1:\n"							\
+		: "=&r" (__retx), "=&r" (__rc), "+A" (*(__ptr32b))	\
+		: "rJ" ((long)__oldx), "rJ" (__newx),			\
+		  "rJ" (__mask), "rJ" (~__mask)				\
+		: "memory");						\
+									\
+	r = (__typeof__(*(p)))((__retx & __mask) >> __s);		\
+})
 
 #define __arch_cmpxchg(lr_sfx, sc_sfx, prepend, append, r, p, co, o, n)	\
 ({									\
@@ -98,6 +127,11 @@
 	__typeof__(*(__ptr)) __ret;					\
 									\
 	switch (sizeof(*__ptr)) {					\
+	case 1:								\
+	case 2:								\
+		__arch_cmpxchg_masked(sc_sfx, prepend, append,		\
+					__ret, __ptr, __old, __new);	\
+		break;							\
 	case 4:								\
 		__arch_cmpxchg(".w", ".w" sc_sfx, prepend, append,	\
 				__ret, __ptr, (long), __old, __new);	\
Signature · 7 lines
2.41.0


_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
#5 lore raw

RFC PATCH v5 5/5 riscv/cmpxchg: Implement xchg for variables of size 1 and 2

In reply to Leonardo Bras

xchg for variables of size 1-byte and 2-bytes is not yet available for riscv, even though its present in other architectures such as arm64 and x86. This could lead to not being able to implement some locking mechanisms or requiring some rework to make it work properly.

Implement 1-byte and 2-bytes xchg in order to achieve parity with other architectures.

---
 arch/riscv/include/asm/cmpxchg.h | 31 +++++++++++++++++++++++++++++++
 1 file changed, 31 insertions(+)
arch/riscv/include/asm/cmpxchg.h +31 -0
index ac9d0eeb74e6..26cea2395aae 100644
@@ -11,6 +11,31 @@
 #include <asm/barrier.h>
 #include <asm/fence.h>
 
+#define __arch_xchg_masked(prepend, append, r, p, n)			\
+({									\
+	u32 *__ptr32b = (u32 *)((ulong)(p) & ~0x3);			\
+	ulong __s = ((ulong)(p) & (0x4 - sizeof(*p))) * BITS_PER_BYTE;	\
+	ulong __mask = GENMASK(((sizeof(*p)) * BITS_PER_BYTE) - 1, 0)	\
+			<< __s;						\
+	ulong __newx = (ulong)(n) << __s;				\
+	ulong __retx;							\
+	ulong __rc;							\
+									\
+	__asm__ __volatile__ (						\
+	       prepend							\
+	       "0:	lr.w %0, %2\n"					\
+	       "	and  %1, %0, %z4\n"				\
+	       "	or   %1, %1, %z3\n"				\
+	       "	sc.w %1, %1, %2\n"				\
+	       "	bnez %1, 0b\n"					\
+	       append							\
+	       : "=&r" (__retx), "=&r" (__rc), "+A" (*(__ptr32b))	\
+	       : "rJ" (__newx), "rJ" (~__mask)				\
+	       : "memory");						\
+									\
+	r = (__typeof__(*(p)))((__retx & __mask) >> __s);		\
+})
+
 #define __arch_xchg(sfx, prepend, append, r, p, n)			\
 ({									\
 	__asm__ __volatile__ (						\
@@ -27,7 +52,13 @@
 	__typeof__(ptr) __ptr = (ptr);					\
 	__typeof__(*(__ptr)) __new = (new);				\
 	__typeof__(*(__ptr)) __ret;					\
+									\
 	switch (sizeof(*__ptr)) {					\
+	case 1:								\
+	case 2:								\
+		__arch_xchg_masked(prepend, append,			\
+				   __ret, __ptr, __new);		\
+		break;							\
 	case 4:								\
 		__arch_xchg(".w" sfx, prepend, append,			\
 			      __ret, __ptr, __new);			\
Signature · 7 lines
2.41.0


_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
#6 lore raw

Re: [RFC PATCH v5 5/5] riscv/cmpxchg: Implement xchg for variables of size 1 and 2

Quoted 9 lines · Leonardo Bras

xchg for variables of size 1-byte and 2-bytes is not yet available for riscv, even though its present in other architectures such as arm64 and x86. This could lead to not being able to implement some locking mechanisms or requiring some rework to make it work properly.

Implement 1-byte and 2-bytes xchg in order to achieve parity with other architectures.

Parity with other architectures by itself is not a reason to do this, in particular the other architectures you listed have the instructions in hardware while riscv does not.

Emulating the small xchg() through cmpxchg() is particularly tricky since it's easy to run into a case where this does not guarantee forward progress. This is also something that almost no architecture specific code relies on (generic qspinlock being a notable exception).

I would recommend just dropping this patch from the series, at least until there is a need for it.

Arnd

#7 lore raw

Re: [RFC PATCH v5 5/5] riscv/cmpxchg: Implement xchg for variables of size 1 and 2

Quoted 11 lines · Arnd Bergmann
Quoted 9 lines · Leonardo Bras

xchg for variables of size 1-byte and 2-bytes is not yet available for riscv, even though its present in other architectures such as arm64 and x86. This could lead to not being able to implement some locking mechanisms or requiring some rework to make it work properly.

Implement 1-byte and 2-bytes xchg in order to achieve parity with other architectures.

Hello Arnd Bergmann, thanks for reviewing!

Quoted 3 lines

Parity with other architectures by itself is not a reason to do this, in particular the other architectures you listed have the instructions in hardware while riscv does not.

Sure, I understand RISC-V don't have native support for xchg on variables of size < 4B. My argument is that it's nice to have even an emulated version for this in case any future mechanism wants to use it.

Not having it may mean we won't be able to enable given mechanism in RISC-V.

Quoted 5 lines

Emulating the small xchg() through cmpxchg() is particularly tricky since it's easy to run into a case where this does not guarantee forward progress.

Didn't get this part:
By "emulating small xchg() through cmpxchg()", did you mean like emulating an xchg (usually 1 instruction) with lr & sc (same used in cmpxchg) ?

If so, yeah, it's a fair point: in some extreme case we could have multiple threads accessing given cacheline and have sc always failing. On the other hand, there are 2 arguments on that:

1 - Other architectures, (such as powerpc, arm and arm64 without LSE atomics) also seem to rely in this mechanism for every xchg size. Another archs like csky and loongarch use asm that look like mine to handle size < 4B xchg.

Quoted 3 lines

This is also something that almost no architecture
specific code relies on (generic qspinlock being a notable exception).

2 - As you mentioned, there should be very little code that will actually make use of xchg for vars < 4B, so it should be safe to assume its fine to not guarantee forward progress for those rare usages (like some of above mentioned archs).

Quoted 2 lines

I would recommend just dropping this patch from the series, at least until there is a need for it.

While I agree this is a valid point, I believe its more interesting to have it implemented if any future mechanism wants to make use of this.

Thanks!
Leo

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

#8 lore raw

Re: [RFC PATCH v5 5/5] riscv/cmpxchg: Implement xchg for variables of size 1 and 2

Quoted 19 lines · On Thu, 10 Aug 2023 09:04:04 PDT (-0700), [email protected] wrote:
Quoted 11 lines · Arnd Bergmann
Quoted 9 lines · Leonardo Bras

xchg for variables of size 1-byte and 2-bytes is not yet available for riscv, even though its present in other architectures such as arm64 and x86. This could lead to not being able to implement some locking mechanisms or requiring some rework to make it work properly.

Implement 1-byte and 2-bytes xchg in order to achieve parity with other architectures.

Hello Arnd Bergmann, thanks for reviewing!

Quoted 3 lines

Parity with other architectures by itself is not a reason to do this, in particular the other architectures you listed have the instructions in hardware while riscv does not.

Quoted 5 lines

Sure, I understand RISC-V don't have native support for xchg on variables of size < 4B. My argument is that it's nice to have even an emulated version for this in case any future mechanism wants to use it.

Not having it may mean we won't be able to enable given mechanism in RISC-V.

IIUC the ask is to have a user within the kernel for these functions. That's the general thing to do, and last time this came up there was no in-kernel use of it -- the qspinlock stuff would, but we haven't enabled it yet because we're worried about the performance/fairness stuff that other ports have seen and nobody's got concrete benchmarks yet (though there's another patch set out that I haven't had time to look through, so that may have changed).

So if something uses these I'm happy to go look closer.

Quoted 36 lines
Quoted 4 lines

Emulating the small xchg() through cmpxchg() is particularly tricky since it's easy to run into a case where this does not guarantee forward progress.

Didn't get this part:
By "emulating small xchg() through cmpxchg()", did you mean like emulating an xchg (usually 1 instruction) with lr & sc (same used in cmpxchg) ?

If so, yeah, it's a fair point: in some extreme case we could have multiple threads accessing given cacheline and have sc always failing. On the other hand, there are 2 arguments on that:

1 - Other architectures, (such as powerpc, arm and arm64 without LSE atomics) also seem to rely in this mechanism for every xchg size. Another archs like csky and loongarch use asm that look like mine to handle size < 4B xchg.

Quoted 3 lines

This is also something that almost no architecture
specific code relies on (generic qspinlock being a notable exception).

2 - As you mentioned, there should be very little code that will actually make use of xchg for vars < 4B, so it should be safe to assume its fine to not guarantee forward progress for those rare usages (like some of above mentioned archs).

Quoted 2 lines

I would recommend just dropping this patch from the series, at least until there is a need for it.

While I agree this is a valid point, I believe its more interesting to have it implemented if any future mechanism wants to make use of this.

Thanks!
Leo

#9 lore raw

Re: [RFC PATCH v5 5/5] riscv/cmpxchg: Implement xchg for variables of size 1 and 2

Quoted 21 lines · Palmer Dabbelt
Quoted 14 lines · On Thu, 10 Aug 2023 09:04:04 PDT (-0700), [email protected] wrote:
Quoted 6 lines · Arnd Bergmann
Quoted 5 lines · Leonardo Bras

xchg for variables of size 1-byte and 2-bytes is not yet available for riscv, even though its present in other architectures such as arm64 and x86. This could lead to not being able to implement some locking mechanisms or requiring some rework to make it work properly.

Quoted 6 lines
Quoted 2 lines

Implement 1-byte and 2-bytes xchg in order to achieve parity with other architectures.

Parity with other architectures by itself is not a reason to do this, in particular the other architectures you listed have the instructions in hardware while riscv does not.

Quoted 5 lines

Sure, I understand RISC-V don't have native support for xchg on variables of size < 4B. My argument is that it's nice to have even an emulated version for this in case any future mechanism wants to use it.

Not having it may mean we won't be able to enable given mechanism in RISC-V.

Quoted 7 lines

IIUC the ask is to have a user within the kernel for these functions. That's the general thing to do, and last time this came up there was no in-kernel use of it -- the qspinlock stuff would, but we haven't enabled it yet because we're worried about the performance/fairness stuff that other ports have seen and nobody's got concrete benchmarks yet (though there's another patch set out that I haven't had time to look through, so that may have changed).

Right. In particular the qspinlock is a good example for something where having the emulated 16-bit xchg() may end up less efficient than a natively supported instruction.

The xchg() here is a performance optimization for CPUs that can do this without touching the other half of the 32-bit word.

Quoted 5 lines
Quoted 5 lines

Didn't get this part:
By "emulating small xchg() through cmpxchg()", did you mean like emulating an xchg (usually 1 instruction) with lr & sc (same used in cmpxchg) ?

Quoted 7 lines
Quoted 7 lines

If so, yeah, it's a fair point: in some extreme case we could have multiple threads accessing given cacheline and have sc always failing. On the other hand, there are 2 arguments on that:

1 - Other architectures, (such as powerpc, arm and arm64 without LSE atomics) also seem to rely in this mechanism for every xchg size. Another archs like csky and loongarch use asm that look like mine to handle size < 4B xchg.

I think you misread the arm64 code, which should use native instructions for all sizes, in both the armv8.0 and LSE atomics.

PowerPC does use the masking for xchg, but I suspect there are no actual users, at least it actually has its own qspinlock implementation that avoids xchg().

Quoted 8 lines
Quoted 8 lines
Quoted 3 lines

This is also something that almost no architecture
specific code relies on (generic qspinlock being a notable exception).

2 - As you mentioned, there should be very little code that will actually make use of xchg for vars < 4B, so it should be safe to assume its fine to not guarantee forward progress for those rare usages (like some of above mentioned archs).

I don't this this is a safe assumption, we've had endless discussions about using qspinlock on architectures without a native xchg(), which needs either hardware guarantees or special countermeasures in xchg() itself to avoid this.

What I'd actually like to do here is to remove the special 8-bit and 16-bit cases from the xchg() and cmpxchg() interfaces at all, leaving only fixed 32-bit and native wordsize (either 32 or 64) as the option, while dealing with the others the same way we treat the fixed 64-bit cases that hardcode the 64-bit argument types and are only usable on architectures that provide them.

Arnd

#10 lore raw

Re: [RFC PATCH v5 5/5] riscv/cmpxchg: Implement xchg for variables of size 1 and 2

Quoted 31 lines · Arnd Bergmann
Quoted 21 lines · Palmer Dabbelt
Quoted 14 lines · On Thu, 10 Aug 2023 09:04:04 PDT (-0700), [email protected] wrote:
Quoted 6 lines · Arnd Bergmann
Quoted 5 lines · Leonardo Bras

xchg for variables of size 1-byte and 2-bytes is not yet available for riscv, even though its present in other architectures such as arm64 and x86. This could lead to not being able to implement some locking mechanisms or requiring some rework to make it work properly.

Quoted 6 lines
Quoted 2 lines

Implement 1-byte and 2-bytes xchg in order to achieve parity with other architectures.

Parity with other architectures by itself is not a reason to do this, in particular the other architectures you listed have the instructions in hardware while riscv does not.

Quoted 5 lines

Sure, I understand RISC-V don't have native support for xchg on variables of size < 4B. My argument is that it's nice to have even an emulated version for this in case any future mechanism wants to use it.

Not having it may mean we won't be able to enable given mechanism in RISC-V.

Quoted 7 lines

IIUC the ask is to have a user within the kernel for these functions. That's the general thing to do, and last time this came up there was no in-kernel use of it -- the qspinlock stuff would, but we haven't enabled it yet because we're worried about the performance/fairness stuff that other ports have seen and nobody's got concrete benchmarks yet (though there's another patch set out that I haven't had time to look through, so that may have changed).

Quoted 3 lines

Right. In particular the qspinlock is a good example for something where having the emulated 16-bit xchg() may end up less efficient than a natively supported instruction.

The xchg() efficiency depends on micro-architecture. and the number of instructions is not the key, even one instruction would be separated into several micro-ops. I thought the Power guys won't agree with this view :)

Quoted 3 lines

The xchg() here is a performance optimization for CPUs that can do this without touching the other half of the 32-bit word.

It's useless on a non-SMT system because all operations are cacheline based. (Ps: Because xchg() has a load semantic, CHI's "Dirty Partial" & "Clean Empty" can't help anymore.)

Quoted 14 lines
Quoted 6 lines
Quoted 6 lines

Didn't get this part:
By "emulating small xchg() through cmpxchg()", did you mean like emulating an xchg (usually 1 instruction) with lr & sc (same used in cmpxchg) ?

Quoted 7 lines
Quoted 7 lines

If so, yeah, it's a fair point: in some extreme case we could have multiple threads accessing given cacheline and have sc always failing. On the other hand, there are 2 arguments on that:

1 - Other architectures, (such as powerpc, arm and arm64 without LSE atomics) also seem to rely in this mechanism for every xchg size. Another archs like csky and loongarch use asm that look like mine to handle size < 4B xchg.

Quoted 6 lines

I think you misread the arm64 code, which should use native instructions for all sizes, in both the armv8.0 and LSE atomics.

PowerPC does use the masking for xchg, but I suspect there are no actual users, at least it actually has its own qspinlock implementation that avoids xchg().

PowerPC still needs similar things, see publish_tail_cpu(), and more complex cmpxchg semantics.

Paravrit qspinlock and CNA qspinlock still need more:
- xchg8 (RCsc)
- cmpxchg8/16_relaxed
- cmpxchg8/16_release (Rcpc)
- cmpxchg8_acquire (RCpc)
- cmpxchg8 (RCsc)

Quoted 10 lines
Quoted 9 lines
Quoted 9 lines
Quoted 4 lines

This is also something that almost no architecture
specific code relies on (generic qspinlock being a notable exception).

2 - As you mentioned, there should be very little code that will actually make use of xchg for vars < 4B, so it should be safe to assume its fine to not guarantee forward progress for those rare usages (like some of above mentioned archs).

Quoted 7 lines

I don't this this is a safe assumption, we've had endless discussions about using qspinlock on architectures without a native xchg(), which needs either hardware guarantees or special countermeasures in xchg() itself to avoid this.

What I'd actually like to do here is to remove the special 8-bit and 16-bit cases from the xchg() and cmpxchg() interfaces at all, leaving

It needs to modify qspinlock, paravirt_qspinlock, and CNA_qspinlock code to prevent using 8-bit/16-bit xchg/cmpxchg, and cleanup all architectures' cmpxchg.h. What you do is just get them out of the common atomic.h, but architectures still need to solve them and connect to the qspinlock series.

Quoted 6 lines

only fixed 32-bit and native wordsize (either 32 or 64) as the option, while dealing with the others the same way we treat the fixed 64-bit cases that hardcode the 64-bit argument types and are only usable on architectures that provide them.

Arnd

Signature · 2 lines
Best Regards
 Guo Ren
#11 lore raw

Re: [RFC PATCH v5 5/5] riscv/cmpxchg: Implement xchg for variables of size 1 and 2

Hello everyone,

Sorry for the delay, I was out of office for a while.

Quoted 41 lines · Guo Ren
Quoted 31 lines · Arnd Bergmann
Quoted 21 lines · Palmer Dabbelt
Quoted 14 lines · On Thu, 10 Aug 2023 09:04:04 PDT (-0700), [email protected] wrote:
Quoted 6 lines · Arnd Bergmann
Quoted 5 lines · Leonardo Bras

xchg for variables of size 1-byte and 2-bytes is not yet available for riscv, even though its present in other architectures such as arm64 and x86. This could lead to not being able to implement some locking mechanisms or requiring some rework to make it work properly.

Quoted 6 lines
Quoted 2 lines

Implement 1-byte and 2-bytes xchg in order to achieve parity with other architectures.

Parity with other architectures by itself is not a reason to do this, in particular the other architectures you listed have the instructions in hardware while riscv does not.

Quoted 5 lines

Sure, I understand RISC-V don't have native support for xchg on variables of size < 4B. My argument is that it's nice to have even an emulated version for this in case any future mechanism wants to use it.

Not having it may mean we won't be able to enable given mechanism in RISC-V.

Quoted 7 lines

IIUC the ask is to have a user within the kernel for these functions. That's the general thing to do, and last time this came up there was no in-kernel use of it -- the qspinlock stuff would, but we haven't enabled it yet because we're worried about the performance/fairness stuff that other ports have seen and nobody's got concrete benchmarks yet (though there's another patch set out that I haven't had time to look through, so that may have changed).

Quoted 3 lines

Right. In particular the qspinlock is a good example for something where having the emulated 16-bit xchg() may end up less efficient than a natively supported instruction.

The xchg() efficiency depends on micro-architecture. and the number of instructions is not the key, even one instruction would be separated into several micro-ops. I thought the Power guys won't agree with this view :)

Quoted 1 line
Quoted 22 lines
Quoted 2 lines

The xchg() here is a performance optimization for CPUs that can do this without touching the other half of the 32-bit word.

It's useless on a non-SMT system because all operations are cacheline based. (Ps: Because xchg() has a load semantic, CHI's "Dirty Partial" & "Clean Empty" can't help anymore.)

Quoted 16 lines
Quoted 6 lines
Quoted 6 lines

Didn't get this part:
By "emulating small xchg() through cmpxchg()", did you mean like emulating an xchg (usually 1 instruction) with lr & sc (same used in cmpxchg) ?

Quoted 7 lines
Quoted 7 lines

If so, yeah, it's a fair point: in some extreme case we could have multiple threads accessing given cacheline and have sc always failing. On the other hand, there are 2 arguments on that:

1 - Other architectures, (such as powerpc, arm and arm64 without LSE atomics) also seem to rely in this mechanism for every xchg size. Another archs like csky and loongarch use asm that look like mine to handle size < 4B xchg.

I think you misread the arm64 code, which should use native instructions for all sizes, in both the armv8.0 and LSE atomics.

By native I understand you mean swp instead of ll/sc, right?

Well, that's right only if the kernel is compiled with LSE, and the ll/sc option for is available for other arm64 that don't.

Also, according to Kconfig, it seems to have been introduced in ARMv8.1, meaning arm64 for (at least some) ARMv8.0 use ll/sc, and this is why xchg with the ll/sc code is available for 1, 2, 4 and 8 bytes in arch/arm64/include/asm/cmpxchg.h:

#define __XCHG_CASE(w, sfx, name, sz, mb, nop_lse, acq, acq_lse, rel,
cl) \static inline u##sz __xchg_case_##name##sz(u##sz x, volatile void *ptr) \{ \ u##szret; \ unsignedlongtmp; \ \
\ asm
volatile(ARM64_LSE_ATOMIC_INSN( \ /* LL/SC */ \ " prfm pstl1strm, %2\n" \ "1: ld"#acq"xr"#sfx"\t%"#w"0,%2\n" \ " st"#rel"xr"#sfx"\t%w1,%"#w"3,%2\n" \ " cbnz %w1,1b\n" \ " "#mb, \ /*LSEatomics*/ \ " swp"#acq_lse#rel#sfx"\t%"#w"3,%"#w"0,%2\n" \[...]

__XCHG_CASE(w, b,     ,  8,        ,    ,  ,  ,  ,         )
__XCHG_CASE(w, h,     , 16,        ,    ,  ,  ,  ,         )
__XCHG_CASE(w,  ,     , 32,        ,    ,  ,  ,  ,         )
Quoted 7 lines
Quoted 4 lines

PowerPC does use the masking for xchg, but I suspect there are no actual users, at least it actually has its own qspinlock implementation that avoids xchg().

PowerPC still needs similar things, see publish_tail_cpu(), and more complex cmpxchg semantics.

Quoted 21 lines

Paravrit qspinlock and CNA qspinlock still need more:
- xchg8 (RCsc)
- cmpxchg8/16_relaxed
- cmpxchg8/16_release (Rcpc)
- cmpxchg8_acquire (RCpc)
- cmpxchg8 (RCsc)

Quoted 14 lines
Quoted 9 lines
Quoted 9 lines
Quoted 4 lines

This is also something that almost no architecture
specific code relies on (generic qspinlock being a notable exception).

2 - As you mentioned, there should be very little code that will actually make use of xchg for vars < 4B, so it should be safe to assume its fine to not guarantee forward progress for those rare usages (like some of above mentioned archs).

I don't this this is a safe assumption, we've had endless discussions about using qspinlock on architectures without a native xchg(), which needs either hardware guarantees or special countermeasures in xchg() itself to avoid this.

That seems a nice discussion, do you have a link for this?

By what I could see, Guo Ren is doing a great work on proving that using qspinlock (with smaller xchg) performs better on RISC-V.

Quoted 18 lines
Quoted 3 lines

What I'd actually like to do here is to remove the special 8-bit and 16-bit cases from the xchg() and cmpxchg() interfaces at all, leaving

It needs to modify qspinlock, paravirt_qspinlock, and CNA_qspinlock code to prevent using 8-bit/16-bit xchg/cmpxchg, and cleanup all architectures' cmpxchg.h. What you do is just get them out of the common atomic.h, but architectures still need to solve them and connect to the qspinlock series.

Quoted 6 lines

only fixed 32-bit and native wordsize (either 32 or 64) as the option, while dealing with the others the same way we treat the fixed 64-bit cases that hardcode the 64-bit argument types and are only usable on architectures that provide them.

Arnd

IIUC, xchg for size 1 & 2 can still be useful if having the lock variable bigger causes the target struct to use more than a cacheline. This could reduce cache usage and avoid some cacheline misses.

Even though in some arches those 'non-native' xchg can take longer, it can be perceived as a valid tradeoff for some scenarios.

Thanks,
Leo

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

#12 lore raw

Re: [RFC PATCH v5 5/5] riscv/cmpxchg: Implement xchg for variables of size 1 and 2

In reply to Palmer Dabbelt
Quoted 27 lines · Palmer Dabbelt
Quoted 19 lines · On Thu, 10 Aug 2023 09:04:04 PDT (-0700), [email protected] wrote:
Quoted 11 lines · Arnd Bergmann
Quoted 9 lines · Leonardo Bras

xchg for variables of size 1-byte and 2-bytes is not yet available for riscv, even though its present in other architectures such as arm64 and x86. This could lead to not being able to implement some locking mechanisms or requiring some rework to make it work properly.

Implement 1-byte and 2-bytes xchg in order to achieve parity with other architectures.

Hello Arnd Bergmann, thanks for reviewing!

Quoted 3 lines

Parity with other architectures by itself is not a reason to do this, in particular the other architectures you listed have the instructions in hardware while riscv does not.

Quoted 5 lines

Sure, I understand RISC-V don't have native support for xchg on variables of size < 4B. My argument is that it's nice to have even an emulated version for this in case any future mechanism wants to use it.

Not having it may mean we won't be able to enable given mechanism in RISC-V.

Quoted 7 lines

IIUC the ask is to have a user within the kernel for these functions. That's the general thing to do, and last time this came up there was no in-kernel use of it -- the qspinlock stuff would, but we haven't enabled it yet because we're worried about the performance/fairness stuff that other ports have seen and nobody's got concrete benchmarks yet (though there's another patch set out that I haven't had time to look through, so that may have changed).

Conor doesn't agree with using an alternative as a detour mechanism between qspinlock & ticket lock. So I'm preparing V11 with static_key (jump_label) style. Next version, I would separate paravirt_qspinlock & CNA_qspinlock from V10. That would make it easy to review the qspinlock patch series. You can review the next version V11. Now I'm debugging a static_key init problem when load_modules, which is triggered by our combo_qspinlock.

The qspinlock is being tested on the riscv platform [1] with 128 cores with 8 NUMA nodes, next, I would update the comparison results of qspinlock & ticket lock.

[1]: https://www.sophon.ai/

Quoted 39 lines

So if something uses these I'm happy to go look closer.

Quoted 36 lines
Quoted 4 lines

Emulating the small xchg() through cmpxchg() is particularly tricky since it's easy to run into a case where this does not guarantee forward progress.

Didn't get this part:
By "emulating small xchg() through cmpxchg()", did you mean like emulating an xchg (usually 1 instruction) with lr & sc (same used in cmpxchg) ?

If so, yeah, it's a fair point: in some extreme case we could have multiple threads accessing given cacheline and have sc always failing. On the other hand, there are 2 arguments on that:

1 - Other architectures, (such as powerpc, arm and arm64 without LSE atomics) also seem to rely in this mechanism for every xchg size. Another archs like csky and loongarch use asm that look like mine to handle size < 4B xchg.

Quoted 3 lines

This is also something that almost no architecture
specific code relies on (generic qspinlock being a notable exception).

2 - As you mentioned, there should be very little code that will actually make use of xchg for vars < 4B, so it should be safe to assume its fine to not guarantee forward progress for those rare usages (like some of above mentioned archs).

Quoted 2 lines

I would recommend just dropping this patch from the series, at least until there is a need for it.

While I agree this is a valid point, I believe its more interesting to have it implemented if any future mechanism wants to make use of this.

Thanks!
Leo

Signature · 2 lines
Best Regards
 Guo Ren
#13 lore raw

Re: [RFC PATCH v5 5/5] riscv/cmpxchg: Implement xchg for variables of size 1 and 2

Quoted 35 lines · Guo Ren
Quoted 27 lines · Palmer Dabbelt
Quoted 19 lines · On Thu, 10 Aug 2023 09:04:04 PDT (-0700), [email protected] wrote:
Quoted 11 lines · Arnd Bergmann
Quoted 9 lines · Leonardo Bras

xchg for variables of size 1-byte and 2-bytes is not yet available for riscv, even though its present in other architectures such as arm64 and x86. This could lead to not being able to implement some locking mechanisms or requiring some rework to make it work properly.

Implement 1-byte and 2-bytes xchg in order to achieve parity with other architectures.

Hello Arnd Bergmann, thanks for reviewing!

Quoted 3 lines

Parity with other architectures by itself is not a reason to do this, in particular the other architectures you listed have the instructions in hardware while riscv does not.

Quoted 5 lines

Sure, I understand RISC-V don't have native support for xchg on variables of size < 4B. My argument is that it's nice to have even an emulated version for this in case any future mechanism wants to use it.

Not having it may mean we won't be able to enable given mechanism in RISC-V.

Quoted 7 lines

IIUC the ask is to have a user within the kernel for these functions. That's the general thing to do, and last time this came up there was no in-kernel use of it -- the qspinlock stuff would, but we haven't enabled it yet because we're worried about the performance/fairness stuff that other ports have seen and nobody's got concrete benchmarks yet (though there's another patch set out that I haven't had time to look through, so that may have changed).

Quoted 2 lines

Conor doesn't agree with using an alternative as a detour mechanism between qspinlock & ticket lock.

Hold on a sec, I don't recall having a problem with alternatives - it was calling the stronger forward progress guarantee an erratum (which it isn't) and an ISA extension w/o any "abusing" that framework that I did not like.

Quoted 2 lines

So I'm preparing V11 with static_key
(jump_label) style.

I don't think there's much point rushing into making it based on static keys when no progress has been made on implementing support for non-standard extensions. Changing to a static key doesn't change the detection mechanism, I've not got a problem with using alternatives for this stuff.

Thanks,
Conor.

Quoted 57 lines

Next version, I would separate paravirt_qspinlock
& CNA_qspinlock from V10. That would make it easy to review the qspinlock patch series. You can review the next version V11. Now I'm debugging a static_key init problem when load_modules, which is triggered by our combo_qspinlock.

The qspinlock is being tested on the riscv platform [1] with 128 cores with 8 NUMA nodes, next, I would update the comparison results of qspinlock & ticket lock.

[1]: https://www.sophon.ai/

Quoted 39 lines

So if something uses these I'm happy to go look closer.

Quoted 36 lines
Quoted 4 lines

Emulating the small xchg() through cmpxchg() is particularly tricky since it's easy to run into a case where this does not guarantee forward progress.

Didn't get this part:
By "emulating small xchg() through cmpxchg()", did you mean like emulating an xchg (usually 1 instruction) with lr & sc (same used in cmpxchg) ?

If so, yeah, it's a fair point: in some extreme case we could have multiple threads accessing given cacheline and have sc always failing. On the other hand, there are 2 arguments on that:

1 - Other architectures, (such as powerpc, arm and arm64 without LSE atomics) also seem to rely in this mechanism for every xchg size. Another archs like csky and loongarch use asm that look like mine to handle size < 4B xchg.

Quoted 3 lines

This is also something that almost no architecture
specific code relies on (generic qspinlock being a notable exception).

2 - As you mentioned, there should be very little code that will actually make use of xchg for vars < 4B, so it should be safe to assume its fine to not guarantee forward progress for those rare usages (like some of above mentioned archs).

Quoted 2 lines

I would recommend just dropping this patch from the series, at least until there is a need for it.

While I agree this is a valid point, I believe its more interesting to have it implemented if any future mechanism wants to make use of this.

Thanks!
Leo

Signature · 2 lines
Best Regards
 Guo Ren
#14 lore raw

Re: [RFC PATCH v5 5/5] riscv/cmpxchg: Implement xchg for variables of size 1 and 2

Quoted 36 lines · Conor Dooley
Quoted 35 lines · Guo Ren
Quoted 34 lines · Palmer Dabbelt
Quoted 19 lines · On Thu, 10 Aug 2023 09:04:04 PDT (-0700), [email protected] wrote:
Quoted 11 lines · Arnd Bergmann
Quoted 9 lines · Leonardo Bras

xchg for variables of size 1-byte and 2-bytes is not yet available for riscv, even though its present in other architectures such as arm64 and x86. This could lead to not being able to implement some locking mechanisms or requiring some rework to make it work properly.

Implement 1-byte and 2-bytes xchg in order to achieve parity with other architectures.

Hello Arnd Bergmann, thanks for reviewing!

Quoted 3 lines

Parity with other architectures by itself is not a reason to do this, in particular the other architectures you listed have the instructions in hardware while riscv does not.

Quoted 5 lines

Sure, I understand RISC-V don't have native support for xchg on variables of size < 4B. My argument is that it's nice to have even an emulated version for this in case any future mechanism wants to use it.

Not having it may mean we won't be able to enable given mechanism in RISC-V.

IIUC the ask is to have a user within the kernel for these functions. That's the general thing to do, and last time this came up there was no in-kernel use of it -- the qspinlock stuff would, but we haven't enabled it yet because we're worried about the performance/fairness stuff that other ports have seen and nobody's got concrete benchmarks yet (though there's another patch set out that I haven't had time to look through, so that may have changed).

Quoted 7 lines
Quoted 2 lines

Conor doesn't agree with using an alternative as a detour mechanism between qspinlock & ticket lock.

Hold on a sec, I don't recall having a problem with alternatives - it was calling the stronger forward progress guarantee an erratum (which it isn't) and an ISA extension w/o any "abusing" that framework that I did not like.

Hmm, what I wrote here makes no sense reading it back. Let me try again:

Hold on a sec, I don't recall having a problem with alternatives - it was calling the stronger forward progress guarantee an erratum (which it isn't) and "abusing" that framework that I did not like. The series effectively mixed and matched whichever part of ISA extensions and errata that was convenient.

Quoted 70 lines
Quoted 3 lines

So I'm preparing V11 with static_key
(jump_label) style.

I don't think there's much point rushing into making it based on static keys when no progress has been made on implementing support for non-standard extensions. Changing to a static key doesn't change the detection mechanism, I've not got a problem with using alternatives for this stuff.

Thanks,
Conor.

Quoted 57 lines

Next version, I would separate paravirt_qspinlock
& CNA_qspinlock from V10. That would make it easy to review the qspinlock patch series. You can review the next version V11. Now I'm debugging a static_key init problem when load_modules, which is triggered by our combo_qspinlock.

The qspinlock is being tested on the riscv platform [1] with 128 cores with 8 NUMA nodes, next, I would update the comparison results of qspinlock & ticket lock.

[1]: https://www.sophon.ai/

Quoted 39 lines

So if something uses these I'm happy to go look closer.

Quoted 36 lines
Quoted 4 lines

Emulating the small xchg() through cmpxchg() is particularly tricky since it's easy to run into a case where this does not guarantee forward progress.

Didn't get this part:
By "emulating small xchg() through cmpxchg()", did you mean like emulating an xchg (usually 1 instruction) with lr & sc (same used in cmpxchg) ?

If so, yeah, it's a fair point: in some extreme case we could have multiple threads accessing given cacheline and have sc always failing. On the other hand, there are 2 arguments on that:

1 - Other architectures, (such as powerpc, arm and arm64 without LSE atomics) also seem to rely in this mechanism for every xchg size. Another archs like csky and loongarch use asm that look like mine to handle size < 4B xchg.

Quoted 3 lines

This is also something that almost no architecture
specific code relies on (generic qspinlock being a notable exception).

2 - As you mentioned, there should be very little code that will actually make use of xchg for vars < 4B, so it should be safe to assume its fine to not guarantee forward progress for those rare usages (like some of above mentioned archs).

Quoted 2 lines

I would recommend just dropping this patch from the series, at least until there is a need for it.

While I agree this is a valid point, I believe its more interesting to have it implemented if any future mechanism wants to make use of this.

Thanks!
Leo

Signature · 2 lines
Best Regards
 Guo Ren
#15 lore raw

Re: [RFC PATCH v5 5/5] riscv/cmpxchg: Implement xchg for variables of size 1 and 2

In reply to Guo Ren
Quoted 35 lines · Guo Ren
Quoted 27 lines · Palmer Dabbelt
Quoted 19 lines · On Thu, 10 Aug 2023 09:04:04 PDT (-0700), [email protected] wrote:
Quoted 11 lines · Arnd Bergmann
Quoted 9 lines · Leonardo Bras

xchg for variables of size 1-byte and 2-bytes is not yet available for riscv, even though its present in other architectures such as arm64 and x86. This could lead to not being able to implement some locking mechanisms or requiring some rework to make it work properly.

Implement 1-byte and 2-bytes xchg in order to achieve parity with other architectures.

Hello Arnd Bergmann, thanks for reviewing!

Quoted 3 lines

Parity with other architectures by itself is not a reason to do this, in particular the other architectures you listed have the instructions in hardware while riscv does not.

Quoted 5 lines

Sure, I understand RISC-V don't have native support for xchg on variables of size < 4B. My argument is that it's nice to have even an emulated version for this in case any future mechanism wants to use it.

Not having it may mean we won't be able to enable given mechanism in RISC-V.

Quoted 7 lines

IIUC the ask is to have a user within the kernel for these functions. That's the general thing to do, and last time this came up there was no in-kernel use of it -- the qspinlock stuff would, but we haven't enabled it yet because we're worried about the performance/fairness stuff that other ports have seen and nobody's got concrete benchmarks yet (though there's another patch set out that I haven't had time to look through, so that may have changed).

Quoted 7 lines

Conor doesn't agree with using an alternative as a detour mechanism between qspinlock & ticket lock. So I'm preparing V11 with static_key (jump_label) style. Next version, I would separate paravirt_qspinlock & CNA_qspinlock from V10. That would make it easy to review the qspinlock patch series. You can review the next version V11. Now I'm debugging a static_key init problem when load_modules, which is triggered by our combo_qspinlock.

We've seen problems with static keys and module loading in the past. You may want to take a look at commit eb6354e11630 ("riscv: Ensure isa-ext static keys are writable")

Thanks,
drew

Quoted 57 lines

The qspinlock is being tested on the riscv platform [1] with 128 cores with 8 NUMA nodes, next, I would update the comparison results of qspinlock & ticket lock.

[1]: https://www.sophon.ai/

Quoted 39 lines

So if something uses these I'm happy to go look closer.

Quoted 36 lines
Quoted 4 lines

Emulating the small xchg() through cmpxchg() is particularly tricky since it's easy to run into a case where this does not guarantee forward progress.

Didn't get this part:
By "emulating small xchg() through cmpxchg()", did you mean like emulating an xchg (usually 1 instruction) with lr & sc (same used in cmpxchg) ?

If so, yeah, it's a fair point: in some extreme case we could have multiple threads accessing given cacheline and have sc always failing. On the other hand, there are 2 arguments on that:

1 - Other architectures, (such as powerpc, arm and arm64 without LSE atomics) also seem to rely in this mechanism for every xchg size. Another archs like csky and loongarch use asm that look like mine to handle size < 4B xchg.

Quoted 3 lines

This is also something that almost no architecture
specific code relies on (generic qspinlock being a notable exception).

2 - As you mentioned, there should be very little code that will actually make use of xchg for vars < 4B, so it should be safe to assume its fine to not guarantee forward progress for those rare usages (like some of above mentioned archs).

Quoted 2 lines

I would recommend just dropping this patch from the series, at least until there is a need for it.

While I agree this is a valid point, I believe its more interesting to have it implemented if any future mechanism wants to make use of this.

Thanks!
Leo

Signature · 7 lines
Best Regards
 Guo Ren

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
#16 lore raw

Re: [RFC PATCH v5 5/5] riscv/cmpxchg: Implement xchg for variables of size 1 and 2

Quoted 45 lines · Andrew Jones
Quoted 35 lines · Guo Ren
Quoted 27 lines · Palmer Dabbelt
Quoted 19 lines · On Thu, 10 Aug 2023 09:04:04 PDT (-0700), [email protected] wrote:
Quoted 11 lines · Arnd Bergmann
Quoted 9 lines · Leonardo Bras

xchg for variables of size 1-byte and 2-bytes is not yet available for riscv, even though its present in other architectures such as arm64 and x86. This could lead to not being able to implement some locking mechanisms or requiring some rework to make it work properly.

Implement 1-byte and 2-bytes xchg in order to achieve parity with other architectures.

Hello Arnd Bergmann, thanks for reviewing!

Quoted 3 lines

Parity with other architectures by itself is not a reason to do this, in particular the other architectures you listed have the instructions in hardware while riscv does not.

Quoted 5 lines

Sure, I understand RISC-V don't have native support for xchg on variables of size < 4B. My argument is that it's nice to have even an emulated version for this in case any future mechanism wants to use it.

Not having it may mean we won't be able to enable given mechanism in RISC-V.

Quoted 7 lines

IIUC the ask is to have a user within the kernel for these functions. That's the general thing to do, and last time this came up there was no in-kernel use of it -- the qspinlock stuff would, but we haven't enabled it yet because we're worried about the performance/fairness stuff that other ports have seen and nobody's got concrete benchmarks yet (though there's another patch set out that I haven't had time to look through, so that may have changed).

Quoted 7 lines

Conor doesn't agree with using an alternative as a detour mechanism between qspinlock & ticket lock. So I'm preparing V11 with static_key (jump_label) style. Next version, I would separate paravirt_qspinlock & CNA_qspinlock from V10. That would make it easy to review the qspinlock patch series. You can review the next version V11. Now I'm debugging a static_key init problem when load_modules, which is triggered by our combo_qspinlock.

Quoted 3 lines

We've seen problems with static keys and module loading in the past. You may want to take a look at commit eb6354e11630 ("riscv: Ensure isa-ext static keys are writable")

Thank you for the tip. I would take a look.

Quoted 61 lines

Thanks,
drew

Quoted 57 lines

The qspinlock is being tested on the riscv platform [1] with 128 cores with 8 NUMA nodes, next, I would update the comparison results of qspinlock & ticket lock.

[1]: https://www.sophon.ai/

Quoted 39 lines

So if something uses these I'm happy to go look closer.

Quoted 36 lines
Quoted 4 lines

Emulating the small xchg() through cmpxchg() is particularly tricky since it's easy to run into a case where this does not guarantee forward progress.

Didn't get this part:
By "emulating small xchg() through cmpxchg()", did you mean like emulating an xchg (usually 1 instruction) with lr & sc (same used in cmpxchg) ?

If so, yeah, it's a fair point: in some extreme case we could have multiple threads accessing given cacheline and have sc always failing. On the other hand, there are 2 arguments on that:

1 - Other architectures, (such as powerpc, arm and arm64 without LSE atomics) also seem to rely in this mechanism for every xchg size. Another archs like csky and loongarch use asm that look like mine to handle size < 4B xchg.

Quoted 3 lines

This is also something that almost no architecture
specific code relies on (generic qspinlock being a notable exception).

2 - As you mentioned, there should be very little code that will actually make use of xchg for vars < 4B, so it should be safe to assume its fine to not guarantee forward progress for those rare usages (like some of above mentioned archs).

Quoted 2 lines

I would recommend just dropping this patch from the series, at least until there is a need for it.

While I agree this is a valid point, I believe its more interesting to have it implemented if any future mechanism wants to make use of this.

Thanks!
Leo

--
Best Regards
Guo Ren

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

Signature · 2 lines
Best Regards
 Guo Ren
#17 lore raw

Re: [RFC PATCH v5 5/5] riscv/cmpxchg: Implement xchg for variables of size 1 and 2

In reply to Palmer Dabbelt
Quoted 34 lines · Palmer Dabbelt
Quoted 19 lines · On Thu, 10 Aug 2023 09:04:04 PDT (-0700), [email protected] wrote:
Quoted 11 lines · Arnd Bergmann
Quoted 9 lines · Leonardo Bras

xchg for variables of size 1-byte and 2-bytes is not yet available for riscv, even though its present in other architectures such as arm64 and x86. This could lead to not being able to implement some locking mechanisms or requiring some rework to make it work properly.

Implement 1-byte and 2-bytes xchg in order to achieve parity with other architectures.

Hello Arnd Bergmann, thanks for reviewing!

Quoted 3 lines

Parity with other architectures by itself is not a reason to do this, in particular the other architectures you listed have the instructions in hardware while riscv does not.

Quoted 5 lines

Sure, I understand RISC-V don't have native support for xchg on variables of size < 4B. My argument is that it's nice to have even an emulated version for this in case any future mechanism wants to use it.

Not having it may mean we won't be able to enable given mechanism in RISC-V.

IIUC the ask is to have a user within the kernel for these functions. That's the general thing to do, and last time this came up there was no in-kernel use of it -- the qspinlock stuff would, but we haven't enabled it yet because we're worried about the performance/fairness stuff that other ports have seen and nobody's got concrete benchmarks yet (though there's another patch set out that I haven't had time to look through, so that may have changed).

Quoted 1 line

So if something uses these I'm happy to go look closer.

IIUC patches 4 & 5 will be used by qspinlock, which may not be done yet, so we don't have an use for them for the time being.

Otherwise, any comments on patches 1, 2 & 3?

Quoted 31 lines
Quoted 31 lines
Quoted 5 lines

Emulating the small xchg() through cmpxchg() is particularly tricky since it's easy to run into a case where this does not guarantee forward progress.

Didn't get this part:
By "emulating small xchg() through cmpxchg()", did you mean like emulating an xchg (usually 1 instruction) with lr & sc (same used in cmpxchg) ?

If so, yeah, it's a fair point: in some extreme case we could have multiple threads accessing given cacheline and have sc always failing. On the other hand, there are 2 arguments on that:

1 - Other architectures, (such as powerpc, arm and arm64 without LSE atomics) also seem to rely in this mechanism for every xchg size. Another archs like csky and loongarch use asm that look like mine to handle size < 4B xchg.

Quoted 3 lines

This is also something that almost no architecture
specific code relies on (generic qspinlock being a notable exception).

2 - As you mentioned, there should be very little code that will actually make use of xchg for vars < 4B, so it should be safe to assume its fine to not guarantee forward progress for those rare usages (like some of above mentioned archs).

Quoted 2 lines

I would recommend just dropping this patch from the series, at least until there is a need for it.

Quoted 7 lines
Quoted 6 lines

While I agree this is a valid point, I believe its more interesting to have it implemented if any future mechanism wants to make use of this.

Thanks!
Leo

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

#18 lore raw

Re: [RFC PATCH v5 5/5] riscv/cmpxchg: Implement xchg for variables of size 1 and 2

Quoted 36 lines · Leonardo Brás
Quoted 34 lines · Palmer Dabbelt
Quoted 19 lines · On Thu, 10 Aug 2023 09:04:04 PDT (-0700), [email protected] wrote:
Quoted 11 lines · Arnd Bergmann
Quoted 9 lines · Leonardo Bras

xchg for variables of size 1-byte and 2-bytes is not yet available for riscv, even though its present in other architectures such as arm64 and x86. This could lead to not being able to implement some locking mechanisms or requiring some rework to make it work properly.

Implement 1-byte and 2-bytes xchg in order to achieve parity with other architectures.

Hello Arnd Bergmann, thanks for reviewing!

Quoted 3 lines

Parity with other architectures by itself is not a reason to do this, in particular the other architectures you listed have the instructions in hardware while riscv does not.

Quoted 5 lines

Sure, I understand RISC-V don't have native support for xchg on variables of size < 4B. My argument is that it's nice to have even an emulated version for this in case any future mechanism wants to use it.

Not having it may mean we won't be able to enable given mechanism in RISC-V.

IIUC the ask is to have a user within the kernel for these functions. That's the general thing to do, and last time this came up there was no in-kernel use of it -- the qspinlock stuff would, but we haven't enabled it yet because we're worried about the performance/fairness stuff that other ports have seen and nobody's got concrete benchmarks yet (though there's another patch set out that I haven't had time to look through, so that may have changed).

Quoted 6 lines
Quoted 1 line

So if something uses these I'm happy to go look closer.

IIUC patches 4 & 5 will be used by qspinlock, which may not be done yet, so we don't have an use for them for the time being.

Otherwise, any comments on patches 1, 2 & 3?

Hello Palmer,
Any chance of patches 1, 2 & 3 being merged in this merge window?

Thanks!

Quoted 32 lines
Quoted 32 lines
Quoted 32 lines
Quoted 6 lines

Emulating the small xchg() through cmpxchg() is particularly tricky since it's easy to run into a case where this does not guarantee forward progress.

Didn't get this part:
By "emulating small xchg() through cmpxchg()", did you mean like emulating an xchg (usually 1 instruction) with lr & sc (same used in cmpxchg) ?

If so, yeah, it's a fair point: in some extreme case we could have multiple threads accessing given cacheline and have sc always failing. On the other hand, there are 2 arguments on that:

1 - Other architectures, (such as powerpc, arm and arm64 without LSE atomics) also seem to rely in this mechanism for every xchg size. Another archs like csky and loongarch use asm that look like mine to handle size < 4B xchg.

Quoted 3 lines

This is also something that almost no architecture
specific code relies on (generic qspinlock being a notable exception).

2 - As you mentioned, there should be very little code that will actually make use of xchg for vars < 4B, so it should be safe to assume its fine to not guarantee forward progress for those rare usages (like some of above mentioned archs).

Quoted 2 lines

I would recommend just dropping this patch from the series, at least until there is a need for it.

Quoted 8 lines
Quoted 7 lines
Quoted 6 lines

While I agree this is a valid point, I believe its more interesting to have it implemented if any future mechanism wants to make use of this.

Thanks!
Leo

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

#19 lore raw

Re: [RFC PATCH v5 5/5] riscv/cmpxchg: Implement xchg for variables of size 1 and 2

In reply to Leonardo Brás

From: Leonardo Bras <[email protected]>

Quoted 35 lines · Leonardo Brás
Quoted 34 lines · Palmer Dabbelt
Quoted 19 lines · On Thu, 10 Aug 2023 09:04:04 PDT (-0700), [email protected] wrote:
Quoted 11 lines · Arnd Bergmann
Quoted 9 lines · Leonardo Bras

xchg for variables of size 1-byte and 2-bytes is not yet available for riscv, even though its present in other architectures such as arm64 and x86. This could lead to not being able to implement some locking mechanisms or requiring some rework to make it work properly.

Implement 1-byte and 2-bytes xchg in order to achieve parity with other architectures.

Hello Arnd Bergmann, thanks for reviewing!

Quoted 3 lines

Parity with other architectures by itself is not a reason to do this, in particular the other architectures you listed have the instructions in hardware while riscv does not.

Quoted 5 lines

Sure, I understand RISC-V don't have native support for xchg on variables of size < 4B. My argument is that it's nice to have even an emulated version for this in case any future mechanism wants to use it.

Not having it may mean we won't be able to enable given mechanism in RISC-V.

IIUC the ask is to have a user within the kernel for these functions. That's the general thing to do, and last time this came up there was no in-kernel use of it -- the qspinlock stuff would, but we haven't enabled it yet because we're worried about the performance/fairness stuff that other ports have seen and nobody's got concrete benchmarks yet (though there's another patch set out that I haven't had time to look through, so that may have changed).

Quoted 6 lines
Quoted 1 line

So if something uses these I'm happy to go look closer.

IIUC patches 4 & 5 will be used by qspinlock, which may not be done yet, so we don't have an use for them for the time being.

Otherwise, any comments on patches 1, 2 & 3?

ping

Quoted 40 lines
Quoted 39 lines
Quoted 38 lines
Quoted 6 lines

Emulating the small xchg() through cmpxchg() is particularly tricky since it's easy to run into a case where this does not guarantee forward progress.

Didn't get this part:
By "emulating small xchg() through cmpxchg()", did you mean like emulating an xchg (usually 1 instruction) with lr & sc (same used in cmpxchg) ?

If so, yeah, it's a fair point: in some extreme case we could have multiple threads accessing given cacheline and have sc always failing. On the other hand, there are 2 arguments on that:

1 - Other architectures, (such as powerpc, arm and arm64 without LSE atomics) also seem to rely in this mechanism for every xchg size. Another archs like csky and loongarch use asm that look like mine to handle size < 4B xchg.

Quoted 3 lines

This is also something that almost no architecture
specific code relies on (generic qspinlock being a notable exception).

2 - As you mentioned, there should be very little code that will actually make use of xchg for vars < 4B, so it should be safe to assume its fine to not guarantee forward progress for those rare usages (like some of above mentioned archs).

Quoted 2 lines

I would recommend just dropping this patch from the series, at least until there is a need for it.

While I agree this is a valid point, I believe its more interesting to have it implemented if any future mechanism wants to make use of this.

Thanks!
Leo

#20 lore raw

Re: [RFC PATCH v5 5/5] riscv/cmpxchg: Implement xchg for variables of size 1 and 2

Quoted 38 lines · On Tue, Dec 05, 2023 at 09:56:44PM -0300, [email protected] wrote:

From: Leonardo Bras <[email protected]>

Quoted 35 lines · Leonardo Brás
Quoted 34 lines · Palmer Dabbelt
Quoted 19 lines · On Thu, 10 Aug 2023 09:04:04 PDT (-0700), [email protected] wrote:
Quoted 11 lines · Arnd Bergmann
Quoted 9 lines · Leonardo Bras

xchg for variables of size 1-byte and 2-bytes is not yet available for riscv, even though its present in other architectures such as arm64 and x86. This could lead to not being able to implement some locking mechanisms or requiring some rework to make it work properly.

Implement 1-byte and 2-bytes xchg in order to achieve parity with other architectures.

Hello Arnd Bergmann, thanks for reviewing!

Quoted 3 lines

Parity with other architectures by itself is not a reason to do this, in particular the other architectures you listed have the instructions in hardware while riscv does not.

Quoted 5 lines

Sure, I understand RISC-V don't have native support for xchg on variables of size < 4B. My argument is that it's nice to have even an emulated version for this in case any future mechanism wants to use it.

Not having it may mean we won't be able to enable given mechanism in RISC-V.

IIUC the ask is to have a user within the kernel for these functions. That's the general thing to do, and last time this came up there was no in-kernel use of it -- the qspinlock stuff would, but we haven't enabled it yet because we're worried about the performance/fairness stuff that other ports have seen and nobody's got concrete benchmarks yet (though there's another patch set out that I haven't had time to look through, so that may have changed).

Quoted 8 lines
Quoted 6 lines
Quoted 1 line

So if something uses these I'm happy to go look closer.

IIUC patches 4 & 5 will be used by qspinlock, which may not be done yet, so we don't have an use for them for the time being.

Otherwise, any comments on patches 1, 2 & 3?

ping

Hi,

I believe the "RFC" makes some reviewers think the series isn't ready for review, so could you please send a new one w/o RFC?

thanks

Quoted 46 lines
Quoted 33 lines
Quoted 33 lines
Quoted 33 lines
Quoted 7 lines

Emulating the small xchg() through cmpxchg() is particularly tricky since it's easy to run into a case where this does not guarantee forward progress.

Didn't get this part:
By "emulating small xchg() through cmpxchg()", did you mean like emulating an xchg (usually 1 instruction) with lr & sc (same used in cmpxchg) ?

If so, yeah, it's a fair point: in some extreme case we could have multiple threads accessing given cacheline and have sc always failing. On the other hand, there are 2 arguments on that:

1 - Other architectures, (such as powerpc, arm and arm64 without LSE atomics) also seem to rely in this mechanism for every xchg size. Another archs like csky and loongarch use asm that look like mine to handle size < 4B xchg.

Quoted 3 lines

This is also something that almost no architecture
specific code relies on (generic qspinlock being a notable exception).

2 - As you mentioned, there should be very little code that will actually make use of xchg for vars < 4B, so it should be safe to assume its fine to not guarantee forward progress for those rare usages (like some of above mentioned archs).

Quoted 2 lines

I would recommend just dropping this patch from the series, at least until there is a need for it.

Quoted 8 lines
Quoted 7 lines
Quoted 6 lines

While I agree this is a valid point, I believe its more interesting to have it implemented if any future mechanism wants to make use of this.

Thanks!
Leo

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

#21 lore raw

Re: [RFC PATCH v5 5/5] riscv/cmpxchg: Implement xchg for variables of size 1 and 2

Quoted 46 lines · Jisheng Zhang
Quoted 45 lines · On Tue, Dec 05, 2023 at 09:56:44PM -0300, [email protected] wrote:

From: Leonardo Bras <[email protected]>

Quoted 41 lines · Leonardo Brás
Quoted 34 lines · Palmer Dabbelt
Quoted 19 lines · On Thu, 10 Aug 2023 09:04:04 PDT (-0700), [email protected] wrote:
Quoted 11 lines · Arnd Bergmann
Quoted 9 lines · Leonardo Bras

xchg for variables of size 1-byte and 2-bytes is not yet available for riscv, even though its present in other architectures such as arm64 and x86. This could lead to not being able to implement some locking mechanisms or requiring some rework to make it work properly.

Implement 1-byte and 2-bytes xchg in order to achieve parity with other architectures.

Hello Arnd Bergmann, thanks for reviewing!

Quoted 3 lines

Parity with other architectures by itself is not a reason to do this, in particular the other architectures you listed have the instructions in hardware while riscv does not.

Quoted 5 lines

Sure, I understand RISC-V don't have native support for xchg on variables of size < 4B. My argument is that it's nice to have even an emulated version for this in case any future mechanism wants to use it.

Not having it may mean we won't be able to enable given mechanism in RISC-V.

IIUC the ask is to have a user within the kernel for these functions. That's the general thing to do, and last time this came up there was no in-kernel use of it -- the qspinlock stuff would, but we haven't enabled it yet because we're worried about the performance/fairness stuff that other ports have seen and nobody's got concrete benchmarks yet (though there's another patch set out that I haven't had time to look through, so that may have changed).

Quoted 1 line

So if something uses these I'm happy to go look closer.

IIUC patches 4 & 5 will be used by qspinlock, which may not be done yet, so we don't have an use for them for the time being.

Otherwise, any comments on patches 1, 2 & 3?

Quoted 8 lines
Quoted 1 line

ping

Hi,

I believe the "RFC" makes some reviewers think the series isn't ready for review, so could you please send a new one w/o RFC?

thanks

Sure, will do.

Thanks!
Leo

Quoted 48 lines
Quoted 47 lines
Quoted 34 lines
Quoted 34 lines
Quoted 34 lines
Quoted 8 lines

Emulating the small xchg() through cmpxchg() is particularly tricky since it's easy to run into a case where this does not guarantee forward progress.

Didn't get this part:
By "emulating small xchg() through cmpxchg()", did you mean like emulating an xchg (usually 1 instruction) with lr & sc (same used in cmpxchg) ?

If so, yeah, it's a fair point: in some extreme case we could have multiple threads accessing given cacheline and have sc always failing. On the other hand, there are 2 arguments on that:

1 - Other architectures, (such as powerpc, arm and arm64 without LSE atomics) also seem to rely in this mechanism for every xchg size. Another archs like csky and loongarch use asm that look like mine to handle size < 4B xchg.

Quoted 3 lines

This is also something that almost no architecture
specific code relies on (generic qspinlock being a notable exception).

2 - As you mentioned, there should be very little code that will actually make use of xchg for vars < 4B, so it should be safe to assume its fine to not guarantee forward progress for those rare usages (like some of above mentioned archs).

Quoted 2 lines

I would recommend just dropping this patch from the series, at least until there is a need for it.

Quoted 8 lines
Quoted 7 lines
Quoted 6 lines

While I agree this is a valid point, I believe its more interesting to have it implemented if any future mechanism wants to make use of this.

Thanks!
Leo

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

Tested-by on the v5 series
#22 lore raw
In reply to Leonardo Bras
Quoted 17 lines · Leonardo Bras

While studying riscv's cmpxchg.h file, I got really interested in understanding how RISCV asm implemented the different versions of {cmp,}xchg.

When I understood the pattern, it made sense for me to remove the duplications and create macros to make it easier to understand what exactly changes between the versions: Instruction sufixes & barriers.

Also, did the same kind of work on atomic.c.

After that, I noted both cmpxchg and xchg only accept variables of size 4 and 8, compared to x86 and arm64 which do 1,2,4,8.

Now that deduplication is done, it is quite direct to implement them for variable sizes 1 and 2, so I did it. Then Guo Ren already presented me some possible users :)

Quoted 8 lines

I did compare the generated asm on a test.c that contained usage for every changed function, and could not detect any change on patches 1 + 2 + 3 compared with upstream.

Pathes 4 & 5 were compiled-tested, merged with guoren/qspinlock_v11 and booted just fine with qemu -machine virt -append "qspinlock".

(tree: https://gitlab.com/LeoBras/linux/-/commits/guo_qspinlock_v11)

Sorry for late reply, because we are stress testing CNA qspinlock on sg2042 128 cores hardware platform. This series has passed our test for several weeks. For more detail, ref:
https://lore.kernel.org/linux-riscv/[email protected]/

Quoted 49 lines

Thanks!
Leo

Changes since squashed cmpxchg RFCv4:
- Added (__typeof__(*(p))) before returning from {cmp,}xchg, as done in current upstream, (possibly) fixing the bug from kernel test robot
https://lore.kernel.org/all/[email protected]/

Changes since squashed cmpxchg RFCv3:
- Fixed bug on cmpxchg macro for var size 1 & 2: now working
- Macros for var size 1 & 2's lr.w and sc.w now are guaranteed to receive input of a 32-bit aligned address
- Renamed internal macros from _mask to _masked for patches 4 & 5
- __rc variable on macros for var size 1 & 2 changed from register to ulong
https://lore.kernel.org/all/[email protected]/

Changes since squashed cmpxchg RFCv2:
- Removed rc parameter from the new macro: it can be internal to the macro
- 2 new patches: cmpxchg size 1 and 2, xchg size 1 and 2
https://lore.kernel.org/all/[email protected]/

Changes since squashed cmpxchg RFCv1:
- Unified with atomic.c patchset
- Rebased on top of torvalds/master (thanks Andrea Parri!)
- Removed helper macros that were not being used elsewhere in the kernel.
https://lore.kernel.org/all/[email protected]/
https://lore.kernel.org/all/[email protected]/

Changes since (cmpxchg) RFCv3:
- Squashed the 6 original patches in 2: one for cmpxchg and one for xchg
https://lore.kernel.org/all/[email protected]/

Changes since (cmpxchg) RFCv2:
- Fixed macros that depend on having a local variable with a magic name
- Previous cast to (long) is now only applied on 4-bytes cmpxchg
https://lore.kernel.org/all/[email protected]/

Changes since (cmpxchg) RFCv1:
- Fixed patch 4/6 suffix from 'w.aqrl' to '.w.aqrl', to avoid build error
https://lore.kernel.org/all/[email protected]/

Leonardo Bras (5):
riscv/cmpxchg: Deduplicate xchg() asm functions riscv/cmpxchg: Deduplicate cmpxchg() asm and macros riscv/atomic.h : Deduplicate arch_atomic.* riscv/cmpxchg: Implement cmpxchg for variables of size 1 and 2 riscv/cmpxchg: Implement xchg for variables of size 1 and 2

Quoted 15 lines
 arch/riscv/include/asm/atomic.h  | 164 ++++++-------
 arch/riscv/include/asm/cmpxchg.h | 404 ++++++++++---------------------
 2 files changed, 200 insertions(+), 368 deletions(-)

base-commit: cacc6e22932f373a91d7be55a9b992dc77f4c59b

Signature · 7 lines
2.41.0


_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

#23 lore raw
Quoted 26 lines · Guo Ren
Quoted 24 lines · Leonardo Bras

While studying riscv's cmpxchg.h file, I got really interested in understanding how RISCV asm implemented the different versions of {cmp,}xchg.

When I understood the pattern, it made sense for me to remove the duplications and create macros to make it easier to understand what exactly changes between the versions: Instruction sufixes & barriers.

Also, did the same kind of work on atomic.c.

After that, I noted both cmpxchg and xchg only accept variables of size 4 and 8, compared to x86 and arm64 which do 1,2,4,8.

Now that deduplication is done, it is quite direct to implement them for variable sizes 1 and 2, so I did it. Then Guo Ren already presented me some possible users :)

I did compare the generated asm on a test.c that contained usage for every changed function, and could not detect any change on patches 1 + 2 + 3 compared with upstream.

Pathes 4 & 5 were compiled-tested, merged with guoren/qspinlock_v11 and booted just fine with qemu -machine virt -append "qspinlock".

Quoted 8 lines
Quoted 1 line

(tree: https://gitlab.com/LeoBras/linux/-/commits/guo_qspinlock_v11)

Sorry for late reply, because we are stress testing CNA qspinlock on sg2042 128 cores hardware platform. This series has passed our test for several weeks. For more detail, ref:
https://lore.kernel.org/linux-riscv/[email protected]/

That's awesome!
Thanks for testing!

Leo

Quoted 59 lines
Quoted 59 lines

Thanks!
Leo

Changes since squashed cmpxchg RFCv4:
- Added (__typeof__(*(p))) before returning from {cmp,}xchg, as done in current upstream, (possibly) fixing the bug from kernel test robot
https://lore.kernel.org/all/[email protected]/

Changes since squashed cmpxchg RFCv3:
- Fixed bug on cmpxchg macro for var size 1 & 2: now working
- Macros for var size 1 & 2's lr.w and sc.w now are guaranteed to receive input of a 32-bit aligned address
- Renamed internal macros from _mask to _masked for patches 4 & 5
- __rc variable on macros for var size 1 & 2 changed from register to ulong
https://lore.kernel.org/all/[email protected]/

Changes since squashed cmpxchg RFCv2:
- Removed rc parameter from the new macro: it can be internal to the macro
- 2 new patches: cmpxchg size 1 and 2, xchg size 1 and 2
https://lore.kernel.org/all/[email protected]/

Changes since squashed cmpxchg RFCv1:
- Unified with atomic.c patchset
- Rebased on top of torvalds/master (thanks Andrea Parri!)
- Removed helper macros that were not being used elsewhere in the kernel.
https://lore.kernel.org/all/[email protected]/
https://lore.kernel.org/all/[email protected]/

Changes since (cmpxchg) RFCv3:
- Squashed the 6 original patches in 2: one for cmpxchg and one for xchg
https://lore.kernel.org/all/[email protected]/

Changes since (cmpxchg) RFCv2:
- Fixed macros that depend on having a local variable with a magic name
- Previous cast to (long) is now only applied on 4-bytes cmpxchg
https://lore.kernel.org/all/[email protected]/

Changes since (cmpxchg) RFCv1:
- Fixed patch 4/6 suffix from 'w.aqrl' to '.w.aqrl', to avoid build error
https://lore.kernel.org/all/[email protected]/

Leonardo Bras (5):
riscv/cmpxchg: Deduplicate xchg() asm functions riscv/cmpxchg: Deduplicate cmpxchg() asm and macros riscv/atomic.h : Deduplicate arch_atomic.* riscv/cmpxchg: Implement cmpxchg for variables of size 1 and 2 riscv/cmpxchg: Implement xchg for variables of size 1 and 2

 arch/riscv/include/asm/atomic.h  | 164 ++++++-------
 arch/riscv/include/asm/cmpxchg.h | 404 ++++++++++---------------------
 2 files changed, 200 insertions(+), 368 deletions(-)

base-commit: cacc6e22932f373a91d7be55a9b992dc77f4c59b --
2.41.0

Quoted 6 lines
Quoted 5 lines

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

#24 lore raw
In reply to Guo Ren
Quoted 22 lines · Guo Ren
Quoted 21 lines · Leonardo Bras

While studying riscv's cmpxchg.h file, I got really interested in understanding how RISCV asm implemented the different versions of {cmp,}xchg.

When I understood the pattern, it made sense for me to remove the duplications and create macros to make it easier to understand what exactly changes between the versions: Instruction sufixes & barriers.

Also, did the same kind of work on atomic.c.

After that, I noted both cmpxchg and xchg only accept variables of size 4 and 8, compared to x86 and arm64 which do 1,2,4,8.

Now that deduplication is done, it is quite direct to implement them for variable sizes 1 and 2, so I did it. Then Guo Ren already presented me some possible users :)

I did compare the generated asm on a test.c that contained usage for every changed function, and could not detect any change on patches 1 + 2 + 3 compared with upstream.

Quoted 6 lines
Quoted 4 lines

Pathes 4 & 5 were compiled-tested, merged with guoren/qspinlock_v11 and booted just fine with qemu -machine virt -append "qspinlock".

(tree: https://gitlab.com/LeoBras/linux/-/commits/guo_qspinlock_v11)

Hello Guo Ren, thanks for testing!

I will resend this series, and I would like to understand how should I put your Tested-by over this patchset:

Is it ok if I add it on each patch of this series?

Thanks!
Leo

Quoted 70 lines

Sorry for late reply, because we are stress testing CNA qspinlock on sg2042 128 cores hardware platform. This series has passed our test for several weeks. For more detail, ref:
https://lore.kernel.org/linux-riscv/[email protected]/

Quoted 64 lines

Thanks!
Leo

Changes since squashed cmpxchg RFCv4:
- Added (__typeof__(*(p))) before returning from {cmp,}xchg, as done in current upstream, (possibly) fixing the bug from kernel test robot
https://lore.kernel.org/all/[email protected]/

Changes since squashed cmpxchg RFCv3:
- Fixed bug on cmpxchg macro for var size 1 & 2: now working
- Macros for var size 1 & 2's lr.w and sc.w now are guaranteed to receive input of a 32-bit aligned address
- Renamed internal macros from _mask to _masked for patches 4 & 5
- __rc variable on macros for var size 1 & 2 changed from register to ulong
https://lore.kernel.org/all/[email protected]/

Changes since squashed cmpxchg RFCv2:
- Removed rc parameter from the new macro: it can be internal to the macro
- 2 new patches: cmpxchg size 1 and 2, xchg size 1 and 2
https://lore.kernel.org/all/[email protected]/

Changes since squashed cmpxchg RFCv1:
- Unified with atomic.c patchset
- Rebased on top of torvalds/master (thanks Andrea Parri!)
- Removed helper macros that were not being used elsewhere in the kernel.
https://lore.kernel.org/all/[email protected]/
https://lore.kernel.org/all/[email protected]/

Changes since (cmpxchg) RFCv3:
- Squashed the 6 original patches in 2: one for cmpxchg and one for xchg
https://lore.kernel.org/all/[email protected]/

Changes since (cmpxchg) RFCv2:
- Fixed macros that depend on having a local variable with a magic name
- Previous cast to (long) is now only applied on 4-bytes cmpxchg
https://lore.kernel.org/all/[email protected]/

Changes since (cmpxchg) RFCv1:
- Fixed patch 4/6 suffix from 'w.aqrl' to '.w.aqrl', to avoid build error
https://lore.kernel.org/all/[email protected]/

Leonardo Bras (5):
riscv/cmpxchg: Deduplicate xchg() asm functions riscv/cmpxchg: Deduplicate cmpxchg() asm and macros riscv/atomic.h : Deduplicate arch_atomic.* riscv/cmpxchg: Implement cmpxchg for variables of size 1 and 2 riscv/cmpxchg: Implement xchg for variables of size 1 and 2

 arch/riscv/include/asm/atomic.h  | 164 ++++++-------
 arch/riscv/include/asm/cmpxchg.h | 404 ++++++++++---------------------
 2 files changed, 200 insertions(+), 368 deletions(-)

base-commit: cacc6e22932f373a91d7be55a9b992dc77f4c59b

Signature · 7 lines
2.41.0


_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
#25 lore raw
Quoted 31 lines · Leonardo Bras
Quoted 22 lines · Guo Ren
Quoted 21 lines · Leonardo Bras

While studying riscv's cmpxchg.h file, I got really interested in understanding how RISCV asm implemented the different versions of {cmp,}xchg.

When I understood the pattern, it made sense for me to remove the duplications and create macros to make it easier to understand what exactly changes between the versions: Instruction sufixes & barriers.

Also, did the same kind of work on atomic.c.

After that, I noted both cmpxchg and xchg only accept variables of size 4 and 8, compared to x86 and arm64 which do 1,2,4,8.

Now that deduplication is done, it is quite direct to implement them for variable sizes 1 and 2, so I did it. Then Guo Ren already presented me some possible users :)

I did compare the generated asm on a test.c that contained usage for every changed function, and could not detect any change on patches 1 + 2 + 3 compared with upstream.

Quoted 6 lines
Quoted 4 lines

Pathes 4 & 5 were compiled-tested, merged with guoren/qspinlock_v11 and booted just fine with qemu -machine virt -append "qspinlock".

(tree: https://gitlab.com/LeoBras/linux/-/commits/guo_qspinlock_v11)

Quoted 6 lines

Hello Guo Ren, thanks for testing!

I will resend this series, and I would like to understand how should I put your Tested-by over this patchset:

Is it ok if I add it on each patch of this series?

Yes, my qspinlock_v12 based on your patch series.
https://github.com/guoren83/linux/tree/qspinlock_v12

Some people tell me paravirt-qspinlock can't work with nested virtualization, but I haven't found a way to set up a test environment. I'm working on that.

Quoted 76 lines

Thanks!
Leo

Quoted 70 lines

Sorry for late reply, because we are stress testing CNA qspinlock on sg2042 128 cores hardware platform. This series has passed our test for several weeks. For more detail, ref:
https://lore.kernel.org/linux-riscv/[email protected]/

Quoted 64 lines

Thanks!
Leo

Changes since squashed cmpxchg RFCv4:
- Added (__typeof__(*(p))) before returning from {cmp,}xchg, as done in current upstream, (possibly) fixing the bug from kernel test robot
https://lore.kernel.org/all/[email protected]/

Changes since squashed cmpxchg RFCv3:
- Fixed bug on cmpxchg macro for var size 1 & 2: now working
- Macros for var size 1 & 2's lr.w and sc.w now are guaranteed to receive input of a 32-bit aligned address
- Renamed internal macros from _mask to _masked for patches 4 & 5
- __rc variable on macros for var size 1 & 2 changed from register to ulong
https://lore.kernel.org/all/[email protected]/

Changes since squashed cmpxchg RFCv2:
- Removed rc parameter from the new macro: it can be internal to the macro
- 2 new patches: cmpxchg size 1 and 2, xchg size 1 and 2
https://lore.kernel.org/all/[email protected]/

Changes since squashed cmpxchg RFCv1:
- Unified with atomic.c patchset
- Rebased on top of torvalds/master (thanks Andrea Parri!)
- Removed helper macros that were not being used elsewhere in the kernel.
https://lore.kernel.org/all/[email protected]/
https://lore.kernel.org/all/[email protected]/

Changes since (cmpxchg) RFCv3:
- Squashed the 6 original patches in 2: one for cmpxchg and one for xchg
https://lore.kernel.org/all/[email protected]/

Changes since (cmpxchg) RFCv2:
- Fixed macros that depend on having a local variable with a magic name
- Previous cast to (long) is now only applied on 4-bytes cmpxchg
https://lore.kernel.org/all/[email protected]/

Changes since (cmpxchg) RFCv1:
- Fixed patch 4/6 suffix from 'w.aqrl' to '.w.aqrl', to avoid build error
https://lore.kernel.org/all/[email protected]/

Leonardo Bras (5):
riscv/cmpxchg: Deduplicate xchg() asm functions riscv/cmpxchg: Deduplicate cmpxchg() asm and macros riscv/atomic.h : Deduplicate arch_atomic.* riscv/cmpxchg: Implement cmpxchg for variables of size 1 and 2 riscv/cmpxchg: Implement xchg for variables of size 1 and 2

 arch/riscv/include/asm/atomic.h  | 164 ++++++-------
 arch/riscv/include/asm/cmpxchg.h | 404 ++++++++++---------------------
 2 files changed, 200 insertions(+), 368 deletions(-)

base-commit: cacc6e22932f373a91d7be55a9b992dc77f4c59b --
2.41.0

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

--
Best Regards
Guo Ren