Skip to contentSkip to conversation

riscv: Add Zalasr ISA extension support

PATCH Superseded Xu Lu 3 participants 15 messages Sep 19, 2025 – Sep 20, 2025

Subsystems (7)

v3 of 4 Superseded Latest is v4 →

lore
Share & tools
lore

Xu Lu

author
#0 lore raw

v3 cover riscv: Add Zalasr ISA extension support

This patch adds support for the Zalasr ISA extension, which supplies the real load acquire/store release instructions.

The specification can be found here:
https://github.com/riscv/riscv-zalasr/blob/main/chapter2.adoc

This patch seires has been tested with ltp on Qemu with Brensan's zalasr support patch[1].

Some false positive spacing error happens during patch checking. Thus I CCed maintainers of checkpatch.pl as well.

[1] https://lore.kernel.org/all/CAGPSXwJEdtqW=nx71oufZp64nK6tK=0rytVEcz4F-gfvCOXk2w@mail.gmail.com/

v3:
- Apply acquire/release semantics to arch_xchg/arch_cmpxchg operations so as to ensure FENCE.TSO ordering between operations which precede the UNLOCK+LOCK sequence and operations which follow the sequence. Thanks to Andrea.
- Support hwprobe of Zalasr.
- Allow Zalasr extensions for Guest/VM.

v2:
- Adjust the order of Zalasr and Zalrsc in dt-bindings. Thanks to Conor.

Xu Lu (8):
riscv: add ISA extension parsing for Zalasr
dt-bindings: riscv: Add Zalasr ISA extension description
riscv: hwprobe: Export Zalasr extension
riscv: Introduce Zalasr instructions
riscv: Use Zalasr for smp_load_acquire/smp_store_release
riscv: Apply acquire/release semantics to arch_xchg/arch_cmpxchg
operations
RISC-V: KVM: Allow Zalasr extensions for Guest/VM
KVM: riscv: selftests: Add Zalasr extensions to get-reg-list test

 Documentation/arch/riscv/hwprobe.rst          |   5 +-
 .../devicetree/bindings/riscv/extensions.yaml |   5 +
 arch/riscv/include/asm/atomic.h               |   6 -
 arch/riscv/include/asm/barrier.h              |  91 ++++++++++--
 arch/riscv/include/asm/cmpxchg.h              | 136 ++++++++----------
 arch/riscv/include/asm/hwcap.h                |   1 +
 arch/riscv/include/asm/insn-def.h             |  79 ++++++++++
 arch/riscv/include/uapi/asm/hwprobe.h         |   1 +
 arch/riscv/include/uapi/asm/kvm.h             |   1 +
 arch/riscv/kernel/cpufeature.c                |   1 +
 arch/riscv/kernel/sys_hwprobe.c               |   1 +
 arch/riscv/kvm/vcpu_onereg.c                  |   2 +
 .../selftests/kvm/riscv/get-reg-list.c        |   4 +
 13 files changed, 242 insertions(+), 91 deletions(-)
Signature · 1 line
2.20.1
author
#1 lore raw

PATCH v3 1/8 riscv: add ISA extension parsing for Zalasr

Add parsing for Zalasr ISA extension.

---
 arch/riscv/include/asm/hwcap.h | 1 +
 arch/riscv/kernel/cpufeature.c | 1 +
 2 files changed, 2 insertions(+)
arch/riscv/include/asm/hwcap.h +1 -0
index affd63e11b0a3..ae3852c4f2ca2 100644
@@ -106,6 +106,7 @@
 #define RISCV_ISA_EXT_ZAAMO		97
 #define RISCV_ISA_EXT_ZALRSC		98
 #define RISCV_ISA_EXT_ZICBOP		99
+#define RISCV_ISA_EXT_ZALASR		100
 
 #define RISCV_ISA_EXT_XLINUXENVCFG	127
 
arch/riscv/kernel/cpufeature.c +1 -0
index 743d53415572e..bf9d3d92bf372 100644
@@ -472,6 +472,7 @@ const struct riscv_isa_ext_data riscv_isa_ext[] = {
 	__RISCV_ISA_EXT_DATA(zaamo, RISCV_ISA_EXT_ZAAMO),
 	__RISCV_ISA_EXT_DATA(zabha, RISCV_ISA_EXT_ZABHA),
 	__RISCV_ISA_EXT_DATA(zacas, RISCV_ISA_EXT_ZACAS),
+	__RISCV_ISA_EXT_DATA(zalasr, RISCV_ISA_EXT_ZALASR),
 	__RISCV_ISA_EXT_DATA(zalrsc, RISCV_ISA_EXT_ZALRSC),
 	__RISCV_ISA_EXT_DATA(zawrs, RISCV_ISA_EXT_ZAWRS),
 	__RISCV_ISA_EXT_DATA(zfa, RISCV_ISA_EXT_ZFA),
Signature · 1 line
2.20.1
author
#2 lore raw

PATCH v3 2/8 dt-bindings: riscv: Add Zalasr ISA extension description

In reply to Xu Lu

Add description for the Zalasr ISA extension

---
 Documentation/devicetree/bindings/riscv/extensions.yaml | 5 +++++
 1 file changed, 5 insertions(+)
Documentation/devicetree/bindings/riscv/extensions.yaml +5 -0
index ede6a58ccf534..100fe53fb0731 100644
@@ -242,6 +242,11 @@ properties:
             is supported as ratified at commit 5059e0ca641c ("update to
             ratified") of the riscv-zacas.
 
+        - const: zalasr
+          description: |
+            The standard Zalasr extension for load-acquire/store-release as frozen
+            at commit 194f0094 ("Version 0.9 for freeze") of riscv-zalasr.
+
         - const: zalrsc
           description: |
             The standard Zalrsc extension for load-reserved/store-conditional as
Signature · 1 line
2.20.1
author
#3 lore raw

PATCH v3 3/8 riscv: hwprobe: Export Zalasr extension

In reply to Xu Lu

Export the Zalasr extension to userspace using hwprobe.

---
 Documentation/arch/riscv/hwprobe.rst  | 5 ++++-
 arch/riscv/include/uapi/asm/hwprobe.h | 1 +
 arch/riscv/kernel/sys_hwprobe.c       | 1 +
 3 files changed, 6 insertions(+), 1 deletion(-)
Documentation/arch/riscv/hwprobe.rst +4 -1
index 2aa9be272d5de..067a3595fb9d5 100644
@@ -249,6 +249,9 @@ The following keys are defined:
        defined in the in the RISC-V ISA manual starting from commit e87412e621f1
        ("integrate Zaamo and Zalrsc text (#1304)").
 
+  * :c:macro:`RISCV_HWPROBE_EXT_ZALASR`: The Zalasr extension is supported as
+       frozen at commit 194f0094 ("Version 0.9 for freeze") of riscv-zalasr.
+
   * :c:macro:`RISCV_HWPROBE_EXT_ZALRSC`: The Zalrsc extension is supported as
        defined in the in the RISC-V ISA manual starting from commit e87412e621f1
        ("integrate Zaamo and Zalrsc text (#1304)").
@@ -360,4 +363,4 @@ The following keys are defined:
 
     * :c:macro:`RISCV_HWPROBE_VENDOR_EXT_XSFVFWMACCQQQ`: The Xsfvfwmaccqqq
         vendor extension is supported in version 1.0 of Matrix Multiply Accumulate
-	Instruction Extensions Specification.
\ No newline at end of file
+	Instruction Extensions Specification.
arch/riscv/include/uapi/asm/hwprobe.h +1 -0
index aaf6ad9704993..d3a65f8ff7da4 100644
@@ -82,6 +82,7 @@ struct riscv_hwprobe {
 #define		RISCV_HWPROBE_EXT_ZAAMO		(1ULL << 56)
 #define		RISCV_HWPROBE_EXT_ZALRSC	(1ULL << 57)
 #define		RISCV_HWPROBE_EXT_ZABHA		(1ULL << 58)
+#define		RISCV_HWPROBE_EXT_ZALASR	(1ULL << 59)
 #define RISCV_HWPROBE_KEY_CPUPERF_0	5
 #define		RISCV_HWPROBE_MISALIGNED_UNKNOWN	(0 << 0)
 #define		RISCV_HWPROBE_MISALIGNED_EMULATED	(1 << 0)
arch/riscv/kernel/sys_hwprobe.c +1 -0
index 0b170e18a2beb..0529e692b1173 100644
@@ -99,6 +99,7 @@ static void hwprobe_isa_ext0(struct riscv_hwprobe *pair,
 		EXT_KEY(ZAAMO);
 		EXT_KEY(ZABHA);
 		EXT_KEY(ZACAS);
+		EXT_KEY(ZALASR);
 		EXT_KEY(ZALRSC);
 		EXT_KEY(ZAWRS);
 		EXT_KEY(ZBA);
Signature · 1 line
2.20.1
author
#4 lore raw

PATCH v3 4/8 riscv: Introduce Zalasr instructions

In reply to Xu Lu

Introduce l{b|h|w|d}.{aq|aqrl} and s{b|h|w|d}.{rl|aqrl} instruction encodings.

---
 arch/riscv/include/asm/insn-def.h | 79 +++++++++++++++++++++++++++++++
 1 file changed, 79 insertions(+)
arch/riscv/include/asm/insn-def.h +79 -0
index d5adbaec1d010..3fec7e66ce50f 100644
@@ -179,6 +179,7 @@
 #define RV___RS1(v)		__RV_REG(v)
 #define RV___RS2(v)		__RV_REG(v)
 
+#define RV_OPCODE_AMO		RV_OPCODE(47)
 #define RV_OPCODE_MISC_MEM	RV_OPCODE(15)
 #define RV_OPCODE_OP_IMM	RV_OPCODE(19)
 #define RV_OPCODE_SYSTEM	RV_OPCODE(115)
@@ -208,6 +209,84 @@
 	__ASM_STR(.error "hlv.d requires 64-bit support")
 #endif
 
+#define LB_AQ(dest, addr)					\
+	INSN_R(OPCODE_AMO, FUNC3(0), FUNC7(26),			\
+	       RD(dest), RS1(addr), __RS2(0))
+
+#define LB_AQRL(dest, addr)					\
+	INSN_R(OPCODE_AMO, FUNC3(0), FUNC7(27),			\
+	       RD(dest), RS1(addr), __RS2(0))
+
+#define LH_AQ(dest, addr)					\
+	INSN_R(OPCODE_AMO, FUNC3(1), FUNC7(26),			\
+	       RD(dest), RS1(addr), __RS2(0))
+
+#define LH_AQRL(dest, addr)					\
+	INSN_R(OPCODE_AMO, FUNC3(1), FUNC7(27),			\
+	       RD(dest), RS1(addr), __RS2(0))
+
+#define LW_AQ(dest, addr)					\
+	INSN_R(OPCODE_AMO, FUNC3(2), FUNC7(26),			\
+	       RD(dest), RS1(addr), __RS2(0))
+
+#define LW_AQRL(dest, addr)					\
+	INSN_R(OPCODE_AMO, FUNC3(2), FUNC7(27),			\
+	       RD(dest), RS1(addr), __RS2(0))
+
+#define SB_RL(src, addr)					\
+	INSN_R(OPCODE_AMO, FUNC3(0), FUNC7(29),			\
+	       __RD(0), RS1(addr), RS2(src))
+
+#define SB_AQRL(src, addr)					\
+	INSN_R(OPCODE_AMO, FUNC3(0), FUNC7(31),			\
+	       __RD(0), RS1(addr), RS2(src))
+
+#define SH_RL(src, addr)					\
+	INSN_R(OPCODE_AMO, FUNC3(1), FUNC7(29),			\
+	       __RD(0), RS1(addr), RS2(src))
+
+#define SH_AQRL(src, addr)					\
+	INSN_R(OPCODE_AMO, FUNC3(1), FUNC7(31),			\
+	       __RD(0), RS1(addr), RS2(src))
+
+#define SW_RL(src, addr)					\
+	INSN_R(OPCODE_AMO, FUNC3(2), FUNC7(29),			\
+	       __RD(0), RS1(addr), RS2(src))
+
+#define SW_AQRL(src, addr)					\
+	INSN_R(OPCODE_AMO, FUNC3(2), FUNC7(31),			\
+	       __RD(0), RS1(addr), RS2(src))
+
+#ifdef CONFIG_64BIT
+#define LD_AQ(dest, addr)					\
+	INSN_R(OPCODE_AMO, FUNC3(3), FUNC7(26),			\
+	       RD(dest), RS1(addr), __RS2(0))
+
+#define LD_AQRL(dest, addr)					\
+	INSN_R(OPCODE_AMO, FUNC3(3), FUNC7(27),			\
+	       RD(dest), RS1(addr), __RS2(0))
+
+#define SD_RL(src, addr)					\
+	INSN_R(OPCODE_AMO, FUNC3(3), FUNC7(29),			\
+	       __RD(0), RS1(addr), RS2(src))
+
+#define SD_AQRL(src, addr)					\
+	INSN_R(OPCODE_AMO, FUNC3(3), FUNC7(31),			\
+	       __RD(0), RS1(addr), RS2(src))
+#else
+#define LD_AQ(dest, addr)					\
+	__ASM_STR(.error "ld.aq requires 64-bit support")
+
+#define LD_AQRL(dest, addr)					\
+	__ASM_STR(.error "ld.aqrl requires 64-bit support")
+
+#define SD_RL(dest, addr)					\
+	__ASM_STR(.error "sd.rl requires 64-bit support")
+
+#define SD_AQRL(dest, addr)					\
+	__ASM_STR(.error "sd.aqrl requires 64-bit support")
+#endif
+
 #define SINVAL_VMA(vaddr, asid)					\
 	INSN_R(OPCODE_SYSTEM, FUNC3(0), FUNC7(11),		\
 	       __RD(0), RS1(vaddr), RS2(asid))
Signature · 1 line
2.20.1
author
#5 lore raw

PATCH v3 5/8 riscv: Use Zalasr for smp_load_acquire/smp_store_release

In reply to Xu Lu

Replace fence instructions with Zalasr instructions during smp_load_acquire() and smp_store_release() operations.

|----------------------------------|
|    |     __smp_store_release     |
|    |-----------------------------|
|    | zalasr        | !zalasr     |
| rl |-----------------------------|
|    | s{b|h|w|d}.rl | fence rw, w |
|    |               | s{b|h|w|d}  |
|----------------------------------|
|    |    __smp_load_acquire       |
|    |-----------------------------|
|    | zalasr        | !zalasr     |
| aq |-----------------------------|
|    | l{b|h|w|d}.rl | l{b|h|w|d}  |
|    |               | fence r, rw |
|----------------------------------|
---
 arch/riscv/include/asm/barrier.h | 91 ++++++++++++++++++++++++++++----
 1 file changed, 80 insertions(+), 11 deletions(-)
arch/riscv/include/asm/barrier.h +80 -11
index b8c5726d86acb..9eaf94a028096 100644
@@ -51,19 +51,88 @@
  */
 #define smp_mb__after_spinlock()	RISCV_FENCE(iorw, iorw)
 
-#define __smp_store_release(p, v)					\
-do {									\
-	compiletime_assert_atomic_type(*p);				\
-	RISCV_FENCE(rw, w);						\
-	WRITE_ONCE(*p, v);						\
+extern void __bad_size_call_parameter(void);
+
+#define __smp_store_release(p, v)						\
+do {										\
+	typeof(p) __p = (p);							\
+	union { typeof(*p) __val; char __c[1]; } __u =				\
+		{ .__val = (__force typeof(*p)) (v) };				\
+	compiletime_assert_atomic_type(*p);					\
+	switch (sizeof(*p)) {							\
+	case 1:									\
+		asm volatile(ALTERNATIVE("fence rw, w;\t\nsb %0, 0(%1)\t\n",	\
+					 SB_RL(%0, %1) "\t\nnop\t\n",		\
+					 0, RISCV_ISA_EXT_ZALASR, 1)		\
+					 : : "r" (*(__u8 *)__u.__c), "r" (__p)	\
+					 : "memory");				\
+		break;								\
+	case 2:									\
+		asm volatile(ALTERNATIVE("fence rw, w;\t\nsh %0, 0(%1)\t\n",	\
+					 SH_RL(%0, %1) "\t\nnop\t\n",		\
+					 0, RISCV_ISA_EXT_ZALASR, 1)		\
+					 : : "r" (*(__u16 *)__u.__c), "r" (__p)	\
+					 : "memory");				\
+		break;								\
+	case 4:									\
+		asm volatile(ALTERNATIVE("fence rw, w;\t\nsw %0, 0(%1)\t\n",	\
+					 SW_RL(%0, %1) "\t\nnop\t\n",		\
+					 0, RISCV_ISA_EXT_ZALASR, 1)		\
+					 : : "r" (*(__u32 *)__u.__c), "r" (__p)	\
+					 : "memory");				\
+		break;								\
+	case 8:									\
+		asm volatile(ALTERNATIVE("fence rw, w;\t\nsd %0, 0(%1)\t\n",	\
+					 SD_RL(%0, %1) "\t\nnop\t\n",		\
+					 0, RISCV_ISA_EXT_ZALASR, 1)		\
+					 : : "r" (*(__u64 *)__u.__c), "r" (__p)	\
+					 : "memory");				\
+		break;								\
+	default:								\
+		__bad_size_call_parameter();					\
+		break;								\
+	}									\
 } while (0)
 
-#define __smp_load_acquire(p)						\
-({									\
-	typeof(*p) ___p1 = READ_ONCE(*p);				\
-	compiletime_assert_atomic_type(*p);				\
-	RISCV_FENCE(r, rw);						\
-	___p1;								\
+#define __smp_load_acquire(p)							\
+({										\
+	union { typeof(*p) __val; char __c[1]; } __u;				\
+	typeof(p) __p = (p);							\
+	compiletime_assert_atomic_type(*p);					\
+	switch (sizeof(*p)) {							\
+	case 1:									\
+		asm volatile(ALTERNATIVE("lb %0, 0(%1)\t\nfence r, rw\t\n",	\
+					 LB_AQ(%0, %1) "\t\nnop\t\n",		\
+					 0, RISCV_ISA_EXT_ZALASR, 1)		\
+					 : "=r" (*(__u8 *)__u.__c) : "r" (__p)	\
+					 : "memory");				\
+		break;								\
+	case 2:									\
+		asm volatile(ALTERNATIVE("lh %0, 0(%1)\t\nfence r, rw\t\n",	\
+					 LH_AQ(%0, %1) "\t\nnop\t\n",		\
+					 0, RISCV_ISA_EXT_ZALASR, 1)		\
+					 : "=r" (*(__u16 *)__u.__c) : "r" (__p)	\
+					 : "memory");				\
+		break;								\
+	case 4:									\
+		asm volatile(ALTERNATIVE("lw %0, 0(%1)\t\nfence r, rw\t\n",	\
+					 LW_AQ(%0, %1) "\t\nnop\t\n",		\
+					 0, RISCV_ISA_EXT_ZALASR, 1)		\
+					 : "=r" (*(__u32 *)__u.__c) : "r" (__p)	\
+					 : "memory");				\
+		break;								\
+	case 8:									\
+		asm volatile(ALTERNATIVE("ld %0, 0(%1)\t\nfence r, rw\t\n",	\
+					 LD_AQ(%0, %1) "\t\nnop\t\n",		\
+					 0, RISCV_ISA_EXT_ZALASR, 1)		\
+					 : "=r" (*(__u64 *)__u.__c) : "r" (__p)	\
+					 : "memory");				\
+		break;								\
+	default:								\
+		__bad_size_call_parameter();					\
+		break;								\
+	}									\
+	__u.__val;								\
 })
 
 #ifdef CONFIG_RISCV_ISA_ZAWRS
Signature · 1 line
2.20.1
author
#6 lore raw

PATCH v3 6/8 riscv: Apply acquire/release semantics to arch_xchg/arch_cmpxchg operations

In reply to Xu Lu

The existing arch_xchg/arch_cmpxchg operations are implemented by inserting fence instructions before or after atomic instructions. This commit replaces them with real acquire/release semantics.

|----------------------------------------------------------------|
|    |    arch_xchg_release       |     arch_cmpxchg_release     |
|    |-----------------------------------------------------------|
|    | zabha      | !zabha        | zabha+zacas | !(zabha+zacas) |
| rl |-----------------------------------------------------------|
|    |            | (fence rw, w) |             | (fence rw, w)  |
|    | amoswap.rl | lr.w          | amocas.rl   | lr.w           |
|    |            | sc.w.rl       |             | sc.w.rl        |
|----------------------------------------------------------------|
|    |    arch_xchg_acquire       |     arch_cmpxchg_acquire     |
|    |-----------------------------------------------------------|
|    | zabha      | !zabha        | zabha+zacas | !(zabha+zacas) |
| aq |-----------------------------------------------------------|
|    |            | lr.w.aq       |             | lr.w.aq        |
|    | amoswap.aq | sc.w          | amocas.aq   | sc.w           |
|    |            | (fence r, rw) |             | (fence r, rw)  |
|----------------------------------------------------------------|

(fence rw, w), (fence r, rw) here means such instructions will only be inserted when zalasr is not implemented.

---
 arch/riscv/include/asm/atomic.h  |   6 --
 arch/riscv/include/asm/cmpxchg.h | 136 ++++++++++++++-----------------
 2 files changed, 63 insertions(+), 79 deletions(-)
arch/riscv/include/asm/atomic.h +0 -6
index 5b96c2f61adb5..b79a4f889f339 100644
@@ -18,12 +18,6 @@
 
 #include <asm/cmpxchg.h>
 
-#define __atomic_acquire_fence()					\
-	__asm__ __volatile__(RISCV_ACQUIRE_BARRIER "" ::: "memory")
-
-#define __atomic_release_fence()					\
-	__asm__ __volatile__(RISCV_RELEASE_BARRIER "" ::: "memory");
-
 static __always_inline int arch_atomic_read(const atomic_t *v)
 {
 	return READ_ONCE(v->counter);
arch/riscv/include/asm/cmpxchg.h +63 -73

289 lines not on this page · raw message

Signature · 1 line
2.20.1
#7 lore raw

Re: [PATCH v3 6/8] riscv: Apply acquire/release semantics to arch_xchg/arch_cmpxchg operations

kernel test robot noticed the following build errors:

Hi Xu,

kernel test robot noticed the following build errors:

[auto build test ERROR on robh/for-next]
[also build test ERROR on kvm/queue kvm/next linus/master v6.17-rc6] [cannot apply to kvm/linux-next]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Xu-Lu/riscv-add-ISA-extension-parsing-for-Zalasr/20250919-154304
base:   https://git.kernel.org/pub/scm/linux/kernel/git/robh/linux.git for-next
patch link:    https://lore.kernel.org/r/20250919073714.83063-7-luxu.kernel%40bytedance.com
patch subject: [PATCH v3 6/8] riscv: Apply acquire/release semantics to arch_xchg/arch_cmpxchg operations
config: riscv-randconfig-002-20250920 (https://download.01.org/0day-ci/archive/20250920/[email protected]/config)
compiler: clang version 22.0.0git (https://github.com/llvm/llvm-project 7c861bcedf61607b6c087380ac711eb7ff918ca6)
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20250920/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

All errors (new ones prefixed by >>):

In file included from lib/objpool.c:3:

>> include/linux/objpool.h:156:7: error: invalid .org offset '1528' (at offset '1532')
     156 |                 if (try_cmpxchg_release(&slot->head, &head, head + 1))
         |                     ^
   include/linux/atomic/atomic-instrumented.h:4899:2: note: expanded from macro 'try_cmpxchg_release'
    4899 |         raw_try_cmpxchg_release(__ai_ptr, __ai_oldp, __VA_ARGS__); \
         |         ^
   include/linux/atomic/atomic-arch-fallback.h:228:9: note: expanded from macro 'raw_try_cmpxchg_release'
     228 |         ___r = raw_cmpxchg_release((_ptr), ___o, (_new)); \
         |                ^
   include/linux/atomic/atomic-arch-fallback.h:77:29: note: expanded from macro 'raw_cmpxchg_release'
      77 | #define raw_cmpxchg_release arch_cmpxchg_release
         |                             ^
   note: (skipping 4 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
   arch/riscv/include/asm/alternative-macros.h:104:2: note: expanded from macro '_ALTERNATIVE_CFG'
     104 |         __ALTERNATIVE_CFG(old_c, new_c, vendor_id, patch_id, IS_ENABLED(CONFIG_k))
         |         ^
   arch/riscv/include/asm/alternative-macros.h:94:2: note: expanded from macro '__ALTERNATIVE_CFG'
      94 |         ALT_NEW_CONTENT(vendor_id, patch_id, enable, new_c)
         |         ^
   arch/riscv/include/asm/alternative-macros.h:81:3: note: expanded from macro 'ALT_NEW_CONTENT'
      81 |         ".org   . - (887b - 886b) + (889b - 888b)\n"                    \
         |          ^
   <inline asm>:27:6: note: instantiated into assembly here
      27 | .org    . - (887b - 886b) + (889b - 888b)
         |         ^
   In file included from lib/objpool.c:3:
>> include/linux/objpool.h:156:7: error: invalid .org offset '1528' (at offset '1532')
     156 |                 if (try_cmpxchg_release(&slot->head, &head, head + 1))
         |                     ^
   include/linux/atomic/atomic-instrumented.h:4899:2: note: expanded from macro 'try_cmpxchg_release'
    4899 |         raw_try_cmpxchg_release(__ai_ptr, __ai_oldp, __VA_ARGS__); \
         |         ^
   include/linux/atomic/atomic-arch-fallback.h:228:9: note: expanded from macro 'raw_try_cmpxchg_release'
     228 |         ___r = raw_cmpxchg_release((_ptr), ___o, (_new)); \
         |                ^
   include/linux/atomic/atomic-arch-fallback.h:77:29: note: expanded from macro 'raw_cmpxchg_release'
      77 | #define raw_cmpxchg_release arch_cmpxchg_release
         |                             ^
   note: (skipping 4 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
   arch/riscv/include/asm/alternative-macros.h:104:2: note: expanded from macro '_ALTERNATIVE_CFG'
     104 |         __ALTERNATIVE_CFG(old_c, new_c, vendor_id, patch_id, IS_ENABLED(CONFIG_k))
         |         ^
   arch/riscv/include/asm/alternative-macros.h:94:2: note: expanded from macro '__ALTERNATIVE_CFG'
      94 |         ALT_NEW_CONTENT(vendor_id, patch_id, enable, new_c)
         |         ^
   arch/riscv/include/asm/alternative-macros.h:81:3: note: expanded from macro 'ALT_NEW_CONTENT'
      81 |         ".org   . - (887b - 886b) + (889b - 888b)\n"                    \
         |          ^
   <inline asm>:27:6: note: instantiated into assembly here
      27 | .org    . - (887b - 886b) + (889b - 888b)
         |         ^
   2 errors generated.
--
>> lib/generic-radix-tree.c:53:12: error: invalid .org offset '1850' (at offset '1854')
      53 |                 if ((v = cmpxchg_release(&radix->root, r, new_root)) == r) {
         |                          ^
   include/linux/atomic/atomic-instrumented.h:4803:2: note: expanded from macro 'cmpxchg_release'
    4803 |         raw_cmpxchg_release(__ai_ptr, __VA_ARGS__); \
         |         ^
   include/linux/atomic/atomic-arch-fallback.h:77:29: note: expanded from macro 'raw_cmpxchg_release'
      77 | #define raw_cmpxchg_release arch_cmpxchg_release
         |                             ^
   arch/riscv/include/asm/cmpxchg.h:265:2: note: expanded from macro 'arch_cmpxchg_release'
     265 |         _arch_cmpxchg((ptr), (o), (n),                                  \
         |         ^
   note: (skipping 3 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
   arch/riscv/include/asm/alternative-macros.h:104:2: note: expanded from macro '_ALTERNATIVE_CFG'
     104 |         __ALTERNATIVE_CFG(old_c, new_c, vendor_id, patch_id, IS_ENABLED(CONFIG_k))
         |         ^
   arch/riscv/include/asm/alternative-macros.h:94:2: note: expanded from macro '__ALTERNATIVE_CFG'
      94 |         ALT_NEW_CONTENT(vendor_id, patch_id, enable, new_c)
         |         ^
   arch/riscv/include/asm/alternative-macros.h:81:3: note: expanded from macro 'ALT_NEW_CONTENT'
      81 |         ".org   . - (887b - 886b) + (889b - 888b)\n"                    \
         |          ^
   <inline asm>:27:6: note: instantiated into assembly here
      27 | .org    . - (887b - 886b) + (889b - 888b)
         |         ^
   lib/generic-radix-tree.c:74:14: error: invalid .org offset '1862' (at offset '1866')
      74 |                         if (!(n = cmpxchg_release(p, NULL, new_node)))
         |                                   ^
   include/linux/atomic/atomic-instrumented.h:4803:2: note: expanded from macro 'cmpxchg_release'
    4803 |         raw_cmpxchg_release(__ai_ptr, __VA_ARGS__); \
         |         ^
   include/linux/atomic/atomic-arch-fallback.h:77:29: note: expanded from macro 'raw_cmpxchg_release'
      77 | #define raw_cmpxchg_release arch_cmpxchg_release
         |                             ^
   arch/riscv/include/asm/cmpxchg.h:265:2: note: expanded from macro 'arch_cmpxchg_release'
     265 |         _arch_cmpxchg((ptr), (o), (n),                                  \
         |         ^
   note: (skipping 3 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
   arch/riscv/include/asm/alternative-macros.h:104:2: note: expanded from macro '_ALTERNATIVE_CFG'
     104 |         __ALTERNATIVE_CFG(old_c, new_c, vendor_id, patch_id, IS_ENABLED(CONFIG_k))
         |         ^
   arch/riscv/include/asm/alternative-macros.h:94:2: note: expanded from macro '__ALTERNATIVE_CFG'
      94 |         ALT_NEW_CONTENT(vendor_id, patch_id, enable, new_c)
         |         ^
   arch/riscv/include/asm/alternative-macros.h:81:3: note: expanded from macro 'ALT_NEW_CONTENT'
      81 |         ".org   . - (887b - 886b) + (889b - 888b)\n"                    \
         |          ^
   <inline asm>:27:6: note: instantiated into assembly here
      27 | .org    . - (887b - 886b) + (889b - 888b)
         |         ^
>> lib/generic-radix-tree.c:53:12: error: invalid .org offset '1850' (at offset '1854')
      53 |                 if ((v = cmpxchg_release(&radix->root, r, new_root)) == r) {
         |                          ^
   include/linux/atomic/atomic-instrumented.h:4803:2: note: expanded from macro 'cmpxchg_release'
    4803 |         raw_cmpxchg_release(__ai_ptr, __VA_ARGS__); \
         |         ^
   include/linux/atomic/atomic-arch-fallback.h:77:29: note: expanded from macro 'raw_cmpxchg_release'
      77 | #define raw_cmpxchg_release arch_cmpxchg_release
         |                             ^
   arch/riscv/include/asm/cmpxchg.h:265:2: note: expanded from macro 'arch_cmpxchg_release'
     265 |         _arch_cmpxchg((ptr), (o), (n),                                  \
         |         ^
   note: (skipping 3 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
   arch/riscv/include/asm/alternative-macros.h:104:2: note: expanded from macro '_ALTERNATIVE_CFG'
     104 |         __ALTERNATIVE_CFG(old_c, new_c, vendor_id, patch_id, IS_ENABLED(CONFIG_k))
         |         ^
   arch/riscv/include/asm/alternative-macros.h:94:2: note: expanded from macro '__ALTERNATIVE_CFG'
      94 |         ALT_NEW_CONTENT(vendor_id, patch_id, enable, new_c)
         |         ^
   arch/riscv/include/asm/alternative-macros.h:81:3: note: expanded from macro 'ALT_NEW_CONTENT'
      81 |         ".org   . - (887b - 886b) + (889b - 888b)\n"                    \
         |          ^
   <inline asm>:27:6: note: instantiated into assembly here
      27 | .org    . - (887b - 886b) + (889b - 888b)
         |         ^
   lib/generic-radix-tree.c:74:14: error: invalid .org offset '1862' (at offset '1866')
      74 |                         if (!(n = cmpxchg_release(p, NULL, new_node)))
         |                                   ^
   include/linux/atomic/atomic-instrumented.h:4803:2: note: expanded from macro 'cmpxchg_release'
    4803 |         raw_cmpxchg_release(__ai_ptr, __VA_ARGS__); \
         |         ^
   include/linux/atomic/atomic-arch-fallback.h:77:29: note: expanded from macro 'raw_cmpxchg_release'
      77 | #define raw_cmpxchg_release arch_cmpxchg_release
         |                             ^
   arch/riscv/include/asm/cmpxchg.h:265:2: note: expanded from macro 'arch_cmpxchg_release'
     265 |         _arch_cmpxchg((ptr), (o), (n),                                  \
         |         ^
   note: (skipping 3 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
   arch/riscv/include/asm/alternative-macros.h:104:2: note: expanded from macro '_ALTERNATIVE_CFG'
     104 |         __ALTERNATIVE_CFG(old_c, new_c, vendor_id, patch_id, IS_ENABLED(CONFIG_k))
         |         ^
   arch/riscv/include/asm/alternative-macros.h:94:2: note: expanded from macro '__ALTERNATIVE_CFG'
      94 |         ALT_NEW_CONTENT(vendor_id, patch_id, enable, new_c)
         |         ^
   arch/riscv/include/asm/alternative-macros.h:81:3: note: expanded from macro 'ALT_NEW_CONTENT'
      81 |         ".org   . - (887b - 886b) + (889b - 888b)\n"                    \
         |          ^
   <inline asm>:27:6: note: instantiated into assembly here
      27 | .org    . - (887b - 886b) + (889b - 888b)
         |         ^

4 errors generated.
--
In file included from lib/refcount.c:6:
In file included from include/linux/mutex.h:17:
In file included from include/linux/lockdep.h:14:
In file included from include/linux/smp.h:13:
In file included from include/linux/cpumask.h:14:
In file included from include/linux/atomic.h:80:

>> include/linux/atomic/atomic-arch-fallback.h:2083:9: error: invalid .org offset '528' (at offset '532')
    2083 |         return raw_cmpxchg_release(&v->counter, old, new);
         |                ^
   include/linux/atomic/atomic-arch-fallback.h:77:29: note: expanded from macro 'raw_cmpxchg_release'
      77 | #define raw_cmpxchg_release arch_cmpxchg_release
         |                             ^
   arch/riscv/include/asm/cmpxchg.h:265:2: note: expanded from macro 'arch_cmpxchg_release'
     265 |         _arch_cmpxchg((ptr), (o), (n),                                  \
         |         ^
   arch/riscv/include/asm/cmpxchg.h:238:3: note: expanded from macro '_arch_cmpxchg'
     238 |                 __arch_cmpxchg(".w" lr_sfx, ".w" sc_sfx, ".w" cas_sfx,  \
         |                 ^
   note: (skipping 2 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
   arch/riscv/include/asm/alternative-macros.h:104:2: note: expanded from macro '_ALTERNATIVE_CFG'
     104 |         __ALTERNATIVE_CFG(old_c, new_c, vendor_id, patch_id, IS_ENABLED(CONFIG_k))
         |         ^
   arch/riscv/include/asm/alternative-macros.h:94:2: note: expanded from macro '__ALTERNATIVE_CFG'
      94 |         ALT_NEW_CONTENT(vendor_id, patch_id, enable, new_c)
         |         ^
   arch/riscv/include/asm/alternative-macros.h:81:3: note: expanded from macro 'ALT_NEW_CONTENT'
      81 |         ".org   . - (887b - 886b) + (889b - 888b)\n"                    \
         |          ^
   <inline asm>:27:6: note: instantiated into assembly here
      27 | .org    . - (887b - 886b) + (889b - 888b)
         |         ^
   In file included from lib/refcount.c:6:
   In file included from include/linux/mutex.h:17:
   In file included from include/linux/lockdep.h:14:
   In file included from include/linux/smp.h:13:
   In file included from include/linux/cpumask.h:14:
   In file included from include/linux/atomic.h:80:
   include/linux/atomic/atomic-arch-fallback.h:2083:9: error: invalid .org offset '540' (at offset '544')
    2083 |         return raw_cmpxchg_release(&v->counter, old, new);
         |                ^
   include/linux/atomic/atomic-arch-fallback.h:77:29: note: expanded from macro 'raw_cmpxchg_release'
      77 | #define raw_cmpxchg_release arch_cmpxchg_release
         |                             ^
   arch/riscv/include/asm/cmpxchg.h:265:2: note: expanded from macro 'arch_cmpxchg_release'
     265 |         _arch_cmpxchg((ptr), (o), (n),                                  \
         |         ^
   arch/riscv/include/asm/cmpxchg.h:238:3: note: expanded from macro '_arch_cmpxchg'
     238 |                 __arch_cmpxchg(".w" lr_sfx, ".w" sc_sfx, ".w" cas_sfx,  \
         |                 ^
   note: (skipping 2 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
   arch/riscv/include/asm/alternative-macros.h:104:2: note: expanded from macro '_ALTERNATIVE_CFG'
     104 |         __ALTERNATIVE_CFG(old_c, new_c, vendor_id, patch_id, IS_ENABLED(CONFIG_k))
         |         ^
   arch/riscv/include/asm/alternative-macros.h:94:2: note: expanded from macro '__ALTERNATIVE_CFG'
      94 |         ALT_NEW_CONTENT(vendor_id, patch_id, enable, new_c)
         |         ^
   arch/riscv/include/asm/alternative-macros.h:81:3: note: expanded from macro 'ALT_NEW_CONTENT'
      81 |         ".org   . - (887b - 886b) + (889b - 888b)\n"                    \
         |          ^
   <inline asm>:27:6: note: instantiated into assembly here
      27 | .org    . - (887b - 886b) + (889b - 888b)
         |         ^
   In file included from lib/refcount.c:6:
   In file included from include/linux/mutex.h:17:
   In file included from include/linux/lockdep.h:14:
   In file included from include/linux/smp.h:13:
   In file included from include/linux/cpumask.h:14:
   In file included from include/linux/atomic.h:80:
>> include/linux/atomic/atomic-arch-fallback.h:2083:9: error: invalid .org offset '528' (at offset '532')
    2083 |         return raw_cmpxchg_release(&v->counter, old, new);
         |                ^
   include/linux/atomic/atomic-arch-fallback.h:77:29: note: expanded from macro 'raw_cmpxchg_release'
      77 | #define raw_cmpxchg_release arch_cmpxchg_release
         |                             ^
   arch/riscv/include/asm/cmpxchg.h:265:2: note: expanded from macro 'arch_cmpxchg_release'
     265 |         _arch_cmpxchg((ptr), (o), (n),                                  \
         |         ^
   arch/riscv/include/asm/cmpxchg.h:238:3: note: expanded from macro '_arch_cmpxchg'
     238 |                 __arch_cmpxchg(".w" lr_sfx, ".w" sc_sfx, ".w" cas_sfx,  \
         |                 ^
   note: (skipping 2 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
   arch/riscv/include/asm/alternative-macros.h:104:2: note: expanded from macro '_ALTERNATIVE_CFG'
     104 |         __ALTERNATIVE_CFG(old_c, new_c, vendor_id, patch_id, IS_ENABLED(CONFIG_k))
         |         ^
   arch/riscv/include/asm/alternative-macros.h:94:2: note: expanded from macro '__ALTERNATIVE_CFG'
      94 |         ALT_NEW_CONTENT(vendor_id, patch_id, enable, new_c)
         |         ^
   arch/riscv/include/asm/alternative-macros.h:81:3: note: expanded from macro 'ALT_NEW_CONTENT'
      81 |         ".org   . - (887b - 886b) + (889b - 888b)\n"                    \
         |          ^
   <inline asm>:27:6: note: instantiated into assembly here
      27 | .org    . - (887b - 886b) + (889b - 888b)
         |         ^
   In file included from lib/refcount.c:6:
   In file included from include/linux/mutex.h:17:
   In file included from include/linux/lockdep.h:14:
   In file included from include/linux/smp.h:13:
   In file included from include/linux/cpumask.h:14:
   In file included from include/linux/atomic.h:80:
   include/linux/atomic/atomic-arch-fallback.h:2083:9: error: invalid .org offset '540' (at offset '544')
    2083 |         return raw_cmpxchg_release(&v->counter, old, new);
         |                ^
   include/linux/atomic/atomic-arch-fallback.h:77:29: note: expanded from macro 'raw_cmpxchg_release'
      77 | #define raw_cmpxchg_release arch_cmpxchg_release
         |                             ^
   arch/riscv/include/asm/cmpxchg.h:265:2: note: expanded from macro 'arch_cmpxchg_release'
     265 |         _arch_cmpxchg((ptr), (o), (n),                                  \
         |         ^
   arch/riscv/include/asm/cmpxchg.h:238:3: note: expanded from macro '_arch_cmpxchg'
     238 |                 __arch_cmpxchg(".w" lr_sfx, ".w" sc_sfx, ".w" cas_sfx,  \
         |                 ^
   note: (skipping 2 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
   arch/riscv/include/asm/alternative-macros.h:104:2: note: expanded from macro '_ALTERNATIVE_CFG'
     104 |         __ALTERNATIVE_CFG(old_c, new_c, vendor_id, patch_id, IS_ENABLED(CONFIG_k))
         |         ^
   arch/riscv/include/asm/alternative-macros.h:94:2: note: expanded from macro '__ALTERNATIVE_CFG'
      94 |         ALT_NEW_CONTENT(vendor_id, patch_id, enable, new_c)
         |         ^
   arch/riscv/include/asm/alternative-macros.h:81:3: note: expanded from macro 'ALT_NEW_CONTENT'
      81 |         ".org   . - (887b - 886b) + (889b - 888b)\n"                    \
         |          ^
   <inline asm>:27:6: note: instantiated into assembly here
      27 | .org    . - (887b - 886b) + (889b - 888b)
         |         ^
   4 errors generated.
--
>> fs/overlayfs/file.c:147:10: error: invalid .org offset '3640' (at offset '3644')
     147 |                         old = cmpxchg_release(&of->upperfile, NULL, upperfile);
         |                               ^
   include/linux/atomic/atomic-instrumented.h:4803:2: note: expanded from macro 'cmpxchg_release'
    4803 |         raw_cmpxchg_release(__ai_ptr, __VA_ARGS__); \
         |         ^
   include/linux/atomic/atomic-arch-fallback.h:77:29: note: expanded from macro 'raw_cmpxchg_release'
      77 | #define raw_cmpxchg_release arch_cmpxchg_release
         |                             ^
   arch/riscv/include/asm/cmpxchg.h:265:2: note: expanded from macro 'arch_cmpxchg_release'
     265 |         _arch_cmpxchg((ptr), (o), (n),                                  \
         |         ^
   note: (skipping 3 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
   arch/riscv/include/asm/alternative-macros.h:104:2: note: expanded from macro '_ALTERNATIVE_CFG'
     104 |         __ALTERNATIVE_CFG(old_c, new_c, vendor_id, patch_id, IS_ENABLED(CONFIG_k))
         |         ^
   arch/riscv/include/asm/alternative-macros.h:94:2: note: expanded from macro '__ALTERNATIVE_CFG'
      94 |         ALT_NEW_CONTENT(vendor_id, patch_id, enable, new_c)
         |         ^
   arch/riscv/include/asm/alternative-macros.h:81:3: note: expanded from macro 'ALT_NEW_CONTENT'
      81 |         ".org   . - (887b - 886b) + (889b - 888b)\n"                    \
         |          ^
   <inline asm>:27:6: note: instantiated into assembly here
      27 | .org    . - (887b - 886b) + (889b - 888b)
         |         ^
>> fs/overlayfs/file.c:147:10: error: invalid .org offset '3640' (at offset '3644')
     147 |                         old = cmpxchg_release(&of->upperfile, NULL, upperfile);
         |                               ^
   include/linux/atomic/atomic-instrumented.h:4803:2: note: expanded from macro 'cmpxchg_release'
    4803 |         raw_cmpxchg_release(__ai_ptr, __VA_ARGS__); \
         |         ^
   include/linux/atomic/atomic-arch-fallback.h:77:29: note: expanded from macro 'raw_cmpxchg_release'
      77 | #define raw_cmpxchg_release arch_cmpxchg_release
         |                             ^
   arch/riscv/include/asm/cmpxchg.h:265:2: note: expanded from macro 'arch_cmpxchg_release'
     265 |         _arch_cmpxchg((ptr), (o), (n),                                  \
         |         ^
   note: (skipping 3 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
   arch/riscv/include/asm/alternative-macros.h:104:2: note: expanded from macro '_ALTERNATIVE_CFG'
     104 |         __ALTERNATIVE_CFG(old_c, new_c, vendor_id, patch_id, IS_ENABLED(CONFIG_k))
         |         ^
   arch/riscv/include/asm/alternative-macros.h:94:2: note: expanded from macro '__ALTERNATIVE_CFG'
      94 |         ALT_NEW_CONTENT(vendor_id, patch_id, enable, new_c)
         |         ^
   arch/riscv/include/asm/alternative-macros.h:81:3: note: expanded from macro 'ALT_NEW_CONTENT'
      81 |         ".org   . - (887b - 886b) + (889b - 888b)\n"                    \
         |          ^
   <inline asm>:27:6: note: instantiated into assembly here
      27 | .org    . - (887b - 886b) + (889b - 888b)
         |         ^

2 errors generated.
..

vim +156 include/linux/objpool.h

b4edb8d2d4647a wuqiang.matt    2023-10-17  100  
b4edb8d2d4647a wuqiang.matt    2023-10-17  101  /**
b4edb8d2d4647a wuqiang.matt    2023-10-17  102   * objpool_init() - initialize objpool and pre-allocated objects
b4edb8d2d4647a wuqiang.matt    2023-10-17  103   * @pool:    the object pool to be initialized, declared by caller
b4edb8d2d4647a wuqiang.matt    2023-10-17  104   * @nr_objs: total objects to be pre-allocated by this object pool
b4edb8d2d4647a wuqiang.matt    2023-10-17  105   * @object_size: size of an object (should be > 0)
b4edb8d2d4647a wuqiang.matt    2023-10-17  106   * @gfp:     flags for memory allocation (via kmalloc or vmalloc)
b4edb8d2d4647a wuqiang.matt    2023-10-17  107   * @context: user context for object initialization callback
b4edb8d2d4647a wuqiang.matt    2023-10-17  108   * @objinit: object initialization callback for extra setup
b4edb8d2d4647a wuqiang.matt    2023-10-17  109   * @release: cleanup callback for extra cleanup task
b4edb8d2d4647a wuqiang.matt    2023-10-17  110   *
b4edb8d2d4647a wuqiang.matt    2023-10-17  111   * return value: 0 for success, otherwise error code
b4edb8d2d4647a wuqiang.matt    2023-10-17  112   *
b4edb8d2d4647a wuqiang.matt    2023-10-17  113   * All pre-allocated objects are to be zeroed after memory allocation.
b4edb8d2d4647a wuqiang.matt    2023-10-17  114   * Caller could do extra initialization in objinit callback. objinit()
b4edb8d2d4647a wuqiang.matt    2023-10-17  115   * will be called just after slot allocation and called only once for
b4edb8d2d4647a wuqiang.matt    2023-10-17  116   * each object. After that the objpool won't touch any content of the
b4edb8d2d4647a wuqiang.matt    2023-10-17  117   * objects. It's caller's duty to perform reinitialization after each
b4edb8d2d4647a wuqiang.matt    2023-10-17  118   * pop (object allocation) or do clearance before each push (object
b4edb8d2d4647a wuqiang.matt    2023-10-17  119   * reclamation).
b4edb8d2d4647a wuqiang.matt    2023-10-17  120   */
b4edb8d2d4647a wuqiang.matt    2023-10-17  121  int objpool_init(struct objpool_head *pool, int nr_objs, int object_size,
b4edb8d2d4647a wuqiang.matt    2023-10-17  122  		 gfp_t gfp, void *context, objpool_init_obj_cb objinit,
b4edb8d2d4647a wuqiang.matt    2023-10-17  123  		 objpool_fini_cb release);
b4edb8d2d4647a wuqiang.matt    2023-10-17  124  
a3b00f10da808b Andrii Nakryiko 2024-04-24  125  /* try to retrieve object from slot */
a3b00f10da808b Andrii Nakryiko 2024-04-24  126  static inline void *__objpool_try_get_slot(struct objpool_head *pool, int cpu)
a3b00f10da808b Andrii Nakryiko 2024-04-24  127  {
a3b00f10da808b Andrii Nakryiko 2024-04-24  128  	struct objpool_slot *slot = pool->cpu_slots[cpu];
a3b00f10da808b Andrii Nakryiko 2024-04-24  129  	/* load head snapshot, other cpus may change it */
a3b00f10da808b Andrii Nakryiko 2024-04-24  130  	uint32_t head = smp_load_acquire(&slot->head);
a3b00f10da808b Andrii Nakryiko 2024-04-24  131  
a3b00f10da808b Andrii Nakryiko 2024-04-24  132  	while (head != READ_ONCE(slot->last)) {
a3b00f10da808b Andrii Nakryiko 2024-04-24  133  		void *obj;
a3b00f10da808b Andrii Nakryiko 2024-04-24  134  
a3b00f10da808b Andrii Nakryiko 2024-04-24  135  		/*
a3b00f10da808b Andrii Nakryiko 2024-04-24  136  		 * data visibility of 'last' and 'head' could be out of
a3b00f10da808b Andrii Nakryiko 2024-04-24  137  		 * order since memory updating of 'last' and 'head' are
a3b00f10da808b Andrii Nakryiko 2024-04-24  138  		 * performed in push() and pop() independently
a3b00f10da808b Andrii Nakryiko 2024-04-24  139  		 *
a3b00f10da808b Andrii Nakryiko 2024-04-24  140  		 * before any retrieving attempts, pop() must guarantee
a3b00f10da808b Andrii Nakryiko 2024-04-24  141  		 * 'last' is behind 'head', that is to say, there must
a3b00f10da808b Andrii Nakryiko 2024-04-24  142  		 * be available objects in slot, which could be ensured
a3b00f10da808b Andrii Nakryiko 2024-04-24  143  		 * by condition 'last != head && last - head <= nr_objs'
a3b00f10da808b Andrii Nakryiko 2024-04-24  144  		 * that is equivalent to 'last - head - 1 < nr_objs' as
a3b00f10da808b Andrii Nakryiko 2024-04-24  145  		 * 'last' and 'head' are both unsigned int32
a3b00f10da808b Andrii Nakryiko 2024-04-24  146  		 */
a3b00f10da808b Andrii Nakryiko 2024-04-24  147  		if (READ_ONCE(slot->last) - head - 1 >= pool->nr_objs) {
a3b00f10da808b Andrii Nakryiko 2024-04-24  148  			head = READ_ONCE(slot->head);
a3b00f10da808b Andrii Nakryiko 2024-04-24  149  			continue;
a3b00f10da808b Andrii Nakryiko 2024-04-24  150  		}
a3b00f10da808b Andrii Nakryiko 2024-04-24  151  
a3b00f10da808b Andrii Nakryiko 2024-04-24  152  		/* obj must be retrieved before moving forward head */
a3b00f10da808b Andrii Nakryiko 2024-04-24  153  		obj = READ_ONCE(slot->entries[head & slot->mask]);
a3b00f10da808b Andrii Nakryiko 2024-04-24  154  
a3b00f10da808b Andrii Nakryiko 2024-04-24  155  		/* move head forward to mark it's consumption */
a3b00f10da808b Andrii Nakryiko 2024-04-24 @156  		if (try_cmpxchg_release(&slot->head, &head, head + 1))
a3b00f10da808b Andrii Nakryiko 2024-04-24  157  			return obj;
a3b00f10da808b Andrii Nakryiko 2024-04-24  158  	}
a3b00f10da808b Andrii Nakryiko 2024-04-24  159  
a3b00f10da808b Andrii Nakryiko 2024-04-24  160  	return NULL;
a3b00f10da808b Andrii Nakryiko 2024-04-24  161  }
a3b00f10da808b Andrii Nakryiko 2024-04-24  162  
Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
author
#8 lore raw

PATCH v3 7/8 RISC-V: KVM: Allow Zalasr extensions for Guest/VM

In reply to Xu Lu

Extend the KVM ISA extension ONE_REG interface to allow KVM user space to detect and enable Zalasr extensions for Guest/VM.

---
 arch/riscv/include/uapi/asm/kvm.h | 1 +
 arch/riscv/kvm/vcpu_onereg.c      | 2 ++
 2 files changed, 3 insertions(+)
arch/riscv/include/uapi/asm/kvm.h +1 -0
index ef27d4289da11..4fbc32ef888fa 100644
@@ -185,6 +185,7 @@ enum KVM_RISCV_ISA_EXT_ID {
 	KVM_RISCV_ISA_EXT_ZICCRSE,
 	KVM_RISCV_ISA_EXT_ZAAMO,
 	KVM_RISCV_ISA_EXT_ZALRSC,
+	KVM_RISCV_ISA_EXT_ZALASR,
 	KVM_RISCV_ISA_EXT_MAX,
 };
 
arch/riscv/kvm/vcpu_onereg.c +2 -0
index cce6a38ea54f2..6ae5f9859f25b 100644
@@ -50,6 +50,7 @@ static const unsigned long kvm_isa_ext_arr[] = {
 	KVM_ISA_EXT_ARR(ZAAMO),
 	KVM_ISA_EXT_ARR(ZABHA),
 	KVM_ISA_EXT_ARR(ZACAS),
+	KVM_ISA_EXT_ARR(ZALASR),
 	KVM_ISA_EXT_ARR(ZALRSC),
 	KVM_ISA_EXT_ARR(ZAWRS),
 	KVM_ISA_EXT_ARR(ZBA),
@@ -184,6 +185,7 @@ static bool kvm_riscv_vcpu_isa_disable_allowed(unsigned long ext)
 	case KVM_RISCV_ISA_EXT_ZAAMO:
 	case KVM_RISCV_ISA_EXT_ZABHA:
 	case KVM_RISCV_ISA_EXT_ZACAS:
+	case KVM_RISCV_ISA_EXT_ZALASR:
 	case KVM_RISCV_ISA_EXT_ZALRSC:
 	case KVM_RISCV_ISA_EXT_ZAWRS:
 	case KVM_RISCV_ISA_EXT_ZBA:
Signature · 1 line
2.20.1
author
#9 lore raw

PATCH v3 8/8 KVM: riscv: selftests: Add Zalasr extensions to get-reg-list test

In reply to Xu Lu

The KVM RISC-V allows Zalasr extensions for Guest/VM so add these extensions to get-reg-list test.

---
 tools/testing/selftests/kvm/riscv/get-reg-list.c | 4 ++++
 1 file changed, 4 insertions(+)
tools/testing/selftests/kvm/riscv/get-reg-list.c +4 -0
index a0b7dabb50406..3020e37f621ba 100644
@@ -65,6 +65,7 @@ bool filter_reg(__u64 reg)
 	case KVM_REG_RISCV_ISA_EXT | KVM_REG_RISCV_ISA_SINGLE | KVM_RISCV_ISA_EXT_ZAAMO:
 	case KVM_REG_RISCV_ISA_EXT | KVM_REG_RISCV_ISA_SINGLE | KVM_RISCV_ISA_EXT_ZABHA:
 	case KVM_REG_RISCV_ISA_EXT | KVM_REG_RISCV_ISA_SINGLE | KVM_RISCV_ISA_EXT_ZACAS:
+	case KVM_REG_RISCV_ISA_EXT | KVM_REG_RISCV_ISA_SINGLE | KVM_RISCV_ISA_EXT_ZALASR:
 	case KVM_REG_RISCV_ISA_EXT | KVM_REG_RISCV_ISA_SINGLE | KVM_RISCV_ISA_EXT_ZALRSC:
 	case KVM_REG_RISCV_ISA_EXT | KVM_REG_RISCV_ISA_SINGLE | KVM_RISCV_ISA_EXT_ZAWRS:
 	case KVM_REG_RISCV_ISA_EXT | KVM_REG_RISCV_ISA_SINGLE | KVM_RISCV_ISA_EXT_ZBA:
@@ -517,6 +518,7 @@ static const char *isa_ext_single_id_to_str(__u64 reg_off)
 		KVM_ISA_EXT_ARR(ZAAMO),
 		KVM_ISA_EXT_ARR(ZABHA),
 		KVM_ISA_EXT_ARR(ZACAS),
+		KVM_ISA_EXT_ARR(ZALASR),
 		KVM_ISA_EXT_ARR(ZALRSC),
 		KVM_ISA_EXT_ARR(ZAWRS),
 		KVM_ISA_EXT_ARR(ZBA),
@@ -1112,6 +1114,7 @@ KVM_ISA_EXT_SIMPLE_CONFIG(svvptc, SVVPTC);
 KVM_ISA_EXT_SIMPLE_CONFIG(zaamo, ZAAMO);
 KVM_ISA_EXT_SIMPLE_CONFIG(zabha, ZABHA);
 KVM_ISA_EXT_SIMPLE_CONFIG(zacas, ZACAS);
+KVM_ISA_EXT_SIMPLE_CONFIG(zalasr, ZALASR);
 KVM_ISA_EXT_SIMPLE_CONFIG(zalrsc, ZALRSC);
 KVM_ISA_EXT_SIMPLE_CONFIG(zawrs, ZAWRS);
 KVM_ISA_EXT_SIMPLE_CONFIG(zba, ZBA);
@@ -1187,6 +1190,7 @@ struct vcpu_reg_list *vcpu_configs[] = {
 	&config_zabha,
 	&config_zacas,
 	&config_zalrsc,
+	&config_zalasr,
 	&config_zawrs,
 	&config_zba,
 	&config_zbb,
Signature · 1 line
2.20.1
#10 lore raw
In reply to Xu Lu
Quoted 37 lines · Xu Lu

This patch adds support for the Zalasr ISA extension, which supplies the real load acquire/store release instructions.

The specification can be found here:
https://github.com/riscv/riscv-zalasr/blob/main/chapter2.adoc

This patch seires has been tested with ltp on Qemu with Brensan's zalasr support patch[1].

Some false positive spacing error happens during patch checking. Thus I CCed maintainers of checkpatch.pl as well.

[1] https://lore.kernel.org/all/CAGPSXwJEdtqW=nx71oufZp64nK6tK=0rytVEcz4F-gfvCOXk2w@mail.gmail.com/

v3:
- Apply acquire/release semantics to arch_xchg/arch_cmpxchg operations so as to ensure FENCE.TSO ordering between operations which precede the UNLOCK+LOCK sequence and operations which follow the sequence. Thanks to Andrea.
- Support hwprobe of Zalasr.
- Allow Zalasr extensions for Guest/VM.

v2:
- Adjust the order of Zalasr and Zalrsc in dt-bindings. Thanks to Conor.

Xu Lu (8):
riscv: add ISA extension parsing for Zalasr
dt-bindings: riscv: Add Zalasr ISA extension description
riscv: hwprobe: Export Zalasr extension
riscv: Introduce Zalasr instructions
riscv: Use Zalasr for smp_load_acquire/smp_store_release
riscv: Apply acquire/release semantics to arch_xchg/arch_cmpxchg
operations
RISC-V: KVM: Allow Zalasr extensions for Guest/VM
KVM: riscv: selftests: Add Zalasr extensions to get-reg-list test

Quoted 14 lines
 Documentation/arch/riscv/hwprobe.rst          |   5 +-
 .../devicetree/bindings/riscv/extensions.yaml |   5 +
 arch/riscv/include/asm/atomic.h               |   6 -
 arch/riscv/include/asm/barrier.h              |  91 ++++++++++--
 arch/riscv/include/asm/cmpxchg.h              | 136 ++++++++----------
 arch/riscv/include/asm/hwcap.h                |   1 +
 arch/riscv/include/asm/insn-def.h             |  79 ++++++++++
 arch/riscv/include/uapi/asm/hwprobe.h         |   1 +
 arch/riscv/include/uapi/asm/kvm.h             |   1 +
 arch/riscv/kernel/cpufeature.c                |   1 +
 arch/riscv/kernel/sys_hwprobe.c               |   1 +
 arch/riscv/kvm/vcpu_onereg.c                  |   2 +
 .../selftests/kvm/riscv/get-reg-list.c        |   4 +
 13 files changed, 242 insertions(+), 91 deletions(-)

I wouldn't have rushed this submission while the discussion on v2 seems so much alive; IAC, to add and link to that discussion, this version (not a review, just looking at this diff stat) is changing the fastpath

read_unlock()
read_lock()

from something like

fence rw,w
amodadd.w
amoadd.w
fence r,rw

to

fence rw,rw
amoadd.w
amoadd.w
fence rw,rw

no matter Zalasr or !Zalasr. Similarly for other atomic operations with release or acquire semantics. I guess the change was not intentional? If it was intentional, it should be properly mentioned in the changelog.

Andrea

author
#11 lore raw

Hi Andrea,

Quoted 54 lines · Andrea Parri
Quoted 37 lines · Xu Lu

This patch adds support for the Zalasr ISA extension, which supplies the real load acquire/store release instructions.

The specification can be found here:
https://github.com/riscv/riscv-zalasr/blob/main/chapter2.adoc

This patch seires has been tested with ltp on Qemu with Brensan's zalasr support patch[1].

Some false positive spacing error happens during patch checking. Thus I CCed maintainers of checkpatch.pl as well.

[1] https://lore.kernel.org/all/CAGPSXwJEdtqW=nx71oufZp64nK6tK=0rytVEcz4F-gfvCOXk2w@mail.gmail.com/

v3:
- Apply acquire/release semantics to arch_xchg/arch_cmpxchg operations so as to ensure FENCE.TSO ordering between operations which precede the UNLOCK+LOCK sequence and operations which follow the sequence. Thanks to Andrea.
- Support hwprobe of Zalasr.
- Allow Zalasr extensions for Guest/VM.

v2:
- Adjust the order of Zalasr and Zalrsc in dt-bindings. Thanks to Conor.

Xu Lu (8):
riscv: add ISA extension parsing for Zalasr
dt-bindings: riscv: Add Zalasr ISA extension description
riscv: hwprobe: Export Zalasr extension
riscv: Introduce Zalasr instructions
riscv: Use Zalasr for smp_load_acquire/smp_store_release
riscv: Apply acquire/release semantics to arch_xchg/arch_cmpxchg
operations
RISC-V: KVM: Allow Zalasr extensions for Guest/VM
KVM: riscv: selftests: Add Zalasr extensions to get-reg-list test

Quoted 14 lines
 Documentation/arch/riscv/hwprobe.rst          |   5 +-
 .../devicetree/bindings/riscv/extensions.yaml |   5 +
 arch/riscv/include/asm/atomic.h               |   6 -
 arch/riscv/include/asm/barrier.h              |  91 ++++++++++--
 arch/riscv/include/asm/cmpxchg.h              | 136 ++++++++----------
 arch/riscv/include/asm/hwcap.h                |   1 +
 arch/riscv/include/asm/insn-def.h             |  79 ++++++++++
 arch/riscv/include/uapi/asm/hwprobe.h         |   1 +
 arch/riscv/include/uapi/asm/kvm.h             |   1 +
 arch/riscv/kernel/cpufeature.c                |   1 +
 arch/riscv/kernel/sys_hwprobe.c               |   1 +
 arch/riscv/kvm/vcpu_onereg.c                  |   2 +
 .../selftests/kvm/riscv/get-reg-list.c        |   4 +
 13 files changed, 242 insertions(+), 91 deletions(-)
Quoted 2 lines

I wouldn't have rushed this submission while the discussion on v2 seems so much alive; IAC, to add and link to that discussion, this version

Thanks. This version is sent out to show my solution to the FENCE.TSO problem you pointed out before. I will continue to improve it. Look forward to more suggestions from you.

Quoted 14 lines

(not a review, just looking at this diff stat) is changing the fastpath

read_unlock()
read_lock()

from something like

fence rw,w
amodadd.w
amoadd.w
fence r,rw

to

Quoted 8 lines

fence rw,rw
amoadd.w
amoadd.w
fence rw,rw

no matter Zalasr or !Zalasr. Similarly for other atomic operations with release or acquire semantics. I guess the change was not intentional? If it was intentional, it should be properly mentioned in the changelog.

Sorry about that. It is intended. The atomic operation before __atomic_acquire_fence or operation after __atomic_release_fence can be just a single ld or sd instruction instead of amocas or amoswap. In such cases, when the store release operation becomes 'sd.rl', the __atomic_acquire_fence via 'fence r, rw' can not ensure FENCE.TSO anymore. Thus I replace it with 'fence rw, rw'.

I will make it a separate commit and provide more messages in the changelog. Maybe alternative mechanism can be applied to accelerate it.

Best Regards,
Xu Lu

Quoted 2 lines

Andrea

author
#12 lore raw
Quoted 88 lines · Xu Lu

Hi Andrea,

Quoted 54 lines · Andrea Parri
Quoted 37 lines · Xu Lu

This patch adds support for the Zalasr ISA extension, which supplies the real load acquire/store release instructions.

The specification can be found here:
https://github.com/riscv/riscv-zalasr/blob/main/chapter2.adoc

This patch seires has been tested with ltp on Qemu with Brensan's zalasr support patch[1].

Some false positive spacing error happens during patch checking. Thus I CCed maintainers of checkpatch.pl as well.

[1] https://lore.kernel.org/all/CAGPSXwJEdtqW=nx71oufZp64nK6tK=0rytVEcz4F-gfvCOXk2w@mail.gmail.com/

v3:
- Apply acquire/release semantics to arch_xchg/arch_cmpxchg operations so as to ensure FENCE.TSO ordering between operations which precede the UNLOCK+LOCK sequence and operations which follow the sequence. Thanks to Andrea.
- Support hwprobe of Zalasr.
- Allow Zalasr extensions for Guest/VM.

v2:
- Adjust the order of Zalasr and Zalrsc in dt-bindings. Thanks to Conor.

Xu Lu (8):
riscv: add ISA extension parsing for Zalasr
dt-bindings: riscv: Add Zalasr ISA extension description
riscv: hwprobe: Export Zalasr extension
riscv: Introduce Zalasr instructions
riscv: Use Zalasr for smp_load_acquire/smp_store_release
riscv: Apply acquire/release semantics to arch_xchg/arch_cmpxchg
operations
RISC-V: KVM: Allow Zalasr extensions for Guest/VM
KVM: riscv: selftests: Add Zalasr extensions to get-reg-list test

Quoted 14 lines
 Documentation/arch/riscv/hwprobe.rst          |   5 +-
 .../devicetree/bindings/riscv/extensions.yaml |   5 +
 arch/riscv/include/asm/atomic.h               |   6 -
 arch/riscv/include/asm/barrier.h              |  91 ++++++++++--
 arch/riscv/include/asm/cmpxchg.h              | 136 ++++++++----------
 arch/riscv/include/asm/hwcap.h                |   1 +
 arch/riscv/include/asm/insn-def.h             |  79 ++++++++++
 arch/riscv/include/uapi/asm/hwprobe.h         |   1 +
 arch/riscv/include/uapi/asm/kvm.h             |   1 +
 arch/riscv/kernel/cpufeature.c                |   1 +
 arch/riscv/kernel/sys_hwprobe.c               |   1 +
 arch/riscv/kvm/vcpu_onereg.c                  |   2 +
 .../selftests/kvm/riscv/get-reg-list.c        |   4 +
 13 files changed, 242 insertions(+), 91 deletions(-)
Quoted 2 lines

I wouldn't have rushed this submission while the discussion on v2 seems so much alive; IAC, to add and link to that discussion, this version

Thanks. This version is sent out to show my solution to the FENCE.TSO problem you pointed out before. I will continue to improve it. Look forward to more suggestions from you.

Quoted 14 lines

(not a review, just looking at this diff stat) is changing the fastpath

read_unlock()
read_lock()

from something like

fence rw,w
amodadd.w
amoadd.w
fence r,rw

to

Quoted 8 lines

fence rw,rw
amoadd.w
amoadd.w
fence rw,rw

no matter Zalasr or !Zalasr. Similarly for other atomic operations with release or acquire semantics. I guess the change was not intentional? If it was intentional, it should be properly mentioned in the changelog.

Quoted 6 lines

Sorry about that. It is intended. The atomic operation before __atomic_acquire_fence or operation after __atomic_release_fence can be just a single ld or sd instruction instead of amocas or amoswap. In such cases, when the store release operation becomes 'sd.rl', the __atomic_acquire_fence via 'fence r, rw' can not ensure FENCE.TSO anymore. Thus I replace it with 'fence rw, rw'.

This is also the common implementation on other architectures who use aq/rl instructions like ARM. And you certainly already knew it~

Quoted 10 lines

I will make it a separate commit and provide more messages in the changelog. Maybe alternative mechanism can be applied to accelerate it.

Best Regards,
Xu Lu

Quoted 2 lines

Andrea

#13 lore raw
Quoted 23 lines
Quoted 23 lines
Quoted 22 lines

(not a review, just looking at this diff stat) is changing the fastpath

read_unlock()
read_lock()

from something like

fence rw,w
amodadd.w
amoadd.w
fence r,rw

to

fence rw,rw
amoadd.w
amoadd.w
fence rw,rw

no matter Zalasr or !Zalasr. Similarly for other atomic operations with release or acquire semantics. I guess the change was not intentional? If it was intentional, it should be properly mentioned in the changelog.

Quoted 6 lines
Quoted 6 lines

Sorry about that. It is intended. The atomic operation before __atomic_acquire_fence or operation after __atomic_release_fence can be just a single ld or sd instruction instead of amocas or amoswap. In such cases, when the store release operation becomes 'sd.rl', the __atomic_acquire_fence via 'fence r, rw' can not ensure FENCE.TSO anymore. Thus I replace it with 'fence rw, rw'.

But you could apply similar changes you performed for xchg & cmpxchg: use .AQ and .RL for other atomic RMW operations as well, no? AFAICS, that is what arm64 actually does in arch/arm64/include/asm/atomic_{ll_sc,lse}.h .

Andrea

Quoted 2 lines

This is also the common implementation on other architectures who use aq/rl instructions like ARM. And you certainly already knew it~

author
#14 lore raw
Quoted 31 lines · Andrea Parri
Quoted 24 lines
Quoted 24 lines
Quoted 23 lines

(not a review, just looking at this diff stat) is changing the fastpath

read_unlock()
read_lock()

from something like

fence rw,w
amodadd.w
amoadd.w
fence r,rw

to

fence rw,rw
amoadd.w
amoadd.w
fence rw,rw

no matter Zalasr or !Zalasr. Similarly for other atomic operations with release or acquire semantics. I guess the change was not intentional? If it was intentional, it should be properly mentioned in the changelog.

Quoted 6 lines
Quoted 6 lines

Sorry about that. It is intended. The atomic operation before __atomic_acquire_fence or operation after __atomic_release_fence can be just a single ld or sd instruction instead of amocas or amoswap. In such cases, when the store release operation becomes 'sd.rl', the __atomic_acquire_fence via 'fence r, rw' can not ensure FENCE.TSO anymore. Thus I replace it with 'fence rw, rw'.

Quoted 3 lines

But you could apply similar changes you performed for xchg & cmpxchg: use .AQ and .RL for other atomic RMW operations as well, no? AFAICS, that is what arm64 actually does in arch/arm64/include/asm/atomic_{ll_sc,lse}.h .

I see. I will study the implementation of ARM and refine my patch. Thanks a lot.

Best regards,
Xu Lu

Quoted 6 lines

Andrea

Quoted 2 lines

This is also the common implementation on other architectures who use aq/rl instructions like ARM. And you certainly already knew it~