riscv: add support for restartable sequence
Subsystems (4)
v2 cover riscv: add support for restartable sequence
Add RSEQ, restartable sequence, support and related selftest to RISCV. The Kconfig option HAVE_REGS_AND_STACK_ACCESS_API is also required by RSEQ because RSEQ will modify the content of pt_regs.sepc through instruction_pointer_set() during the fixup procedure. In order to select the config HAVE_REGS_AND_STACK_ACCESS_API, the missing APIs for accessing pt_regs are also added in this patch set.
The relevant RSEQ tests in kselftest require the Binutils patch "RISC-V:
Fix linker problems with TLS copy relocs" to avoid placing PREINIT_ARRAY and TLS variable of librseq.so at the same address.
https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;a=commit;h=3e7bd7f24146f162565edf878840449f36a8d974
A segmental fault will happen if binutils misses this patch.
Patrick Stählin (1):
riscv: add required functions to enable HAVE_REGS_AND_STACK_ACCESS_API
Vincent Chen (2):
riscv: Add support for restartable sequence
rseq/selftests: Add support for riscv
Changes since v1:
1. Use the correct register name to access pt_regs
arch/riscv/Kconfig | 2 + arch/riscv/include/asm/ptrace.h | 29 +- arch/riscv/kernel/entry.S | 4 + arch/riscv/kernel/ptrace.c | 99 +++++ arch/riscv/kernel/signal.c | 2 + tools/testing/selftests/rseq/param_test.c | 23 ++ tools/testing/selftests/rseq/rseq-riscv.h | 622 ++++++++++++++++++++++++++++++ tools/testing/selftests/rseq/rseq.h | 2 + 8 files changed, 782 insertions(+), 1 deletion(-) create mode 100644 tools/testing/selftests/rseq/rseq-riscv.h
Signature · 1 line
2.7.4
PATCH v2 1/3 riscv: add required functions to enable HAVE_REGS_AND_STACK_ACCESS_API
From: Patrick Stählin <[email protected]>
In order to select HAVE_REGS_AND_STACK_ACCESS_API, adding the APIs required by kprobes to access pt_regs and stack entries to the RISC-V ports.
--- arch/riscv/Kconfig | 1 + arch/riscv/include/asm/ptrace.h | 29 +++++++++++- arch/riscv/kernel/ptrace.c | 99 +++++++++++++++++++++++++++++++++++++++++ 3 files changed, 128 insertions(+), 1 deletion(-)
arch/riscv/Kconfig +1 -0
@@ -66,6 +66,7 @@ config RISCV select ARCH_HAS_GCOV_PROFILE_ALL select HAVE_COPY_THREAD_TLS select HAVE_ARCH_KASAN if MMU && 64BIT + select HAVE_REGS_AND_STACK_ACCESS_API config ARCH_MMAP_RND_BITS_MIN default 18 if 64BIT
arch/riscv/include/asm/ptrace.h +28 -1
@@ -8,6 +8,7 @@#include <uapi/asm/ptrace.h> #include <asm/csr.h> +#include <linux/compiler.h> #ifndef __ASSEMBLY__
@@ -59,7 +60,7 @@ struct pt_regs {#endif #define user_mode (regs) (((regs)->status & SR_PP) == 0) - +#define MAX_REG_OFFSET offsetof(struct pt_regs, orig_a0)/* Helpers for working with the instruction pointer */ static inline unsigned long instruction_pointer (struct pt_regs *regs)
@@ -79,6 +80,12 @@ static inline unsigned long user_stack_pointer(struct pt_regs *regs) {return regs->sp; } + +static inline unsigned long kernel_stack_pointer (struct pt_regs *regs) +{ +return regs->sp; +} +static inline void user_stack_pointer_set (struct pt_regs *regs,unsigned long val) {
@@ -101,6 +108,26 @@ static inline unsigned long regs_return_value(struct pt_regs *regs)return regs->a0; } +int regs_query_register_offset (const char *name); + +unsigned long regs_get_kernel_stack_nth (struct pt_regs *regs,unsigned int n); +/** +* regs_get_register() - get register value from its offset +* @regs: pt_regs from which register value is gotten +* @offset: offset of the register. +* +* regs_get_register returns the value of a register whose offset from @regs. +* The @offset is the offset of the register in struct pt_regs. +* If @offset is bigger than MAX_REG_OFFSET, this returns 0. +*/ +static inline unsigned long regs_get_register (struct pt_regs *regs, +unsigned int offset) +{ +if (unlikely (offset >MAX_REG_OFFSET )) +return 0 ; + +return *(unsigned long *)((unsigned long )regs + offset); +}#endif /* __ASSEMBLY__ */ #endif /* _ASM_RISCV_PTRACE_H */
arch/riscv/kernel/ptrace.c +99 -0
105 lines not on this page · raw message
Signature · 1 line
2.7.4
PATCH v2 2/3 riscv: Add support for restartable sequence
In reply to Vincent ChenAdd calls to rseq_signal_deliver(), rseq_handle_notify_resume() and rseq_syscall() to introduce RSEQ support.
1. Call the rseq_handle_notify_resume() function on return to userspace if TIF_NOTIFY_RESUME thread flag is set.
2. Call the rseq_signal_deliver() function to fixup on the pre-signal frame when a signal is delivered on top of a restartable sequence critical section.
3. Check that system calls are not invoked from within rseq critical sections by invoking rseq_signal() from ret_from_syscall(). With CONFIG_DEBUG_RSEQ, such behavior results in termination of the process with SIGSEGV.
--- arch/riscv/Kconfig | 1 + arch/riscv/kernel/entry.S | 4 ++++ arch/riscv/kernel/signal.c | 2 ++ 3 files changed, 7 insertions(+)
arch/riscv/Kconfig +1 -0
@@ -67,6 +67,7 @@ config RISCV select HAVE_COPY_THREAD_TLS select HAVE_ARCH_KASAN if MMU && 64BIT select HAVE_REGS_AND_STACK_ACCESS_API + select HAVE_RSEQ config ARCH_MMAP_RND_BITS_MIN default 18 if 64BIT
arch/riscv/kernel/entry.S +4 -0
@@ -214,6 +214,10 @@ ENTRY(handle_exception) handle_syscall: /* save the initial A0 value (needed in signal handlers) */ REG_S a0, PT_ORIG_A0(sp) +#ifdef CONFIG_RSEQ_DEBUG + move a0, sp + call rseq_syscall +#endif /* * Advance SEPC to avoid executing the original * scall instruction on sret
arch/riscv/kernel/signal.c +2 -0
@@ -234,6 +234,7 @@ static void handle_signal(struct ksignal *ksig, struct pt_regs *regs)sigset_t *oldset =sigmask_to_save ();int ret; +rseq_signal_deliver (ksig, regs);/* Are we from a system call? */ if (regs->cause ==EXC_SYSCALL ) {/* Avoid additional syscall restarting via ret_from_exception */
@@ -316,5 +317,6 @@ asmlinkage __visible void do_notify_resume(struct pt_regs *regs,if (thread_info_flags & _TIF_NOTIFY_RESUME) {clear_thread_flag (TIF_NOTIFY_RESUME );tracehook_notify_resume (regs); +rseq_handle_notify_resume (NULL , regs); } }
Signature · 1 line
2.7.4
Mathieu Desnoyers
maintainerRe: [PATCH V2 2/3] riscv: Add support for restartable sequence
----- On Mar 9, 2020, at 1:59 AM, Vincent Chen [email protected] wrote:
[...]
arch/riscv/kernel/signal.c static void handle_signal(struct ksignal *ksig, struct · line 240
Quoted 10 lines
arch/riscv/kernel/signal.c +1
-0@@ -234,6 +234,7 @@ static void handle_signal(struct ksignal *ksig, struct pt_regs *regs) sigset_t *oldset = sigmask_to_save();int ret; +rseq_signal_deliver (ksig, regs);/* Are we from a system call? */ if (regs->cause ==EXC_SYSCALL ) {
[...]
As Al Viro pointed out on IRC, the rseq_signal_deliver() should go after syscall restart handling, similarly to what is done on every other supported architecture.
Note that there is already an upstream commit derived on this non-upstream patch:
commit 9866d141a097 ("csky: Add support for restartable sequence")
which is broken in the same way.
I'm not sure why I was never CC'd on the csky patch. Considering that nobody bothered to implement the rseq selftests for csky, I don't see how any of this is tested. I would favor a revert of that commit until the testing glue is contributed. Unfortunately, the csky commit has been upstream since v5.7.
Thanks,
Mathieu
Signature · 3 lines
Mathieu Desnoyers EfficiOS Inc. http://www.efficios.com
Re: [PATCH V2 2/3] riscv: Add support for restartable sequence
Quoted 10 lines · Mathieu Desnoyers
----- On Mar 9, 2020, at 1:59 AM, Vincent Chen [email protected] wrote:
[...]Quoted 7 lines
arch/riscv/kernel/signal.c +0
-0@@ -234,6 +234,7 @@ static void handle_signal(struct ksignal *ksig, struct pt_regs *regs) sigset_t *oldset = sigmask_to_save();int ret;
Quoted 8 lines
arch/riscv/kernel/signal.c static void handle_signal(struct ksignal *ksig, struct pt_regs *regs) · line 239
Quoted 3 lines
arch/riscv/kernel/signal.c +1
-0+rseq_signal_deliver (ksig, regs);/* Are we from a system call? */ if (regs->cause ==EXC_SYSCALL ) {[...]
As Al Viro pointed out on IRC, the
rseq_signal_deliver()should go after syscall restart handling, similarly to what is done on every other supported architecture.
Thanks for the notification. I will adjust the porting and try to send the patch again for review.
Quoted 21 lines
Note that there is already an upstream commit derived on this non-upstream patch:
commit
9866d141a097("csky: Add support for restartable sequence")which is broken in the same way.
I'm not sure why I was never CC'd on the csky patch. Considering that nobody bothered to implement the rseq selftests for csky, I don't see how any of this is tested. I would favor a revert of that commit until the testing glue is contributed. Unfortunately, the csky commit has been upstream since v5.7.
Thanks,
Mathieu
--
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
PATCH v2 3/3 rseq/selftests: Add support for riscv
In reply to Vincent ChenAdd support for risc-v in the rseq selftests, which covers both 64-bit and 32-bit ISA with little endian mode.
--- tools/testing/selftests/rseq/param_test.c | 23 ++ tools/testing/selftests/rseq/rseq-riscv.h | 622 ++++++++++++++++++++++++++++++ tools/testing/selftests/rseq/rseq.h | 2 + 3 files changed, 647 insertions(+) create mode 100644 tools/testing/selftests/rseq/rseq-riscv.h
tools/testing/selftests/rseq/param_test.c +23 -0
@@ -205,6 +205,29 @@ unsigned int yield_mod_cnt, nr_abort;"addiu " INJECT_ASM_REG ", -1\n\t" \"bnez " INJECT_ASM_REG ", 222b\n\t" \ "333:\n\t" +#elif defined(__riscv) + +#define RSEQ_INJECT_INPUT \ + , [loop_cnt_1]"m"(loop_cnt[1]) \ + , [loop_cnt_2]"m"(loop_cnt[2]) \ + , [loop_cnt_3]"m"(loop_cnt[3]) \ + , [loop_cnt_4]"m"(loop_cnt[4]) \ + , [loop_cnt_5]"m"(loop_cnt[5]) \ + , [loop_cnt_6]"m"(loop_cnt[6]) + +#define INJECT_ASM_REG "t1" + +#define RSEQ_INJECT_CLOBBER \ + , INJECT_ASM_REG + +#define RSEQ_INJECT_ASM (n) \ + "lw " INJECT_ASM_REG ", %[loop_cnt_" #n "]\n\t" \ + "beqz " INJECT_ASM_REG ", 333f\n\t" \ + "222:\n\t" \ + "addi " INJECT_ASM_REG "," INJECT_ASM_REG ", -1\n\t" \ + "bnez " INJECT_ASM_REG ", 222b\n\t" \ + "333:\n\t" +#else #error unsupported target
tools/testing/selftests/rseq/rseq-riscv.h new file +622 -0
622 lines not on this page · raw message
tools/testing/selftests/rseq/rseq.h +2 -0
@@ -79,6 +79,8 @@ extern int __rseq_handled;#include <rseq-mips.h> #elif defined(__s390__)#include <rseq-s390.h> +#elif defined(__riscv) +#include <rseq-riscv.h> #else #error unsupported target#endif
Signature · 1 line
2.7.4
Palmer Dabbelt
maintainerRe: [PATCH V2 3/3] rseq/selftests: Add support for riscv
Quoted 10 lines · On Sun, 08 Mar 2020 22:59:52 PDT (-0700), [email protected] wrote:
Add support for risc-v in the rseq selftests, which covers both 64-bit and 32-bit ISA with little endian mode.
Signed-off-by Vincent Chen <[email protected]>--- tools/testing/selftests/rseq/param_test.c | 23 ++ tools/testing/selftests/rseq/rseq-riscv.h | 622 ++++++++++++++++++++++++++++++ tools/testing/selftests/rseq/rseq.h | 2 + 3 files changed, 647 insertions(+) create mode 100644 tools/testing/selftests/rseq/rseq-riscv.h
There are a ton of checkpatch errors in here.
Quoted 676 lines
tools/testing/selftests/rseq/param_test.c +23
-0@@ -205,6 +205,29 @@ unsigned int yield_mod_cnt, nr_abort;"addiu " INJECT_ASM_REG ", -1\n\t" \"bnez " INJECT_ASM_REG ", 222b\n\t" \ "333:\n\t" +#elif defined(__riscv) + +#define RSEQ_INJECT_INPUT \ + , [loop_cnt_1]"m"(loop_cnt[1]) \ + , [loop_cnt_2]"m"(loop_cnt[2]) \ + , [loop_cnt_3]"m"(loop_cnt[3]) \ + , [loop_cnt_4]"m"(loop_cnt[4]) \ + , [loop_cnt_5]"m"(loop_cnt[5]) \ + , [loop_cnt_6]"m"(loop_cnt[6]) + +#define INJECT_ASM_REG "t1" + +#define RSEQ_INJECT_CLOBBER \ + , INJECT_ASM_REG + +#define RSEQ_INJECT_ASM (n) \ + "lw " INJECT_ASM_REG ", %[loop_cnt_" #n "]\n\t" \ + "beqz " INJECT_ASM_REG ", 333f\n\t" \ + "222:\n\t" \ + "addi " INJECT_ASM_REG "," INJECT_ASM_REG ", -1\n\t" \ + "bnez " INJECT_ASM_REG ", 222b\n\t" \ + "333:\n\t" +#else #error unsupported targettools/testing/selftests/rseq/rseq-riscv.h new file +622
-0@@ -0,0 +1,622 @@ +/* SPDX-License-Identifier: LGPL-2.1 OR MIT */ +/* +* Select the instruction "csrw mhartid, x0" as the RSEQ_SIG. Unlike +* other architecture, the ebreak instruction has no immediate field for +* distinguishing purposes. Hence, ebreak is not suitable as RSEQ_SIG. +* "csrw mhartid, x0" can also satisfy the RSEQ requirement because it +* is an uncommon instruction and will raise an illegal instruction +* exception when executed in all modes. +*/ + +#if __ORDER_LITTLE_ENDIAN__ ==1234 +#define RSEQ_SIG 0xf1401073/* csrr mhartid, x0 */ +#else +#error "Currently, RSEQ only supports Little-Endian version" +#endif + +#if __riscv_xlen ==64 +#define __REG_SEL (a, b) a +#elif __riscv_xlen ==32 +#define __REG_SEL (a, b) b +#endif + +#define REG_L __REG_SEL("ld ", "lw ") +#define REG_S __REG_SEL("sd ", "sw ") + +#define RISCV_FENCE (p, s) \ + __asm__ __volatile__ ("fence " #p "," #s : : : "memory") +#define rseq_smp_mb () RISCV_FENCE(rw, rw) +#define rseq_smp_rmb () RISCV_FENCE(r, r) +#define rseq_smp_wmb () RISCV_FENCE(w, w) +#define RSEQ_ASM_TMP_REG_1 "t6" +#define RSEQ_ASM_TMP_REG_2 "t5" +#define RSEQ_ASM_TMP_REG_3 "t4" +#define RSEQ_ASM_TMP_REG_4 "t3" + +#define rseq_smp_load_acquire (p) \ +__extension__ ({ \ + __typeof(*p) ____p1 = RSEQ_READ_ONCE(*p); \ + RISCV_FENCE(r, rw) \ + ____p1; \ +}) + +#define rseq_smp_acquire__after_ctrl_dep () rseq_smp_rmb() + +#define rseq_smp_store_release (p, v) \ +do { \ + RISCV_FENCE(rw, w); \ + RSEQ_WRITE_ONCE(*p, v); \ +} while (0) + + +#ifdef RSEQ_SKIP_FASTPATH +#include "rseq-skip.h" +#else /* !RSEQ_SKIP_FASTPATH */ + +#define __RSEQ_ASM_DEFINE_TABLE (label, version, flags, start_ip, \ + post_commit_offset, abort_ip) \ + ".pushsection __rseq_cs, \"aw\"\n" \ + ".balign 32\n" \ + __rseq_str(label) ":\n" \ + ".long " __rseq_str(version) ", " __rseq_str(flags) "\n" \ + ".quad " __rseq_str(start_ip) ", " \ + __rseq_str(post_commit_offset) ", " \ + __rseq_str(abort_ip) "\n" \ + ".popsection\n\t" \ + ".pushsection __rseq_cs_ptr_array, \"aw\"\n" \ + ".quad " __rseq_str(label) "b\n" \ + ".popsection\n" + +#define RSEQ_ASM_DEFINE_TABLE (label, start_ip, post_commit_ip, abort_ip) \ + __RSEQ_ASM_DEFINE_TABLE(label, 0x0, 0x0, start_ip, \ + (post_commit_ip - start_ip), abort_ip) + + +/* +* Exit points of a rseq critical section consist of all instructions outside +* of the critical section where a critical section can either branch to or +* reach through the normal course of its execution. The abort IP and the +* post-commit IP are already part of the __rseq_cs section and should not be +* explicitly defined as additional exit points. Knowing all exit points is +* useful to assist debuggers stepping over the critical section. +*/ +#define RSEQ_ASM_DEFINE_EXIT_POINT (start_ip, exit_ip) \ + ".pushsection __rseq_exit_point_array, \"aw\"\n" \ + ".quad " __rseq_str(start_ip) ", " __rseq_str(exit_ip) "\n" \ + ".popsection\n" + +#define RSEQ_ASM_STORE_RSEQ_CS (label, cs_label, rseq_cs) \ + RSEQ_INJECT_ASM(1) \ + "la "RSEQ_ASM_TMP_REG_1 ", " __rseq_str(cs_label) "\n" \ + REG_S RSEQ_ASM_TMP_REG_1 ", %[" __rseq_str(rseq_cs) "]\n" \ + __rseq_str(label) ":\n" + +#define RSEQ_ASM_DEFINE_ABORT (label, abort_label) \ + "j 222f\n" \ + ".balign 4\n" \ + ".long " __rseq_str(RSEQ_SIG) "\n" \ + __rseq_str(label) ":\n" \ + "j %l[" __rseq_str(abort_label) "]\n" \ + "222:\n" + +#define RSEQ_ASM_OP_STORE (value, var) \ + REG_S "%[" __rseq_str(value) "], %[" __rseq_str(var) "]\n" + +#define RSEQ_ASM_OP_CMPEQ (var, expect, label) \ + REG_L RSEQ_ASM_TMP_REG_1 ", %[" __rseq_str(var) "]\n" \ + "bne "RSEQ_ASM_TMP_REG_1 ", %[" __rseq_str(expect) "] ," \ + __rseq_str(label) "\n" + +#define RSEQ_ASM_OP_CMPEQ32 (var, expect, label) \ + "lw "RSEQ_ASM_TMP_REG_1 ", %[" __rseq_str(var) "]\n" \ + "bne "RSEQ_ASM_TMP_REG_1 ", %[" __rseq_str(expect) "] ," \ + __rseq_str(label) "\n" + +#define RSEQ_ASM_OP_CMPNE (var, expect, label) \ + REG_L RSEQ_ASM_TMP_REG_1 ", %[" __rseq_str(var) "]\n" \ + "beq "RSEQ_ASM_TMP_REG_1 ", %[" __rseq_str(expect) "] ," \ + __rseq_str(label) "\n" + +#define RSEQ_ASM_CMP_CPU_ID (cpu_id, current_cpu_id, label) \ + RSEQ_INJECT_ASM(2) \ + RSEQ_ASM_OP_CMPEQ32(current_cpu_id, cpu_id, label) + +#define RSEQ_ASM_OP_R_LOAD (var) \ + REG_L RSEQ_ASM_TMP_REG_1 ", %[" __rseq_str(var) "]\n" + +#define RSEQ_ASM_OP_R_STORE (var) \ + REG_S RSEQ_ASM_TMP_REG_1 ", %[" __rseq_str(var) "]\n" + +#define RSEQ_ASM_OP_R_LOAD_OFF (offset) \ + "add "RSEQ_ASM_TMP_REG_1 ", %[" __rseq_str(offset) "], " \ + RSEQ_ASM_TMP_REG_1 "\n" \ + REG_L RSEQ_ASM_TMP_REG_1 ", (" RSEQ_ASM_TMP_REG_1 ")\n" + +#define RSEQ_ASM_OP_R_ADD (count) \ + "add "RSEQ_ASM_TMP_REG_1 ", " RSEQ_ASM_TMP_REG_1 \ + ", %[" __rseq_str(count) "]\n" + +#define RSEQ_ASM_OP_FINAL_STORE (value, var, post_commit_label) \ + RSEQ_ASM_OP_STORE(value, var) \ + __rseq_str(post_commit_label) ":\n" + +#define RSEQ_ASM_OP_FINAL_STORE_RELEASE (value, var, post_commit_label) \ + "fence rw, w\n" \ + RSEQ_ASM_OP_STORE(value, var) \ + __rseq_str(post_commit_label) ":\n" + +#define RSEQ_ASM_OP_R_FINAL_STORE (var, post_commit_label) \ + REG_S RSEQ_ASM_TMP_REG_1 ", %[" __rseq_str(var) "]\n" \ + __rseq_str(post_commit_label) ":\n" + +#define RSEQ_ASM_OP_R_BAD_MEMCPY (dst, src, len) \ + "beqz %[" __rseq_str(len) "], 333f\n" \ + "mv " RSEQ_ASM_TMP_REG_1 ", %[" __rseq_str(len) "]\n" \ + "mv " RSEQ_ASM_TMP_REG_2 ", %[" __rseq_str(src) "]\n" \ + "mv " RSEQ_ASM_TMP_REG_3 ", %[" __rseq_str(dst) "]\n" \ + "222:\n" \ + "lb " RSEQ_ASM_TMP_REG_4 ", 0(" RSEQ_ASM_TMP_REG_2 ")\n" \ + "sb " RSEQ_ASM_TMP_REG_4 ", 0(" RSEQ_ASM_TMP_REG_3 ")\n" \ + "addi " RSEQ_ASM_TMP_REG_1 ", " RSEQ_ASM_TMP_REG_1 ", -1\n" \ + "addi " RSEQ_ASM_TMP_REG_2 ", " RSEQ_ASM_TMP_REG_2 ", 1\n" \ + "addi " RSEQ_ASM_TMP_REG_3 ", " RSEQ_ASM_TMP_REG_3 ", 1\n" \ + "bnez " RSEQ_ASM_TMP_REG_1 ", 222b\n" \ + "333:\n" + +static inline __attribute__((always_inline)) +int rseq_cmpeqv_storev (intptr_t *v,intptr_t expect,intptr_t newv,int cpu) +{ + RSEQ_INJECT_C(9) + + __asm__ __volatile__ goto ( + RSEQ_ASM_DEFINE_TABLE(1, 2f, 3f, 4f) + RSEQ_ASM_DEFINE_EXIT_POINT(2f, %l[cmpfail]) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_DEFINE_EXIT_POINT(2f, %l[error1]) + RSEQ_ASM_DEFINE_EXIT_POINT(2f, %l[error2]) +#endif + RSEQ_ASM_STORE_RSEQ_CS(2, 1b, rseq_cs) + RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f) + RSEQ_INJECT_ASM(3) + RSEQ_ASM_OP_CMPEQ(v, expect, %l[cmpfail]) + RSEQ_INJECT_ASM(4) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, %l[error1]) + RSEQ_ASM_OP_CMPEQ(v, expect, %l[error2]) +#endif + RSEQ_ASM_OP_FINAL_STORE(newv, v, 3) + RSEQ_INJECT_ASM(5) +RSEQ_ASM_DEFINE_ABORT (4 , abort) + : /* gcc asm goto does not allow outputs */ + : [cpu_id] "r" (cpu), + [current_cpu_id] "m" (__rseq_abi.cpu_id), + [rseq_cs] "m" (__rseq_abi.rseq_cs), + [v] "m" (*v), + [expect] "r" (expect), + [newv] "r" (newv) + RSEQ_INJECT_INPUT + : "memory", RSEQ_ASM_TMP_REG_1 + : abort, cmpfail +#ifdef RSEQ_COMPARE_TWICE + , error1, error2 +#endif + ); + +return 0 ; +abort: +RSEQ_INJECT_FAILED + return -1; +cmpfail: +return 1 ; +#ifdef RSEQ_COMPARE_TWICE +error1: +rseq_bug ("cpu_id comparison failed" ); +error2: +rseq_bug ("expected value comparison failed" ); +#endif +} + +static inline __attribute__((always_inline)) +int rseq_cmpnev_storeoffp_load (intptr_t *v,intptr_t expectnot, +off_t voffp,intptr_t *load,int cpu) +{ + RSEQ_INJECT_C(9) + + __asm__ __volatile__ goto ( + RSEQ_ASM_DEFINE_TABLE(1, 2f, 3f, 4f) + RSEQ_ASM_DEFINE_EXIT_POINT(2f, %l[cmpfail]) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_DEFINE_EXIT_POINT(2f, %l[error1]) + RSEQ_ASM_DEFINE_EXIT_POINT(2f, %l[error2]) +#endif + RSEQ_ASM_STORE_RSEQ_CS(2, 1b, rseq_cs) + RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f) + RSEQ_INJECT_ASM(3) + RSEQ_ASM_OP_CMPNE(v, expectnot, %l[cmpfail]) + RSEQ_INJECT_ASM(4) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, %l[error1]) + RSEQ_ASM_OP_CMPNE(v, expectnot, %l[error2]) +#endif +RSEQ_ASM_OP_R_LOAD (v) +RSEQ_ASM_OP_R_STORE (load ) +RSEQ_ASM_OP_R_LOAD_OFF (voffp) +RSEQ_ASM_OP_R_FINAL_STORE (v,3 ) +RSEQ_INJECT_ASM (5 ) +RSEQ_ASM_DEFINE_ABORT (4 , abort) + : /* gcc asm goto does not allow outputs */ + : [cpu_id] "r" (cpu), + [current_cpu_id] "m" (__rseq_abi.cpu_id), + [rseq_cs] "m" (__rseq_abi.rseq_cs), + [v] "m" (*v), + [expectnot] "r" (expectnot), + [load] "m" (*load), + [voffp] "r" (voffp) + RSEQ_INJECT_INPUT + : "memory", RSEQ_ASM_TMP_REG_1 + : abort, cmpfail +#ifdef RSEQ_COMPARE_TWICE + , error1, error2 +#endif + ); +return 0 ; +abort: +RSEQ_INJECT_FAILED + return -1; +cmpfail: +return 1 ; +#ifdef RSEQ_COMPARE_TWICE +error1: +rseq_bug ("cpu_id comparison failed" ); +error2: +rseq_bug ("expected value comparison failed" ); +#endif +} + +static inline __attribute__((always_inline)) +int rseq_addv (intptr_t *v,intptr_t count,int cpu) +{ + RSEQ_INJECT_C(9) + + __asm__ __volatile__ goto ( + RSEQ_ASM_DEFINE_TABLE(1, 2f, 3f, 4f) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_DEFINE_EXIT_POINT(2f, %l[error1]) +#endif + RSEQ_ASM_STORE_RSEQ_CS(2, 1b, rseq_cs) + RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f) + RSEQ_INJECT_ASM(3) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, %l[error1]) +#endif +RSEQ_ASM_OP_R_LOAD (v) +RSEQ_ASM_OP_R_ADD (count ) +RSEQ_ASM_OP_R_FINAL_STORE (v,3 ) +RSEQ_INJECT_ASM (4 ) +RSEQ_ASM_DEFINE_ABORT (4 , abort) + : /* gcc asm goto does not allow outputs */ + : [cpu_id] "r" (cpu), + [current_cpu_id] "m" (__rseq_abi.cpu_id), + [rseq_cs] "m" (__rseq_abi.rseq_cs), + [v] "m" (*v), + [count] "r" (count) + RSEQ_INJECT_INPUT + : "memory", RSEQ_ASM_TMP_REG_1 + : abort +#ifdef RSEQ_COMPARE_TWICE + , error1 +#endif + ); +return 0 ; +abort: +RSEQ_INJECT_FAILED + return -1; +#ifdef RSEQ_COMPARE_TWICE +error1: +rseq_bug ("cpu_id comparison failed" ); +#endif +} + +static inline __attribute__((always_inline)) +int rseq_cmpeqv_trystorev_storev (intptr_t *v,intptr_t expect, +intptr_t *v2,intptr_t newv2, +intptr_t newv,int cpu) +{ + RSEQ_INJECT_C(9) + + __asm__ __volatile__ goto ( + RSEQ_ASM_DEFINE_TABLE(1, 2f, 3f, 4f) + RSEQ_ASM_DEFINE_EXIT_POINT(2f, %l[cmpfail]) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_DEFINE_EXIT_POINT(2f, %l[error1]) + RSEQ_ASM_DEFINE_EXIT_POINT(2f, %l[error2]) +#endif + RSEQ_ASM_STORE_RSEQ_CS(2, 1b, rseq_cs) + RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f) + RSEQ_INJECT_ASM(3) + RSEQ_ASM_OP_CMPEQ(v, expect, %l[cmpfail]) + RSEQ_INJECT_ASM(4) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, %l[error1]) + RSEQ_ASM_OP_CMPEQ(v, expect, %l[error2]) +#endif + RSEQ_ASM_OP_STORE(newv2, v2) + RSEQ_INJECT_ASM(5) + RSEQ_ASM_OP_FINAL_STORE(newv, v, 3) + RSEQ_INJECT_ASM(6) +RSEQ_ASM_DEFINE_ABORT (4 , abort) + : /* gcc asm goto does not allow outputs */ + : [cpu_id] "r" (cpu), + [current_cpu_id] "m" (__rseq_abi.cpu_id), + [rseq_cs] "m" (__rseq_abi.rseq_cs), + [expect] "r" (expect), + [v] "m" (*v), + [newv] "r" (newv), + [v2] "m" (*v2), + [newv2] "r" (newv2) + RSEQ_INJECT_INPUT + : "memory", RSEQ_ASM_TMP_REG_1 + : abort, cmpfail +#ifdef RSEQ_COMPARE_TWICE + , error1, error2 +#endif + ); + +return 0 ; +abort: +RSEQ_INJECT_FAILED + return -1; +cmpfail: +return 1 ; +#ifdef RSEQ_COMPARE_TWICE +error1: +rseq_bug ("cpu_id comparison failed" ); +error2: +rseq_bug ("expected value comparison failed" ); +#endif +} + +static inline __attribute__((always_inline)) +int rseq_cmpeqv_trystorev_storev_release (intptr_t *v,intptr_t expect, +intptr_t *v2,intptr_t newv2, +intptr_t newv,int cpu) +{ + RSEQ_INJECT_C(9) + + __asm__ __volatile__ goto ( + RSEQ_ASM_DEFINE_TABLE(1, 2f, 3f, 4f) + RSEQ_ASM_DEFINE_EXIT_POINT(2f, %l[cmpfail]) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_DEFINE_EXIT_POINT(2f, %l[error1]) + RSEQ_ASM_DEFINE_EXIT_POINT(2f, %l[error2]) +#endif + RSEQ_ASM_STORE_RSEQ_CS(2, 1b, rseq_cs) + RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f) + RSEQ_INJECT_ASM(3) + RSEQ_ASM_OP_CMPEQ(v, expect, %l[cmpfail]) + RSEQ_INJECT_ASM(4) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, %l[error1]) + RSEQ_ASM_OP_CMPEQ(v, expect, %l[error2]) +#endif + RSEQ_ASM_OP_STORE(newv2, v2) + RSEQ_INJECT_ASM(5) + RSEQ_ASM_OP_FINAL_STORE_RELEASE(newv, v, 3) + RSEQ_INJECT_ASM(6) +RSEQ_ASM_DEFINE_ABORT (4 , abort) + : /* gcc asm goto does not allow outputs */ + : [cpu_id] "r" (cpu), + [current_cpu_id] "m" (__rseq_abi.cpu_id), + [rseq_cs] "m" (__rseq_abi.rseq_cs), + [expect] "r" (expect), + [v] "m" (*v), + [newv] "r" (newv), + [v2] "m" (*v2), + [newv2] "r" (newv2) + RSEQ_INJECT_INPUT + : "memory", RSEQ_ASM_TMP_REG_1 + : abort, cmpfail +#ifdef RSEQ_COMPARE_TWICE + , error1, error2 +#endif + ); + +return 0 ; +abort: +RSEQ_INJECT_FAILED + return -1; +cmpfail: +return 1 ; +#ifdef RSEQ_COMPARE_TWICE +error1: +rseq_bug ("cpu_id comparison failed" ); +error2: +rseq_bug ("expected value comparison failed" ); +#endif +} + +static inline __attribute__((always_inline)) +int rseq_cmpeqv_cmpeqv_storev (intptr_t *v,intptr_t expect, +intptr_t *v2,intptr_t expect2, +intptr_t newv,int cpu) +{ + RSEQ_INJECT_C(9) + + __asm__ __volatile__ goto ( + RSEQ_ASM_DEFINE_TABLE(1, 2f, 3f, 4f) + RSEQ_ASM_DEFINE_EXIT_POINT(2f, %l[cmpfail]) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_DEFINE_EXIT_POINT(2f, %l[error1]) + RSEQ_ASM_DEFINE_EXIT_POINT(2f, %l[error2]) + RSEQ_ASM_DEFINE_EXIT_POINT(2f, %l[error3]) +#endif + RSEQ_ASM_STORE_RSEQ_CS(2, 1b, rseq_cs) + RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f) + RSEQ_INJECT_ASM(3) + RSEQ_ASM_OP_CMPEQ(v, expect, %l[cmpfail]) + RSEQ_INJECT_ASM(4) + RSEQ_ASM_OP_CMPEQ(v2, expect2, %l[cmpfail]) + RSEQ_INJECT_ASM(5) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, %l[error1]) + RSEQ_ASM_OP_CMPEQ(v, expect, %l[error2]) + RSEQ_ASM_OP_CMPEQ(v2, expect2, %l[error3]) +#endif + RSEQ_ASM_OP_FINAL_STORE(newv, v, 3) + RSEQ_INJECT_ASM(6) +RSEQ_ASM_DEFINE_ABORT (4 , abort) + : /* gcc asm goto does not allow outputs */ + : [cpu_id] "r" (cpu), + [current_cpu_id] "m" (__rseq_abi.cpu_id), + [rseq_cs] "m" (__rseq_abi.rseq_cs), + [v] "m" (*v), + [expect] "r" (expect), + [v2] "m" (*v2), + [expect2] "r" (expect2), + [newv] "r" (newv) + RSEQ_INJECT_INPUT + : "memory", RSEQ_ASM_TMP_REG_1 + : abort, cmpfail +#ifdef RSEQ_COMPARE_TWICE + , error1, error2, error3 +#endif + ); + +return 0 ; +abort: +RSEQ_INJECT_FAILED + return -1; +cmpfail: +return 1 ; +#ifdef RSEQ_COMPARE_TWICE +error1: +rseq_bug ("cpu_id comparison failed" ); +error2: +rseq_bug ("expected value comparison failed" ); +error3: +rseq_bug ("2nd expected value comparison failed" ); +#endif +} + +static inline __attribute__((always_inline)) +int rseq_cmpeqv_trymemcpy_storev (intptr_t *v,intptr_t expect, +void *dst,void *src,size_t len, +intptr_t newv,int cpu) +{ + RSEQ_INJECT_C(9) + __asm__ __volatile__ goto ( + RSEQ_ASM_DEFINE_TABLE(1, 2f, 3f, 4f) + RSEQ_ASM_DEFINE_EXIT_POINT(2f, %l[cmpfail]) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_DEFINE_EXIT_POINT(2f, %l[error1]) + RSEQ_ASM_DEFINE_EXIT_POINT(2f, %l[error2]) +#endif + RSEQ_ASM_STORE_RSEQ_CS(2, 1b, rseq_cs) + RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f) + RSEQ_INJECT_ASM(3) + RSEQ_ASM_OP_CMPEQ(v, expect, %l[cmpfail]) + RSEQ_INJECT_ASM(4) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, %l[error1]) + RSEQ_ASM_OP_CMPEQ(v, expect, %l[error2]) +#endif + RSEQ_ASM_OP_R_BAD_MEMCPY(dst, src, len) + RSEQ_INJECT_ASM(5) + RSEQ_ASM_OP_FINAL_STORE(newv, v, 3) + RSEQ_INJECT_ASM(6) +RSEQ_ASM_DEFINE_ABORT (4 , abort) + : /* gcc asm goto does not allow outputs */ + : [cpu_id] "r" (cpu), + [current_cpu_id] "m" (__rseq_abi.cpu_id), + [rseq_cs] "m" (__rseq_abi.rseq_cs), + [expect] "r" (expect), + [v] "m" (*v), + [newv] "r" (newv), + [dst] "r" (dst), + [src] "r" (src), + [len] "r" (len) + RSEQ_INJECT_INPUT + : "memory", RSEQ_ASM_TMP_REG_1, RSEQ_ASM_TMP_REG_2, + RSEQ_ASM_TMP_REG_3, RSEQ_ASM_TMP_REG_4 + : abort, cmpfail +#ifdef RSEQ_COMPARE_TWICE + , error1, error2 +#endif + ); + +return 0 ; +abort: +RSEQ_INJECT_FAILED + return -1; +cmpfail: +return 1 ; +#ifdef RSEQ_COMPARE_TWICE +error1: +rseq_bug ("cpu_id comparison failed" ); +error2: +rseq_bug ("expected value comparison failed" ); +#endif +} + +static inline __attribute__((always_inline)) +int rseq_cmpeqv_trymemcpy_storev_release (intptr_t *v,intptr_t expect, +void *dst,void *src,size_t len, +intptr_t newv,int cpu) +{ + RSEQ_INJECT_C(9) + + __asm__ __volatile__ goto ( + RSEQ_ASM_DEFINE_TABLE(1, 2f, 3f, 4f) + RSEQ_ASM_DEFINE_EXIT_POINT(2f, %l[cmpfail]) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_DEFINE_EXIT_POINT(2f, %l[error1]) + RSEQ_ASM_DEFINE_EXIT_POINT(2f, %l[error2]) +#endif + RSEQ_ASM_STORE_RSEQ_CS(2, 1b, rseq_cs) + RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f) + RSEQ_INJECT_ASM(3) + RSEQ_ASM_OP_CMPEQ(v, expect, %l[cmpfail]) + RSEQ_INJECT_ASM(4) +#ifdef RSEQ_COMPARE_TWICE + RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, %l[error1]) + RSEQ_ASM_OP_CMPEQ(v, expect, %l[error2]) +#endif + RSEQ_ASM_OP_R_BAD_MEMCPY(dst, src, len) + RSEQ_INJECT_ASM(5) + RSEQ_ASM_OP_FINAL_STORE_RELEASE(newv, v, 3) + RSEQ_INJECT_ASM(6) +RSEQ_ASM_DEFINE_ABORT (4 , abort) + : /* gcc asm goto does not allow outputs */ + : [cpu_id] "r" (cpu), + [current_cpu_id] "m" (__rseq_abi.cpu_id), + [rseq_cs] "m" (__rseq_abi.rseq_cs), + [expect] "r" (expect), + [v] "m" (*v), + [newv] "r" (newv), + [dst] "r" (dst), + [src] "r" (src), + [len] "r" (len) + RSEQ_INJECT_INPUT + : "memory", RSEQ_ASM_TMP_REG_1, RSEQ_ASM_TMP_REG_2, + RSEQ_ASM_TMP_REG_3, RSEQ_ASM_TMP_REG_4 + : abort, cmpfail +#ifdef RSEQ_COMPARE_TWICE + , error1, error2 +#endif + ); + +return 0 ; +abort: +RSEQ_INJECT_FAILED + return -1; +cmpfail: +return 1 ; +#ifdef RSEQ_COMPARE_TWICE +error1: +rseq_bug ("cpu_id comparison failed" ); +error2: +rseq_bug ("expected value comparison failed" ); +#endif +} + +#endif /* !RSEQ_SKIP_FASTPATH */ tools/testing/selftests/rseq/rseq.h +2
-0@@ -79,6 +79,8 @@ extern int __rseq_handled;#include <rseq-mips.h> #elif defined(__s390__)#include <rseq-s390.h> +#elif defined(__riscv) +#include <rseq-riscv.h> #else #error unsupported target#endif
Mathieu Desnoyers
maintainerRe: [PATCH V2 3/3] rseq/selftests: Add support for riscv
Quoted 12 lines · ----- On Mar 26, 2020, at 11:49 AM, Palmer Dabbelt [email protected] wrote:
Quoted 10 lines · On Sun, 08 Mar 2020 22:59:52 PDT (-0700), [email protected] wrote:
Add support for risc-v in the rseq selftests, which covers both 64-bit and 32-bit ISA with little endian mode.
Signed-off-by Vincent Chen <[email protected]>--- tools/testing/selftests/rseq/param_test.c | 23 ++ tools/testing/selftests/rseq/rseq-riscv.h | 622 ++++++++++++++++++++++++++++++ tools/testing/selftests/rseq/rseq.h | 2 + 3 files changed, 647 insertions(+) create mode 100644 tools/testing/selftests/rseq/rseq-riscv.h
Quoted 1 line
There are a ton of checkpatch errors in here.
Is it just my mail client or the main issue is:
ERROR: DOS line endings
?
As far as other issues are concerned, I know there are a few checkpatch false-positives that trigger for my rseq-{$ARCH}.h header, from which rseq-riscv.h is derived, because it has issues with extensive use of inline assembly.
Thanks,
Mathieu
Signature · 3 lines
Mathieu Desnoyers EfficiOS Inc. http://www.efficios.com
Re: [PATCH V2 3/3] rseq/selftests: Add support for riscv
Quoted 15 lines · Mathieu Desnoyers
Quoted 12 lines · ----- On Mar 26, 2020, at 11:49 AM, Palmer Dabbelt [email protected] wrote:
Quoted 10 lines · On Sun, 08 Mar 2020 22:59:52 PDT (-0700), [email protected] wrote:
Add support for risc-v in the rseq selftests, which covers both 64-bit and 32-bit ISA with little endian mode.
Signed-off-by Vincent Chen <[email protected]>--- tools/testing/selftests/rseq/param_test.c | 23 ++ tools/testing/selftests/rseq/rseq-riscv.h | 622 ++++++++++++++++++++++++++++++ tools/testing/selftests/rseq/rseq.h | 2 + 3 files changed, 647 insertions(+) create mode 100644 tools/testing/selftests/rseq/rseq-riscv.h
Quoted 7 lines
Quoted 1 line
There are a ton of checkpatch errors in here.
Is it just my mail client or the main issue is:
ERROR: DOS line endings
?
I am not sure, but I did not run into this error in my environment.
Quoted 9 lines
As far as other issues are concerned, I know there are a few checkpatch false-positives that trigger for my rseq-{$ARCH}.h header, from which
rseq-riscv.his derived, because it has issues with extensive use of inline assembly.Thanks,
Mathieu
Thank Mathieu for your explanation.
The errors reported by checkpatch.pl can be categorized into two cases. The first one is "need consistent spacing around %". such as
ERROR: need consistent spacing around '%' (ctx:WxV) #628: FILE: tools/testing/selftests/rseq/rseq-riscv.h:572:
+ RSEQ_ASM_DEFINE_EXIT_POINT(2f, %l[error2])
where RSEQ_ASM_DEFINE_EXIT_POINT is defined as below
#define RSEQ_ASM_DEFINE_EXIT_POINT(start_ip, exit_ip) \
".pushsection __rseq_exit_point_array, \"aw\"\n" \
".quad " __rseq_str(start_ip) ", " __rseq_str(exit_ip) "\n" \
".popsection\n"These errors were mainly found in the rseq-riscv.h. As Mathieu mentioned, the RSEQ_ASM_DEFINE_EXIT_POINT macro is used in the inline assembly, which the second argument %l[error2] indicates the error2 is a label and it locates outside of the inline assembly. To obey the syntax, I cannot add a space after % to fix this bug.
The second kind of error is "Macros with complex values should be enclosed in parentheses" such as
ERROR: Macros with complex values should be enclosed in parentheses #27: FILE: tools/testing/selftests/rseq/param_test.c:210:
+#define RSEQ_INJECT_INPUT \
+ , [loop_cnt_1]"m"(loop_cnt[1]) \ + , [loop_cnt_2]"m"(loop_cnt[2]) \ + , [loop_cnt_3]"m"(loop_cnt[3]) \ + , [loop_cnt_4]"m"(loop_cnt[4]) \ + , [loop_cnt_5]"m"(loop_cnt[5]) \ + , [loop_cnt_6]"m"(loop_cnt[6])
In this case, it was a input operand list of inline assembly, so I could not add parentheses to enclose them. Except for these two kinds of error, there are two erros could be solved by adding parentheses. I also checked it and I think it would be safe. So, I mimic the implementations of other architecture without using parenthese. If needed, I think can add the parenthese to solve these two errors. Thanks