Skip to contentSkip to conversation

Restartable Sequences selftests updates

PATCH In review Mathieu Desnoyers 4 participants 26 messages Apr 29, 2019 – Jun 8, 2019

Subsystems (3)
lore
Share & tools
lore

Mathieu Desnoyers

author maintainer
#0 lore raw

cover Restartable Sequences selftests updates

Those rseq selftests updates are hereby submitted to Shuah Khan, maintainer of kernel selftests, for the next merge window (5.2).

They change the per-architecture pre-abort signatures to ensure those are valid trap instructions.

The way exit points are presented to debuggers is enhanced, ensuring all exit points are present, so debuggers don't have to disassemble rseq critical section to properly skip over them.

Discussions with the glibc community is reaching a concensus of exposing a __rseq_handled symbol from glibc to coexist with rseq early adopters. Update the rseq selftest code to expose and use this symbol.

Support for compiling asm goto with clang is added with the "-no-integrated-as" compiler switch, similarly to the toplevel kernel Makefile.

Thanks,

Mathieu

Martin Schwidefsky (1):
rseq/selftests: s390: use trap4 for RSEQ_SIG

Mathieu Desnoyers (11):
rseq/selftests: x86: Work-around bogus gcc-8 optimisation rseq/selftests: Add __rseq_exit_point_array section for debuggers rseq/selftests: Introduce __rseq_cs_ptr_array, rename __rseq_table to

    __rseq_cs

rseq/selftests: Use __rseq_handled symbol to coexist with glibc rseq/selftests: s390: use jg instruction for jumps outside of the asm rseq/selftests: x86: use ud1 instruction as RSEQ_SIG opcode rseq/selftests: arm: use udf instruction for RSEQ_SIG rseq/selftests: aarch64 code signature: handle big-endian environment rseq/selftests: powerpc code signature: generate valid instructions rseq/selftests: mips: use break instruction for RSEQ_SIG rseq/selftests: add -no-integrated-as for clang

 tools/testing/selftests/rseq/Makefile     |   8 +-
 tools/testing/selftests/rseq/rseq-arm.h   | 132 +++++++++++++--
 tools/testing/selftests/rseq/rseq-arm64.h |  74 ++++++++-
 tools/testing/selftests/rseq/rseq-mips.h  | 115 +++++++++++--
 tools/testing/selftests/rseq/rseq-ppc.h   |  90 +++++++++-
 tools/testing/selftests/rseq/rseq-s390.h  |  78 ++++++++-
 tools/testing/selftests/rseq/rseq-x86.h   | 264 +++++++++++++++++++++---------
 tools/testing/selftests/rseq/rseq.c       |  55 ++++++-
 tools/testing/selftests/rseq/rseq.h       |   1 +
 9 files changed, 688 insertions(+), 129 deletions(-)
Signature · 1 line
2.11.0
author maintainer
#1 lore raw

PATCH 1/12 rseq/selftests: x86: Work-around bogus gcc-8 optimisation

gcc-8 version 8.1.0, 8.2.0, and 8.3.0 generate broken assembler with asm goto that have a thread-local storage "m" input operand on both x86-32 and x86-64. For instance:

__thread int var;

static int fct(void)

{
        asm goto (      "jmp %l[testlabel]\n\t"
                        : : [var] "m" (var) : : testlabel);
        return 0;
testlabel:
        return 1;
}

int main()

{
        return fct();
}

% gcc-8 -O2 -o test-asm-goto test-asm-goto.c /tmp/ccAdHJbe.o: In function `main':
test-asm-goto.c:(.text.startup+0x1): undefined reference to `.L2'
collect2: error: ld returned 1 exit status

% gcc-8 -m32 -O2 -o test-asm-goto test-asm-goto.c /tmp/ccREsVXA.o: In function `main':
test-asm-goto.c:(.text.startup+0x1): undefined reference to `.L2'
collect2: error: ld returned 1 exit status

Work-around this compiler bug in the rseq-x86.h header by passing the address of the __rseq_abi TLS as a register operand rather than using the "m" input operand.

CC Ingo Molnar <[email protected]>
CC Peter Zijlstra <[email protected]>
CC Thomas Gleixner <[email protected]>
CC Joel Fernandes <[email protected]>
CC Catalin Marinas <[email protected]>
CC Dave Watson <[email protected]>
CC Will Deacon <[email protected]>
CC Shuah Khan <[email protected]>
CC Andi Kleen <[email protected]>
CC "H . Peter Anvin" <[email protected]>
CC Chris Lameter <[email protected]>
CC Russell King <[email protected]>
CC Michael Kerrisk <[email protected]>
CC "Paul E . McKenney" <[email protected]>
CC Paul Turner <[email protected]>
CC Boqun Feng <[email protected]>
CC Josh Triplett <[email protected]>
CC Steven Rostedt <[email protected]>
CC Ben Maurer <[email protected]>
CC Andy Lutomirski <[email protected]>
CC Andrew Morton <[email protected]>
CC Linus Torvalds <[email protected]>
---
 tools/testing/selftests/rseq/rseq-x86.h | 144 ++++++++++++++++----------------
 1 file changed, 70 insertions(+), 74 deletions(-)
tools/testing/selftests/rseq/rseq-x86.h +70 -74
index 089410a314e9..a5341044a2f5 100644
@@ -9,6 +9,16 @@
 
 #define RSEQ_SIG	0x53053053
 
+/*
+ * Due to a compiler optimization bug in gcc-8 with asm goto and TLS asm input
+ * operands, we cannot use "m" input operands, and rather pass the __rseq_abi
+ * address through a "r" input operand.
+ */
+
+/* Offset of cpu_id and rseq_cs fields in struct rseq. */
+#define RSEQ_CPU_ID_OFFSET	4
+#define RSEQ_CS_OFFSET		8
+
 #ifdef __x86_64__
 
 #define rseq_smp_mb()	\
@@ -51,12 +61,12 @@ do {									\
 #define RSEQ_ASM_STORE_RSEQ_CS(label, cs_label, rseq_cs)		\
 		RSEQ_INJECT_ASM(1)					\
 		"leaq " __rseq_str(cs_label) "(%%rip), %%rax\n\t"	\
-		"movq %%rax, %[" __rseq_str(rseq_cs) "]\n\t"		\
+		"movq %%rax, " __rseq_str(rseq_cs) "\n\t"		\
 		__rseq_str(label) ":\n\t"
 
 #define RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, label)		\
 		RSEQ_INJECT_ASM(2)					\
-		"cmpl %[" __rseq_str(cpu_id) "], %[" __rseq_str(current_cpu_id) "]\n\t" \
+		"cmpl %[" __rseq_str(cpu_id) "], " __rseq_str(current_cpu_id) "\n\t" \
 		"jnz " __rseq_str(label) "\n\t"
 
 #define RSEQ_ASM_DEFINE_ABORT(label, teardown, abort_label)		\
@@ -84,14 +94,14 @@ int rseq_cmpeqv_storev(intptr_t *v, intptr_t expect, intptr_t newv, int cpu)
 	__asm__ __volatile__ goto (
 		RSEQ_ASM_DEFINE_TABLE(3, 1f, 2f, 4f) /* start, commit, abort */
 		/* Start rseq by storing table entry pointer into rseq_cs. */
-		RSEQ_ASM_STORE_RSEQ_CS(1, 3b, rseq_cs)
-		RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f)
+		RSEQ_ASM_STORE_RSEQ_CS(1, 3b, RSEQ_CS_OFFSET(%[rseq_abi]))
+		RSEQ_ASM_CMP_CPU_ID(cpu_id, RSEQ_CPU_ID_OFFSET(%[rseq_abi]), 4f)
 		RSEQ_INJECT_ASM(3)
 		"cmpq %[v], %[expect]\n\t"
 		"jnz %l[cmpfail]\n\t"
 		RSEQ_INJECT_ASM(4)
 #ifdef RSEQ_COMPARE_TWICE
-		RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, %l[error1])
+		RSEQ_ASM_CMP_CPU_ID(cpu_id, RSEQ_CPU_ID_OFFSET(%[rseq_abi]), %l[error1])
 		"cmpq %[v], %[expect]\n\t"
 		"jnz %l[error2]\n\t"
 #endif
@@ -102,8 +112,7 @@ int rseq_cmpeqv_storev(intptr_t *v, intptr_t expect, intptr_t newv, int cpu)
 		RSEQ_ASM_DEFINE_ABORT(4, "", abort)
 		: /* gcc asm goto does not allow outputs */
 		: [cpu_id]		"r" (cpu),
-		  [current_cpu_id]	"m" (__rseq_abi.cpu_id),
-		  [rseq_cs]		"m" (__rseq_abi.rseq_cs),
+		  [rseq_abi]		"r" (&__rseq_abi),
 		  [v]			"m" (*v),
 		  [expect]		"r" (expect),
 		  [newv]		"r" (newv)
@@ -141,15 +150,15 @@ int rseq_cmpnev_storeoffp_load(intptr_t *v, intptr_t expectnot,
 	__asm__ __volatile__ goto (
 		RSEQ_ASM_DEFINE_TABLE(3, 1f, 2f, 4f) /* start, commit, abort */
 		/* Start rseq by storing table entry pointer into rseq_cs. */
-		RSEQ_ASM_STORE_RSEQ_CS(1, 3b, rseq_cs)
-		RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f)
+		RSEQ_ASM_STORE_RSEQ_CS(1, 3b, RSEQ_CS_OFFSET(%[rseq_abi]))
+		RSEQ_ASM_CMP_CPU_ID(cpu_id, RSEQ_CPU_ID_OFFSET(%[rseq_abi]), 4f)
 		RSEQ_INJECT_ASM(3)
 		"movq %[v], %%rbx\n\t"
 		"cmpq %%rbx, %[expectnot]\n\t"
 		"je %l[cmpfail]\n\t"
 		RSEQ_INJECT_ASM(4)
 #ifdef RSEQ_COMPARE_TWICE
-		RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, %l[error1])
+		RSEQ_ASM_CMP_CPU_ID(cpu_id, RSEQ_CPU_ID_OFFSET(%[rseq_abi]), %l[error1])
 		"movq %[v], %%rbx\n\t"
 		"cmpq %%rbx, %[expectnot]\n\t"
 		"je %l[error2]\n\t"
@@ -164,8 +173,7 @@ int rseq_cmpnev_storeoffp_load(intptr_t *v, intptr_t expectnot,
 		RSEQ_ASM_DEFINE_ABORT(4, "", abort)
 		: /* gcc asm goto does not allow outputs */
 		: [cpu_id]		"r" (cpu),
-		  [current_cpu_id]	"m" (__rseq_abi.cpu_id),
-		  [rseq_cs]		"m" (__rseq_abi.rseq_cs),
+		  [rseq_abi]		"r" (&__rseq_abi),
 		  /* final store input */
 		  [v]			"m" (*v),
 		  [expectnot]		"r" (expectnot),
@@ -200,11 +208,11 @@ int rseq_addv(intptr_t *v, intptr_t count, int cpu)
 	__asm__ __volatile__ goto (
 		RSEQ_ASM_DEFINE_TABLE(3, 1f, 2f, 4f) /* start, commit, abort */
 		/* Start rseq by storing table entry pointer into rseq_cs. */
-		RSEQ_ASM_STORE_RSEQ_CS(1, 3b, rseq_cs)
-		RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f)
+		RSEQ_ASM_STORE_RSEQ_CS(1, 3b, RSEQ_CS_OFFSET(%[rseq_abi]))
+		RSEQ_ASM_CMP_CPU_ID(cpu_id, RSEQ_CPU_ID_OFFSET(%[rseq_abi]), 4f)
 		RSEQ_INJECT_ASM(3)
 #ifdef RSEQ_COMPARE_TWICE
-		RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, %l[error1])
+		RSEQ_ASM_CMP_CPU_ID(cpu_id, RSEQ_CPU_ID_OFFSET(%[rseq_abi]), %l[error1])
 #endif
 		/* final store */
 		"addq %[count], %[v]\n\t"
@@ -213,8 +221,7 @@ int rseq_addv(intptr_t *v, intptr_t count, int cpu)
 		RSEQ_ASM_DEFINE_ABORT(4, "", abort)
 		: /* gcc asm goto does not allow outputs */
 		: [cpu_id]		"r" (cpu),
-		  [current_cpu_id]	"m" (__rseq_abi.cpu_id),
-		  [rseq_cs]		"m" (__rseq_abi.rseq_cs),
+		  [rseq_abi]		"r" (&__rseq_abi),
 		  /* final store input */
 		  [v]			"m" (*v),
 		  [count]		"er" (count)
@@ -245,14 +252,14 @@ int rseq_cmpeqv_trystorev_storev(intptr_t *v, intptr_t expect,
 	__asm__ __volatile__ goto (
 		RSEQ_ASM_DEFINE_TABLE(3, 1f, 2f, 4f) /* start, commit, abort */
 		/* Start rseq by storing table entry pointer into rseq_cs. */
-		RSEQ_ASM_STORE_RSEQ_CS(1, 3b, rseq_cs)
-		RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f)
+		RSEQ_ASM_STORE_RSEQ_CS(1, 3b, RSEQ_CS_OFFSET(%[rseq_abi]))
+		RSEQ_ASM_CMP_CPU_ID(cpu_id, RSEQ_CPU_ID_OFFSET(%[rseq_abi]), 4f)
 		RSEQ_INJECT_ASM(3)
 		"cmpq %[v], %[expect]\n\t"
 		"jnz %l[cmpfail]\n\t"
 		RSEQ_INJECT_ASM(4)
 #ifdef RSEQ_COMPARE_TWICE
-		RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, %l[error1])
+		RSEQ_ASM_CMP_CPU_ID(cpu_id, RSEQ_CPU_ID_OFFSET(%[rseq_abi]), %l[error1])
 		"cmpq %[v], %[expect]\n\t"
 		"jnz %l[error2]\n\t"
 #endif
@@ -266,8 +273,7 @@ int rseq_cmpeqv_trystorev_storev(intptr_t *v, intptr_t expect,
 		RSEQ_ASM_DEFINE_ABORT(4, "", abort)
 		: /* gcc asm goto does not allow outputs */
 		: [cpu_id]		"r" (cpu),
-		  [current_cpu_id]	"m" (__rseq_abi.cpu_id),
-		  [rseq_cs]		"m" (__rseq_abi.rseq_cs),
+		  [rseq_abi]		"r" (&__rseq_abi),
 		  /* try store input */
 		  [v2]			"m" (*v2),
 		  [newv2]		"r" (newv2),
@@ -315,8 +321,8 @@ int rseq_cmpeqv_cmpeqv_storev(intptr_t *v, intptr_t expect,
 	__asm__ __volatile__ goto (
 		RSEQ_ASM_DEFINE_TABLE(3, 1f, 2f, 4f) /* start, commit, abort */
 		/* Start rseq by storing table entry pointer into rseq_cs. */
-		RSEQ_ASM_STORE_RSEQ_CS(1, 3b, rseq_cs)
-		RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f)
+		RSEQ_ASM_STORE_RSEQ_CS(1, 3b, RSEQ_CS_OFFSET(%[rseq_abi]))
+		RSEQ_ASM_CMP_CPU_ID(cpu_id, RSEQ_CPU_ID_OFFSET(%[rseq_abi]), 4f)
 		RSEQ_INJECT_ASM(3)
 		"cmpq %[v], %[expect]\n\t"
 		"jnz %l[cmpfail]\n\t"
@@ -325,7 +331,7 @@ int rseq_cmpeqv_cmpeqv_storev(intptr_t *v, intptr_t expect,
 		"jnz %l[cmpfail]\n\t"
 		RSEQ_INJECT_ASM(5)
 #ifdef RSEQ_COMPARE_TWICE
-		RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, %l[error1])
+		RSEQ_ASM_CMP_CPU_ID(cpu_id, RSEQ_CPU_ID_OFFSET(%[rseq_abi]), %l[error1])
 		"cmpq %[v], %[expect]\n\t"
 		"jnz %l[error2]\n\t"
 		"cmpq %[v2], %[expect2]\n\t"
@@ -338,8 +344,7 @@ int rseq_cmpeqv_cmpeqv_storev(intptr_t *v, intptr_t expect,
 		RSEQ_ASM_DEFINE_ABORT(4, "", abort)
 		: /* gcc asm goto does not allow outputs */
 		: [cpu_id]		"r" (cpu),
-		  [current_cpu_id]	"m" (__rseq_abi.cpu_id),
-		  [rseq_cs]		"m" (__rseq_abi.rseq_cs),
+		  [rseq_abi]		"r" (&__rseq_abi),
 		  /* cmp2 input */
 		  [v2]			"m" (*v2),
 		  [expect2]		"r" (expect2),
@@ -385,14 +390,14 @@ int rseq_cmpeqv_trymemcpy_storev(intptr_t *v, intptr_t expect,
 		"movq %[dst], %[rseq_scratch1]\n\t"
 		"movq %[len], %[rseq_scratch2]\n\t"
 		/* Start rseq by storing table entry pointer into rseq_cs. */
-		RSEQ_ASM_STORE_RSEQ_CS(1, 3b, rseq_cs)
-		RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f)
+		RSEQ_ASM_STORE_RSEQ_CS(1, 3b, RSEQ_CS_OFFSET(%[rseq_abi]))
+		RSEQ_ASM_CMP_CPU_ID(cpu_id, RSEQ_CPU_ID_OFFSET(%[rseq_abi]), 4f)
 		RSEQ_INJECT_ASM(3)
 		"cmpq %[v], %[expect]\n\t"
 		"jnz 5f\n\t"
 		RSEQ_INJECT_ASM(4)
 #ifdef RSEQ_COMPARE_TWICE
-		RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 6f)
+		RSEQ_ASM_CMP_CPU_ID(cpu_id, RSEQ_CPU_ID_OFFSET(%[rseq_abi]), 6f)
 		"cmpq %[v], %[expect]\n\t"
 		"jnz 7f\n\t"
 #endif
@@ -440,8 +445,7 @@ int rseq_cmpeqv_trymemcpy_storev(intptr_t *v, intptr_t expect,
 #endif
 		: /* gcc asm goto does not allow outputs */
 		: [cpu_id]		"r" (cpu),
-		  [current_cpu_id]	"m" (__rseq_abi.cpu_id),
-		  [rseq_cs]		"m" (__rseq_abi.rseq_cs),
+		  [rseq_abi]		"r" (&__rseq_abi),
 		  /* final store input */
 		  [v]			"m" (*v),
 		  [expect]		"r" (expect),
@@ -533,12 +537,12 @@ do {									\
 
 #define RSEQ_ASM_STORE_RSEQ_CS(label, cs_label, rseq_cs)		\
 		RSEQ_INJECT_ASM(1)					\
-		"movl $" __rseq_str(cs_label) ", %[rseq_cs]\n\t"	\
+		"movl $" __rseq_str(cs_label) ", " __rseq_str(rseq_cs) "\n\t"	\
 		__rseq_str(label) ":\n\t"
 
 #define RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, label)		\
 		RSEQ_INJECT_ASM(2)					\
-		"cmpl %[" __rseq_str(cpu_id) "], %[" __rseq_str(current_cpu_id) "]\n\t" \
+		"cmpl %[" __rseq_str(cpu_id) "], " __rseq_str(current_cpu_id) "\n\t" \
 		"jnz " __rseq_str(label) "\n\t"
 
 #define RSEQ_ASM_DEFINE_ABORT(label, teardown, abort_label)		\
@@ -566,14 +570,14 @@ int rseq_cmpeqv_storev(intptr_t *v, intptr_t expect, intptr_t newv, int cpu)
 	__asm__ __volatile__ goto (
 		RSEQ_ASM_DEFINE_TABLE(3, 1f, 2f, 4f) /* start, commit, abort */
 		/* Start rseq by storing table entry pointer into rseq_cs. */
-		RSEQ_ASM_STORE_RSEQ_CS(1, 3b, rseq_cs)
-		RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f)
+		RSEQ_ASM_STORE_RSEQ_CS(1, 3b, RSEQ_CS_OFFSET(%[rseq_abi]))
+		RSEQ_ASM_CMP_CPU_ID(cpu_id, RSEQ_CPU_ID_OFFSET(%[rseq_abi]), 4f)
 		RSEQ_INJECT_ASM(3)
 		"cmpl %[v], %[expect]\n\t"
 		"jnz %l[cmpfail]\n\t"
 		RSEQ_INJECT_ASM(4)
 #ifdef RSEQ_COMPARE_TWICE
-		RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, %l[error1])
+		RSEQ_ASM_CMP_CPU_ID(cpu_id, RSEQ_CPU_ID_OFFSET(%[rseq_abi]), %l[error1])
 		"cmpl %[v], %[expect]\n\t"
 		"jnz %l[error2]\n\t"
 #endif
@@ -584,8 +588,7 @@ int rseq_cmpeqv_storev(intptr_t *v, intptr_t expect, intptr_t newv, int cpu)
 		RSEQ_ASM_DEFINE_ABORT(4, "", abort)
 		: /* gcc asm goto does not allow outputs */
 		: [cpu_id]		"r" (cpu),
-		  [current_cpu_id]	"m" (__rseq_abi.cpu_id),
-		  [rseq_cs]		"m" (__rseq_abi.rseq_cs),
+		  [rseq_abi]		"r" (&__rseq_abi),
 		  [v]			"m" (*v),
 		  [expect]		"r" (expect),
 		  [newv]		"r" (newv)
@@ -623,15 +626,15 @@ int rseq_cmpnev_storeoffp_load(intptr_t *v, intptr_t expectnot,
 	__asm__ __volatile__ goto (
 		RSEQ_ASM_DEFINE_TABLE(3, 1f, 2f, 4f) /* start, commit, abort */
 		/* Start rseq by storing table entry pointer into rseq_cs. */
-		RSEQ_ASM_STORE_RSEQ_CS(1, 3b, rseq_cs)
-		RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f)
+		RSEQ_ASM_STORE_RSEQ_CS(1, 3b, RSEQ_CS_OFFSET(%[rseq_abi]))
+		RSEQ_ASM_CMP_CPU_ID(cpu_id, RSEQ_CPU_ID_OFFSET(%[rseq_abi]), 4f)
 		RSEQ_INJECT_ASM(3)
 		"movl %[v], %%ebx\n\t"
 		"cmpl %%ebx, %[expectnot]\n\t"
 		"je %l[cmpfail]\n\t"
 		RSEQ_INJECT_ASM(4)
 #ifdef RSEQ_COMPARE_TWICE
-		RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, %l[error1])
+		RSEQ_ASM_CMP_CPU_ID(cpu_id, RSEQ_CPU_ID_OFFSET(%[rseq_abi]), %l[error1])
 		"movl %[v], %%ebx\n\t"
 		"cmpl %%ebx, %[expectnot]\n\t"
 		"je %l[error2]\n\t"
@@ -646,8 +649,7 @@ int rseq_cmpnev_storeoffp_load(intptr_t *v, intptr_t expectnot,
 		RSEQ_ASM_DEFINE_ABORT(4, "", abort)
 		: /* gcc asm goto does not allow outputs */
 		: [cpu_id]		"r" (cpu),
-		  [current_cpu_id]	"m" (__rseq_abi.cpu_id),
-		  [rseq_cs]		"m" (__rseq_abi.rseq_cs),
+		  [rseq_abi]		"r" (&__rseq_abi),
 		  /* final store input */
 		  [v]			"m" (*v),
 		  [expectnot]		"r" (expectnot),
@@ -682,11 +684,11 @@ int rseq_addv(intptr_t *v, intptr_t count, int cpu)
 	__asm__ __volatile__ goto (
 		RSEQ_ASM_DEFINE_TABLE(3, 1f, 2f, 4f) /* start, commit, abort */
 		/* Start rseq by storing table entry pointer into rseq_cs. */
-		RSEQ_ASM_STORE_RSEQ_CS(1, 3b, rseq_cs)
-		RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f)
+		RSEQ_ASM_STORE_RSEQ_CS(1, 3b, RSEQ_CS_OFFSET(%[rseq_abi]))
+		RSEQ_ASM_CMP_CPU_ID(cpu_id, RSEQ_CPU_ID_OFFSET(%[rseq_abi]), 4f)
 		RSEQ_INJECT_ASM(3)
 #ifdef RSEQ_COMPARE_TWICE
-		RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, %l[error1])
+		RSEQ_ASM_CMP_CPU_ID(cpu_id, RSEQ_CPU_ID_OFFSET(%[rseq_abi]), %l[error1])
 #endif
 		/* final store */
 		"addl %[count], %[v]\n\t"
@@ -695,8 +697,7 @@ int rseq_addv(intptr_t *v, intptr_t count, int cpu)
 		RSEQ_ASM_DEFINE_ABORT(4, "", abort)
 		: /* gcc asm goto does not allow outputs */
 		: [cpu_id]		"r" (cpu),
-		  [current_cpu_id]	"m" (__rseq_abi.cpu_id),
-		  [rseq_cs]		"m" (__rseq_abi.rseq_cs),
+		  [rseq_abi]		"r" (&__rseq_abi),
 		  /* final store input */
 		  [v]			"m" (*v),
 		  [count]		"ir" (count)
@@ -727,14 +728,14 @@ int rseq_cmpeqv_trystorev_storev(intptr_t *v, intptr_t expect,
 	__asm__ __volatile__ goto (
 		RSEQ_ASM_DEFINE_TABLE(3, 1f, 2f, 4f) /* start, commit, abort */
 		/* Start rseq by storing table entry pointer into rseq_cs. */
-		RSEQ_ASM_STORE_RSEQ_CS(1, 3b, rseq_cs)
-		RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f)
+		RSEQ_ASM_STORE_RSEQ_CS(1, 3b, RSEQ_CS_OFFSET(%[rseq_abi]))
+		RSEQ_ASM_CMP_CPU_ID(cpu_id, RSEQ_CPU_ID_OFFSET(%[rseq_abi]), 4f)
 		RSEQ_INJECT_ASM(3)
 		"cmpl %[v], %[expect]\n\t"
 		"jnz %l[cmpfail]\n\t"
 		RSEQ_INJECT_ASM(4)
 #ifdef RSEQ_COMPARE_TWICE
-		RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, %l[error1])
+		RSEQ_ASM_CMP_CPU_ID(cpu_id, RSEQ_CPU_ID_OFFSET(%[rseq_abi]), %l[error1])
 		"cmpl %[v], %[expect]\n\t"
 		"jnz %l[error2]\n\t"
 #endif
@@ -749,8 +750,7 @@ int rseq_cmpeqv_trystorev_storev(intptr_t *v, intptr_t expect,
 		RSEQ_ASM_DEFINE_ABORT(4, "", abort)
 		: /* gcc asm goto does not allow outputs */
 		: [cpu_id]		"r" (cpu),
-		  [current_cpu_id]	"m" (__rseq_abi.cpu_id),
-		  [rseq_cs]		"m" (__rseq_abi.rseq_cs),
+		  [rseq_abi]		"r" (&__rseq_abi),
 		  /* try store input */
 		  [v2]			"m" (*v2),
 		  [newv2]		"m" (newv2),
@@ -789,15 +789,15 @@ int rseq_cmpeqv_trystorev_storev_release(intptr_t *v, intptr_t expect,
 	__asm__ __volatile__ goto (
 		RSEQ_ASM_DEFINE_TABLE(3, 1f, 2f, 4f) /* start, commit, abort */
 		/* Start rseq by storing table entry pointer into rseq_cs. */
-		RSEQ_ASM_STORE_RSEQ_CS(1, 3b, rseq_cs)
-		RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f)
+		RSEQ_ASM_STORE_RSEQ_CS(1, 3b, RSEQ_CS_OFFSET(%[rseq_abi]))
+		RSEQ_ASM_CMP_CPU_ID(cpu_id, RSEQ_CPU_ID_OFFSET(%[rseq_abi]), 4f)
 		RSEQ_INJECT_ASM(3)
 		"movl %[expect], %%eax\n\t"
 		"cmpl %[v], %%eax\n\t"
 		"jnz %l[cmpfail]\n\t"
 		RSEQ_INJECT_ASM(4)
 #ifdef RSEQ_COMPARE_TWICE
-		RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, %l[error1])
+		RSEQ_ASM_CMP_CPU_ID(cpu_id, RSEQ_CPU_ID_OFFSET(%[rseq_abi]), %l[error1])
 		"movl %[expect], %%eax\n\t"
 		"cmpl %[v], %%eax\n\t"
 		"jnz %l[error2]\n\t"
@@ -813,8 +813,7 @@ int rseq_cmpeqv_trystorev_storev_release(intptr_t *v, intptr_t expect,
 		RSEQ_ASM_DEFINE_ABORT(4, "", abort)
 		: /* gcc asm goto does not allow outputs */
 		: [cpu_id]		"r" (cpu),
-		  [current_cpu_id]	"m" (__rseq_abi.cpu_id),
-		  [rseq_cs]		"m" (__rseq_abi.rseq_cs),
+		  [rseq_abi]		"r" (&__rseq_abi),
 		  /* try store input */
 		  [v2]			"m" (*v2),
 		  [newv2]		"r" (newv2),
@@ -854,8 +853,8 @@ int rseq_cmpeqv_cmpeqv_storev(intptr_t *v, intptr_t expect,
 	__asm__ __volatile__ goto (
 		RSEQ_ASM_DEFINE_TABLE(3, 1f, 2f, 4f) /* start, commit, abort */
 		/* Start rseq by storing table entry pointer into rseq_cs. */
-		RSEQ_ASM_STORE_RSEQ_CS(1, 3b, rseq_cs)
-		RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f)
+		RSEQ_ASM_STORE_RSEQ_CS(1, 3b, RSEQ_CS_OFFSET(%[rseq_abi]))
+		RSEQ_ASM_CMP_CPU_ID(cpu_id, RSEQ_CPU_ID_OFFSET(%[rseq_abi]), 4f)
 		RSEQ_INJECT_ASM(3)
 		"cmpl %[v], %[expect]\n\t"
 		"jnz %l[cmpfail]\n\t"
@@ -864,7 +863,7 @@ int rseq_cmpeqv_cmpeqv_storev(intptr_t *v, intptr_t expect,
 		"jnz %l[cmpfail]\n\t"
 		RSEQ_INJECT_ASM(5)
 #ifdef RSEQ_COMPARE_TWICE
-		RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, %l[error1])
+		RSEQ_ASM_CMP_CPU_ID(cpu_id, RSEQ_CPU_ID_OFFSET(%[rseq_abi]), %l[error1])
 		"cmpl %[v], %[expect]\n\t"
 		"jnz %l[error2]\n\t"
 		"cmpl %[expect2], %[v2]\n\t"
@@ -878,8 +877,7 @@ int rseq_cmpeqv_cmpeqv_storev(intptr_t *v, intptr_t expect,
 		RSEQ_ASM_DEFINE_ABORT(4, "", abort)
 		: /* gcc asm goto does not allow outputs */
 		: [cpu_id]		"r" (cpu),
-		  [current_cpu_id]	"m" (__rseq_abi.cpu_id),
-		  [rseq_cs]		"m" (__rseq_abi.rseq_cs),
+		  [rseq_abi]		"r" (&__rseq_abi),
 		  /* cmp2 input */
 		  [v2]			"m" (*v2),
 		  [expect2]		"r" (expect2),
@@ -926,15 +924,15 @@ int rseq_cmpeqv_trymemcpy_storev(intptr_t *v, intptr_t expect,
 		"movl %[dst], %[rseq_scratch1]\n\t"
 		"movl %[len], %[rseq_scratch2]\n\t"
 		/* Start rseq by storing table entry pointer into rseq_cs. */
-		RSEQ_ASM_STORE_RSEQ_CS(1, 3b, rseq_cs)
-		RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f)
+		RSEQ_ASM_STORE_RSEQ_CS(1, 3b, RSEQ_CS_OFFSET(%[rseq_abi]))
+		RSEQ_ASM_CMP_CPU_ID(cpu_id, RSEQ_CPU_ID_OFFSET(%[rseq_abi]), 4f)
 		RSEQ_INJECT_ASM(3)
 		"movl %[expect], %%eax\n\t"
 		"cmpl %%eax, %[v]\n\t"
 		"jnz 5f\n\t"
 		RSEQ_INJECT_ASM(4)
 #ifdef RSEQ_COMPARE_TWICE
-		RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 6f)
+		RSEQ_ASM_CMP_CPU_ID(cpu_id, RSEQ_CPU_ID_OFFSET(%[rseq_abi]), 6f)
 		"movl %[expect], %%eax\n\t"
 		"cmpl %%eax, %[v]\n\t"
 		"jnz 7f\n\t"
@@ -984,8 +982,7 @@ int rseq_cmpeqv_trymemcpy_storev(intptr_t *v, intptr_t expect,
 #endif
 		: /* gcc asm goto does not allow outputs */
 		: [cpu_id]		"r" (cpu),
-		  [current_cpu_id]	"m" (__rseq_abi.cpu_id),
-		  [rseq_cs]		"m" (__rseq_abi.rseq_cs),
+		  [rseq_abi]		"r" (&__rseq_abi),
 		  /* final store input */
 		  [v]			"m" (*v),
 		  [expect]		"m" (expect),
@@ -1034,15 +1031,15 @@ int rseq_cmpeqv_trymemcpy_storev_release(intptr_t *v, intptr_t expect,
 		"movl %[dst], %[rseq_scratch1]\n\t"
 		"movl %[len], %[rseq_scratch2]\n\t"
 		/* Start rseq by storing table entry pointer into rseq_cs. */
-		RSEQ_ASM_STORE_RSEQ_CS(1, 3b, rseq_cs)
-		RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f)
+		RSEQ_ASM_STORE_RSEQ_CS(1, 3b, RSEQ_CS_OFFSET(%[rseq_abi]))
+		RSEQ_ASM_CMP_CPU_ID(cpu_id, RSEQ_CPU_ID_OFFSET(%[rseq_abi]), 4f)
 		RSEQ_INJECT_ASM(3)
 		"movl %[expect], %%eax\n\t"
 		"cmpl %%eax, %[v]\n\t"
 		"jnz 5f\n\t"
 		RSEQ_INJECT_ASM(4)
 #ifdef RSEQ_COMPARE_TWICE
-		RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 6f)
+		RSEQ_ASM_CMP_CPU_ID(cpu_id, RSEQ_CPU_ID_OFFSET(%[rseq_abi]), 6f)
 		"movl %[expect], %%eax\n\t"
 		"cmpl %%eax, %[v]\n\t"
 		"jnz 7f\n\t"
@@ -1093,8 +1090,7 @@ int rseq_cmpeqv_trymemcpy_storev_release(intptr_t *v, intptr_t expect,
 #endif
 		: /* gcc asm goto does not allow outputs */
 		: [cpu_id]		"r" (cpu),
-		  [current_cpu_id]	"m" (__rseq_abi.cpu_id),
-		  [rseq_cs]		"m" (__rseq_abi.rseq_cs),
+		  [rseq_abi]		"r" (&__rseq_abi),
 		  /* final store input */
 		  [v]			"m" (*v),
 		  [expect]		"m" (expect),
Signature · 1 line
2.11.0
author maintainer
#2 lore raw

PATCH 2/12 rseq/selftests: Add __rseq_exit_point_array section for debuggers

In reply to Mathieu Desnoyers

Knowing all exit points is useful to assist debuggers stepping over the rseq critical sections without requiring them to disassemble the content of the critical section to figure out the exit points.

CC Thomas Gleixner <[email protected]>
CC Joel Fernandes <[email protected]>
CC Peter Zijlstra <[email protected]>
CC Catalin Marinas <[email protected]>
CC Dave Watson <[email protected]>
CC Will Deacon <[email protected]>
CC Shuah Khan <[email protected]>
CC Andi Kleen <[email protected]>
CC "H . Peter Anvin" <[email protected]>
CC Chris Lameter <[email protected]>
CC Russell King <[email protected]>
CC Michael Kerrisk <[email protected]>
CC "Paul E . McKenney" <[email protected]>
CC Paul Turner <[email protected]>
CC Boqun Feng <[email protected]>
CC Josh Triplett <[email protected]>
CC Steven Rostedt <[email protected]>
CC Ben Maurer <[email protected]>
CC Andy Lutomirski <[email protected]>
CC Andrew Morton <[email protected]>
CC Linus Torvalds <[email protected]>
---
 tools/testing/selftests/rseq/rseq-arm.h   | 52 +++++++++++++++++
 tools/testing/selftests/rseq/rseq-arm64.h | 52 +++++++++++++++++
 tools/testing/selftests/rseq/rseq-mips.h  | 53 +++++++++++++++++
 tools/testing/selftests/rseq/rseq-ppc.h   | 66 ++++++++++++++++++++++
 tools/testing/selftests/rseq/rseq-s390.h  | 55 ++++++++++++++++++
 tools/testing/selftests/rseq/rseq-x86.h   | 94 +++++++++++++++++++++++++++++++
 6 files changed, 372 insertions(+)
tools/testing/selftests/rseq/rseq-arm.h +52 -0
index 3cea19877227..17e8d231943a 100644
@@ -42,6 +42,19 @@ do {									\
 	__RSEQ_ASM_DEFINE_TABLE(0x0, 0x0, start_ip,			\
 				(post_commit_ip - start_ip), abort_ip)
 
+/*
+ * Exit points of a rseq critical section consist of all instructions outside
+ * of the critical section where a critical section can either branch to or
+ * reach through the normal course of its execution. The abort IP and the
+ * post-commit IP are already part of the __rseq_table section and should not
+ * be explicitly defined as additional exit points. Knowing all exit points is
+ * useful to assist debuggers stepping over the critical section.
+ */
+#define RSEQ_ASM_DEFINE_EXIT_POINT(start_ip, exit_ip)			\
+		".pushsection __rseq_exit_point_array, \"aw\"\n\t"	\
+		".word " __rseq_str(start_ip) ", 0x0, " __rseq_str(exit_ip) ", 0x0\n\t" \
+		".popsection\n\t"
+
 #define RSEQ_ASM_STORE_RSEQ_CS(label, cs_label, rseq_cs)		\
 		RSEQ_INJECT_ASM(1)					\
 		"adr r0, " __rseq_str(cs_label) "\n\t"			\
@@ -87,6 +100,11 @@ int rseq_cmpeqv_storev(intptr_t *v, intptr_t expect, intptr_t newv, int cpu)
 	rseq_workaround_gcc_asm_size_guess();
 	__asm__ __volatile__ goto (
 		RSEQ_ASM_DEFINE_TABLE(1f, 2f, 4f) /* start, commit, abort */
+		RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail])
+#ifdef RSEQ_COMPARE_TWICE
+		RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1])
+		RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error2])
+#endif
 		/* Start rseq by storing table entry pointer into rseq_cs. */
 		RSEQ_ASM_STORE_RSEQ_CS(1, 3f, rseq_cs)
 		RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f)
@@ -149,6 +167,11 @@ int rseq_cmpnev_storeoffp_load(intptr_t *v, intptr_t expectnot,
 	rseq_workaround_gcc_asm_size_guess();
 	__asm__ __volatile__ goto (
 		RSEQ_ASM_DEFINE_TABLE(1f, 2f, 4f) /* start, commit, abort */
+		RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail])
+#ifdef RSEQ_COMPARE_TWICE
+		RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1])
+		RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error2])
+#endif
 		/* Start rseq by storing table entry pointer into rseq_cs. */
 		RSEQ_ASM_STORE_RSEQ_CS(1, 3f, rseq_cs)
 		RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f)
@@ -215,6 +238,9 @@ int rseq_addv(intptr_t *v, intptr_t count, int cpu)
 	rseq_workaround_gcc_asm_size_guess();
 	__asm__ __volatile__ goto (
 		RSEQ_ASM_DEFINE_TABLE(1f, 2f, 4f) /* start, commit, abort */
+#ifdef RSEQ_COMPARE_TWICE
+		RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1])
+#endif
 		/* Start rseq by storing table entry pointer into rseq_cs. */
 		RSEQ_ASM_STORE_RSEQ_CS(1, 3f, rseq_cs)
 		RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f)
@@ -267,6 +293,11 @@ int rseq_cmpeqv_trystorev_storev(intptr_t *v, intptr_t expect,
 	rseq_workaround_gcc_asm_size_guess();
 	__asm__ __volatile__ goto (
 		RSEQ_ASM_DEFINE_TABLE(1f, 2f, 4f) /* start, commit, abort */
+		RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail])
+#ifdef RSEQ_COMPARE_TWICE
+		RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1])
+		RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error2])
+#endif
 		/* Start rseq by storing table entry pointer into rseq_cs. */
 		RSEQ_ASM_STORE_RSEQ_CS(1, 3f, rseq_cs)
 		RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f)
@@ -337,6 +368,11 @@ int rseq_cmpeqv_trystorev_storev_release(intptr_t *v, intptr_t expect,
 	rseq_workaround_gcc_asm_size_guess();
 	__asm__ __volatile__ goto (
 		RSEQ_ASM_DEFINE_TABLE(1f, 2f, 4f) /* start, commit, abort */
+		RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail])
+#ifdef RSEQ_COMPARE_TWICE
+		RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1])
+		RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error2])
+#endif
 		/* Start rseq by storing table entry pointer into rseq_cs. */
 		RSEQ_ASM_STORE_RSEQ_CS(1, 3f, rseq_cs)
 		RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f)
@@ -408,6 +444,12 @@ int rseq_cmpeqv_cmpeqv_storev(intptr_t *v, intptr_t expect,
 	rseq_workaround_gcc_asm_size_guess();
 	__asm__ __volatile__ goto (
 		RSEQ_ASM_DEFINE_TABLE(1f, 2f, 4f) /* start, commit, abort */
+		RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail])
+#ifdef RSEQ_COMPARE_TWICE
+		RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1])
+		RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error2])
+		RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error3])
+#endif
 		/* Start rseq by storing table entry pointer into rseq_cs. */
 		RSEQ_ASM_STORE_RSEQ_CS(1, 3f, rseq_cs)
 		RSEQ_ASM_CMP_CPU_ID(cpu_id, current_cpu_id, 4f)
@@ -486,6 +528,11 @@ int rseq_cmpeqv_trymemcpy_storev(intptr_t *v, intptr_t expect,
 	rseq_workaround_gcc_asm_size_guess();
 	__asm__ __volatile__ goto (
 		RSEQ_ASM_DEFINE_TABLE(1f, 2f, 4f) /* start, commit, abort */
+		RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail])
+#ifdef RSEQ_COMPARE_TWICE
+		RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1])
+		RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error2])
+#endif
 		"str %[src], %[rseq_scratch0]\n\t"
 		"str %[dst], %[rseq_scratch1]\n\t"
 		"str %[len], %[rseq_scratch2]\n\t"
@@ -605,6 +652,11 @@ int rseq_cmpeqv_trymemcpy_storev_release(intptr_t *v, intptr_t expect,
 	rseq_workaround_gcc_asm_size_guess();
 	__asm__ __volatile__ goto (
 		RSEQ_ASM_DEFINE_TABLE(1f, 2f, 4f) /* start, commit, abort */
+		RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail])
+#ifdef RSEQ_COMPARE_TWICE
+		RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1])
+		RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error2])
+#endif
 		"str %[src], %[rseq_scratch0]\n\t"
 		"str %[dst], %[rseq_scratch1]\n\t"
 		"str %[len], %[rseq_scratch2]\n\t"
tools/testing/selftests/rseq/rseq-arm64.h +52 -0

106 lines not on this page · raw message

tools/testing/selftests/rseq/rseq-mips.h +53 -0

107 lines not on this page · raw message

tools/testing/selftests/rseq/rseq-ppc.h +66 -0

126 lines not on this page · raw message

tools/testing/selftests/rseq/rseq-s390.h +55 -0

103 lines not on this page · raw message

tools/testing/selftests/rseq/rseq-x86.h +94 -0

190 lines not on this page · raw message

Signature · 1 line
2.11.0
author maintainer
#3 lore raw

PATCH 3/12 rseq/selftests: Introduce __rseq_cs_ptr_array, rename __rseq_table to __rseq_cs

In reply to Mathieu Desnoyers

The entries within __rseq_table are aligned on 32 bytes due to linux/rseq.h struct rseq_cs uapi requirements, but the start of the __rseq_table section is not guaranteed to be 32-byte aligned. It can cause padding to be added at the start of the section, which makes it hard to use as an array of items by debuggers.

Considering that __rseq_table does not really consist of a table due to the presence of padding, rename this section to __rseq_cs.

Create a new __rseq_cs_ptr_array section which contains 64-bit packed pointers to entries within the __rseq_cs section.

CC Thomas Gleixner <[email protected]>
CC Joel Fernandes <[email protected]>
CC Peter Zijlstra <[email protected]>
CC Catalin Marinas <[email protected]>
CC Dave Watson <[email protected]>
CC Will Deacon <[email protected]>
CC Shuah Khan <[email protected]>
CC Andi Kleen <[email protected]>
CC "H . Peter Anvin" <[email protected]>
CC Chris Lameter <[email protected]>
CC Russell King <[email protected]>
CC Michael Kerrisk <[email protected]>
CC "Paul E . McKenney" <[email protected]>
CC Paul Turner <[email protected]>
CC Boqun Feng <[email protected]>
CC Josh Triplett <[email protected]>
CC Steven Rostedt <[email protected]>
CC Ben Maurer <[email protected]>
CC Andy Lutomirski <[email protected]>
CC Andrew Morton <[email protected]>
CC Linus Torvalds <[email protected]>
---
 tools/testing/selftests/rseq/rseq-arm.h   | 32 +++++++++++++++++--------------
 tools/testing/selftests/rseq/rseq-arm64.h |  9 ++++++---
 tools/testing/selftests/rseq/rseq-mips.h  | 32 +++++++++++++++++--------------
 tools/testing/selftests/rseq/rseq-ppc.h   | 22 +++++++++++++--------
 tools/testing/selftests/rseq/rseq-s390.h  | 18 +++++++++++------
 tools/testing/selftests/rseq/rseq-x86.h   | 19 ++++++++++++------
 6 files changed, 81 insertions(+), 51 deletions(-)
tools/testing/selftests/rseq/rseq-arm.h +18 -14
index 17e8d231943a..5f262c54364f 100644
@@ -30,24 +30,28 @@ do {									\
 #include "rseq-skip.h"
 #else /* !RSEQ_SKIP_FASTPATH */
 
-#define __RSEQ_ASM_DEFINE_TABLE(version, flags,	start_ip,		\
+#define __RSEQ_ASM_DEFINE_TABLE(label, version, flags, start_ip,	\
 				post_commit_offset, abort_ip)		\
-		".pushsection __rseq_table, \"aw\"\n\t"			\
+		".pushsection __rseq_cs, \"aw\"\n\t"			\
 		".balign 32\n\t"					\
+		__rseq_str(label) ":\n\t"					\
 		".word " __rseq_str(version) ", " __rseq_str(flags) "\n\t" \
 		".word " __rseq_str(start_ip) ", 0x0, " __rseq_str(post_commit_offset) ", 0x0, " __rseq_str(abort_ip) ", 0x0\n\t" \
+		".popsection\n\t"					\
+		".pushsection __rseq_cs_ptr_array, \"aw\"\n\t"		\
+		".word " __rseq_str(label) "b, 0x0\n\t"			\
 		".popsection\n\t"
 
-#define RSEQ_ASM_DEFINE_TABLE(start_ip, post_commit_ip, abort_ip)	\
-	__RSEQ_ASM_DEFINE_TABLE(0x0, 0x0, start_ip,			\
+#define RSEQ_ASM_DEFINE_TABLE(label, start_ip, post_commit_ip, abort_ip) \
+	__RSEQ_ASM_DEFINE_TABLE(label, 0x0, 0x0, start_ip,		\
 				(post_commit_ip - start_ip), abort_ip)
 
 /*
  * Exit points of a rseq critical section consist of all instructions outside
  * of the critical section where a critical section can either branch to or
  * reach through the normal course of its execution. The abort IP and the
- * post-commit IP are already part of the __rseq_table section and should not
- * be explicitly defined as additional exit points. Knowing all exit points is
+ * post-commit IP are already part of the __rseq_cs section and should not be
+ * explicitly defined as additional exit points. Knowing all exit points is
  * useful to assist debuggers stepping over the critical section.
  */
 #define RSEQ_ASM_DEFINE_EXIT_POINT(start_ip, exit_ip)			\
@@ -99,7 +103,7 @@ int rseq_cmpeqv_storev(intptr_t *v, intptr_t expect, intptr_t newv, int cpu)
 
 	rseq_workaround_gcc_asm_size_guess();
 	__asm__ __volatile__ goto (
-		RSEQ_ASM_DEFINE_TABLE(1f, 2f, 4f) /* start, commit, abort */
+		RSEQ_ASM_DEFINE_TABLE(9, 1f, 2f, 4f) /* start, commit, abort */
 		RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail])
 #ifdef RSEQ_COMPARE_TWICE
 		RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1])
@@ -166,7 +170,7 @@ int rseq_cmpnev_storeoffp_load(intptr_t *v, intptr_t expectnot,
 
 	rseq_workaround_gcc_asm_size_guess();
 	__asm__ __volatile__ goto (
-		RSEQ_ASM_DEFINE_TABLE(1f, 2f, 4f) /* start, commit, abort */
+		RSEQ_ASM_DEFINE_TABLE(9, 1f, 2f, 4f) /* start, commit, abort */
 		RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail])
 #ifdef RSEQ_COMPARE_TWICE
 		RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1])
@@ -237,7 +241,7 @@ int rseq_addv(intptr_t *v, intptr_t count, int cpu)
 
 	rseq_workaround_gcc_asm_size_guess();
 	__asm__ __volatile__ goto (
-		RSEQ_ASM_DEFINE_TABLE(1f, 2f, 4f) /* start, commit, abort */
+		RSEQ_ASM_DEFINE_TABLE(9, 1f, 2f, 4f) /* start, commit, abort */
 #ifdef RSEQ_COMPARE_TWICE
 		RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1])
 #endif
@@ -292,7 +296,7 @@ int rseq_cmpeqv_trystorev_storev(intptr_t *v, intptr_t expect,
 
 	rseq_workaround_gcc_asm_size_guess();
 	__asm__ __volatile__ goto (
-		RSEQ_ASM_DEFINE_TABLE(1f, 2f, 4f) /* start, commit, abort */
+		RSEQ_ASM_DEFINE_TABLE(9, 1f, 2f, 4f) /* start, commit, abort */
 		RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail])
 #ifdef RSEQ_COMPARE_TWICE
 		RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1])
@@ -367,7 +371,7 @@ int rseq_cmpeqv_trystorev_storev_release(intptr_t *v, intptr_t expect,
 
 	rseq_workaround_gcc_asm_size_guess();
 	__asm__ __volatile__ goto (
-		RSEQ_ASM_DEFINE_TABLE(1f, 2f, 4f) /* start, commit, abort */
+		RSEQ_ASM_DEFINE_TABLE(9, 1f, 2f, 4f) /* start, commit, abort */
 		RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail])
 #ifdef RSEQ_COMPARE_TWICE
 		RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1])
@@ -443,7 +447,7 @@ int rseq_cmpeqv_cmpeqv_storev(intptr_t *v, intptr_t expect,
 
 	rseq_workaround_gcc_asm_size_guess();
 	__asm__ __volatile__ goto (
-		RSEQ_ASM_DEFINE_TABLE(1f, 2f, 4f) /* start, commit, abort */
+		RSEQ_ASM_DEFINE_TABLE(9, 1f, 2f, 4f) /* start, commit, abort */
 		RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail])
 #ifdef RSEQ_COMPARE_TWICE
 		RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1])
@@ -527,7 +531,7 @@ int rseq_cmpeqv_trymemcpy_storev(intptr_t *v, intptr_t expect,
 
 	rseq_workaround_gcc_asm_size_guess();
 	__asm__ __volatile__ goto (
-		RSEQ_ASM_DEFINE_TABLE(1f, 2f, 4f) /* start, commit, abort */
+		RSEQ_ASM_DEFINE_TABLE(9, 1f, 2f, 4f) /* start, commit, abort */
 		RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail])
 #ifdef RSEQ_COMPARE_TWICE
 		RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1])
@@ -651,7 +655,7 @@ int rseq_cmpeqv_trymemcpy_storev_release(intptr_t *v, intptr_t expect,
 
 	rseq_workaround_gcc_asm_size_guess();
 	__asm__ __volatile__ goto (
-		RSEQ_ASM_DEFINE_TABLE(1f, 2f, 4f) /* start, commit, abort */
+		RSEQ_ASM_DEFINE_TABLE(9, 1f, 2f, 4f) /* start, commit, abort */
 		RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail])
 #ifdef RSEQ_COMPARE_TWICE
 		RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1])
tools/testing/selftests/rseq/rseq-arm64.h +6 -3
index 2079f71e0ca2..b41a2a48e965 100644
@@ -82,13 +82,16 @@ do {										\
 
 #define __RSEQ_ASM_DEFINE_TABLE(label, version, flags, start_ip,		\
 				post_commit_offset, abort_ip)			\
-	"	.pushsection	__rseq_table, \"aw\"\n"				\
+	"	.pushsection	__rseq_cs, \"aw\"\n"				\
 	"	.balign	32\n"							\
 	__rseq_str(label) ":\n"							\
 	"	.long	" __rseq_str(version) ", " __rseq_str(flags) "\n"	\
 	"	.quad	" __rseq_str(start_ip) ", "				\
 			  __rseq_str(post_commit_offset) ", "			\
 			  __rseq_str(abort_ip) "\n"				\
+	"	.popsection\n\t"						\
+	"	.pushsection __rseq_cs_ptr_array, \"aw\"\n"				\
+	"	.quad " __rseq_str(label) "b\n"					\
 	"	.popsection\n"
 
 #define RSEQ_ASM_DEFINE_TABLE(label, start_ip, post_commit_ip, abort_ip)	\
@@ -99,8 +102,8 @@ do {										\
  * Exit points of a rseq critical section consist of all instructions outside
  * of the critical section where a critical section can either branch to or
  * reach through the normal course of its execution. The abort IP and the
- * post-commit IP are already part of the __rseq_table section and should not
- * be explicitly defined as additional exit points. Knowing all exit points is
+ * post-commit IP are already part of the __rseq_cs section and should not be
+ * explicitly defined as additional exit points. Knowing all exit points is
  * useful to assist debuggers stepping over the critical section.
  */
 #define RSEQ_ASM_DEFINE_EXIT_POINT(start_ip, exit_ip)				\
tools/testing/selftests/rseq/rseq-mips.h +18 -14
index 25d10ff54769..fe3eabcdcbe5 100644
@@ -54,26 +54,30 @@ do {									\
 # error unsupported _MIPS_SZLONG
 #endif
 
-#define __RSEQ_ASM_DEFINE_TABLE(version, flags,	start_ip, \
+#define __RSEQ_ASM_DEFINE_TABLE(label, version, flags, start_ip, \
 				post_commit_offset, abort_ip) \
-		".pushsection __rseq_table, \"aw\"\n\t" \
+		".pushsection __rseq_cs, \"aw\"\n\t" \
 		".balign 32\n\t" \
+		__rseq_str(label) ":\n\t"					\
 		".word " __rseq_str(version) ", " __rseq_str(flags) "\n\t" \
 		LONG " " U32_U64_PAD(__rseq_str(start_ip)) "\n\t" \
 		LONG " " U32_U64_PAD(__rseq_str(post_commit_offset)) "\n\t" \
 		LONG " " U32_U64_PAD(__rseq_str(abort_ip)) "\n\t" \
+		".popsection\n\t" \
+		".pushsection __rseq_cs_ptr_array, \"aw\"\n\t" \
+		LONG " " U32_U64_PAD(__rseq_str(label) "b") "\n\t" \
 		".popsection\n\t"
 
-#define RSEQ_ASM_DEFINE_TABLE(start_ip, post_commit_ip, abort_ip) \
-	__RSEQ_ASM_DEFINE_TABLE(0x0, 0x0, start_ip, \
+#define RSEQ_ASM_DEFINE_TABLE(label, start_ip, post_commit_ip, abort_ip) \
+	__RSEQ_ASM_DEFINE_TABLE(label, 0x0, 0x0, start_ip, \
 				(post_commit_ip - start_ip), abort_ip)
 
 /*
  * Exit points of a rseq critical section consist of all instructions outside
  * of the critical section where a critical section can either branch to or
  * reach through the normal course of its execution. The abort IP and the
- * post-commit IP are already part of the __rseq_table section and should not
- * be explicitly defined as additional exit points. Knowing all exit points is
+ * post-commit IP are already part of the __rseq_cs section and should not be
+ * explicitly defined as additional exit points. Knowing all exit points is
  * useful to assist debuggers stepping over the critical section.
  */
 #define RSEQ_ASM_DEFINE_EXIT_POINT(start_ip, exit_ip) \
@@ -127,7 +131,7 @@ int rseq_cmpeqv_storev(intptr_t *v, intptr_t expect, intptr_t newv, int cpu)
 
 	rseq_workaround_gcc_asm_size_guess();
 	__asm__ __volatile__ goto (
-		RSEQ_ASM_DEFINE_TABLE(1f, 2f, 4f) /* start, commit, abort */
+		RSEQ_ASM_DEFINE_TABLE(9, 1f, 2f, 4f) /* start, commit, abort */
 		RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail])
 #ifdef RSEQ_COMPARE_TWICE
 		RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1])
@@ -192,7 +196,7 @@ int rseq_cmpnev_storeoffp_load(intptr_t *v, intptr_t expectnot,
 
 	rseq_workaround_gcc_asm_size_guess();
 	__asm__ __volatile__ goto (
-		RSEQ_ASM_DEFINE_TABLE(1f, 2f, 4f) /* start, commit, abort */
+		RSEQ_ASM_DEFINE_TABLE(9, 1f, 2f, 4f) /* start, commit, abort */
 		RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail])
 #ifdef RSEQ_COMPARE_TWICE
 		RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1])
@@ -261,7 +265,7 @@ int rseq_addv(intptr_t *v, intptr_t count, int cpu)
 
 	rseq_workaround_gcc_asm_size_guess();
 	__asm__ __volatile__ goto (
-		RSEQ_ASM_DEFINE_TABLE(1f, 2f, 4f) /* start, commit, abort */
+		RSEQ_ASM_DEFINE_TABLE(9, 1f, 2f, 4f) /* start, commit, abort */
 #ifdef RSEQ_COMPARE_TWICE
 		RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1])
 #endif
@@ -316,7 +320,7 @@ int rseq_cmpeqv_trystorev_storev(intptr_t *v, intptr_t expect,
 
 	rseq_workaround_gcc_asm_size_guess();
 	__asm__ __volatile__ goto (
-		RSEQ_ASM_DEFINE_TABLE(1f, 2f, 4f) /* start, commit, abort */
+		RSEQ_ASM_DEFINE_TABLE(9, 1f, 2f, 4f) /* start, commit, abort */
 		RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail])
 #ifdef RSEQ_COMPARE_TWICE
 		RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1])
@@ -389,7 +393,7 @@ int rseq_cmpeqv_trystorev_storev_release(intptr_t *v, intptr_t expect,
 
 	rseq_workaround_gcc_asm_size_guess();
 	__asm__ __volatile__ goto (
-		RSEQ_ASM_DEFINE_TABLE(1f, 2f, 4f) /* start, commit, abort */
+		RSEQ_ASM_DEFINE_TABLE(9, 1f, 2f, 4f) /* start, commit, abort */
 		RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail])
 #ifdef RSEQ_COMPARE_TWICE
 		RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1])
@@ -463,7 +467,7 @@ int rseq_cmpeqv_cmpeqv_storev(intptr_t *v, intptr_t expect,
 
 	rseq_workaround_gcc_asm_size_guess();
 	__asm__ __volatile__ goto (
-		RSEQ_ASM_DEFINE_TABLE(1f, 2f, 4f) /* start, commit, abort */
+		RSEQ_ASM_DEFINE_TABLE(9, 1f, 2f, 4f) /* start, commit, abort */
 		RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail])
 #ifdef RSEQ_COMPARE_TWICE
 		RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1])
@@ -543,7 +547,7 @@ int rseq_cmpeqv_trymemcpy_storev(intptr_t *v, intptr_t expect,
 
 	rseq_workaround_gcc_asm_size_guess();
 	__asm__ __volatile__ goto (
-		RSEQ_ASM_DEFINE_TABLE(1f, 2f, 4f) /* start, commit, abort */
+		RSEQ_ASM_DEFINE_TABLE(9, 1f, 2f, 4f) /* start, commit, abort */
 		RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail])
 #ifdef RSEQ_COMPARE_TWICE
 		RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1])
@@ -664,7 +668,7 @@ int rseq_cmpeqv_trymemcpy_storev_release(intptr_t *v, intptr_t expect,
 
 	rseq_workaround_gcc_asm_size_guess();
 	__asm__ __volatile__ goto (
-		RSEQ_ASM_DEFINE_TABLE(1f, 2f, 4f) /* start, commit, abort */
+		RSEQ_ASM_DEFINE_TABLE(9, 1f, 2f, 4f) /* start, commit, abort */
 		RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[cmpfail])
 #ifdef RSEQ_COMPARE_TWICE
 		RSEQ_ASM_DEFINE_EXIT_POINT(1f, %l[error1])
tools/testing/selftests/rseq/rseq-ppc.h +14 -8
index 24f95649d71e..9df18487fa9f 100644
@@ -33,8 +33,8 @@ do {									\
 #else /* !RSEQ_SKIP_FASTPATH */
 
 /*
- * The __rseq_table section can be used by debuggers to better handle
- * single-stepping through the restartable critical sections.
+ * The __rseq_cs_ptr_array and __rseq_cs sections can be used by debuggers to
+ * better handle single-stepping through the restartable critical sections.
  */
 
 #ifdef __PPC64__
@@ -46,11 +46,14 @@ do {									\
 
 #define __RSEQ_ASM_DEFINE_TABLE(label, version, flags,				\
 			start_ip, post_commit_offset, abort_ip)			\
-		".pushsection __rseq_table, \"aw\"\n\t"				\
+		".pushsection __rseq_cs, \"aw\"\n\t"				\
 		".balign 32\n\t"						\
 		__rseq_str(label) ":\n\t"					\
 		".long " __rseq_str(version) ", " __rseq_str(flags) "\n\t"	\
 		".quad " __rseq_str(start_ip) ", " __rseq_str(post_commit_offset) ", " __rseq_str(abort_ip) "\n\t" \
+		".popsection\n\t"						\
+		".pushsection __rseq_cs_ptr_array, \"aw\"\n\t"			\
+		".quad " __rseq_str(label) "b\n\t"				\
 		".popsection\n\t"
 
 #define RSEQ_ASM_STORE_RSEQ_CS(label, cs_label, rseq_cs)			\
@@ -67,8 +70,8 @@ do {									\
  * Exit points of a rseq critical section consist of all instructions outside
  * of the critical section where a critical section can either branch to or
  * reach through the normal course of its execution. The abort IP and the
- * post-commit IP are already part of the __rseq_table section and should not
- * be explicitly defined as additional exit points. Knowing all exit points is
+ * post-commit IP are already part of the __rseq_cs section and should not be
+ * explicitly defined as additional exit points. Knowing all exit points is
  * useful to assist debuggers stepping over the critical section.
  */
 #define RSEQ_ASM_DEFINE_EXIT_POINT(start_ip, exit_ip)			\
@@ -85,20 +88,23 @@ do {									\
 
 #define __RSEQ_ASM_DEFINE_TABLE(label, version, flags,				\
 			start_ip, post_commit_offset, abort_ip)			\
-		".pushsection __rseq_table, \"aw\"\n\t"				\
+		".pushsection __rseq_cs, \"aw\"\n\t"				\
 		".balign 32\n\t"						\
 		__rseq_str(label) ":\n\t"					\
 		".long " __rseq_str(version) ", " __rseq_str(flags) "\n\t"	\
 		/* 32-bit only supported on BE */				\
 		".long 0x0, " __rseq_str(start_ip) ", 0x0, " __rseq_str(post_commit_offset) ", 0x0, " __rseq_str(abort_ip) "\n\t" \
+		".popsection\n\t"					\
+		".pushsection __rseq_cs_ptr_array, \"aw\"\n\t"		\
+		".long 0x0, " __rseq_str(label) "b\n\t"			\
 		".popsection\n\t"
 
 /*
  * Exit points of a rseq critical section consist of all instructions outside
  * of the critical section where a critical section can either branch to or
  * reach through the normal course of its execution. The abort IP and the
- * post-commit IP are already part of the __rseq_table section and should not
- * be explicitly defined as additional exit points. Knowing all exit points is
+ * post-commit IP are already part of the __rseq_cs section and should not be
+ * explicitly defined as additional exit points. Knowing all exit points is
  * useful to assist debuggers stepping over the critical section.
  */
 #define RSEQ_ASM_DEFINE_EXIT_POINT(start_ip, exit_ip)				\
tools/testing/selftests/rseq/rseq-s390.h +12 -6
index b8b5b6f900af..fbb97815d71c 100644
@@ -37,19 +37,22 @@ do {									\
 
 #define __RSEQ_ASM_DEFINE_TABLE(label, version, flags,			\
 				start_ip, post_commit_offset, abort_ip)	\
-		".pushsection __rseq_table, \"aw\"\n\t"			\
+		".pushsection __rseq_cs, \"aw\"\n\t"			\
 		".balign 32\n\t"					\
 		__rseq_str(label) ":\n\t"				\
 		".long " __rseq_str(version) ", " __rseq_str(flags) "\n\t" \
 		".quad " __rseq_str(start_ip) ", " __rseq_str(post_commit_offset) ", " __rseq_str(abort_ip) "\n\t" \
+		".popsection\n\t"					\
+		".pushsection __rseq_cs_ptr_array, \"aw\"\n\t"		\
+		".quad " __rseq_str(label) "b\n\t"			\
 		".popsection\n\t"
 
 /*
  * Exit points of a rseq critical section consist of all instructions outside
  * of the critical section where a critical section can either branch to or
  * reach through the normal course of its execution. The abort IP and the
- * post-commit IP are already part of the __rseq_table section and should not
- * be explicitly defined as additional exit points. Knowing all exit points is
+ * post-commit IP are already part of the __rseq_cs section and should not be
+ * explicitly defined as additional exit points. Knowing all exit points is
  * useful to assist debuggers stepping over the critical section.
  */
 #define RSEQ_ASM_DEFINE_EXIT_POINT(start_ip, exit_ip)			\
@@ -61,19 +64,22 @@ do {									\
 
 #define __RSEQ_ASM_DEFINE_TABLE(label, version, flags,			\
 				start_ip, post_commit_offset, abort_ip)	\
-		".pushsection __rseq_table, \"aw\"\n\t"			\
+		".pushsection __rseq_cs, \"aw\"\n\t"			\
 		".balign 32\n\t"					\
 		__rseq_str(label) ":\n\t"				\
 		".long " __rseq_str(version) ", " __rseq_str(flags) "\n\t" \
 		".long 0x0, " __rseq_str(start_ip) ", 0x0, " __rseq_str(post_commit_offset) ", 0x0, " __rseq_str(abort_ip) "\n\t" \
+		".popsection\n\t"					\
+		".pushsection __rseq_cs_ptr_array, \"aw\"\n\t"		\
+		".long 0x0, " __rseq_str(label) "b\n\t"			\
 		".popsection\n\t"
 
 /*
  * Exit points of a rseq critical section consist of all instructions outside
  * of the critical section where a critical section can either branch to or
  * reach through the normal course of its execution. The abort IP and the
- * post-commit IP are already part of the __rseq_table section and should not
- * be explicitly defined as additional exit points. Knowing all exit points is
+ * post-commit IP are already part of the __rseq_cs section and should not be
+ * explicitly defined as additional exit points. Knowing all exit points is
  * useful to assist debuggers stepping over the critical section.
  */
 #define RSEQ_ASM_DEFINE_EXIT_POINT(start_ip, exit_ip)			\
tools/testing/selftests/rseq/rseq-x86.h +13 -6
index 0668608d3674..03095236f6fa 100644
@@ -47,13 +47,17 @@ do {									\
 
 #define __RSEQ_ASM_DEFINE_TABLE(label, version, flags,			\
 				start_ip, post_commit_offset, abort_ip)	\
-		".pushsection __rseq_table, \"aw\"\n\t"			\
+		".pushsection __rseq_cs, \"aw\"\n\t"			\
 		".balign 32\n\t"					\
 		__rseq_str(label) ":\n\t"				\
 		".long " __rseq_str(version) ", " __rseq_str(flags) "\n\t" \
 		".quad " __rseq_str(start_ip) ", " __rseq_str(post_commit_offset) ", " __rseq_str(abort_ip) "\n\t" \
+		".popsection\n\t"					\
+		".pushsection __rseq_cs_ptr_array, \"aw\"\n\t"		\
+		".quad " __rseq_str(label) "b\n\t"			\
 		".popsection\n\t"
 
+
 #define RSEQ_ASM_DEFINE_TABLE(label, start_ip, post_commit_ip, abort_ip) \
 	__RSEQ_ASM_DEFINE_TABLE(label, 0x0, 0x0, start_ip,		\
 				(post_commit_ip - start_ip), abort_ip)
@@ -62,8 +66,8 @@ do {									\
  * Exit points of a rseq critical section consist of all instructions outside
  * of the critical section where a critical section can either branch to or
  * reach through the normal course of its execution. The abort IP and the
- * post-commit IP are already part of the __rseq_table section and should not
- * be explicitly defined as additional exit points. Knowing all exit points is
+ * post-commit IP are already part of the __rseq_cs section and should not be
+ * explicitly defined as additional exit points. Knowing all exit points is
  * useful to assist debuggers stepping over the critical section.
  */
 #define RSEQ_ASM_DEFINE_EXIT_POINT(start_ip, exit_ip)			\
@@ -566,11 +570,14 @@ do {									\
  */
 #define __RSEQ_ASM_DEFINE_TABLE(label, version, flags,			\
 				start_ip, post_commit_offset, abort_ip)	\
-		".pushsection __rseq_table, \"aw\"\n\t"			\
+		".pushsection __rseq_cs, \"aw\"\n\t"			\
 		".balign 32\n\t"					\
 		__rseq_str(label) ":\n\t"				\
 		".long " __rseq_str(version) ", " __rseq_str(flags) "\n\t" \
 		".long " __rseq_str(start_ip) ", 0x0, " __rseq_str(post_commit_offset) ", 0x0, " __rseq_str(abort_ip) ", 0x0\n\t" \
+		".popsection\n\t"					\
+		".pushsection __rseq_cs_ptr_array, \"aw\"\n\t"		\
+		".long " __rseq_str(label) "b, 0x0\n\t"			\
 		".popsection\n\t"
 
 #define RSEQ_ASM_DEFINE_TABLE(label, start_ip, post_commit_ip, abort_ip) \
@@ -581,8 +588,8 @@ do {									\
  * Exit points of a rseq critical section consist of all instructions outside
  * of the critical section where a critical section can either branch to or
  * reach through the normal course of its execution. The abort IP and the
- * post-commit IP are already part of the __rseq_table section and should not
- * be explicitly defined as additional exit points. Knowing all exit points is
+ * post-commit IP are already part of the __rseq_cs section and should not be
+ * explicitly defined as additional exit points. Knowing all exit points is
  * useful to assist debuggers stepping over the critical section.
  */
 #define RSEQ_ASM_DEFINE_EXIT_POINT(start_ip, exit_ip)			\
Signature · 1 line
2.11.0
author maintainer
#4 lore raw

PATCH 4/12 rseq/selftests: Use __rseq_handled symbol to coexist with glibc

In reply to Mathieu Desnoyers

In order to integrate rseq into user-space applications, expose a __rseq_handled symbol so many rseq users can be linked into the same application (e.g. librseq and glibc).

The __rseq_refcount TLS variable is static to the librseq library. It ensures that rseq syscall registration/unregistration happens only for the most early/late caller to rseq_{,un}register_current_thread for each thread, thus ensuring that rseq is registered across the lifetime of all rseq users for a given thread.

CC Shuah Khan <[email protected]>
CC Carlos O'Donell <[email protected]>
CC Florian Weimer <[email protected]>
CC Joseph Myers <[email protected]>
CC Szabolcs Nagy <[email protected]>
CC Thomas Gleixner <[email protected]>
CC Ben Maurer <[email protected]>
CC Peter Zijlstra <[email protected]>
CC "Paul E. McKenney" <[email protected]>
CC Boqun Feng <[email protected]>
CC Will Deacon <[email protected]>
CC Dave Watson <[email protected]>
CC Paul Turner <[email protected]>
---
 tools/testing/selftests/rseq/rseq.c | 55 +++++++++++++++++++++++++++++++------
 tools/testing/selftests/rseq/rseq.h |  1 +
 2 files changed, 48 insertions(+), 8 deletions(-)
tools/testing/selftests/rseq/rseq.c +47 -8
index 4847e97ed049..7159eb777fd3 100644
@@ -25,18 +25,27 @@
 #include <syscall.h>
 #include <assert.h>
 #include <signal.h>
+#include <limits.h>
 
 #include "rseq.h"
 
 #define ARRAY_SIZE(arr)	(sizeof(arr) / sizeof((arr)[0]))
 
-__attribute__((tls_model("initial-exec"))) __thread
-volatile struct rseq __rseq_abi = {
+__thread volatile struct rseq __rseq_abi = {
 	.cpu_id = RSEQ_CPU_ID_UNINITIALIZED,
 };
 
-static __attribute__((tls_model("initial-exec"))) __thread
-volatile int refcount;
+/*
+ * Shared with other libraries. This library may take rseq ownership if it is
+ * still 0 when executing the library constructor. Set to 1 by library
+ * constructor when handling rseq. Set to 0 in destructor if handling rseq.
+ */
+int __rseq_handled;
+
+/* Whether this library have ownership of rseq registration. */
+static int rseq_ownership;
+
+static __thread volatile uint32_t __rseq_refcount;
 
 static void signal_off_save(sigset_t *oldset)
 {
@@ -69,8 +78,14 @@ int rseq_register_current_thread(void)
 	int rc, ret = 0;
 	sigset_t oldset;
 
+	if (!rseq_ownership)
+		return 0;
 	signal_off_save(&oldset);
-	if (refcount++)
+	if (__rseq_refcount == UINT_MAX) {
+		ret = -1;
+		goto end;
+	}
+	if (__rseq_refcount++)
 		goto end;
 	rc = sys_rseq(&__rseq_abi, sizeof(struct rseq), 0, RSEQ_SIG);
 	if (!rc) {
@@ -78,9 +93,9 @@ int rseq_register_current_thread(void)
 		goto end;
 	}
 	if (errno != EBUSY)
-		__rseq_abi.cpu_id = -2;
+		__rseq_abi.cpu_id = RSEQ_CPU_ID_REGISTRATION_FAILED;
 	ret = -1;
-	refcount--;
+	__rseq_refcount--;
 end:
 	signal_restore(oldset);
 	return ret;
@@ -91,13 +106,20 @@ int rseq_unregister_current_thread(void)
 	int rc, ret = 0;
 	sigset_t oldset;
 
+	if (!rseq_ownership)
+		return 0;
 	signal_off_save(&oldset);
-	if (--refcount)
+	if (!__rseq_refcount) {
+		ret = -1;
+		goto end;
+	}
+	if (--__rseq_refcount)
 		goto end;
 	rc = sys_rseq(&__rseq_abi, sizeof(struct rseq),
 		      RSEQ_FLAG_UNREGISTER, RSEQ_SIG);
 	if (!rc)
 		goto end;
+	__rseq_refcount = 1;
 	ret = -1;
 end:
 	signal_restore(oldset);
@@ -115,3 +137,20 @@ int32_t rseq_fallback_current_cpu(void)
 	}
 	return cpu;
 }
+
+void __attribute__((constructor)) rseq_init(void)
+{
+	/* Check whether rseq is handled by another library. */
+	if (__rseq_handled)
+		return;
+	__rseq_handled = 1;
+	rseq_ownership = 1;
+}
+
+void __attribute__((destructor)) rseq_fini(void)
+{
+	if (!rseq_ownership)
+		return;
+	__rseq_handled = 0;
+	rseq_ownership = 0;
+}
tools/testing/selftests/rseq/rseq.h +1 -0
index c72eb70f9b52..26348e2c44f3 100644
@@ -45,6 +45,7 @@
 #endif
 
 extern __thread volatile struct rseq __rseq_abi;
+extern int __rseq_handled;
 
 #define rseq_likely(x)		__builtin_expect(!!(x), 1)
 #define rseq_unlikely(x)	__builtin_expect(!!(x), 0)
Signature · 1 line
2.11.0
author maintainer
#5 lore raw

PATCH 5/12 rseq/selftests: s390: use jg instruction for jumps outside of the asm

In reply to Mathieu Desnoyers

The branch target range of the "j" instruction is 64K, which is not enough for the general case.

CC Thomas Gleixner <[email protected]>
CC Joel Fernandes <[email protected]>
CC Peter Zijlstra <[email protected]>
CC Catalin Marinas <[email protected]>
CC Dave Watson <[email protected]>
CC Will Deacon <[email protected]>
CC Shuah Khan <[email protected]>
CC Andi Kleen <[email protected]>
CC "H . Peter Anvin" <[email protected]>
CC Chris Lameter <[email protected]>
CC Russell King <[email protected]>
CC Michael Kerrisk <[email protected]>
CC "Paul E . McKenney" <[email protected]>
CC Paul Turner <[email protected]>
CC Boqun Feng <[email protected]>
CC Josh Triplett <[email protected]>
CC Steven Rostedt <[email protected]>
CC Ben Maurer <[email protected]>
CC Andy Lutomirski <[email protected]>
CC Andrew Morton <[email protected]>
CC Linus Torvalds <[email protected]>
---
 tools/testing/selftests/rseq/rseq-s390.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
tools/testing/selftests/rseq/rseq-s390.h +2 -2
index fbb97815d71c..7c4f3a70b6c7 100644
@@ -117,14 +117,14 @@ do {									\
 		".long " __rseq_str(RSEQ_SIG) "\n\t"			\
 		__rseq_str(label) ":\n\t"				\
 		teardown						\
-		"j %l[" __rseq_str(abort_label) "]\n\t"			\
+		"jg %l[" __rseq_str(abort_label) "]\n\t"		\
 		".popsection\n\t"
 
 #define RSEQ_ASM_DEFINE_CMPFAIL(label, teardown, cmpfail_label)		\
 		".pushsection __rseq_failure, \"ax\"\n\t"		\
 		__rseq_str(label) ":\n\t"				\
 		teardown						\
-		"j %l[" __rseq_str(cmpfail_label) "]\n\t"		\
+		"jg %l[" __rseq_str(cmpfail_label) "]\n\t"		\
 		".popsection\n\t"
 
 static inline __attribute__((always_inline))
Signature · 1 line
2.11.0
author maintainer
#6 lore raw

PATCH 6/12 rseq/selftests: x86: use ud1 instruction as RSEQ_SIG opcode

In reply to Mathieu Desnoyers

Use ud1 as the guard instruction for the restartable sequence abort handler. Its benefit compared to nopl is to trap execution if the program ends up trying to execute it by mistake, which makes debugging easier.

The 4-byte signature per se is unchanged (it is the instruction operand). Only the opcode is changed from nopl to ud1.

CC Peter Zijlstra <[email protected]>
CC Thomas Gleixner <[email protected]>
CC Joel Fernandes <[email protected]>
CC Catalin Marinas <[email protected]>
CC Dave Watson <[email protected]>
CC Will Deacon <[email protected]>
CC Shuah Khan <[email protected]>
CC Andi Kleen <[email protected]>
CC "H . Peter Anvin" <[email protected]>
CC Chris Lameter <[email protected]>
CC Russell King <[email protected]>
CC Michael Kerrisk <[email protected]>
CC "Paul E . McKenney" <[email protected]>
CC Paul Turner <[email protected]>
CC Boqun Feng <[email protected]>
CC Josh Triplett <[email protected]>
CC Steven Rostedt <[email protected]>
CC Ben Maurer <[email protected]>
CC Andy Lutomirski <[email protected]>
CC Andrew Morton <[email protected]>
CC Linus Torvalds <[email protected]>
---
 tools/testing/selftests/rseq/rseq-x86.h | 15 +++++++++++----
 1 file changed, 11 insertions(+), 4 deletions(-)
tools/testing/selftests/rseq/rseq-x86.h +11 -4
index 03095236f6fa..b2da6004fe30 100644
@@ -7,6 +7,13 @@
 
 #include <stdint.h>
 
+/*
+ * RSEQ_SIG is used with the following reserved undefined instructions, which
+ * trap in user-space:
+ *
+ * x86-32:    0f b9 3d 53 30 05 53      ud1    0x53053053,%edi
+ * x86-64:    0f b9 3d 53 30 05 53      ud1    0x53053053(%rip),%edi
+ */
 #define RSEQ_SIG	0x53053053
 
 /*
@@ -88,8 +95,8 @@ do {									\
 
 #define RSEQ_ASM_DEFINE_ABORT(label, teardown, abort_label)		\
 		".pushsection __rseq_failure, \"ax\"\n\t"		\
-		/* Disassembler-friendly signature: nopl <sig>(%rip). */\
-		".byte 0x0f, 0x1f, 0x05\n\t"				\
+		/* Disassembler-friendly signature: ud1 <sig>(%rip),%edi. */ \
+		".byte 0x0f, 0xb9, 0x3d\n\t"				\
 		".long " __rseq_str(RSEQ_SIG) "\n\t"			\
 		__rseq_str(label) ":\n\t"				\
 		teardown						\
@@ -609,8 +616,8 @@ do {									\
 
 #define RSEQ_ASM_DEFINE_ABORT(label, teardown, abort_label)		\
 		".pushsection __rseq_failure, \"ax\"\n\t"		\
-		/* Disassembler-friendly signature: nopl <sig>. */	\
-		".byte 0x0f, 0x1f, 0x05\n\t"				\
+		/* Disassembler-friendly signature: ud1 <sig>,%edi. */	\
+		".byte 0x0f, 0xb9, 0x3d\n\t"				\
 		".long " __rseq_str(RSEQ_SIG) "\n\t"			\
 		__rseq_str(label) ":\n\t"				\
 		teardown						\
Signature · 1 line
2.11.0
author maintainer
#7 lore raw

PATCH 7/12 rseq/selftests: s390: use trap4 for RSEQ_SIG

In reply to Mathieu Desnoyers

From: Martin Schwidefsky <[email protected]>

Use trap4 as the guard instruction for the restartable sequence abort handler.

---
 tools/testing/selftests/rseq/rseq-s390.h | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)
tools/testing/selftests/rseq/rseq-s390.h +8 -1
index 7c4f3a70b6c7..1d05c5187ae6 100644
@@ -1,6 +1,13 @@
 /* SPDX-License-Identifier: LGPL-2.1 OR MIT */
 
-#define RSEQ_SIG	0x53053053
+/*
+ * RSEQ_SIG uses the trap4 instruction. As Linux does not make use of the
+ * access-register mode nor the linkage stack this instruction will always
+ * cause a special-operation exception (the trap-enabled bit in the DUCT
+ * is and will stay 0). The instruction pattern is
+ *	b2 ff 0f ff	trap4   4095(%r0)
+ */
+#define RSEQ_SIG	0xB2FF0FFF
 
 #define rseq_smp_mb()	__asm__ __volatile__ ("bcr 15,0" ::: "memory")
 #define rseq_smp_rmb()	rseq_smp_mb()
Signature · 1 line
2.11.0
maintainer
#8 lore raw

Re: [PATCH for 5.2 07/12] rseq/selftests: s390: use trap4 for RSEQ_SIG

Hi Mathieu,

Quoted 21 lines · Mathieu Desnoyers

From: Martin Schwidefsky <[email protected]>

Use trap4 as the guard instruction for the restartable sequence abort handler.

---
  tools/testing/selftests/rseq/rseq-s390.h | 9 ++++++++-
  1 file changed, 8 insertions(+), 1 deletion(-)
tools/testing/selftests/rseq/rseq-s390.h +1 -1
index 7c4f3a70b6c7..1d05c5187ae6 100644
@@ -1,6 +1,13 @@
  /* SPDX-License-Identifier: LGPL-2.1 OR MIT */
 
-#define RSEQ_SIG	0x53053053
+/*

tools/testing/selftests/rseq/rseq-s390.h line 13

Quoted 10 lines
tools/testing/selftests/rseq/rseq-s390.h +7 -0
+ * RSEQ_SIG uses the trap4 instruction. As Linux does not make use of the
+ * access-register mode nor the linkage stack this instruction will always
+ * cause a special-operation exception (the trap-enabled bit in the DUCT
+ * is and will stay 0). The instruction pattern is
+ *	b2 ff 0f ff	trap4   4095(%r0)
+ */
+#define RSEQ_SIG	0xB2FF0FFF
 
  #define rseq_smp_mb()	__asm__ __volatile__ ("bcr 15,0" ::: "memory")
  #define rseq_smp_rmb()	rseq_smp_mb()

I generated my pull request for Linus and did a sanity check and ran into merge conflict on this patch. Looks like this is already in Linus's tree.

Can you confirm!

I have to drop this patch and regenerate my pull request. Can you confirm!

thanks,
-- Shuah

author maintainer
#9 lore raw

Re: [PATCH for 5.2 07/12] rseq/selftests: s390: use trap4 for RSEQ_SIG

Quoted 40 lines · ----- On May 16, 2019, at 4:39 PM, shuah [email protected] wrote:

Hi Mathieu,

Quoted 22 lines · Mathieu Desnoyers

From: Martin Schwidefsky <[email protected]>

Use trap4 as the guard instruction for the restartable sequence abort handler.

---
  tools/testing/selftests/rseq/rseq-s390.h | 9 ++++++++-
  1 file changed, 8 insertions(+), 1 deletion(-)
tools/testing/selftests/rseq/rseq-s390.h +1 -1
index 7c4f3a70b6c7..1d05c5187ae6 100644
@@ -1,6 +1,13 @@
  /* SPDX-License-Identifier: LGPL-2.1 OR MIT */
 
-#define RSEQ_SIG	0x53053053
+/*

tools/testing/selftests/rseq/rseq-s390.h line 13

Quoted 10 lines
tools/testing/selftests/rseq/rseq-s390.h +7 -0
+ * RSEQ_SIG uses the trap4 instruction. As Linux does not make use of the
+ * access-register mode nor the linkage stack this instruction will always
+ * cause a special-operation exception (the trap-enabled bit in the DUCT
+ * is and will stay 0). The instruction pattern is
+ *	b2 ff 0f ff	trap4   4095(%r0)
+ */
+#define RSEQ_SIG	0xB2FF0FFF
 
  #define rseq_smp_mb()	__asm__ __volatile__ ("bcr 15,0" ::: "memory")
  #define rseq_smp_rmb()	rseq_smp_mb()

I generated my pull request for Linus and did a sanity check and ran into merge conflict on this patch. Looks like this is already in Linus's tree.

Quoted 4 lines

Can you confirm!

I have to drop this patch and regenerate my pull request. Can you confirm!

I confirm, it went through the s390 maintainer tree, here is the upstream commit already in Linus' master:

commit e24e4712efad737ca09ff299276737331cd021d9

Author: Martin Schwidefsky <[email protected]>
Date:   Wed Apr 10 12:28:41 2019 +0200

s390/rseq: use trap4 for RSEQ_SIG

Use trap4 as the guard instruction for the restartable sequence abort handler.

So you can either drop it from your end of the pull request, or Linus will deal with the merge, as you prefer.

Thanks!

Mathieu

Quoted 3 lines

thanks,
-- Shuah

Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
author maintainer
#10 lore raw

PATCH 8/12 rseq/selftests: arm: use udf instruction for RSEQ_SIG

In reply to Mathieu Desnoyers

Use udf as the guard instruction for the restartable sequence abort handler.

Previously, the chosen signature was not a valid instruction, based on the assumption that it could always sit in a literal pool. However, there are compilation environments in which literal pools are not availble, for instance execute-only code. Therefore, we need to choose a signature value that is also a valid instruction.

Handle compiling with -mbig-endian on ARMv6+, which generates binaries with mixed code vs data endianness (little endian code, big endian data).

Else mismatch between code endianness for the generated signatures and data endianness for the RSEQ_SIG parameter passed to the rseq registration will trigger application segmentation faults when the kernel try to abort rseq critical sections.

Prior to ARMv6, -mbig-endian generates big-endian code and data, so endianness should not be reversed in that case.

CC Peter Zijlstra <[email protected]>
CC Thomas Gleixner <[email protected]>
CC Joel Fernandes <[email protected]>
CC Catalin Marinas <[email protected]>
CC Dave Watson <[email protected]>
CC Will Deacon <[email protected]>
CC Shuah Khan <[email protected]>
CC Andi Kleen <[email protected]>
CC "H . Peter Anvin" <[email protected]>
CC Chris Lameter <[email protected]>
CC Russell King <[email protected]>
CC Michael Kerrisk <[email protected]>
CC "Paul E . McKenney" <[email protected]>
CC Paul Turner <[email protected]>
CC Boqun Feng <[email protected]>
CC Josh Triplett <[email protected]>
CC Steven Rostedt <[email protected]>
CC Ben Maurer <[email protected]>
CC Andy Lutomirski <[email protected]>
CC Andrew Morton <[email protected]>
CC Linus Torvalds <[email protected]>
---
 tools/testing/selftests/rseq/rseq-arm.h | 52 +++++++++++++++++++++++++++++++--
 1 file changed, 50 insertions(+), 2 deletions(-)
tools/testing/selftests/rseq/rseq-arm.h +50 -2
index 5f262c54364f..e8ccfc37d685 100644
@@ -5,7 +5,54 @@
  * (C) Copyright 2016-2018 - Mathieu Desnoyers <[email protected]>
  */
 
-#define RSEQ_SIG	0x53053053
+/*
+ * RSEQ_SIG uses the udf A32 instruction with an uncommon immediate operand
+ * value 0x5de3. This traps if user-space reaches this instruction by mistake,
+ * and the uncommon operand ensures the kernel does not move the instruction
+ * pointer to attacker-controlled code on rseq abort.
+ *
+ * The instruction pattern in the A32 instruction set is:
+ *
+ * e7f5def3    udf    #24035    ; 0x5de3
+ *
+ * This translates to the following instruction pattern in the T16 instruction
+ * set:
+ *
+ * little endian:
+ * def3        udf    #243      ; 0xf3
+ * e7f5        b.n    <7f5>
+ *
+ * pre-ARMv6 big endian code:
+ * e7f5        b.n    <7f5>
+ * def3        udf    #243      ; 0xf3
+ *
+ * ARMv6+ -mbig-endian generates mixed endianness code vs data: little-endian
+ * code and big-endian data. Ensure the RSEQ_SIG data signature matches code
+ * endianness. Prior to ARMv6, -mbig-endian generates big-endian code and data
+ * (which match), so there is no need to reverse the endianness of the data
+ * representation of the signature. However, the choice between BE32 and BE8
+ * is done by the linker, so we cannot know whether code and data endianness
+ * will be mixed before the linker is invoked.
+ */
+
+#define RSEQ_SIG_CODE	0xe7f5def3
+
+#ifndef __ASSEMBLER__
+
+#define RSEQ_SIG_DATA							\
+	({								\
+		int sig;						\
+		asm volatile (  "b 2f\n\t"				\
+				"1: .inst " __rseq_str(RSEQ_SIG_CODE) "\n\t" \
+				"2:\n\t"				\
+				"ldr %[sig], 1b\n\t"			\
+				: [sig] "=r" (sig));			\
+		sig;							\
+	})
+
+#define RSEQ_SIG	RSEQ_SIG_DATA
+
+#endif
 
 #define rseq_smp_mb()	__asm__ __volatile__ ("dmb" ::: "memory", "cc")
 #define rseq_smp_rmb()	__asm__ __volatile__ ("dmb" ::: "memory", "cc")
@@ -78,7 +125,8 @@ do {									\
 		__rseq_str(table_label) ":\n\t"				\
 		".word " __rseq_str(version) ", " __rseq_str(flags) "\n\t" \
 		".word " __rseq_str(start_ip) ", 0x0, " __rseq_str(post_commit_offset) ", 0x0, " __rseq_str(abort_ip) ", 0x0\n\t" \
-		".word " __rseq_str(RSEQ_SIG) "\n\t"			\
+		".arm\n\t"						\
+		".inst " __rseq_str(RSEQ_SIG_CODE) "\n\t"		\
 		__rseq_str(label) ":\n\t"				\
 		teardown						\
 		"b %l[" __rseq_str(abort_label) "]\n\t"
Signature · 1 line
2.11.0
author maintainer
#11 lore raw

PATCH v2 8/12 rseq/selftests: arm: use udf instruction for RSEQ_SIG

Use udf as the guard instruction for the restartable sequence abort handler.

Previously, the chosen signature was not a valid instruction, based on the assumption that it could always sit in a literal pool. However, there are compilation environments in which literal pools are not available, for instance execute-only code. Therefore, we need to choose a signature value that is also a valid instruction.

Handle compiling with -mbig-endian on ARMv6+, which generates binaries with mixed code vs data endianness (little endian code, big endian data).

Else mismatch between code endianness for the generated signatures and data endianness for the RSEQ_SIG parameter passed to the rseq registration will trigger application segmentation faults when the kernel try to abort rseq critical sections.

Prior to ARMv6, -mbig-endian generates big-endian code and data, so endianness should not be reversed in that case.

CC Peter Zijlstra <[email protected]>
CC Thomas Gleixner <[email protected]>
CC Joel Fernandes <[email protected]>
CC Catalin Marinas <[email protected]>
CC Dave Watson <[email protected]>
CC Will Deacon <[email protected]>
CC Shuah Khan <[email protected]>
CC Andi Kleen <[email protected]>
CC "H . Peter Anvin" <[email protected]>
CC Chris Lameter <[email protected]>
CC Russell King <[email protected]>
CC Michael Kerrisk <[email protected]>
CC "Paul E . McKenney" <[email protected]>
CC Paul Turner <[email protected]>
CC Boqun Feng <[email protected]>
CC Josh Triplett <[email protected]>
CC Steven Rostedt <[email protected]>
CC Ben Maurer <[email protected]>
CC Andy Lutomirski <[email protected]>
CC Andrew Morton <[email protected]>
CC Linus Torvalds <[email protected]>

---
Changes since v1:
- Fix checkpatch error and warning.

---
 tools/testing/selftests/rseq/rseq-arm.h | 52 +++++++++++++++++++++++++++++++--
 1 file changed, 50 insertions(+), 2 deletions(-)
tools/testing/selftests/rseq/rseq-arm.h +50 -2
index 5f262c54364f..84f28f147fb6 100644
@@ -5,7 +5,54 @@
  * (C) Copyright 2016-2018 - Mathieu Desnoyers <[email protected]>
  */
 
-#define RSEQ_SIG	0x53053053
+/*
+ * RSEQ_SIG uses the udf A32 instruction with an uncommon immediate operand
+ * value 0x5de3. This traps if user-space reaches this instruction by mistake,
+ * and the uncommon operand ensures the kernel does not move the instruction
+ * pointer to attacker-controlled code on rseq abort.
+ *
+ * The instruction pattern in the A32 instruction set is:
+ *
+ * e7f5def3    udf    #24035    ; 0x5de3
+ *
+ * This translates to the following instruction pattern in the T16 instruction
+ * set:
+ *
+ * little endian:
+ * def3        udf    #243      ; 0xf3
+ * e7f5        b.n    <7f5>
+ *
+ * pre-ARMv6 big endian code:
+ * e7f5        b.n    <7f5>
+ * def3        udf    #243      ; 0xf3
+ *
+ * ARMv6+ -mbig-endian generates mixed endianness code vs data: little-endian
+ * code and big-endian data. Ensure the RSEQ_SIG data signature matches code
+ * endianness. Prior to ARMv6, -mbig-endian generates big-endian code and data
+ * (which match), so there is no need to reverse the endianness of the data
+ * representation of the signature. However, the choice between BE32 and BE8
+ * is done by the linker, so we cannot know whether code and data endianness
+ * will be mixed before the linker is invoked.
+ */
+
+#define RSEQ_SIG_CODE	0xe7f5def3
+
+#ifndef __ASSEMBLER__
+
+#define RSEQ_SIG_DATA							\
+	({								\
+		int sig;						\
+		asm volatile ("b 2f\n\t"				\
+			      "1: .inst " __rseq_str(RSEQ_SIG_CODE) "\n\t" \
+			      "2:\n\t"					\
+			      "ldr %[sig], 1b\n\t"			\
+			      : [sig] "=r" (sig));			\
+		sig;							\
+	})
+
+#define RSEQ_SIG	RSEQ_SIG_DATA
+
+#endif
 
 #define rseq_smp_mb()	__asm__ __volatile__ ("dmb" ::: "memory", "cc")
 #define rseq_smp_rmb()	__asm__ __volatile__ ("dmb" ::: "memory", "cc")
@@ -78,7 +125,8 @@ do {									\
 		__rseq_str(table_label) ":\n\t"				\
 		".word " __rseq_str(version) ", " __rseq_str(flags) "\n\t" \
 		".word " __rseq_str(start_ip) ", 0x0, " __rseq_str(post_commit_offset) ", 0x0, " __rseq_str(abort_ip) ", 0x0\n\t" \
-		".word " __rseq_str(RSEQ_SIG) "\n\t"			\
+		".arm\n\t"						\
+		".inst " __rseq_str(RSEQ_SIG_CODE) "\n\t"		\
 		__rseq_str(label) ":\n\t"				\
 		teardown						\
 		"b %l[" __rseq_str(abort_label) "]\n\t"
Signature · 1 line
2.11.0
author maintainer
#12 lore raw

Re: [PATCH v2 for 5.2 08/12] rseq/selftests: arm: use udf instruction for RSEQ_SIG

Quoted 13 lines · ----- On May 3, 2019, at 3:38 PM, Mathieu Desnoyers…

Use udf as the guard instruction for the restartable sequence abort handler.

Previously, the chosen signature was not a valid instruction, based on the assumption that it could always sit in a literal pool. However, there are compilation environments in which literal pools are not available, for instance execute-only code. Therefore, we need to choose a signature value that is also a valid instruction.

Handle compiling with -mbig-endian on ARMv6+, which generates binaries with mixed code vs data endianness (little endian code, big endian data).

Quoted 7 lines

Else mismatch between code endianness for the generated signatures and data endianness for the RSEQ_SIG parameter passed to the rseq registration will trigger application segmentation faults when the kernel try to abort rseq critical sections.

Prior to ARMv6, -mbig-endian generates big-endian code and data, so endianness should not be reversed in that case.

And of course it cannot be that easy. This breaks when building in thumb mode (-mthumb). Output from librseq arm32 build [1] (code similar to what is found in the rseq selftests):

CC rseq.lo
/tmp/ccu6Jw1b.s: Assembler messages:
/tmp/ccu6Jw1b.s:297: Error: cannot determine Thumb instruction size. Use .inst.n/.inst.w instead /tmp/ccu6Jw1b.s:490: Error: cannot determine Thumb instruction size. Use .inst.n/.inst.w instead Makefile:460: recipe for target 'rseq.lo' failed

This appears to be caused by a missing .arm directive in RSEQ_SIG_DATA. Fixing with:

-               asm volatile ("b 2f\n\t"                                \
+               asm volatile (".arm\n\t"                                \
+                             "b 2f\n\t"                                \

gets the build to go further, but breaks at:

CC basic_percpu_ops_test.o
/tmp/ccpHOMHZ.s: Assembler messages:
/tmp/ccpHOMHZ.s:148: Error: misaligned branch destination /tmp/ccpHOMHZ.s:956: Error: misaligned branch destination Makefile:378: recipe for target 'basic_percpu_ops_test.o' failed

I suspect it's caused by the change from:

- ".word " __rseq_str(RSEQ_SIG) "\n\t" \

to

+               ".arm\n\t"                                              \
+               ".inst " __rseq_str(RSEQ_SIG_CODE) "\n\t"               \

which changes the mode from thumb to arm for the rest of the inline asm within __RSEQ_ASM_DEFINE_ABORT. Better yet, there appears to be no way to save the arm/thumb state and restore it afterwards.

I'm really starting to wonder if we should go our of our way to try to get this signature to be a valid instruction on arm32. Perhaps we should consider going back to use ".word" on arm32 so it ensures it uses data endianness (which matches the parameter received by the sys_rseq system call), let objdump and friends print it as a literal pool (which it is), and just choose an instruction which has little chances to appear for the cases we care about between ARM32 BE, LE and THUMB. Perhaps a 32-bit palindrome ? Bonus points if this is a trap instruction in common configurations for odd-cases-debugging purposes.

Thoughts ?

Thanks,

Mathieu

[1] https://github.com/compudj/librseq

Quoted 107 lines
CC Peter Zijlstra <[email protected]>
CC Thomas Gleixner <[email protected]>
CC Joel Fernandes <[email protected]>
CC Catalin Marinas <[email protected]>
CC Dave Watson <[email protected]>
CC Will Deacon <[email protected]>
CC Shuah Khan <[email protected]>
CC Andi Kleen <[email protected]>
CC "H . Peter Anvin" <[email protected]>
CC Chris Lameter <[email protected]>
CC Russell King <[email protected]>
CC Michael Kerrisk <[email protected]>
CC "Paul E . McKenney" <[email protected]>
CC Paul Turner <[email protected]>
CC Boqun Feng <[email protected]>
CC Josh Triplett <[email protected]>
CC Steven Rostedt <[email protected]>
CC Ben Maurer <[email protected]>
CC Andy Lutomirski <[email protected]>
CC Andrew Morton <[email protected]>
CC Linus Torvalds <[email protected]>

---
Changes since v1:
- Fix checkpatch error and warning.

---
tools/testing/selftests/rseq/rseq-arm.h | 52 +++++++++++++++++++++++++++++++--

1 file changed, 50 insertions(+), 2 deletions(-)
tools/testing/selftests/rseq/rseq-arm.h +50 -2
index 5f262c54364f..84f28f147fb6 100644
@@ -5,7 +5,54 @@
 * (C) Copyright 2016-2018 - Mathieu Desnoyers <[email protected]>
 */
 
-#define RSEQ_SIG	0x53053053
+/*
+ * RSEQ_SIG uses the udf A32 instruction with an uncommon immediate operand
+ * value 0x5de3. This traps if user-space reaches this instruction by mistake,
+ * and the uncommon operand ensures the kernel does not move the instruction
+ * pointer to attacker-controlled code on rseq abort.
+ *
+ * The instruction pattern in the A32 instruction set is:
+ *
+ * e7f5def3    udf    #24035    ; 0x5de3
+ *
+ * This translates to the following instruction pattern in the T16 instruction
+ * set:
+ *
+ * little endian:
+ * def3        udf    #243      ; 0xf3
+ * e7f5        b.n    <7f5>
+ *
+ * pre-ARMv6 big endian code:
+ * e7f5        b.n    <7f5>
+ * def3        udf    #243      ; 0xf3
+ *
+ * ARMv6+ -mbig-endian generates mixed endianness code vs data: little-endian
+ * code and big-endian data. Ensure the RSEQ_SIG data signature matches code
+ * endianness. Prior to ARMv6, -mbig-endian generates big-endian code and data
+ * (which match), so there is no need to reverse the endianness of the data
+ * representation of the signature. However, the choice between BE32 and BE8
+ * is done by the linker, so we cannot know whether code and data endianness
+ * will be mixed before the linker is invoked.
+ */
+
+#define RSEQ_SIG_CODE	0xe7f5def3
+
+#ifndef __ASSEMBLER__
+
+#define RSEQ_SIG_DATA							\
+	({								\
+		int sig;						\
+		asm volatile ("b 2f\n\t"				\
+			      "1: .inst " __rseq_str(RSEQ_SIG_CODE) "\n\t" \
+			      "2:\n\t"					\
+			      "ldr %[sig], 1b\n\t"			\
+			      : [sig] "=r" (sig));			\
+		sig;							\
+	})
+
+#define RSEQ_SIG	RSEQ_SIG_DATA
+
+#endif
 
 #define rseq_smp_mb()	__asm__ __volatile__ ("dmb" ::: "memory", "cc")
 #define rseq_smp_rmb()	__asm__ __volatile__ ("dmb" ::: "memory", "cc")
@@ -78,7 +125,8 @@ do {									\
 		__rseq_str(table_label) ":\n\t"				\
 		".word " __rseq_str(version) ", " __rseq_str(flags) "\n\t" \
 		".word " __rseq_str(start_ip) ", 0x0, " __rseq_str(post_commit_offset) ", 0x0, "
 		__rseq_str(abort_ip) ", 0x0\n\t" \
-		".word " __rseq_str(RSEQ_SIG) "\n\t"			\
+		".arm\n\t"						\
+		".inst " __rseq_str(RSEQ_SIG_CODE) "\n\t"		\
 		__rseq_str(label) ":\n\t"				\
 		teardown						\
		"b %l[" __rseq_str(abort_label) "]\n\t"

--
2.11.0

Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
author maintainer
#13 lore raw

Re: [PATCH v2 for 5.2 08/12] rseq/selftests: arm: use udf instruction for RSEQ_SIG

Quoted 64 lines · ----- On Jun 6, 2019, at 8:02 PM, Mathieu Desnoyers…

----- On May 3, 2019, at 3:38 PM, Mathieu Desnoyers

Quoted 13 lines · [email protected] wrote:

Use udf as the guard instruction for the restartable sequence abort handler.

Previously, the chosen signature was not a valid instruction, based on the assumption that it could always sit in a literal pool. However, there are compilation environments in which literal pools are not available, for instance execute-only code. Therefore, we need to choose a signature value that is also a valid instruction.

Handle compiling with -mbig-endian on ARMv6+, which generates binaries with mixed code vs data endianness (little endian code, big endian data).

Quoted 7 lines

Else mismatch between code endianness for the generated signatures and data endianness for the RSEQ_SIG parameter passed to the rseq registration will trigger application segmentation faults when the kernel try to abort rseq critical sections.

Prior to ARMv6, -mbig-endian generates big-endian code and data, so endianness should not be reversed in that case.

And of course it cannot be that easy. This breaks when building in thumb mode (-mthumb). Output from librseq arm32 build [1] (code similar to what is found in the rseq selftests):

CC rseq.lo
/tmp/ccu6Jw1b.s: Assembler messages:
/tmp/ccu6Jw1b.s:297: Error: cannot determine Thumb instruction size. Use .inst.n/.inst.w instead
/tmp/ccu6Jw1b.s:490: Error: cannot determine Thumb instruction size. Use .inst.n/.inst.w instead
Makefile:460: recipe for target 'rseq.lo' failed

This appears to be caused by a missing .arm directive in RSEQ_SIG_DATA. Fixing with:

(quoted diff) +2 -1
-               asm volatile ("b 2f\n\t"                                \
+               asm volatile (".arm\n\t"                                \
+                             "b 2f\n\t"                                \

gets the build to go further, but breaks at:

CC basic_percpu_ops_test.o
/tmp/ccpHOMHZ.s: Assembler messages:
/tmp/ccpHOMHZ.s:148: Error: misaligned branch destination /tmp/ccpHOMHZ.s:956: Error: misaligned branch destination Makefile:378: recipe for target 'basic_percpu_ops_test.o' failed

I suspect it's caused by the change from:

- ".word " __rseq_str(RSEQ_SIG) "\n\t" \

to

(quoted diff) +2 -0
+               ".arm\n\t"                                              \
+               ".inst " __rseq_str(RSEQ_SIG_CODE) "\n\t"               \

which changes the mode from thumb to arm for the rest of the inline asm within __RSEQ_ASM_DEFINE_ABORT. Better yet, there appears to be no way to save the arm/thumb state and restore it afterwards.

Quoted 10 lines

I'm really starting to wonder if we should go our of our way to try to get this signature to be a valid instruction on arm32. Perhaps we should consider going back to use ".word" on arm32 so it ensures it uses data endianness (which matches the parameter received by the sys_rseq system call), let objdump and friends print it as a literal pool (which it is), and just choose an instruction which has little chances to appear for the cases we care about between ARM32 BE, LE and THUMB. Perhaps a 32-bit palindrome ? Bonus points if this is a trap instruction in common configurations for odd-cases-debugging purposes.

So I'm not particularly proud of the result, but I found a rather ugly way to figure out if we are currently in thumb mode within an inline asm, and restore that mode: test the length of a nop instruction with a ".if" asm statement.

Do we want to go for this kind of approach, or should we revert back to a ".word" and accept that the rseq signature before the abort handler will be seen as data rather than an instruction on arm32 ?

Is there a better way to do this ?

Thanks,

Mathieu

include/rseq/rseq-arm.h +13 -0
index 1ce9231..b6c36dd 100644
@@ -43,7 +43,14 @@
        ({                                                              \
                int sig;                                                \
                asm volatile ("b 2f\n\t"                                \
+                             "3:\n\t"                                  \
+                             "nop\n\t"                                 \
+                             "4:\n\t"                                  \
+                             ".arm\n\t"                                \
                              "1: .inst " __rseq_str(RSEQ_SIG_CODE) "\n\t" \
+                             ".if ((4b - 3b) == 2)\n\t"                \
+                             ".thumb\n\t"                              \
+                             ".endif\n\t"                              \
                              "2:\n\t"                                  \
                              "ldr %[sig], 1b\n\t"                      \
                              : [sig] "=r" (sig));                      \
@@ -125,8 +132,14 @@ do {                                                                       \
                __rseq_str(table_label) ":\n\t"                         \
                ".word " __rseq_str(version) ", " __rseq_str(flags) "\n\t" \
                ".word " __rseq_str(start_ip) ", 0x0, " __rseq_str(post_commit_offset) ", 0x0, " __rseq_str(abort_ip) ", 0x0\n\t" \
+               "333:\n\t"                                              \
+               "nop\n\t"                                               \
+               "444:\n\t"                                              \
                ".arm\n\t"                                              \
                ".inst " __rseq_str(RSEQ_SIG_CODE) "\n\t"               \
+               ".if ((444b - 333b) == 2)\n\t"                          \
+               ".thumb\n\t"                                            \
+               ".endif\n\t"                                            \
                __rseq_str(label) ":\n\t"                               \
                teardown                                                \
                "b %l[" __rseq_str(abort_label) "]\n\t"
Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
author maintainer
#14 lore raw

PATCH 9/12 rseq/selftests: aarch64 code signature: handle big-endian environment

In reply to Mathieu Desnoyers

Handle compiling with -mbig-endian on aarch64, which generates binaries with mixed code vs data endianness (little endian code, big endian data).

Else mismatch between code endianness for the generated signatures and data endianness for the RSEQ_SIG parameter passed to the rseq registration will trigger application segmentation faults when the kernel try to abort rseq critical sections.

CC Peter Zijlstra <[email protected]>
CC Thomas Gleixner <[email protected]>
CC Joel Fernandes <[email protected]>
CC Catalin Marinas <[email protected]>
CC Dave Watson <[email protected]>
CC Will Deacon <[email protected]>
CC Shuah Khan <[email protected]>
CC Andi Kleen <[email protected]>
CC "H . Peter Anvin" <[email protected]>
CC Chris Lameter <[email protected]>
CC Russell King <[email protected]>
CC Michael Kerrisk <[email protected]>
CC "Paul E . McKenney" <[email protected]>
CC Paul Turner <[email protected]>
CC Boqun Feng <[email protected]>
CC Josh Triplett <[email protected]>
CC Steven Rostedt <[email protected]>
CC Ben Maurer <[email protected]>
CC Andy Lutomirski <[email protected]>
CC Andrew Morton <[email protected]>
CC Linus Torvalds <[email protected]>
---
 tools/testing/selftests/rseq/rseq-arm64.h | 17 +++++++++++++++--
 1 file changed, 15 insertions(+), 2 deletions(-)
tools/testing/selftests/rseq/rseq-arm64.h +15 -2
index b41a2a48e965..200dae9e4208 100644
@@ -6,7 +6,20 @@
  * (C) Copyright 2018 - Will Deacon <[email protected]>
  */
 
-#define RSEQ_SIG	0xd428bc00	/* BRK #0x45E0 */
+/*
+ * aarch64 -mbig-endian generates mixed endianness code vs data:
+ * little-endian code and big-endian data. Ensure the RSEQ_SIG signature
+ * matches code endianness.
+ */
+#define RSEQ_SIG_CODE	0xd428bc00	/* BRK #0x45E0.  */
+
+#ifdef __AARCH64EB__
+#define RSEQ_SIG_DATA	0x00bc28d4	/* BRK #0x45E0.  */
+#else
+#define RSEQ_SIG_DATA	RSEQ_SIG_CODE
+#endif
+
+#define RSEQ_SIG	RSEQ_SIG_DATA
 
 #define rseq_smp_mb()	__asm__ __volatile__ ("dmb ish" ::: "memory")
 #define rseq_smp_rmb()	__asm__ __volatile__ ("dmb ishld" ::: "memory")
@@ -121,7 +134,7 @@ do {										\
 
 #define RSEQ_ASM_DEFINE_ABORT(label, abort_label)				\
 	"	b	222f\n"							\
-	"	.inst 	"	__rseq_str(RSEQ_SIG) "\n"			\
+	"	.inst 	"	__rseq_str(RSEQ_SIG_CODE) "\n"			\
 	__rseq_str(label) ":\n"							\
 	"	b	%l[" __rseq_str(abort_label) "]\n"			\
 	"222:\n"
Signature · 1 line
2.11.0
author maintainer
#15 lore raw

PATCH 10/12 rseq/selftests: powerpc code signature: generate valid instructions

In reply to Mathieu Desnoyers

Use "twui" as the guard instruction for the restartable sequence abort handler.

CC Benjamin Herrenschmidt <[email protected]>
CC Paul Mackerras <[email protected]>
CC Michael Ellerman <[email protected]>
CC Boqun Feng <[email protected]>
CC Peter Zijlstra <[email protected]>
CC "Paul E. McKenney" <[email protected]>
CC Alan Modra <[email protected]>
---
 tools/testing/selftests/rseq/rseq-ppc.h | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)
tools/testing/selftests/rseq/rseq-ppc.h +9 -1
index 9df18487fa9f..76be90196fe4 100644
@@ -6,7 +6,15 @@
  * (C) Copyright 2016-2018 - Boqun Feng <[email protected]>
  */
 
-#define RSEQ_SIG	0x53053053
+/*
+ * RSEQ_SIG is used with the following trap instruction:
+ *
+ * powerpc-be:    0f e5 00 0b           twui   r5,11
+ * powerpc64-le:  0b 00 e5 0f           twui   r5,11
+ * powerpc64-be:  0f e5 00 0b           twui   r5,11
+ */
+
+#define RSEQ_SIG	0x0fe5000b
 
 #define rseq_smp_mb()		__asm__ __volatile__ ("sync"	::: "memory", "cc")
 #define rseq_smp_lwsync()	__asm__ __volatile__ ("lwsync"	::: "memory", "cc")
Signature · 1 line
2.11.0
author maintainer
#16 lore raw

PATCH 11/12 rseq/selftests: mips: use break instruction for RSEQ_SIG

In reply to Mathieu Desnoyers

Use break as guard instruction for the restartable sequence abort handler.

Previously, the chosen signature was simply data, based on the assumption that it could always sit in a literal pool. However, some compilation environments favor disabling literal pool. Therefore, ensure the signature is a valid uncommon trap instruction.

CC Paul Burton <[email protected]>
CC James Hogan <[email protected]>
Cc Ralf Baechle <[email protected]>
Cc Mathieu Desnoyers <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Paul E. McKenney <[email protected]>
Cc Boqun Feng <[email protected]>
---
 tools/testing/selftests/rseq/rseq-mips.h | 34 +++++++++++++++++++++++++++++++-
 1 file changed, 33 insertions(+), 1 deletion(-)
tools/testing/selftests/rseq/rseq-mips.h +33 -1
index fe3eabcdcbe5..e989e7c14b09 100644
@@ -7,7 +7,39 @@
  * (C) Copyright 2016-2018 - Mathieu Desnoyers <[email protected]>
  */
 
-#define RSEQ_SIG	0x53053053
+/*
+ * RSEQ_SIG uses the break instruction. The instruction pattern is:
+ *
+ * On MIPS:
+ *	0350000d        break     0x350
+ *
+ * On nanoMIPS:
+ *      00100350        break     0x350
+ *
+ * On microMIPS:
+ *      0000d407        break     0x350
+ *
+ * For nanoMIPS32 and microMIPS, the instruction stream is encoded as 16-bit
+ * halfwords, so the signature halfwords need to be swapped accordingly for
+ * little-endian.
+ */
+#if defined(__nanomips__)
+# ifdef __MIPSEL__
+#  define RSEQ_SIG	0x03500010
+# else
+#  define RSEQ_SIG	0x00100350
+# endif
+#elif defined(__mips_micromips)
+# ifdef __MIPSEL__
+#  define RSEQ_SIG	0xd4070000
+# else
+#  define RSEQ_SIG	0x0000d407
+# endif
+#elif defined(__mips__)
+# define RSEQ_SIG	0x0350000d
+#else
+/* Unknown MIPS architecture. */
+#endif
 
 #define rseq_smp_mb()	__asm__ __volatile__ ("sync" ::: "memory")
 #define rseq_smp_rmb()	rseq_smp_mb()
Signature · 1 line
2.11.0
author maintainer
#17 lore raw

PATCH 12/12 rseq/selftests: add -no-integrated-as for clang

In reply to Mathieu Desnoyers

Ongoing work for asm goto support from clang requires the -no-integrated-as compiler flag.

This compiler flag is present in the toplevel kernel Makefile, but is not replicated for selftests. Add it specifically for the rseq selftest which requires asm goto.

CC Nick Desaulniers <[email protected]>
CC Thomas Gleixner <[email protected]>
CC Joel Fernandes <[email protected]>
CC Peter Zijlstra <[email protected]>
CC Catalin Marinas <[email protected]>
CC Dave Watson <[email protected]>
CC Will Deacon <[email protected]>
CC Shuah Khan <[email protected]>
CC Andi Kleen <[email protected]>
CC "H . Peter Anvin" <[email protected]>
CC Chris Lameter <[email protected]>
CC Russell King <[email protected]>
CC Michael Kerrisk <[email protected]>
CC "Paul E . McKenney" <[email protected]>
CC Paul Turner <[email protected]>
CC Boqun Feng <[email protected]>
CC Josh Triplett <[email protected]>
CC Steven Rostedt <[email protected]>
CC Ben Maurer <[email protected]>
CC Andy Lutomirski <[email protected]>
CC Andrew Morton <[email protected]>
CC Linus Torvalds <[email protected]>
---
 tools/testing/selftests/rseq/Makefile | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)
tools/testing/selftests/rseq/Makefile +7 -1
index c30c52e1d0d2..d6469535630a 100644
@@ -1,5 +1,11 @@
 # SPDX-License-Identifier: GPL-2.0+ OR MIT
-CFLAGS += -O2 -Wall -g -I./ -I../../../../usr/include/ -L./ -Wl,-rpath=./
+
+ifneq ($(shell $(CC) --version 2>&1 | head -n 1 | grep clang),)
+CLANG_FLAGS += -no-integrated-as
+endif
+
+CFLAGS += -O2 -Wall -g -I./ -I../../../../usr/include/ -L./ -Wl,-rpath=./ \
+	  $(CLANG_FLAGS)
 LDLIBS += -lpthread
 
 # Own dependencies because we only want to build against 1st prerequisite, but
Signature · 1 line
2.11.0
#18 lore raw

Re: [PATCH for 5.2 12/12] rseq/selftests: add -no-integrated-as for clang

Quoted 43 lines · Mathieu Desnoyers

Ongoing work for asm goto support from clang requires the -no-integrated-as compiler flag.

This compiler flag is present in the toplevel kernel Makefile, but is not replicated for selftests. Add it specifically for the rseq selftest which requires asm goto.

CC Nick Desaulniers <[email protected]>
CC Thomas Gleixner <[email protected]>
CC Joel Fernandes <[email protected]>
CC Peter Zijlstra <[email protected]>
CC Catalin Marinas <[email protected]>
CC Dave Watson <[email protected]>
CC Will Deacon <[email protected]>
CC Shuah Khan <[email protected]>
CC Andi Kleen <[email protected]>
CC "H . Peter Anvin" <[email protected]>
CC Chris Lameter <[email protected]>
CC Russell King <[email protected]>
CC Michael Kerrisk <[email protected]>
CC "Paul E . McKenney" <[email protected]>
CC Paul Turner <[email protected]>
CC Boqun Feng <[email protected]>
CC Josh Triplett <[email protected]>
CC Steven Rostedt <[email protected]>
CC Ben Maurer <[email protected]>
CC Andy Lutomirski <[email protected]>
CC Andrew Morton <[email protected]>
CC Linus Torvalds <[email protected]>
---
 tools/testing/selftests/rseq/Makefile | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)
tools/testing/selftests/rseq/Makefile +0 -0
index c30c52e1d0d2..d6469535630a 100644

tools/testing/selftests/rseq/Makefile line 8

Quoted 9 lines
tools/testing/selftests/rseq/Makefile +7 -1
@@ -1,5 +1,11 @@
 # SPDX-License-Identifier: GPL-2.0+ OR MIT
-CFLAGS += -O2 -Wall -g -I./ -I../../../../usr/include/ -L./ -Wl,-rpath=./
+
+ifneq ($(shell $(CC) --version 2>&1 | head -n 1 | grep clang),)
+CLANG_FLAGS += -no-integrated-as
+endif
+
+CFLAGS += -O2 -Wall -g -I./ -I../../../../usr/include/ -L./ -Wl,-rpath=./ \
+         $(CLANG_FLAGS)

The top level Makefile exports $(CLANG_FLAGS), which should contain -no-integrated-as. Is that available here? If so, then you can just add $(CLANG_FLAGS), no compiler check needed.

If not, maybe the test for CONFIG_CC_IS_CLANG is cleaner?

Thanks for the patch, and helping test asm goto in Clang!

Signature · 2 lines
Thanks,
~Nick Desaulniers
author maintainer
#19 lore raw

Re: [PATCH for 5.2 12/12] rseq/selftests: add -no-integrated-as for clang

Quoted 51 lines · ----- On Apr 29, 2019, at 1:03 PM, ndesaulniers [email protected] wrote:
Quoted 49 lines · Mathieu Desnoyers

Ongoing work for asm goto support from clang requires the -no-integrated-as compiler flag.

This compiler flag is present in the toplevel kernel Makefile, but is not replicated for selftests. Add it specifically for the rseq selftest which requires asm goto.

CC Nick Desaulniers <[email protected]>
CC Thomas Gleixner <[email protected]>
CC Joel Fernandes <[email protected]>
CC Peter Zijlstra <[email protected]>
CC Catalin Marinas <[email protected]>
CC Dave Watson <[email protected]>
CC Will Deacon <[email protected]>
CC Shuah Khan <[email protected]>
CC Andi Kleen <[email protected]>
CC "H . Peter Anvin" <[email protected]>
CC Chris Lameter <[email protected]>
CC Russell King <[email protected]>
CC Michael Kerrisk <[email protected]>
CC "Paul E . McKenney" <[email protected]>
CC Paul Turner <[email protected]>
CC Boqun Feng <[email protected]>
CC Josh Triplett <[email protected]>
CC Steven Rostedt <[email protected]>
CC Ben Maurer <[email protected]>
CC Andy Lutomirski <[email protected]>
CC Andrew Morton <[email protected]>
CC Linus Torvalds <[email protected]>
---
 tools/testing/selftests/rseq/Makefile | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)
tools/testing/selftests/rseq/Makefile +3 -1
index c30c52e1d0d2..d6469535630a 100644
@@ -1,5 +1,11 @@
 # SPDX-License-Identifier: GPL-2.0+ OR MIT
-CFLAGS += -O2 -Wall -g -I./ -I../../../../usr/include/ -L./ -Wl,-rpath=./
+
+ifneq ($(shell $(CC) --version 2>&1 | head -n 1 | grep clang),)
+CLANG_FLAGS += -no-integrated-as
Quoted 8 lines

tools/testing/selftests/rseq/Makefile line 8

Quoted 4 lines
tools/testing/selftests/rseq/Makefile +4 -0
+endif
+
+CFLAGS += -O2 -Wall -g -I./ -I../../../../usr/include/ -L./ -Wl,-rpath=./ \
+         $(CLANG_FLAGS)

The top level Makefile exports $(CLANG_FLAGS), which should contain -no-integrated-as. Is that available here? If so, then you can just add $(CLANG_FLAGS), no compiler check needed.

AFAIU, the makefiles under tools/testing/selftests all end up including tools/testing/selftests/lib.mk, which states:

"# This mimics the top-level Makefile. We do it explicitly here so that this
# Makefile can operate with or without the kbuild infrastructure."

So I don't think it's using any of the definitions from the toplevel Makefile.

Quoted 1 line

If not, maybe the test for CONFIG_CC_IS_CLANG is cleaner?

A quick test indicates that the toplevel CONFIG_* definitions are unavailable from the kernel selftests makefiles.

Quoted 1 line

Thanks for the patch, and helping test asm goto in Clang!

You're very welcome! Considering that I intend to have rseq widely adopted in user-space, it's only natural to consider that its user-space side needs to be compiled by clang as well.

Thanks,

Mathieu

Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
Reviewed-by
#20 lore raw

Re: [PATCH for 5.2 12/12] rseq/selftests: add -no-integrated-as for clang

Quoted 31 lines · Mathieu Desnoyers
Quoted 13 lines · ----- On Apr 29, 2019, at 1:03 PM, ndesaulniers [email protected] wrote:
Quoted 11 lines · Mathieu Desnoyers
tools/testing/selftests/rseq/Makefile +3 -1
index c30c52e1d0d2..d6469535630a 100644
@@ -1,5 +1,11 @@
 # SPDX-License-Identifier: GPL-2.0+ OR MIT
-CFLAGS += -O2 -Wall -g -I./ -I../../../../usr/include/ -L./ -Wl,-rpath=./
+
+ifneq ($(shell $(CC) --version 2>&1 | head -n 1 | grep clang),)
+CLANG_FLAGS += -no-integrated-as
Quoted 8 lines

tools/testing/selftests/rseq/Makefile line 8

Quoted 4 lines
tools/testing/selftests/rseq/Makefile +4 -0
+endif
+
+CFLAGS += -O2 -Wall -g -I./ -I../../../../usr/include/ -L./ -Wl,-rpath=./ \
+         $(CLANG_FLAGS)

The top level Makefile exports $(CLANG_FLAGS), which should contain -no-integrated-as. Is that available here? If so, then you can just add $(CLANG_FLAGS), no compiler check needed.

AFAIU, the makefiles under tools/testing/selftests all end up including tools/testing/selftests/lib.mk, which states:

"# This mimics the top-level Makefile. We do it explicitly here so that this
# Makefile can operate with or without the kbuild infrastructure."

Quoted 7 lines

So I don't think it's using any of the definitions from the toplevel Makefile.

Quoted 1 line

If not, maybe the test for CONFIG_CC_IS_CLANG is cleaner?

A quick test indicates that the toplevel CONFIG_* definitions are unavailable from the kernel selftests makefiles.

Ok, in that case...

Thanks again!

Signature · 2 lines
Thanks,
~Nick Desaulniers
author maintainer
#21 lore raw
In reply to Mathieu Desnoyers
Quoted 10 lines · ----- On Apr 29, 2019, at 11:27 AM, Mathieu Desnoyers…

Those rseq selftests updates are hereby submitted to Shuah Khan, maintainer of kernel selftests, for the next merge window (5.2).

They change the per-architecture pre-abort signatures to ensure those are valid trap instructions.

The way exit points are presented to debuggers is enhanced, ensuring all exit points are present, so debuggers don't have to disassemble rseq critical section to properly skip over them.

Quoted 7 lines

Discussions with the glibc community is reaching a concensus of exposing a __rseq_handled symbol from glibc to coexist with rseq early adopters. Update the rseq selftest code to expose and use this symbol.

Support for compiling asm goto with clang is added with the "-no-integrated-as" compiler switch, similarly to the toplevel kernel Makefile.

Hi Shuah,

Is there anything else you need before you can pick up those patches ?

Thanks,

Mathieu

Quoted 35 lines

Thanks,

Mathieu

Martin Schwidefsky (1):
rseq/selftests: s390: use trap4 for RSEQ_SIG

Mathieu Desnoyers (11):
rseq/selftests: x86: Work-around bogus gcc-8 optimisation rseq/selftests: Add __rseq_exit_point_array section for debuggers rseq/selftests: Introduce __rseq_cs_ptr_array, rename __rseq_table to
__rseq_cs
rseq/selftests: Use __rseq_handled symbol to coexist with glibc rseq/selftests: s390: use jg instruction for jumps outside of the asm rseq/selftests: x86: use ud1 instruction as RSEQ_SIG opcode rseq/selftests: arm: use udf instruction for RSEQ_SIG rseq/selftests: aarch64 code signature: handle big-endian environment rseq/selftests: powerpc code signature: generate valid instructions rseq/selftests: mips: use break instruction for RSEQ_SIG rseq/selftests: add -no-integrated-as for clang

tools/testing/selftests/rseq/Makefile     |   8 +-
tools/testing/selftests/rseq/rseq-arm.h   | 132 +++++++++++++--
tools/testing/selftests/rseq/rseq-arm64.h |  74 ++++++++-
tools/testing/selftests/rseq/rseq-mips.h  | 115 +++++++++++--
tools/testing/selftests/rseq/rseq-ppc.h   |  90 +++++++++-
tools/testing/selftests/rseq/rseq-s390.h  |  78 ++++++++-
tools/testing/selftests/rseq/rseq-x86.h   | 264 +++++++++++++++++++++---------
tools/testing/selftests/rseq/rseq.c       |  55 ++++++-
tools/testing/selftests/rseq/rseq.h       |   1 +
9 files changed, 688 insertions(+), 129 deletions(-)

--
2.11.0

Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
maintainer
#22 lore raw
Quoted 16 lines · Mathieu Desnoyers
Quoted 14 lines · ----- On Apr 29, 2019, at 11:27 AM, Mathieu Desnoyers…

Those rseq selftests updates are hereby submitted to Shuah Khan, maintainer of kernel selftests, for the next merge window (5.2).

They change the per-architecture pre-abort signatures to ensure those are valid trap instructions.

The way exit points are presented to debuggers is enhanced, ensuring all exit points are present, so debuggers don't have to disassemble rseq critical section to properly skip over them.

Discussions with the glibc community is reaching a concensus of exposing a __rseq_handled symbol from glibc to coexist with rseq early adopters. Update the rseq selftest code to expose and use this symbol.

Quoted 8 lines
Quoted 3 lines

Support for compiling asm goto with clang is added with the "-no-integrated-as" compiler switch, similarly to the toplevel kernel Makefile.

Hi Shuah,

Is there anything else you need before you can pick up those patches ?

I was going to say "no more work needed" and noticed that the series has checkpatch errors and warns as I was running the series through pre-commit tests.

Patches 1,2,3,8 have errors/warns based
on quick look at the log.

ERROR: need consistent spacing around '%' (ctx:WxV) #227: FILE: tools/testing/selftests/rseq/rseq-x86.h:104:
+ RSEQ_ASM_CMP_CPU_ID(cpu_id, RSEQ_CPU_ID_OFFSET(%[rseq_abi]), %l[error1])

Will you be able to fix them and resend?

thanks,
-- Shuah

author maintainer
#23 lore raw
Quoted 40 lines · ----- On May 3, 2019, at 2:53 PM, shuah [email protected] wrote:
Quoted 17 lines · Mathieu Desnoyers

----- On Apr 29, 2019, at 11:27 AM, Mathieu Desnoyers

Quoted 14 lines · [email protected] wrote:

Those rseq selftests updates are hereby submitted to Shuah Khan, maintainer of kernel selftests, for the next merge window (5.2).

They change the per-architecture pre-abort signatures to ensure those are valid trap instructions.

The way exit points are presented to debuggers is enhanced, ensuring all exit points are present, so debuggers don't have to disassemble rseq critical section to properly skip over them.

Discussions with the glibc community is reaching a concensus of exposing a __rseq_handled symbol from glibc to coexist with rseq early adopters. Update the rseq selftest code to expose and use this symbol.

Quoted 8 lines
Quoted 3 lines

Support for compiling asm goto with clang is added with the "-no-integrated-as" compiler switch, similarly to the toplevel kernel Makefile.

Hi Shuah,

Is there anything else you need before you can pick up those patches ?

I was going to say "no more work needed" and noticed that the series has checkpatch errors and warns as I was running the series through pre-commit tests.

Patches 1,2,3,8 have errors/warns based
on quick look at the log.

ERROR: need consistent spacing around '%' (ctx:WxV) #227: FILE: tools/testing/selftests/rseq/rseq-x86.h:104:

(quoted diff) +1 -0
+		RSEQ_ASM_CMP_CPU_ID(cpu_id, RSEQ_CPU_ID_OFFSET(%[rseq_abi]), %l[error1])
Quoted 1 line

Will you be able to fix them and resend?

(CCing the che checkpatch maintainers)

checkpatch appears to be wrong for these errors. I suspect it thinks those are '%' modulo operators (for which the style requires space before/after), but those are actually part of the asm input and goto target operands.

Most warnings are about some lines over 80 cols. However, the areas where this happens is due to following the style of already upstream code which has the final "\" at the end of line sometimes beyond 80 col to accommodate macros that take a bit of horizontal real estate.

For patch 8, the warning about "availble" being a typo is right. The style error about space after "asm (" is right as well. Should I send only this updated patch to you or should I send the whole patchset again ?

Thanks,

Mathieu

Quoted 3 lines

thanks,
-- Shuah

Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
#24 lore raw
Quoted 6 lines · Mathieu Desnoyers
Quoted 5 lines · ----- On May 3, 2019, at 2:53 PM, shuah [email protected] wrote:

ERROR: need consistent spacing around '%' (ctx:WxV) #227: FILE: tools/testing/selftests/rseq/rseq-x86.h:104:

(quoted diff) +1 -0
+		RSEQ_ASM_CMP_CPU_ID(cpu_id, RSEQ_CPU_ID_OFFSET(%[rseq_abi]), %l[error1])

Will you be able to fix them and resend?

[]

Quoted 5 lines

(CCing the che checkpatch maintainers)

checkpatch appears to be wrong for these errors. I suspect it thinks those are '%' modulo operators (for which the style requires space before/after), but those are actually part of the asm input and goto target operands.

checkpatch doesn't really understand asm.
Ignore checkpatch when it's silly.

maintainer
#25 lore raw
In reply to Mathieu Desnoyers
Quoted 50 lines · Mathieu Desnoyers
Quoted 40 lines · ----- On May 3, 2019, at 2:53 PM, shuah [email protected] wrote:
Quoted 17 lines · Mathieu Desnoyers

----- On Apr 29, 2019, at 11:27 AM, Mathieu Desnoyers

Quoted 14 lines · [email protected] wrote:

Those rseq selftests updates are hereby submitted to Shuah Khan, maintainer of kernel selftests, for the next merge window (5.2).

They change the per-architecture pre-abort signatures to ensure those are valid trap instructions.

The way exit points are presented to debuggers is enhanced, ensuring all exit points are present, so debuggers don't have to disassemble rseq critical section to properly skip over them.

Discussions with the glibc community is reaching a concensus of exposing a __rseq_handled symbol from glibc to coexist with rseq early adopters. Update the rseq selftest code to expose and use this symbol.

Quoted 8 lines
Quoted 3 lines

Support for compiling asm goto with clang is added with the "-no-integrated-as" compiler switch, similarly to the toplevel kernel Makefile.

Hi Shuah,

Is there anything else you need before you can pick up those patches ?

I was going to say "no more work needed" and noticed that the series has checkpatch errors and warns as I was running the series through pre-commit tests.

Patches 1,2,3,8 have errors/warns based
on quick look at the log.

ERROR: need consistent spacing around '%' (ctx:WxV) #227: FILE: tools/testing/selftests/rseq/rseq-x86.h:104:

(quoted diff) +1 -0
+		RSEQ_ASM_CMP_CPU_ID(cpu_id, RSEQ_CPU_ID_OFFSET(%[rseq_abi]), %l[error1])
Quoted 1 line

Will you be able to fix them and resend?

(CCing the che checkpatch maintainers)

checkpatch appears to be wrong for these errors. I suspect it thinks those are '%' modulo operators (for which the style requires space before/after), but those are actually part of the asm input and goto target operands.

Quoted 9 lines

Most warnings are about some lines over 80 cols. However, the areas where this happens is due to following the style of already upstream code which has the final "\" at the end of line sometimes beyond 80 col to accommodate macros that take a bit of horizontal real estate.

For patch 8, the warning about "availble" being a typo is right. The style error about space after "asm (" is right as well. Should I send only this updated patch to you or should I send the whole patchset again ?

No need to send all patches. This is good.

thanks,
-- Shuah