Skip to contentSkip to conversation

riscv: Add reliable stack unwinding for livepatch

PATCH In review Wang Han 2 participants 9 messages Aug 4 – Aug 26

Subsystems (7)
Kselftest perf events RISC-V Tracing +3 more subsystems
lore
Share & tools
lore

Wang Han

author
#0 lore raw

v5 cover riscv: Add reliable stack unwinding for livepatch

Hi,

This is v5 of the RISC-V reliable stack unwinding series for livepatch.

The series adds explicit frame-record metadata, preserves the frame pointer through dynamic ftrace, and replaces the open-coded stack walker with a conservative frame-pointer based unwinder. It then enables the RISC-V livepatch capability and adds the syscall-wrapper selftest prefix.

Changes since v4:
* Use sizeof(struct frame_record) for STACKFRAME_RECORD_SIZE, as requested by Shuai Xue.
* Guard IRQ stack accessors and IRQ stack entries with CONFIG_IRQ_STACKS, addressing the link-time issue reported by Shuai Xue and Sashiko.
* Add the corresponding Reviewed-by tag from Shuai Xue to 1/7.
* Rebase onto the latest riscv/for-next; no other functional changes.

The review replies have been sent in the v4 RESEND thread.

Base:
riscv/for-next ff492e321c47 ("Revert \"riscv: hwprobe: Fix stale vDSO data for late-initialized keys at boot\"")

Previous versions:
v4 RESEND: https://lore.kernel.org/r/[email protected]
v3: https://lore.kernel.org/r/[email protected]
v2: https://lore.kernel.org/r/[email protected]
v1: https://lore.kernel.org/r/[email protected]

Validation:
* Built Linux 7.2.0-rc6 with riscv64-unknown-linux-gnu-gcc 15.2.0

    and binutils 2.46.

* Standard livepatch configuration built successfully, including Image, vmlinux, modules, compile_commands.json and GDB scripts.
* CONFIG_IRQ_STACKS=n variant linked successfully with GCC.
* QEMU 8.2.2, 1 CPU: livepatch selftests PASS: 7, SKIP: 1,
FAIL: 0; ftrace function graph 3 passed, 0 failed, 3 unsupported.
* QEMU 8.2.2, 4 CPUs: livepatch selftests PASS: 7, SKIP: 1,
FAIL: 0; ftrace function graph 3 passed, 0 failed, 3 unsupported.
* No BUG, WARNING, Oops, panic, unreliable stack or Call Trace found by the validation log scans. Expected negative livepatch errors are part of the selftests.

Wang Han (7):
riscv: stacktrace: Add frame record metadata
riscv: stacktrace: disable KASAN and KCOV instrumentation for
stacktrace.o
riscv: ftrace: always preserve s0 in dynamic ftrace register frame
riscv: stacktrace: introduce stack-bound tracking helpers
riscv: stacktrace: switch to frame-pointer based unwinder
riscv: Kconfig: enable HAVE_RELIABLE_STACKTRACE and HAVE_LIVEPATCH selftests/livepatch: Add RISC-V syscall wrapper prefix

 arch/riscv/Kconfig                            |   4 +
 arch/riscv/include/asm/ptrace.h               |   9 +
 arch/riscv/include/asm/stacktrace.h           |  67 ++-
 arch/riscv/include/asm/stacktrace/common.h    | 159 +++++
 arch/riscv/include/asm/stacktrace/frame.h     |  53 ++
 arch/riscv/kernel/Makefile                    |   6 +
 arch/riscv/kernel/asm-offsets.c               |   6 +
 arch/riscv/kernel/entry.S                     |  39 +-
 arch/riscv/kernel/ftrace.c                    |   6 +-
 arch/riscv/kernel/head.S                      |  23 +
 arch/riscv/kernel/mcount-dyn.S                |   4 -
 arch/riscv/kernel/perf_callchain.c            |   2 +-
 arch/riscv/kernel/process.c                   |  33 +-
 arch/riscv/kernel/stacktrace.c                | 561 +++++++++++++++---
 .../livepatch/test_modules/test_klp_syscall.c |   2 +
 15 files changed, 868 insertions(+), 106 deletions(-)
 create mode 100644 arch/riscv/include/asm/stacktrace/common.h
 create mode 100644 arch/riscv/include/asm/stacktrace/frame.h

Range-diff against v4:
1: 8cef363cfed7 ! 1: cbb1dc004d0f riscv: stacktrace: Add frame record metadata
@@ Commit message
the secondary CPU path sets up s0 before smp_callin() so idle-task unwinding does not inherit an undefined caller frame;
* copy_thread creates matching final metadata records for new kernel
- and user tasks, and keeps s0 available for the frame-pointer chain;
- * call_on_irq_stack still reserves an aligned stack slot, but links the
- saved {fp, ra} with the raw frame-record size so s0 points at the
- RISC-V frame record rather than past the alignment padding.
+ and user tasks, and keeps s0 available for the frame-pointer chain.

- The call_on_irq_stack adjustment fixes a latent RV32 issue. On RV64,
- sizeof(struct stackframe) is equal to the stack alignment, so the old
- s0 value happened to point just above the saved {fp, ra}. On RV32, the
- raw frame record is 8 bytes while the reserved stack slot is 16-byte
- aligned, so the old s0 value pointed into the padding. Using the raw
- record size makes s0 point above the saved frame record on both RV32
- and RV64 while still reserving the aligned slot.
+ Keep the embedded metadata-record field offsets distinct from the
+ s0-relative STACKFRAME_* offsets used by call_on_irq_stack(), because
+ the latter describe a frame record relative to s0 rather than to the
+ record base.

These changes keep s0 reserved for the frame-pointer chain at task and
- stack-switch boundaries.
+ exception boundaries.

+ Reviewed-by: Shuai Xue <[email protected]>

## arch/riscv/include/asm/ptrace.h ##
@@ arch/riscv/kernel/asm-offsets.c: void asm_offsets(void)

OFFSET(HIBERN_PBE_ADDR, pbe, address);
@@ arch/riscv/kernel/asm-offsets.c: void asm_offsets(void)
- OFFSET(SBI_HART_BOOT_STACK_PTR_OFFSET, sbi_hart_boot_data, stack_ptr);
-
DEFINE(STACKFRAME_SIZE_ON_STACK, ALIGN(sizeof(struct stackframe), STACK_ALIGN));
-+ DEFINE(STACKFRAME_RECORD_SIZE, sizeof(struct stackframe));
- OFFSET(STACKFRAME_FP, stackframe, fp);
- OFFSET(STACKFRAME_RA, stackframe, ra);
+ DEFINE(STACKFRAME_FP, offsetof(struct stackframe, fp) - sizeof(struct stackframe));
+ DEFINE(STACKFRAME_RA, offsetof(struct stackframe, ra) - sizeof(struct stackframe));
++ DEFINE(STACKFRAME_RECORD_SIZE, sizeof(struct frame_record));
++ OFFSET(FRAME_RECORD_FP, frame_record, fp);
++ OFFSET(FRAME_RECORD_RA, frame_record, ra);
#ifdef CONFIG_FUNCTION_TRACER
+ DEFINE(FTRACE_OPS_FUNC, offsetof(struct ftrace_ops, func));
+ #ifdef CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS

## arch/riscv/kernel/entry.S ##
@@
@@ arch/riscv/kernel/entry.S: SYM_CODE_START(handle_exception)
+ * Create a metadata frame record. The unwinder will use this to
+ * identify and unwind exception boundaries.
+ */
-+ REG_S zero, (S_STACKFRAME + STACKFRAME_FP)(sp) /* stackframe.record.fp = 0 */
-+ REG_S zero, (S_STACKFRAME + STACKFRAME_RA)(sp) /* stackframe.record.ra = 0 */
++ REG_S zero, (S_STACKFRAME + FRAME_RECORD_FP)(sp) /* stackframe.record.fp = 0 */
++ REG_S zero, (S_STACKFRAME + FRAME_RECORD_RA)(sp) /* stackframe.record.ra = 0 */
+#ifdef CONFIG_RISCV_M_MODE
+ li t0, SR_MPP
+ and t0, s1, t0
@@ arch/riscv/kernel/entry.S: SYM_CODE_START_LOCAL(handle_kernel_stack_overflow)
+ * pt_regs boundary and the unwinder can resume from the pre-overflow
+ * frame pointer saved in PT_S0.
+ */
-+ REG_S zero, (S_STACKFRAME + STACKFRAME_FP)(sp)
-+ REG_S zero, (S_STACKFRAME + STACKFRAME_RA)(sp)
++ REG_S zero, (S_STACKFRAME + FRAME_RECORD_FP)(sp)
++ REG_S zero, (S_STACKFRAME + FRAME_RECORD_RA)(sp)
+ li t0, FRAME_META_TYPE_PT_REGS
+ REG_S t0, S_STACKFRAME_TYPE(sp)
+ addi s0, sp, S_STACKFRAME + STACKFRAME_RECORD_SIZE
@@ arch/riscv/kernel/entry.S: ASM_NOKPROBE(handle_kernel_stack_overflow)
move a2, sp /* pt_regs */
call ret_from_fork_kernel
j ret_from_exception
-@@ arch/riscv/kernel/entry.S: SYM_FUNC_START(call_on_irq_stack)
- addi sp, sp, -STACKFRAME_SIZE_ON_STACK
- REG_S ra, STACKFRAME_RA(sp)
- REG_S s0, STACKFRAME_FP(sp)
-- addi s0, sp, STACKFRAME_SIZE_ON_STACK
-+ addi s0, sp, STACKFRAME_RECORD_SIZE
-
- /* Switch to the per-CPU shadow call stack */
- scs_save_current
-@@ arch/riscv/kernel/entry.S: SYM_FUNC_START(call_on_irq_stack)
- scs_load_current
-
- /* Switch back to the thread stack and restore ra and s0 */
-- addi sp, s0, -STACKFRAME_SIZE_ON_STACK
-+ addi sp, s0, -STACKFRAME_RECORD_SIZE
- REG_L ra, STACKFRAME_RA(sp)
- REG_L s0, STACKFRAME_FP(sp)
- addi sp, sp, STACKFRAME_SIZE_ON_STACK

## arch/riscv/kernel/head.S ##
@@
@@ arch/riscv/kernel/head.S: SYM_CODE_START(_start_kernel)
+ * fp/s0 points above the metadata record (RISC-V
+ * convention).
+ */
-+ REG_S zero, (S_STACKFRAME + STACKFRAME_FP)(sp)
-+ REG_S zero, (S_STACKFRAME + STACKFRAME_RA)(sp)
++ REG_S zero, (S_STACKFRAME + FRAME_RECORD_FP)(sp)
++ REG_S zero, (S_STACKFRAME + FRAME_RECORD_RA)(sp)
+ li t0, FRAME_META_TYPE_FINAL
+ REG_S t0, S_STACKFRAME_TYPE(sp)
+ addi s0, sp, S_STACKFRAME + STACKFRAME_RECORD_SIZE

2:  237864b66d78 = 2:  c63daa8759a6 riscv: stacktrace: disable KASAN and KCOV instrumentation for stacktrace.o
3:  e6035966a35a = 3:  352269306907 riscv: ftrace: always preserve s0 in dynamic ftrace register frame
4:  d132087ea01e ! 4:  2cc86f5728c1 riscv: stacktrace: introduce stack-bound tracking helpers
    @@ arch/riscv/include/asm/stacktrace.h: extern void notrace walk_stackframe(struct
     +/*
     + * IRQ stack accessors
     + */
    ++#ifdef CONFIG_IRQ_STACKS
     +static inline struct stack_info stackinfo_get_irq(void)
     +{
     +	unsigned long low = (unsigned long)raw_cpu_read(irq_stack_ptr);
    @@ arch/riscv/include/asm/stacktrace.h: extern void notrace walk_stackframe(struct
     +
     +	return stackinfo_on_stack(&info, sp, size);
     +}
    ++#endif /* CONFIG_IRQ_STACKS */
     +
     +/*
     + * Task stack accessors
5:  02adea3ece82 ! 5:  57c5e7d08cf8 riscv: stacktrace: switch to frame-pointer based unwinder
    @@ arch/riscv/kernel/stacktrace.c: unsigned long __get_wchan(struct task_struct *ta
     +	struct task_struct *tsk = task ?: current;
     +	struct stack_info stacks[] = {
     +		stackinfo_get_task(tsk),
    ++#ifdef CONFIG_IRQ_STACKS
     +		STACKINFO_CPU(tsk, irq),
    ++#endif
     +#ifdef CONFIG_VMAP_STACK
     +		STACKINFO_CPU(tsk, overflow),
     +#endif
6:  c7d7dbe7a8a1 = 6:  9b3379fe9320 riscv: Kconfig: enable HAVE_RELIABLE_STACKTRACE and HAVE_LIVEPATCH
7:  ae94a234b34a < -:  ------------ selftests/livepatch: Add RISC-V syscall wrapper prefix
-:  ------------ > 7:  07dddd25aae1 selftests/livepatch: Add RISC-V syscall wrapper prefix

base-commit: ff492e321c477f0c8899ac2b875635a84e928d78

Signature · 1 line
2.43.0
author
#1 lore raw

PATCH v5 1/7 riscv: stacktrace: Add frame record metadata

Reliable frame-pointer unwinding needs an explicit way to identify exception boundaries and the final entry frame. The existing unwinder infers those boundaries from return addresses, which is too loose for a future reliable unwinder.

Add a small metadata frame record to pt_regs and initialize it on exception entry, kernel stack overflow, kernel thread fork, user fork, and early idle task setup. The record uses a zero {fp, ra} sentinel plus a type field so a later unwinder can distinguish a final user-to-kernel boundary from a nested kernel pt_regs boundary.

This follows the arm64 metadata frame-record model, adapted to the RISC-V {fp, ra} frame record convention.

The metadata is established at the RISC-V entry boundaries that need an explicit unwind marker:

* exception entry clears the metadata {fp, ra} pair and uses SPP (or MPP in M-mode) to record whether the pt_regs frame is the final user-to-kernel boundary or a nested kernel boundary;
* the kernel stack overflow path builds a nested pt_regs metadata record on the overflow stack so an unwinder can resume from the pre-overflow s0 saved in PT_S0;
* _start_kernel builds the init task's final metadata record, while the secondary CPU path sets up s0 before smp_callin() so idle-task unwinding does not inherit an undefined caller frame;
* copy_thread creates matching final metadata records for new kernel and user tasks, and keeps s0 available for the frame-pointer chain.

Keep the embedded metadata-record field offsets distinct from the s0-relative STACKFRAME_* offsets used by call_on_irq_stack(), because the latter describe a frame record relative to s0 rather than to the record base.

These changes keep s0 reserved for the frame-pointer chain at task and exception boundaries.

---
 arch/riscv/include/asm/ptrace.h           |  9 ++++
 arch/riscv/include/asm/stacktrace/frame.h | 53 +++++++++++++++++++++++
 arch/riscv/kernel/asm-offsets.c           |  6 +++
 arch/riscv/kernel/entry.S                 | 39 ++++++++++++++++-
 arch/riscv/kernel/head.S                  | 23 ++++++++++
 arch/riscv/kernel/process.c               | 33 +++++++++++++-
 6 files changed, 159 insertions(+), 4 deletions(-)
 create mode 100644 arch/riscv/include/asm/stacktrace/frame.h
arch/riscv/include/asm/ptrace.h +9 -0
index addc8188152f..4b9b0f279214 100644
@@ -8,6 +8,7 @@
 
 #include <uapi/asm/ptrace.h>
 #include <asm/csr.h>
+#include <asm/stacktrace/frame.h>
 #include <linux/compiler.h>
 
 #ifndef __ASSEMBLER__
@@ -53,6 +54,14 @@ struct pt_regs {
 	unsigned long cause;
 	/* a0 value before the syscall */
 	unsigned long orig_a0;
+
+	/*
+	 * This frame record is entirely zeroed on exception entry, allowing the
+	 * unwinder to identify exception boundaries. The type field encodes
+	 * whether the exception was taken from user (FINAL) or kernel (PT_REGS)
+	 * mode.
+	 */
+	struct frame_record_meta stackframe;
 };
 
 #define PTRACE_SYSEMU			0x1f
arch/riscv/include/asm/stacktrace/frame.h new file +53 -0

53 lines not on this page · raw message

arch/riscv/kernel/asm-offsets.c +6 -0
index a75f0cfea1e9..d0eaf2c902fb 100644
@@ -131,6 +131,9 @@ void asm_offsets(void)
 	OFFSET(PT_BADADDR, pt_regs, badaddr);
 	OFFSET(PT_CAUSE, pt_regs, cause);
 
+	DEFINE(S_STACKFRAME,		offsetof(struct pt_regs, stackframe));
+	DEFINE(S_STACKFRAME_TYPE,	offsetof(struct pt_regs, stackframe.type));
+
 	OFFSET(SUSPEND_CONTEXT_REGS, suspend_context, regs);
 
 	OFFSET(HIBERN_PBE_ADDR, pbe, address);
@@ -503,6 +506,9 @@ void asm_offsets(void)
 	DEFINE(STACKFRAME_SIZE_ON_STACK, ALIGN(sizeof(struct stackframe), STACK_ALIGN));
 	DEFINE(STACKFRAME_FP, offsetof(struct stackframe, fp) - sizeof(struct stackframe));
 	DEFINE(STACKFRAME_RA, offsetof(struct stackframe, ra) - sizeof(struct stackframe));
+	DEFINE(STACKFRAME_RECORD_SIZE, sizeof(struct frame_record));
+	OFFSET(FRAME_RECORD_FP, frame_record, fp);
+	OFFSET(FRAME_RECORD_RA, frame_record, ra);
 #ifdef CONFIG_FUNCTION_TRACER
 	DEFINE(FTRACE_OPS_FUNC,		offsetof(struct ftrace_ops, func));
 #ifdef CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS
arch/riscv/kernel/entry.S +37 -2
index d799c4e56f80..588d5bbe74f5 100644
@@ -11,6 +11,7 @@
 #include <asm/asm.h>
 #include <asm/csr.h>
 #include <asm/scs.h>
+#include <asm/stacktrace/frame.h>
 #include <asm/unistd.h>
 #include <asm/page.h>
 #include <asm/thread_info.h>
@@ -198,6 +199,27 @@ SYM_CODE_START(handle_exception)
 	REG_S s4, PT_CAUSE(sp)
 	REG_S s5, PT_TP(sp)
 
+	/*
+	 * Create a metadata frame record. The unwinder will use this to
+	 * identify and unwind exception boundaries.
+	 */
+	REG_S zero, (S_STACKFRAME + FRAME_RECORD_FP)(sp) /* stackframe.record.fp = 0 */
+	REG_S zero, (S_STACKFRAME + FRAME_RECORD_RA)(sp) /* stackframe.record.ra = 0 */
+#ifdef CONFIG_RISCV_M_MODE
+	li t0, SR_MPP
+	and t0, s1, t0
+#else
+	andi t0, s1, SR_SPP
+#endif
+	bnez t0, 1f
+	li t0, FRAME_META_TYPE_FINAL
+	j 2f
+1:
+	li t0, FRAME_META_TYPE_PT_REGS
+2:
+	REG_S t0, S_STACKFRAME_TYPE(sp)
+	addi s0, sp, S_STACKFRAME + STACKFRAME_RECORD_SIZE
+
 	/*
 	 * Set the scratch register to 0, so that if a recursive exception
 	 * occurs, the exception vector knows it came from the kernel
@@ -354,6 +376,19 @@ SYM_CODE_START_LOCAL(handle_kernel_stack_overflow)
 	REG_S s3, PT_BADADDR(sp)
 	REG_S s4, PT_CAUSE(sp)
 	REG_S s5, PT_TP(sp)
+
+	/*
+	 * Create a metadata frame record for the overflow pt_regs. The
+	 * overflow path is entered from kernel context, so this is a nested
+	 * pt_regs boundary and the unwinder can resume from the pre-overflow
+	 * frame pointer saved in PT_S0.
+	 */
+	REG_S zero, (S_STACKFRAME + FRAME_RECORD_FP)(sp)
+	REG_S zero, (S_STACKFRAME + FRAME_RECORD_RA)(sp)
+	li t0, FRAME_META_TYPE_PT_REGS
+	REG_S t0, S_STACKFRAME_TYPE(sp)
+	addi s0, sp, S_STACKFRAME + STACKFRAME_RECORD_SIZE
+
 	move a0, sp
 	tail handle_bad_stack
 SYM_CODE_END(handle_kernel_stack_overflow)
@@ -362,8 +397,8 @@ ASM_NOKPROBE(handle_kernel_stack_overflow)
 
 SYM_CODE_START(ret_from_fork_kernel_asm)
 	call schedule_tail
-	move a0, s1 /* fn_arg */
-	move a1, s0 /* fn */
+	move a0, s3 /* fn_arg */
+	move a1, s2 /* fn */
 	move a2, sp /* pt_regs */
 	call ret_from_fork_kernel
 	j ret_from_exception
arch/riscv/kernel/head.S +23 -0
index f6a8ca49e627..341b2d3facbc 100644
@@ -14,6 +14,7 @@
 #include <asm/hwcap.h>
 #include <asm/image.h>
 #include <asm/scs.h>
+#include <asm/stacktrace/frame.h>
 #include <asm/usercfi.h>
 #include "efi-header.S"
 
@@ -177,6 +178,14 @@ secondary_start_sbi:
 	REG_S a0, (a1)
 1:
 #endif
+
+	/*
+	 * Set up the frame pointer for the secondary idle task so reliable
+	 * stack unwinding terminates at the metadata frame in task_pt_regs().
+	 * Without this, the first frame records can inherit an undefined caller
+	 * fp and unwind past smp_callin() into .Lsecondary_park.
+	 */
+	addi s0, sp, S_STACKFRAME + STACKFRAME_RECORD_SIZE
 	scs_load_current
 	call smp_callin
 #endif /* CONFIG_SMP */
@@ -305,6 +314,20 @@ SYM_CODE_START(_start_kernel)
 	la tp, init_task
 	la sp, init_thread_union + THREAD_SIZE
 	addi sp, sp, -PT_SIZE_ON_STACK
+
+	/*
+	 * Set up a metadata frame record for the init task so that
+	 * the unwinder can identify the outermost frame by its
+	 * {fp, ra} = {0, 0} sentinel at the bottom of pt_regs.
+	 * fp/s0 points above the metadata record (RISC-V
+	 * convention).
+	 */
+	REG_S zero, (S_STACKFRAME + FRAME_RECORD_FP)(sp)
+	REG_S zero, (S_STACKFRAME + FRAME_RECORD_RA)(sp)
+	li t0, FRAME_META_TYPE_FINAL
+	REG_S t0, S_STACKFRAME_TYPE(sp)
+	addi s0, sp, S_STACKFRAME + STACKFRAME_RECORD_SIZE
+
 #if defined(CONFIG_RISCV_SBI) && defined(CONFIG_RISCV_USER_CFI)
 	li a7, SBI_EXT_FWFT
 	li a6, SBI_EXT_FWFT_SET
arch/riscv/kernel/process.c +31 -2
index b2df7f72241a..0dc90bf7a652 100644
@@ -258,8 +258,23 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args)
 		/* Supervisor/Machine, irqs on: */
 		childregs->status = SR_PP | SR_PIE;
 
-		p->thread.s[0] = (unsigned long)args->fn;
-		p->thread.s[1] = (unsigned long)args->fn_arg;
+		/*
+		 * Set up a metadata frame record at the bottom of the
+		 * stack for the unwinder. Use FRAME_META_TYPE_FINAL
+		 * since this is the outermost kernel entry for the new
+		 * task. The frame_record::{fp,ra} are already zero from
+		 * memset().
+		 *
+		 * fp/s0 points above the metadata record (RISC-V
+		 * convention). fn and fn_arg are passed via s2/s3,
+		 * keeping s0 available for the frame pointer chain.
+		 */
+		childregs->stackframe.type = FRAME_META_TYPE_FINAL;
+
+		p->thread.s[0] = (unsigned long)(&childregs->stackframe)
+				+ sizeof(struct frame_record);
+		p->thread.s[2] = (unsigned long)args->fn;
+		p->thread.s[3] = (unsigned long)args->fn_arg;
 		p->thread.ra = (unsigned long)ret_from_fork_kernel_asm;
 	} else {
 		/* allocate new shadow stack if needed. In case of CLONE_VM we have to */
@@ -278,6 +293,20 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args)
 		if (clone_flags & CLONE_SETTLS)
 			childregs->tp = tls;
 		childregs->a0 = 0; /* Return value of fork() */
+
+		/*
+		 * Set up the unwind boundary: ensure the metadata
+		 * frame record has its {fp,ra} sentinel zeroed and
+		 * point fp/s0 above the metadata record. Mark it as
+		 * FINAL since this is the outermost kernel entry for
+		 * the new task.
+		 */
+		childregs->stackframe.record.fp = 0;
+		childregs->stackframe.record.ra = 0;
+		childregs->stackframe.type = FRAME_META_TYPE_FINAL;
+		p->thread.s[0] = (unsigned long)(&childregs->stackframe)
+				+ sizeof(struct frame_record);
+
 		p->thread.ra = (unsigned long)ret_from_fork_user_asm;
 	}
 	p->thread.riscv_v_flags = 0;
Signature · 1 line
2.43.0
author
#2 lore raw

PATCH v5 2/7 riscv: stacktrace: disable KASAN and KCOV instrumentation for stacktrace.o

In reply to Wang Han

KASAN records stack traces for every alloc/free, which means it walks the unwinder very frequently. Instrumenting the stack trace collection code itself adds substantial overhead and makes the traces themselves noisier.

KCOV instruments every basic-block edge. The unwinder is a hot path, especially with KASAN enabled, so KCOV instrumentation has the same kind of cost and noise problem here.

Mark stacktrace.o as not KASAN- or KCOV-instrumented, matching the x86 treatment of its stack unwinding code. RISC-V keeps the relevant unwinder code in stacktrace.o, so a single translation-unit annotation covers the equivalent scope. This is a prerequisite preference for the upcoming reliable unwinder, but the change is valid on its own.

---
 arch/riscv/kernel/Makefile | 6 ++++++
 1 file changed, 6 insertions(+)
arch/riscv/kernel/Makefile +6 -0
index cabb99cadfb6..c565a72a36f3 100644
@@ -44,6 +44,12 @@ CFLAGS_REMOVE_return_address.o	= $(CC_FLAGS_FTRACE)
 CFLAGS_REMOVE_sbi_ecall.o = $(CC_FLAGS_FTRACE)
 endif
 
+# When KASAN is enabled, a stack trace is recorded for every alloc/free, which
+# can significantly impact performance. Avoid instrumenting the stack trace
+# collection code to minimize this impact.
+KASAN_SANITIZE_stacktrace.o := n
+KCOV_INSTRUMENT_stacktrace.o := n
+
 always-$(KBUILD_BUILTIN) += vmlinux.lds
 
 obj-y	+= head.o
Signature · 1 line
2.43.0
author
#3 lore raw

PATCH v5 3/7 riscv: ftrace: always preserve s0 in dynamic ftrace register frame

In reply to Wang Han

struct __arch_ftrace_regs declares s0 unconditionally, and both ftrace_regs_get_frame_pointer() and ftrace_partial_regs() read it unconditionally. But the SAVE_ABI_REGS / RESTORE_ABI_REGS macros in mcount-dyn.S only stored s0 under HAVE_FUNCTION_GRAPH_FP_TEST (CONFIG_FUNCTION_GRAPH_TRACER && CONFIG_FRAME_POINTER). With CONFIG_FRAME_POINTER=n the slot held whatever was on the stack before, so any callback going through ftrace_partial_regs() saw a garbage regs->s0. RISC-V kernels default to FRAME_POINTER=y, which is why this has not bitten in practice.

Save and restore s0 unconditionally in the dynamic ftrace ABI register frame. This fixes the latent garbage-s0 case, brings the dynamic ftrace path in line with the static _mcount path (mcount.S SAVE_ABI_STATE already saves s0 unconditionally), and matches the frame layout already documented in the comment above SAVE_ABI_REGS. It is also a prerequisite for the upcoming reliable unwinder, which reads ftrace_regs_get_frame_pointer(fregs) directly.

The cost is one extra REG_S/REG_L pair per traced call, negligible compared to the overall ftrace cost; the existing FREGS_SIZE_ON_STACK already reserved the slot, so no extra stack space is used.

---
 arch/riscv/kernel/mcount-dyn.S | 4 ----
 1 file changed, 4 deletions(-)
arch/riscv/kernel/mcount-dyn.S +0 -4
index 082fe0b0e3c0..26c55fba8fec 100644
@@ -85,9 +85,7 @@
 	addi	sp, sp, -FREGS_SIZE_ON_STACK
 	REG_S	t0,  FREGS_EPC(sp)
 	REG_S	x1,  FREGS_RA(sp)
-#ifdef HAVE_FUNCTION_GRAPH_FP_TEST
 	REG_S	x8,  FREGS_S0(sp)
-#endif
 	REG_S	x6,  FREGS_T1(sp)
 #ifdef CONFIG_CC_IS_CLANG
 	REG_S	x7,  FREGS_T2(sp)
@@ -113,9 +111,7 @@
 	.macro RESTORE_ABI_REGS
 	REG_L	t0, FREGS_EPC(sp)
 	REG_L	x1, FREGS_RA(sp)
-#ifdef HAVE_FUNCTION_GRAPH_FP_TEST
 	REG_L	x8, FREGS_S0(sp)
-#endif
 	REG_L	x6,  FREGS_T1(sp)
 #ifdef CONFIG_CC_IS_CLANG
 	REG_L	x7,  FREGS_T2(sp)
Signature · 1 line
2.43.0
author
#4 lore raw

PATCH v5 4/7 riscv: stacktrace: introduce stack-bound tracking helpers

In reply to Wang Han

A reliable unwinder needs to validate that every frame record it reads is fully contained in a known kernel stack, and it needs to refuse to walk back into a stack it has already left. Add the building blocks for that:

* struct stack_info / struct unwind_state in a new asm/stacktrace/common.h, modelled on the arm64 reference implementation.
* stackinfo_get_irq() / stackinfo_get_task() / stackinfo_get_overflow() plus the corresponding on_*_stack() predicates in asm/stacktrace.h, so callers can ask "is this object on stack X?" by stack kind rather than open-coded address arithmetic.
* unwind_init_common(), unwind_find_stack() and unwind_consume_stack() helpers that enforce the forward-progress-only invariant required for reliability.

No existing user is wired up to these helpers in this commit; the unwinder switch comes in a follow-up. The header changes leave on_thread_stack() with the same semantics as before, just expressed in terms of the new helpers.

---
 arch/riscv/include/asm/stacktrace.h        |  67 ++++++++-
 arch/riscv/include/asm/stacktrace/common.h | 159 +++++++++++++++++++++
 2 files changed, 224 insertions(+), 2 deletions(-)
 create mode 100644 arch/riscv/include/asm/stacktrace/common.h
arch/riscv/include/asm/stacktrace.h +65 -2
index b1495a7e06ce..df4d022f6430 100644
@@ -3,8 +3,13 @@
 #ifndef _ASM_RISCV_STACKTRACE_H
 #define _ASM_RISCV_STACKTRACE_H
 
+#include <linux/percpu.h>
 #include <linux/sched.h>
+#include <linux/sched/task_stack.h>
+
+#include <asm/irq_stack.h>
 #include <asm/ptrace.h>
+#include <asm/stacktrace/common.h>
 
 struct stackframe {
 	unsigned long fp;
@@ -16,14 +21,72 @@ extern void notrace walk_stackframe(struct task_struct *task, struct pt_regs *re
 extern void dump_backtrace(struct pt_regs *regs, struct task_struct *task,
 			   const char *loglvl);
 
-static inline bool on_thread_stack(void)
+/*
+ * IRQ stack accessors
+ */
+#ifdef CONFIG_IRQ_STACKS
+static inline struct stack_info stackinfo_get_irq(void)
+{
+	unsigned long low = (unsigned long)raw_cpu_read(irq_stack_ptr);
+	unsigned long high = low + IRQ_STACK_SIZE;
+
+	return (struct stack_info) {
+		.low = low,
+		.high = high,
+	};
+}
+
+static inline bool on_irq_stack(unsigned long sp, unsigned long size)
+{
+	struct stack_info info = stackinfo_get_irq();
+
+	return stackinfo_on_stack(&info, sp, size);
+}
+#endif /* CONFIG_IRQ_STACKS */
+
+/*
+ * Task stack accessors
+ */
+static inline struct stack_info stackinfo_get_task(const struct task_struct *tsk)
 {
-	return !(((unsigned long)(current->stack) ^ current_stack_pointer) & ~(THREAD_SIZE - 1));
+	unsigned long low = (unsigned long)task_stack_page(tsk);
+	unsigned long high = low + THREAD_SIZE;
+
+	return (struct stack_info) {
+		.low = low,
+		.high = high,
+	};
+}
+
+static inline bool on_task_stack(const struct task_struct *tsk,
+				 unsigned long sp, unsigned long size)
+{
+	struct stack_info info = stackinfo_get_task(tsk);
+
+	return stackinfo_on_stack(&info, sp, size);
 }
 
+/*
+ * Cast is necessary since current->stack is an opaque ptr.
+ */
+#define on_thread_stack()	(on_task_stack(current, current_stack_pointer, 1))
 
+/*
+ * Overflow stack accessors
+ */
 #ifdef CONFIG_VMAP_STACK
 DECLARE_PER_CPU(unsigned long [OVERFLOW_STACK_SIZE/sizeof(long)], overflow_stack);
+
+static inline struct stack_info stackinfo_get_overflow(void)
+{
+	unsigned long low = (unsigned long)raw_cpu_ptr(overflow_stack);
+	unsigned long high = low + OVERFLOW_STACK_SIZE;
+
+	return (struct stack_info) {
+		.low = low,
+		.high = high,
+	};
+}
 #endif /* CONFIG_VMAP_STACK */
 
 #endif /* _ASM_RISCV_STACKTRACE_H */
arch/riscv/include/asm/stacktrace/common.h new file +159 -0

159 lines not on this page · raw message

Signature · 1 line
2.43.0
author
#5 lore raw

PATCH v5 5/7 riscv: stacktrace: switch to frame-pointer based unwinder

In reply to Wang Han

Replace the open-coded frame-pointer walker in arch_stack_walk() with a robust kunwind state machine, modelled on arch/arm64/kernel/stacktrace.c and retargeted to the RISC-V {fp, ra} frame record convention. The new walker tracks stack bounds, consumes frame records monotonically, understands the metadata pt_regs records added in the previous frame record metadata patch, and recovers return addresses replaced by function graph tracing and kretprobes.

This commit introduces arch_stack_walk_reliable() but does not yet select HAVE_RELIABLE_STACKTRACE; that is done in a follow-up Kconfig patch so this commit can be reviewed and bisected as a pure unwinder replacement. Until that Kconfig change lands, livepatch is not yet enabled and arch_stack_walk_reliable() has no in-tree caller.

Three related callers are updated to keep the same frame-record assumptions everywhere:

* Function graph tracing: the old RISC-V unwinder matched function graph return-stack entries by the saved return-address slot. That was consistent with the static mcount path, but not with the dynamic ftrace path where the parent slot is ftrace_regs::ra. Use the architectural frame pointer as the function graph return-address cookie, matching the kunwind walker.

* Perf callchains: route kernel callchain collection through arch_stack_walk() so perf sees the same frame-pointer unwind behaviour as dump_stack() and the upcoming livepatch path.

* dump_backtrace() / __get_wchan() / show_stack(): these now go through arch_stack_walk(); the explicit "Call Trace:" header is moved into dump_backtrace() to preserve the original output.

The non-frame-pointer fallback walker is kept untouched for !CONFIG_FRAME_POINTER builds.

---
 arch/riscv/kernel/ftrace.c         |   6 +-
 arch/riscv/kernel/perf_callchain.c |   2 +-
 arch/riscv/kernel/stacktrace.c     | 561 ++++++++++++++++++++++++-----
 3 files changed, 473 insertions(+), 96 deletions(-)
arch/riscv/kernel/ftrace.c +4 -2
index b430edfb83f4..5d55199a9230 100644
@@ -242,7 +242,8 @@ void prepare_ftrace_return(unsigned long *parent, unsigned long self_addr,
 	 */
 	old = *parent;
 
-	if (!function_graph_enter(old, self_addr, frame_pointer, parent))
+	if (!function_graph_enter(old, self_addr, frame_pointer,
+				  (void *)frame_pointer))
 		*parent = return_hooker;
 }
 
@@ -264,7 +265,8 @@ void ftrace_graph_func(unsigned long ip, unsigned long parent_ip,
 	 */
 	old = *parent;
 
-	if (!function_graph_enter_regs(old, ip, frame_pointer, parent, fregs))
+	if (!function_graph_enter_regs(old, ip, frame_pointer,
+				       (void *)frame_pointer, fregs))
 		*parent = return_hooker;
 }
 #endif /* CONFIG_DYNAMIC_FTRACE */
arch/riscv/kernel/perf_callchain.c +1 -1
index b465bc9eb870..436af96ea59c 100644
@@ -44,5 +44,5 @@ void perf_callchain_kernel(struct perf_callchain_entry_ctx *entry,
 		return;
 	}
 
-	walk_stackframe(NULL, regs, fill_callchain, entry);
+	arch_stack_walk(fill_callchain, entry, NULL, regs);
 }
arch/riscv/kernel/stacktrace.c +468 -93

602 lines not on this page · raw message

Signature · 1 line
2.43.0
author
#6 lore raw

PATCH v5 6/7 riscv: Kconfig: enable HAVE_RELIABLE_STACKTRACE and HAVE_LIVEPATCH

In reply to Wang Han

Now that the metadata frame records, the kunwind state machine and arch_stack_walk_reliable() are all in place, advertise the capability to the rest of the kernel:

* select HAVE_RELIABLE_STACKTRACE under FRAME_POINTER && 64BIT, so only the configurations with the tested metadata records and FP-based reliable walker enable it.
* select HAVE_LIVEPATCH under the same condition and source kernel/livepatch/Kconfig so the livepatch menu is reachable from the RISC-V configuration.

The 64BIT dependency is conservative scoping rather than a hard technical requirement: the metadata frame record, kunwind state machine and arch_stack_walk_reliable() also build on RV32, and the IRQ-stack frame-record adjustment fixes a latent RV32 issue. However, the syscall livepatch selftest and module relocation path have only been exercised on RV64 QEMU virt so far. The 64BIT gate can be relaxed in a follow-up once RV32 has equivalent coverage.

This is split out from the unwinder change so the policy decision and the implementation can be reviewed and reverted independently.

---
 arch/riscv/Kconfig | 4 ++++
 1 file changed, 4 insertions(+)
arch/riscv/Kconfig +4 -0
index c2c2f5f3ed5e..b36f711ecf45 100644
@@ -186,6 +186,7 @@ config RISCV
 	select HAVE_KRETPROBES
 	# https://github.com/ClangBuiltLinux/linux/issues/1881
 	select HAVE_LD_DEAD_CODE_DATA_ELIMINATION if !LD_IS_LLD
+	select HAVE_LIVEPATCH if FRAME_POINTER && 64BIT
 	select HAVE_MOVE_PMD
 	select HAVE_MOVE_PUD
 	select HAVE_PAGE_SIZE_4KB
@@ -196,6 +197,7 @@ config RISCV
 	select HAVE_POSIX_CPU_TIMERS_TASK_WORK
 	select HAVE_PREEMPT_DYNAMIC_KEY
 	select HAVE_REGS_AND_STACK_ACCESS_API
+	select HAVE_RELIABLE_STACKTRACE if FRAME_POINTER && 64BIT
 	select HAVE_RETHOOK
 	select HAVE_RSEQ
 	select HAVE_RUST if RUSTC_SUPPORTS_RISCV && CC_IS_CLANG
@@ -1392,3 +1394,5 @@ endmenu # "CPU Power Management"
 source "arch/riscv/kvm/Kconfig"
 
 source "drivers/acpi/Kconfig"
+
+source "kernel/livepatch/Kconfig"
Signature · 1 line
2.43.0
author
#7 lore raw

PATCH v5 7/7 selftests/livepatch: Add RISC-V syscall wrapper prefix

In reply to Wang Han

The syscall livepatch selftest resolves and patches a syscall wrapper symbol. To use that test for RISC-V livepatch validation, add the RISC-V FN_PREFIX definition for ARCH_HAS_SYSCALL_WRAPPER.

Without this macro, the syscall livepatch selftest cannot resolve the RISC-V target symbol, and the syscall-related livepatch test fails on RISC-V.

---
 .../testing/selftests/livepatch/test_modules/test_klp_syscall.c | 2 ++
 1 file changed, 2 insertions(+)
tools/testing/selftests/livepatch/test_modules/test_klp_syscall.c +2 -0
index 08aacc0e14de..9baa2a5f84c9 100644
@@ -24,6 +24,8 @@
     #define FN_PREFIX __s390x_
   #elif defined(__aarch64__)
     #define FN_PREFIX __arm64_
+  #elif defined(__riscv)
+    #define FN_PREFIX __riscv_
   #elif defined(__powerpc__)
     #define FN_PREFIX
   #else
Signature · 1 line
2.43.0
maintainer Acked-by on v5 7/7
#8 lore raw

Re: [PATCH v5 7/7] selftests/livepatch: Add RISC-V syscall wrapper prefix

Quoted 9 lines · Wang Han

The syscall livepatch selftest resolves and patches a syscall wrapper symbol. To use that test for RISC-V livepatch validation, add the RISC-V FN_PREFIX definition for ARCH_HAS_SYSCALL_WRAPPER.

Without this macro, the syscall livepatch selftest cannot resolve the RISC-V target symbol, and the syscall-related livepatch test fails on RISC-V.

[...]

Signature · 1 line
Miroslav