riscv: Add reliable stack unwinding for livepatch
Subsystems (7)
v4 cover riscv: Add reliable stack unwinding for livepatch
Hi,
Sorry for the noise and inconvenience caused by the previous unthreaded v4 submission.
This is a resend of v4 only to fix mail threading. The patch contents are unchanged from the previous v4.
Previous unthreaded v4:
https://lore.kernel.org/r/[email protected]
This series adds the RISC-V architecture pieces needed by livepatch:
dynamic ftrace must preserve the frame pointer context that livepatch uses for redirection, and the stack unwinder must be reliable enough for the livepatch consistency model.
The reliable unwinder is based on frame records and explicit metadata at task and exception boundaries. It is intentionally conservative: it rejects ambiguous states instead of trying to unwind through them. The series then enables HAVE_RELIABLE_STACKTRACE and HAVE_LIVEPATCH for 64-bit RISC-V with dynamic ftrace.
The first patch from v3, "scripts/sorttable: Handle RISC-V patchable ftrace entries", has been picked up by Paul and is already present in riscv/for-next as commit 57ad674d032b. It is therefore dropped from this v4. The remaining patches are rebased on the latest riscv/for-next.
Base:
riscv/for-next 798246e5edfb ("riscv: acpi: Enable ARCH_HAS_ACPI_TABLE_UPGRADE")
Previous versions:
v3: https://lore.kernel.org/r/[email protected]
v2: https://lore.kernel.org/r/[email protected]
v1: https://lore.kernel.org/r/[email protected]
Changes since v3:
* Drop the accepted sorttable fix, now commit 57ad674d032b in riscv/for-next.
* Rebase the remaining 7 patches on the latest riscv/for-next.
* Adapt the frame metadata patch to the existing call_on_irq_stack() RV32 frame-pointer ABI fix by keeping metadata frame-record offsets distinct from the s0-relative STACKFRAME_* offsets.
* Adapt the livepatch syscall selftest prefix change to the current CONFIG_ARCH_HAS_SYSCALL_WRAPPER wrapper logic.
Validation:
* Built with riscv64-unknown-linux-gnu-gcc 15.2.0 and the existing configs/riscv_livepatch_config, including RISCV_ISA_C=y, EFI=y and ACPI=y.
* make -C linux O=$PWD/build/linux/riscv ARCH=riscv \
CROSS_COMPILE=riscv64-unknown-linux-gnu- -j$(nproc) Image modules
passed. * make kernel-debug ARCH=riscv GDB=riscv64-unknown-linux-gnu-gdb passed.
* livepatch selftest modules built successfully via ./test-livepatch.sh riscv.
* QEMU RISC-V livepatch selftests passed with PASS: 7, SKIP: 1,
FAIL: 0. The only skip is test-kprobe.sh because this config does not enable CONFIG_KPROBES_ON_FTRACE.
* The ftrace function graph subset passed with 3 passed, 0 failed and 3 unsupported tests.
Wang Han (7):
riscv: stacktrace: Add frame record metadata
riscv: stacktrace: disable KASAN and KCOV instrumentation for
stacktrace.o
riscv: ftrace: always preserve s0 in dynamic ftrace register frame
riscv: stacktrace: introduce stack-bound tracking helpers
riscv: stacktrace: switch to frame-pointer based unwinder
riscv: Kconfig: enable HAVE_RELIABLE_STACKTRACE and HAVE_LIVEPATCH selftests/livepatch: Add RISC-V syscall wrapper prefix
arch/riscv/Kconfig | 4 + arch/riscv/include/asm/ptrace.h | 9 + arch/riscv/include/asm/stacktrace.h | 65 +- arch/riscv/include/asm/stacktrace/common.h | 159 +++++ arch/riscv/include/asm/stacktrace/frame.h | 53 ++ arch/riscv/kernel/Makefile | 6 + arch/riscv/kernel/asm-offsets.c | 6 + arch/riscv/kernel/entry.S | 39 +- arch/riscv/kernel/ftrace.c | 6 +- arch/riscv/kernel/head.S | 23 + arch/riscv/kernel/mcount-dyn.S | 4 - arch/riscv/kernel/perf_callchain.c | 2 +- arch/riscv/kernel/process.c | 33 +- arch/riscv/kernel/stacktrace.c | 559 +++++++++++++++--- .../livepatch/test_modules/test_klp_syscall.c | 2 + 15 files changed, 864 insertions(+), 106 deletions(-) create mode 100644 arch/riscv/include/asm/stacktrace/common.h create mode 100644 arch/riscv/include/asm/stacktrace/frame.h
Range-diff against v3:
1: 8cef363cfed7 ! 1: efd99ce56c1e riscv: stacktrace: Add frame record metadata
@@ Commit message
the secondary CPU path sets up s0 before smp_callin() so idle-task unwinding does not inherit an undefined caller frame;
* copy_thread creates matching final metadata records for new kernel
- and user tasks, and keeps s0 available for the frame-pointer chain;
- * call_on_irq_stack still reserves an aligned stack slot, but links the
- saved {fp, ra} with the raw frame-record size so s0 points at the
- RISC-V frame record rather than past the alignment padding.
+ and user tasks, and keeps s0 available for the frame-pointer chain.
- The call_on_irq_stack adjustment fixes a latent RV32 issue. On RV64,
- sizeof(struct stackframe) is equal to the stack alignment, so the old
- s0 value happened to point just above the saved {fp, ra}. On RV32, the
- raw frame record is 8 bytes while the reserved stack slot is 16-byte
- aligned, so the old s0 value pointed into the padding. Using the raw
- record size makes s0 point above the saved frame record on both RV32
- and RV64 while still reserving the aligned slot.
+ Keep the embedded metadata-record field offsets distinct from the
+ s0-relative STACKFRAME_* offsets used by call_on_irq_stack(), because
+ the latter describe a frame record relative to s0 rather than to the
+ record base.
These changes keep s0 reserved for the frame-pointer chain at task and
- stack-switch boundaries.
+ exception boundaries.
@@ arch/riscv/kernel/asm-offsets.c: void asm_offsets(void)
OFFSET(HIBERN_PBE_ADDR, pbe, address);
@@ arch/riscv/kernel/asm-offsets.c: void asm_offsets(void)
- OFFSET(SBI_HART_BOOT_STACK_PTR_OFFSET, sbi_hart_boot_data, stack_ptr);
-
DEFINE(STACKFRAME_SIZE_ON_STACK, ALIGN(sizeof(struct stackframe), STACK_ALIGN));
+ DEFINE(STACKFRAME_FP, offsetof(struct stackframe, fp) - sizeof(struct stackframe));
+ DEFINE(STACKFRAME_RA, offsetof(struct stackframe, ra) - sizeof(struct stackframe));
+ DEFINE(STACKFRAME_RECORD_SIZE, sizeof(struct stackframe));
- OFFSET(STACKFRAME_FP, stackframe, fp);
- OFFSET(STACKFRAME_RA, stackframe, ra);
++ OFFSET(FRAME_RECORD_FP, frame_record, fp);
++ OFFSET(FRAME_RECORD_RA, frame_record, ra);
#ifdef CONFIG_FUNCTION_TRACER
+ DEFINE(FTRACE_OPS_FUNC, offsetof(struct ftrace_ops, func));
+ #ifdef CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS
## arch/riscv/kernel/entry.S ##
@@
@@ arch/riscv/kernel/entry.S: SYM_CODE_START(handle_exception)
+ * Create a metadata frame record. The unwinder will use this to
+ * identify and unwind exception boundaries.
+ */
-+ REG_S zero, (S_STACKFRAME + STACKFRAME_FP)(sp) /* stackframe.record.fp = 0 */
-+ REG_S zero, (S_STACKFRAME + STACKFRAME_RA)(sp) /* stackframe.record.ra = 0 */
++ REG_S zero, (S_STACKFRAME + FRAME_RECORD_FP)(sp) /* stackframe.record.fp = 0 */
++ REG_S zero, (S_STACKFRAME + FRAME_RECORD_RA)(sp) /* stackframe.record.ra = 0 */
+#ifdef CONFIG_RISCV_M_MODE
+ li t0, SR_MPP
+ and t0, s1, t0
@@ arch/riscv/kernel/entry.S: SYM_CODE_START_LOCAL(handle_kernel_stack_overflow)
+ * pt_regs boundary and the unwinder can resume from the pre-overflow
+ * frame pointer saved in PT_S0.
+ */
-+ REG_S zero, (S_STACKFRAME + STACKFRAME_FP)(sp)
-+ REG_S zero, (S_STACKFRAME + STACKFRAME_RA)(sp)
++ REG_S zero, (S_STACKFRAME + FRAME_RECORD_FP)(sp)
++ REG_S zero, (S_STACKFRAME + FRAME_RECORD_RA)(sp)
+ li t0, FRAME_META_TYPE_PT_REGS
+ REG_S t0, S_STACKFRAME_TYPE(sp)
+ addi s0, sp, S_STACKFRAME + STACKFRAME_RECORD_SIZE
@@ arch/riscv/kernel/entry.S: ASM_NOKPROBE(handle_kernel_stack_overflow)
move a2, sp /* pt_regs */
call ret_from_fork_kernel
j ret_from_exception
-@@ arch/riscv/kernel/entry.S: SYM_FUNC_START(call_on_irq_stack)
- addi sp, sp, -STACKFRAME_SIZE_ON_STACK
- REG_S ra, STACKFRAME_RA(sp)
- REG_S s0, STACKFRAME_FP(sp)
-- addi s0, sp, STACKFRAME_SIZE_ON_STACK
-+ addi s0, sp, STACKFRAME_RECORD_SIZE
-
- /* Switch to the per-CPU shadow call stack */
- scs_save_current
-@@ arch/riscv/kernel/entry.S: SYM_FUNC_START(call_on_irq_stack)
- scs_load_current
-
- /* Switch back to the thread stack and restore ra and s0 */
-- addi sp, s0, -STACKFRAME_SIZE_ON_STACK
-+ addi sp, s0, -STACKFRAME_RECORD_SIZE
- REG_L ra, STACKFRAME_RA(sp)
- REG_L s0, STACKFRAME_FP(sp)
- addi sp, sp, STACKFRAME_SIZE_ON_STACK
## arch/riscv/kernel/head.S ##
@@
@@ arch/riscv/kernel/head.S: SYM_CODE_START(_start_kernel)
+ * fp/s0 points above the metadata record (RISC-V
+ * convention).
+ */
-+ REG_S zero, (S_STACKFRAME + STACKFRAME_FP)(sp)
-+ REG_S zero, (S_STACKFRAME + STACKFRAME_RA)(sp)
++ REG_S zero, (S_STACKFRAME + FRAME_RECORD_FP)(sp)
++ REG_S zero, (S_STACKFRAME + FRAME_RECORD_RA)(sp)
+ li t0, FRAME_META_TYPE_FINAL
+ REG_S t0, S_STACKFRAME_TYPE(sp)
+ addi s0, sp, S_STACKFRAME + STACKFRAME_RECORD_SIZE
2: 237864b66d78 = 2: bc0af8ec1976 riscv: stacktrace: disable KASAN and KCOV instrumentation for stacktrace.o 3: e6035966a35a = 3: ab3fbed66fff riscv: ftrace: always preserve s0 in dynamic ftrace register frame 4: d132087ea01e = 4: 856d1e31408a riscv: stacktrace: introduce stack-bound tracking helpers 5: 02adea3ece82 = 5: 58aa4435e2ee riscv: stacktrace: switch to frame-pointer based unwinder 6: c7d7dbe7a8a1 = 6: 500f9d9eeac0 riscv: Kconfig: enable HAVE_RELIABLE_STACKTRACE and HAVE_LIVEPATCH 7: ae94a234b34a < -: ------------ selftests/livepatch: Add RISC-V syscall wrapper prefix -: ------------ > 7: 3dcfa694c207 selftests/livepatch: Add RISC-V syscall wrapper prefix
base-commit: 798246e5edfb3aa0b2d6dca46f41014d0b99b209
Signature · 1 line
2.43.0
PATCH RESEND v4 1/7 riscv: stacktrace: Add frame record metadata
Reliable frame-pointer unwinding needs an explicit way to identify exception boundaries and the final entry frame. The existing unwinder infers those boundaries from return addresses, which is too loose for a future reliable unwinder.
Add a small metadata frame record to pt_regs and initialize it on exception entry, kernel stack overflow, kernel thread fork, user fork, and early idle task setup. The record uses a zero {fp, ra} sentinel plus a type field so a later unwinder can distinguish a final user-to-kernel boundary from a nested kernel pt_regs boundary.
This follows the arm64 metadata frame-record model, adapted to the RISC-V {fp, ra} frame record convention.
The metadata is established at the RISC-V entry boundaries that need an explicit unwind marker:
* exception entry clears the metadata {fp, ra} pair and uses SPP (or MPP in M-mode) to record whether the pt_regs frame is the final user-to-kernel boundary or a nested kernel boundary;
* the kernel stack overflow path builds a nested pt_regs metadata record on the overflow stack so an unwinder can resume from the pre-overflow s0 saved in PT_S0;
* _start_kernel builds the init task's final metadata record, while the secondary CPU path sets up s0 before smp_callin() so idle-task unwinding does not inherit an undefined caller frame;
* copy_thread creates matching final metadata records for new kernel and user tasks, and keeps s0 available for the frame-pointer chain.
Keep the embedded metadata-record field offsets distinct from the s0-relative STACKFRAME_* offsets used by call_on_irq_stack(), because the latter describe a frame record relative to s0 rather than to the record base.
These changes keep s0 reserved for the frame-pointer chain at task and exception boundaries.
--- arch/riscv/include/asm/ptrace.h | 9 ++++ arch/riscv/include/asm/stacktrace/frame.h | 53 +++++++++++++++++++++++ arch/riscv/kernel/asm-offsets.c | 6 +++ arch/riscv/kernel/entry.S | 39 ++++++++++++++++- arch/riscv/kernel/head.S | 23 ++++++++++ arch/riscv/kernel/process.c | 33 +++++++++++++- 6 files changed, 159 insertions(+), 4 deletions(-) create mode 100644 arch/riscv/include/asm/stacktrace/frame.h
arch/riscv/include/asm/ptrace.h +9 -0
@@ -8,6 +8,7 @@#include <uapi/asm/ptrace.h> #include <asm/csr.h> +#include <asm/stacktrace/frame.h> #include <linux/compiler.h> #ifndef __ASSEMBLER__
@@ -53,6 +54,14 @@ struct pt_regs {unsigned long cause;/* a0 value before the syscall */ unsigned long orig_a0; + +/* +* This frame record is entirely zeroed on exception entry, allowing the +* unwinder to identify exception boundaries. The type field encodes +* whether the exception was taken from user (FINAL) or kernel (PT_REGS) +* mode. +*/ +struct frame_record_meta stackframe; };#define PTRACE_SYSEMU 0x1f
arch/riscv/include/asm/stacktrace/frame.h new file +53 -0
53 lines not on this page · raw message
arch/riscv/kernel/asm-offsets.c +6 -0
@@ -131,6 +131,9 @@ void asm_offsets(void)OFFSET (PT_BADADDR , pt_regs, badaddr);OFFSET (PT_CAUSE , pt_regs, cause); +DEFINE (S_STACKFRAME , offsetof(struct pt_regs , stackframe)); + DEFINE(S_STACKFRAME_TYPE, offsetof(struct pt_regs, stackframe.type)); +OFFSET (SUSPEND_CONTEXT_REGS , suspend_context, regs);OFFSET (HIBERN_PBE_ADDR , pbe, address);
@@ -503,6 +506,9 @@ void asm_offsets(void)DEFINE (STACKFRAME_SIZE_ON_STACK ,ALIGN (sizeof (struct stackframe ),STACK_ALIGN ));DEFINE (STACKFRAME_FP , offsetof(struct stackframe , fp) -sizeof (struct stackframe ));DEFINE (STACKFRAME_RA , offsetof(struct stackframe , ra) -sizeof (struct stackframe )); +DEFINE (STACKFRAME_RECORD_SIZE ,sizeof (struct stackframe )); +OFFSET (FRAME_RECORD_FP , frame_record, fp); +OFFSET (FRAME_RECORD_RA , frame_record, ra);#ifdef CONFIG_FUNCTION_TRACER DEFINE (FTRACE_OPS_FUNC , offsetof(struct ftrace_ops , func));#ifdef CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS
arch/riscv/kernel/entry.S +37 -2
@@ -11,6 +11,7 @@ #include <asm/asm.h> #include <asm/csr.h> #include <asm/scs.h> +#include <asm/stacktrace/frame.h> #include <asm/unistd.h> #include <asm/page.h> #include <asm/thread_info.h>
@@ -198,6 +199,27 @@ SYM_CODE_START(handle_exception) REG_S s4, PT_CAUSE(sp) REG_S s5, PT_TP(sp) + /* + * Create a metadata frame record. The unwinder will use this to + * identify and unwind exception boundaries. + */ + REG_S zero, (S_STACKFRAME + FRAME_RECORD_FP)(sp) /* stackframe.record.fp = 0 */ + REG_S zero, (S_STACKFRAME + FRAME_RECORD_RA)(sp) /* stackframe.record.ra = 0 */ +#ifdef CONFIG_RISCV_M_MODE + li t0, SR_MPP + and t0, s1, t0 +#else + andi t0, s1, SR_SPP +#endif + bnez t0, 1f + li t0, FRAME_META_TYPE_FINAL + j 2f +1: + li t0, FRAME_META_TYPE_PT_REGS +2: + REG_S t0, S_STACKFRAME_TYPE(sp) + addi s0, sp, S_STACKFRAME + STACKFRAME_RECORD_SIZE + /* * Set the scratch register to 0, so that if a recursive exception * occurs, the exception vector knows it came from the kernel
@@ -354,6 +376,19 @@ SYM_CODE_START_LOCAL(handle_kernel_stack_overflow) REG_S s3, PT_BADADDR(sp) REG_S s4, PT_CAUSE(sp) REG_S s5, PT_TP(sp) + + /* + * Create a metadata frame record for the overflow pt_regs. The + * overflow path is entered from kernel context, so this is a nested + * pt_regs boundary and the unwinder can resume from the pre-overflow + * frame pointer saved in PT_S0. + */ + REG_S zero, (S_STACKFRAME + FRAME_RECORD_FP)(sp) + REG_S zero, (S_STACKFRAME + FRAME_RECORD_RA)(sp) + li t0, FRAME_META_TYPE_PT_REGS + REG_S t0, S_STACKFRAME_TYPE(sp) + addi s0, sp, S_STACKFRAME + STACKFRAME_RECORD_SIZE + move a0, sp tail handle_bad_stack SYM_CODE_END(handle_kernel_stack_overflow)
@@ -362,8 +397,8 @@ ASM_NOKPROBE(handle_kernel_stack_overflow) SYM_CODE_START(ret_from_fork_kernel_asm) call schedule_tail - move a0, s1 /* fn_arg */ - move a1, s0 /* fn */ + move a0, s3 /* fn_arg */ + move a1, s2 /* fn */ move a2, sp /* pt_regs */ call ret_from_fork_kernel j ret_from_exception
arch/riscv/kernel/head.S +23 -0
@@ -14,6 +14,7 @@ #include <asm/hwcap.h> #include <asm/image.h> #include <asm/scs.h> +#include <asm/stacktrace/frame.h> #include <asm/usercfi.h> #include "efi-header.S"
@@ -177,6 +178,14 @@ secondary_start_sbi: REG_S a0, (a1) 1: #endif + + /* + * Set up the frame pointer for the secondary idle task so reliable + * stack unwinding terminates at the metadata frame in task_pt_regs(). + * Without this, the first frame records can inherit an undefined caller + * fp and unwind past smp_callin() into .Lsecondary_park. + */ + addi s0, sp, S_STACKFRAME + STACKFRAME_RECORD_SIZE scs_load_current call smp_callin #endif /* CONFIG_SMP */
@@ -305,6 +314,20 @@ SYM_CODE_START(_start_kernel) la tp, init_task la sp, init_thread_union + THREAD_SIZE addi sp, sp, -PT_SIZE_ON_STACK + + /* + * Set up a metadata frame record for the init task so that + * the unwinder can identify the outermost frame by its + * {fp, ra} = {0, 0} sentinel at the bottom of pt_regs. + * fp/s0 points above the metadata record (RISC-V + * convention). + */ + REG_S zero, (S_STACKFRAME + FRAME_RECORD_FP)(sp) + REG_S zero, (S_STACKFRAME + FRAME_RECORD_RA)(sp) + li t0, FRAME_META_TYPE_FINAL + REG_S t0, S_STACKFRAME_TYPE(sp) + addi s0, sp, S_STACKFRAME + STACKFRAME_RECORD_SIZE + #if defined(CONFIG_RISCV_SBI) && defined(CONFIG_RISCV_USER_CFI) li a7, SBI_EXT_FWFT li a6, SBI_EXT_FWFT_SET
arch/riscv/kernel/process.c +31 -2
@@ -258,8 +258,23 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args)/* Supervisor/Machine, irqs on: */ childregs->status =SR_PP |SR_PIE ; - p->thread.s[0 ] = (unsigned long )args->fn; - p->thread.s[1 ] = (unsigned long )args->fn_arg; +/* +* Set up a metadata frame record at the bottom of the +* stack for the unwinder. Use FRAME_META_TYPE_FINAL +* since this is the outermost kernel entry for the new +* task. The frame_record::{fp,ra} are already zero from +* memset(). +* +* fp/s0 points above the metadata record (RISC-V +* convention). fn and fn_arg are passed via s2/s3, +* keeping s0 available for the frame pointer chain. +*/ + childregs->stackframe.type =FRAME_META_TYPE_FINAL ; + + p->thread.s[0 ] = (unsigned long )(&childregs->stackframe) + +sizeof (struct frame_record ); + p->thread.s[2 ] = (unsigned long )args->fn; + p->thread.s[3 ] = (unsigned long )args->fn_arg; p->thread.ra = (unsigned long )ret_from_fork_kernel_asm; }else {/* allocate new shadow stack if needed. In case of CLONE_VM we have to */
@@ -278,6 +293,20 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args)if (clone_flags &CLONE_SETTLS ) childregs->tp = tls; childregs->a0 =0 ;/* Return value of fork() */ + +/* +* Set up the unwind boundary: ensure the metadata +* frame record has its {fp,ra} sentinel zeroed and +* point fp/s0 above the metadata record. Mark it as +* FINAL since this is the outermost kernel entry for +* the new task. +*/ + childregs->stackframe.record.fp =0 ; + childregs->stackframe.record.ra =0 ; + childregs->stackframe.type =FRAME_META_TYPE_FINAL ; + p->thread.s[0 ] = (unsigned long )(&childregs->stackframe) + +sizeof (struct frame_record ); + p->thread.ra = (unsigned long )ret_from_fork_user_asm; } p->thread.riscv_v_flags =0 ;
Signature · 1 line
2.43.0
Re: [PATCH v4 RESEND 1/7] riscv: stacktrace: Add frame record metadata
Quoted 29 lines · Wang Han
Reliable frame-pointer unwinding needs an explicit way to identify exception boundaries and the final entry frame. The existing unwinder infers those boundaries from return addresses, which is too loose for a future reliable unwinder.
Add a small metadata frame record to
pt_regsand initialize it on exception entry, kernel stack overflow, kernel thread fork, user fork, and early idle task setup. The record uses a zero {fp, ra} sentinel plus a type field so a later unwinder can distinguish a final user-to-kernel boundary from a nested kernelpt_regsboundary.This follows the arm64 metadata frame-record model, adapted to the RISC-V {fp, ra} frame record convention.
The metadata is established at the RISC-V entry boundaries that need an explicit unwind marker:
* exception entry clears the metadata {fp, ra} pair and uses SPP (or MPP in M-mode) to record whether the
pt_regsframe is the final user-to-kernel boundary or a nested kernel boundary;
* the kernel stack overflow path builds a nestedpt_regsmetadata record on the overflow stack so an unwinder can resume from the pre-overflow s0 saved inPT_S0;
*_start_kernelbuilds the init task's final metadata record, while the secondary CPU path sets up s0 beforesmp_callin()so idle-task unwinding does not inherit an undefined caller frame;
*copy_threadcreates matching final metadata records for new kernel and user tasks, and keeps s0 available for the frame-pointer chain.
Quoted 7 lines
Keep the embedded metadata-record field offsets distinct from the s0-relative STACKFRAME_* offsets used by
call_on_irq_stack(), because the latter describe a frame record relative to s0 rather than to the record base.These changes keep s0 reserved for the frame-pointer chain at task and exception boundaries.
Overall the patch looks good to me. One nit below:
Quoted 113 lines
Signed-off-by Wang Han <[email protected]>--- arch/riscv/include/asm/ptrace.h | 9 ++++ arch/riscv/include/asm/stacktrace/frame.h | 53 +++++++++++++++++++++++ arch/riscv/kernel/asm-offsets.c | 6 +++ arch/riscv/kernel/entry.S | 39 ++++++++++++++++- arch/riscv/kernel/head.S | 23 ++++++++++ arch/riscv/kernel/process.c | 33 +++++++++++++- 6 files changed, 159 insertions(+), 4 deletions(-)create mode 100644
arch/riscv/include/asm/stacktrace/frame.harch/riscv/include/asm/ptrace.h +9
-0@@ -8,6 +8,7 @@#include <uapi/asm/ptrace.h> #include <asm/csr.h> +#include <asm/stacktrace/frame.h> #include <linux/compiler.h> #ifndef __ASSEMBLER__@@ -53,6 +54,14 @@ struct pt_regs {unsigned long cause;/* a0 value before the syscall */ unsigned long orig_a0; + +/* +* This frame record is entirely zeroed on exception entry, allowing the +* unwinder to identify exception boundaries. The type field encodes +* whether the exception was taken from user (FINAL) or kernel (PT_REGS) +* mode. +*/ +struct frame_record_meta stackframe; };#define PTRACE_SYSEMU 0x1farch/riscv/include/asm/stacktrace/frame.h new file +53
-0@@ -0,0 +1,53 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +#ifndef __ASM_RISCV_STACKTRACE_FRAME_H +#define __ASM_RISCV_STACKTRACE_FRAME_H + +/* +* See: arch/arm64/include/asm/stacktrace/frame.h for the reference +* implementation. +*/ + +/* +* - FRAME_META_TYPE_NONE +* +* This value is reserved. +* +* - FRAME_META_TYPE_FINAL +* +* The record is the last entry on the stack. +* Unwinding should terminate successfully. +* +* - FRAME_META_TYPE_PT_REGS +* +* The record is embedded within a struct pt_regs, recording the registers at +* an arbitrary point in time. +* Unwinding should consume pt_regs::epc, followed by pt_regs::ra. +* +* Note: all other values are reserved and should result in unwinding +* terminating with an error. +*/ +#define FRAME_META_TYPE_NONE 0 +#define FRAME_META_TYPE_FINAL 1 +#define FRAME_META_TYPE_PT_REGS 2 + +#ifndef __ASSEMBLER__ +/* +* A standard RISC-V frame record. +*/ +struct frame_record { +unsigned long fp; +unsigned long ra; +}; + +/* +* A metadata frame record indicating a special unwind. +* The record::{fp,ra} fields must be zero to indicate the presence of +* metadata. +*/ +struct frame_record_meta { +struct frame_record record; +unsigned long type; +}; +#endif /* __ASSEMBLER__ */ + +#endif /* __ASM_RISCV_STACKTRACE_FRAME_H */ arch/riscv/kernel/asm-offsets.c +3
-0@@ -131,6 +131,9 @@ void asm_offsets(void)OFFSET (PT_BADADDR , pt_regs, badaddr);OFFSET (PT_CAUSE , pt_regs, cause); +DEFINE (S_STACKFRAME , offsetof(struct pt_regs , stackframe)); + DEFINE(S_STACKFRAME_TYPE, offsetof(struct pt_regs, stackframe.type)); +OFFSET (SUSPEND_CONTEXT_REGS , suspend_context, regs);OFFSET (HIBERN_PBE_ADDR , pbe, address);
arch/riscv/kernel/asm-offsets.c void asm_offsets(void) · line 509
Quoted 4 lines
arch/riscv/kernel/asm-offsets.c +1
-0@@ -503,6 +506,9 @@ void asm_offsets(void)DEFINE (STACKFRAME_SIZE_ON_STACK ,ALIGN (sizeof (struct stackframe ),STACK_ALIGN ));DEFINE (STACKFRAME_FP , offsetof(struct stackframe , fp) -sizeof (struct stackframe ));DEFINE (STACKFRAME_RA , offsetof(struct stackframe , ra) -sizeof (struct stackframe )); +DEFINE (STACKFRAME_RECORD_SIZE ,sizeof (struct stackframe ));
This should be sizeof(struct frame_record) rather than sizeof(struct stackframe). STACKFRAME_RECORD_SIZE is used to skip past the embedded frame_record_meta.record field:
addi s0, sp, S_STACKFRAME + STACKFRAME_RECORD_SIZE
The two structs happen to be identical today, but they are independent types serving different purposes — struct stackframe is the existing unwinder frame record, while struct frame_record is the one embedded in struct frame_record_meta introduced by this patch. If struct stackframe ever gains an extra field, this offset would silently break.
With that fixed:
Thanks.
Shuai
PATCH RESEND v4 2/7 riscv: stacktrace: disable KASAN and KCOV instrumentation for stacktrace.o
In reply to Wang HanKASAN records stack traces for every alloc/free, which means it walks the unwinder very frequently. Instrumenting the stack trace collection code itself adds substantial overhead and makes the traces themselves noisier.
KCOV instruments every basic-block edge. The unwinder is a hot path, especially with KASAN enabled, so KCOV instrumentation has the same kind of cost and noise problem here.
Mark stacktrace.o as not KASAN- or KCOV-instrumented, matching the x86 treatment of its stack unwinding code. RISC-V keeps the relevant unwinder code in stacktrace.o, so a single translation-unit annotation covers the equivalent scope. This is a prerequisite preference for the upcoming reliable unwinder, but the change is valid on its own.
--- arch/riscv/kernel/Makefile | 6 ++++++ 1 file changed, 6 insertions(+)
arch/riscv/kernel/Makefile +6 -0
@@ -44,6 +44,12 @@ CFLAGS_REMOVE_return_address.o = $(CC_FLAGS_FTRACE) CFLAGS_REMOVE_sbi_ecall.o = $(CC_FLAGS_FTRACE) endif +# When KASAN is enabled, a stack trace is recorded for every alloc/free, which +# can significantly impact performance. Avoid instrumenting the stack trace +# collection code to minimize this impact. +KASAN_SANITIZE_stacktrace.o := n +KCOV_INSTRUMENT_stacktrace.o := n + always-$(KBUILD_BUILTIN) += vmlinux.lds obj-y += head.o
Signature · 1 line
2.43.0
Re: [PATCH v4 RESEND 2/7] riscv: stacktrace: disable KASAN and KCOV instrumentation for stacktrace.o
Quoted 27 lines · Wang Han
KASAN records stack traces for every alloc/free, which means it walks the unwinder very frequently. Instrumenting the stack trace collection code itself adds substantial overhead and makes the traces themselves noisier.
KCOV instruments every basic-block edge. The unwinder is a hot path, especially with KASAN enabled, so KCOV instrumentation has the same kind of cost and noise problem here.
Mark stacktrace.o as not KASAN- or KCOV-instrumented, matching the x86 treatment of its stack unwinding code. RISC-V keeps the relevant unwinder code in stacktrace.o, so a single translation-unit annotation covers the equivalent scope. This is a prerequisite preference for the upcoming reliable unwinder, but the change is valid on its own.
Signed-off-by Wang Han <[email protected]>--- arch/riscv/kernel/Makefile | 6 ++++++ 1 file changed, 6 insertions(+)arch/riscv/kernel/Makefile +0
-0@@ -44,6 +44,12 @@ CFLAGS_REMOVE_return_address.o = $(CC_FLAGS_FTRACE) CFLAGS_REMOVE_sbi_ecall.o = $(CC_FLAGS_FTRACE) endif
arch/riscv/kernel/Makefile CFLAGS_REMOVE_return_address.o = $(CC_FLAGS_FTRACE) · line 55
Quoted 10 lines
arch/riscv/kernel/Makefile +6
-0+# When KASAN is enabled, a stack trace is recorded for every alloc/free, which +# can significantly impact performance. Avoid instrumenting the stack trace +# collection code to minimize this impact. +KASAN_SANITIZE_stacktrace.o := n +KCOV_INSTRUMENT_stacktrace.o := n + always-$(KBUILD_BUILTIN) += vmlinux.lds obj-y += head.o
Thanks.
Shuai
PATCH RESEND v4 3/7 riscv: ftrace: always preserve s0 in dynamic ftrace register frame
In reply to Wang Hanstruct __arch_ftrace_regs declares s0 unconditionally, and both ftrace_regs_get_frame_pointer() and ftrace_partial_regs() read it unconditionally. But the SAVE_ABI_REGS / RESTORE_ABI_REGS macros in mcount-dyn.S only stored s0 under HAVE_FUNCTION_GRAPH_FP_TEST (CONFIG_FUNCTION_GRAPH_TRACER && CONFIG_FRAME_POINTER). With CONFIG_FRAME_POINTER=n the slot held whatever was on the stack before, so any callback going through ftrace_partial_regs() saw a garbage regs->s0. RISC-V kernels default to FRAME_POINTER=y, which is why this has not bitten in practice.
Save and restore s0 unconditionally in the dynamic ftrace ABI register frame. This fixes the latent garbage-s0 case, brings the dynamic ftrace path in line with the static _mcount path (mcount.S SAVE_ABI_STATE already saves s0 unconditionally), and matches the frame layout already documented in the comment above SAVE_ABI_REGS. It is also a prerequisite for the upcoming reliable unwinder, which reads ftrace_regs_get_frame_pointer(fregs) directly.
The cost is one extra REG_S/REG_L pair per traced call, negligible compared to the overall ftrace cost; the existing FREGS_SIZE_ON_STACK already reserved the slot, so no extra stack space is used.
--- arch/riscv/kernel/mcount-dyn.S | 4 ---- 1 file changed, 4 deletions(-)
arch/riscv/kernel/mcount-dyn.S +0 -4
@@ -85,9 +85,7 @@ addi sp, sp, -FREGS_SIZE_ON_STACK REG_S t0, FREGS_EPC(sp) REG_S x1, FREGS_RA(sp) -#ifdef HAVE_FUNCTION_GRAPH_FP_TEST REG_S x8, FREGS_S0(sp) -#endif REG_S x6, FREGS_T1(sp) #ifdef CONFIG_CC_IS_CLANG REG_S x7, FREGS_T2(sp)
@@ -113,9 +111,7 @@ .macro RESTORE_ABI_REGS REG_L t0, FREGS_EPC(sp) REG_L x1, FREGS_RA(sp) -#ifdef HAVE_FUNCTION_GRAPH_FP_TEST REG_L x8, FREGS_S0(sp) -#endif REG_L x6, FREGS_T1(sp) #ifdef CONFIG_CC_IS_CLANG REG_L x7, FREGS_T2(sp)
Signature · 6 lines
2.43.0 _______________________________________________ linux-riscv mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-riscv
PATCH RESEND v4 4/7 riscv: stacktrace: introduce stack-bound tracking helpers
In reply to Wang HanA reliable unwinder needs to validate that every frame record it reads is fully contained in a known kernel stack, and it needs to refuse to walk back into a stack it has already left. Add the building blocks for that:
* struct stack_info / struct unwind_state in a new asm/stacktrace/common.h, modelled on the arm64 reference implementation.
* stackinfo_get_irq() / stackinfo_get_task() / stackinfo_get_overflow() plus the corresponding on_*_stack() predicates in asm/stacktrace.h, so callers can ask "is this object on stack X?" by stack kind rather than open-coded address arithmetic.
* unwind_init_common(), unwind_find_stack() and unwind_consume_stack() helpers that enforce the forward-progress-only invariant required for reliability.
No existing user is wired up to these helpers in this commit; the unwinder switch comes in a follow-up. The header changes leave on_thread_stack() with the same semantics as before, just expressed in terms of the new helpers.
--- arch/riscv/include/asm/stacktrace.h | 65 ++++++++- arch/riscv/include/asm/stacktrace/common.h | 159 +++++++++++++++++++++ 2 files changed, 222 insertions(+), 2 deletions(-) create mode 100644 arch/riscv/include/asm/stacktrace/common.h
arch/riscv/include/asm/stacktrace.h +63 -2
@@ -3,8 +3,13 @@ #ifndef _ASM_RISCV_STACKTRACE_H #define _ASM_RISCV_STACKTRACE_H +#include <linux/percpu.h> #include <linux/sched.h> +#include <linux/sched/task_stack.h> + +#include <asm/irq_stack.h> #include <asm/ptrace.h> +#include <asm/stacktrace/common.h> struct stackframe { unsigned long fp;
@@ -16,14 +21,70 @@ extern void notrace walk_stackframe(struct task_struct *task, struct pt_regs *reextern void dump_backtrace (struct pt_regs *regs,struct task_struct *task,const char *loglvl); -static inline bool on_thread_stack (void ) +/* +* IRQ stack accessors +*/ +static inline struct stack_info stackinfo_get_irq (void ) +{ +unsigned long low = (unsigned long )raw_cpu_read (irq_stack_ptr); +unsigned long high = low +IRQ_STACK_SIZE ; + +return (struct stack_info ) { + .low = low, + .high = high, + }; +} + +static inline bool on_irq_stack (unsigned long sp,unsigned long size) +{ +struct stack_info info =stackinfo_get_irq (); + +return stackinfo_on_stack (&info, sp, size); +} + +/* +* Task stack accessors +*/ +static inline struct stack_info stackinfo_get_task (const struct task_struct *tsk) { -return !(((unsigned long )(current->stack) ^ current_stack_pointer) & ~(THREAD_SIZE -1 )); +unsigned long low = (unsigned long )task_stack_page (tsk); +unsigned long high = low +THREAD_SIZE ; + +return (struct stack_info ) { + .low = low, + .high = high, + }; +} + +static inline bool on_task_stack (const struct task_struct *tsk, +unsigned long sp,unsigned long size) +{ +struct stack_info info =stackinfo_get_task (tsk); + +return stackinfo_on_stack (&info, sp, size); } +/* +* Cast is necessary since current->stack is an opaque ptr. +*/ +#define on_thread_stack () (on_task_stack(current, current_stack_pointer, 1)) +/* +* Overflow stack accessors +*/ #ifdef CONFIG_VMAP_STACK DECLARE_PER_CPU(unsigned long [OVERFLOW_STACK_SIZE/sizeof(long)], overflow_stack); + +static inline struct stack_info stackinfo_get_overflow (void ) +{ +unsigned long low = (unsigned long )raw_cpu_ptr (overflow_stack); +unsigned long high = low +OVERFLOW_STACK_SIZE ; + +return (struct stack_info ) { + .low = low, + .high = high, + }; +}#endif /* CONFIG_VMAP_STACK */ #endif /* _ASM_RISCV_STACKTRACE_H */
arch/riscv/include/asm/stacktrace/common.h new file +159 -0
159 lines not on this page · raw message
Signature · 1 line
2.43.0
[email protected]
AI reviewRe: [PATCH v4 RESEND 4/7] riscv: stacktrace: introduce stack-bound tracking helpers
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Low] Unconditional reference to conditionally defined irq_stack_ptr causes a link-time error when…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] Unconditional reference to conditionally defined irq_stack_ptr causes a link-time error when CONFIG_IRQ_STACKS is disabled. --
commit 64aa629bbd032157beeb94e2add8faf14c975e32
Author: Wang Han <[email protected]>
riscv: stacktrace: introduce stack-bound tracking helpers
This commit introduces building blocks for a reliable stack unwinder in RISC-V. It adds stack-bound tracking helpers such as stack_info and unwind_state, along with accessors for IRQ, task, and overflow stacks.
Quoted 4 lines
arch/riscv/include/asm/stacktrace.h +0
-0
[ ... ]
arch/riscv/include/asm/stacktrace.h extern void notrace walk_stackframe(struct task_struct *task, struct pt_regs *re · line 29
Quoted 11 lines
arch/riscv/include/asm/stacktrace.h +6
-1@@ -16,14 +21,70 @@ extern void notrace walk_stackframe(struct task_struct *task, struct pt_regs *reextern void dump_backtrace (struct pt_regs *regs,struct task_struct *task,const char *loglvl); -static inline bool on_thread_stack (void ) +/* +* IRQ stack accessors +*/ +static inline struct stack_info stackinfo_get_irq (void ) +{ +unsigned long low = (unsigned long )raw_cpu_read (irq_stack_ptr);
[Severity: Low]
Will this cause a link error when CONFIG_IRQ_STACKS is disabled?
Looking at arch/riscv/kernel/irq.c, irq_stack_ptr is defined inside an #ifdef CONFIG_IRQ_STACKS block. Since stackinfo_get_irq() unconditionally references it here, it seems this might result in an undefined reference to irq_stack_ptr during linking.
Should this accessor be conditionally compiled, or should it provide a fallback when CONFIG_IRQ_STACKS is not set?
Quoted 7 lines
arch/riscv/include/asm/stacktrace.h +7
-0+unsigned long high = low +IRQ_STACK_SIZE ; + +return (struct stack_info ) { + .low = low, + .high = high, + }; +}
[ ... ]
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
Re: [PATCH v4 RESEND 4/7] riscv: stacktrace: introduce stack-bound tracking helpers
In reply to Wang HanQuoted 48 lines · Wang Han
A reliable unwinder needs to validate that every frame record it reads is fully contained in a known kernel stack, and it needs to refuse to walk back into a stack it has already left. Add the building blocks for that:
* struct
stack_info/ structunwind_statein a newasm/stacktrace/common.h, modelled on the arm64 reference implementation.
*stackinfo_get_irq()/stackinfo_get_task()/stackinfo_get_overflow()plus the corresponding on_*_stack() predicates inasm/stacktrace.h, so callers can ask "is this object on stack X?" by stack kind rather than open-coded address arithmetic.
*unwind_init_common(),unwind_find_stack()andunwind_consume_stack()helpers that enforce the forward-progress-only invariant required for reliability.No existing user is wired up to these helpers in this commit; the unwinder switch comes in a follow-up. The header changes leave
on_thread_stack()with the same semantics as before, just expressed in terms of the new helpers.Reviewed-by Shuai Xue <[email protected]>Signed-off-by Wang Han <[email protected]>--- arch/riscv/include/asm/stacktrace.h | 65 ++++++++- arch/riscv/include/asm/stacktrace/common.h | 159 +++++++++++++++++++++ 2 files changed, 222 insertions(+), 2 deletions(-)create mode 100644
arch/riscv/include/asm/stacktrace/common.harch/riscv/include/asm/stacktrace.h +5
-0@@ -3,8 +3,13 @@ #ifndef _ASM_RISCV_STACKTRACE_H #define _ASM_RISCV_STACKTRACE_H +#include <linux/percpu.h> #include <linux/sched.h> +#include <linux/sched/task_stack.h> + +#include <asm/irq_stack.h> #include <asm/ptrace.h> +#include <asm/stacktrace/common.h> struct stackframe { unsigned long fp;
arch/riscv/include/asm/stacktrace.h extern void notrace walk_stackframe(struct task_struct *task, struct pt_regs *re · line 29
Quoted 10 lines
arch/riscv/include/asm/stacktrace.h +6
-1@@ -16,14 +21,70 @@ extern void notrace walk_stackframe(struct task_struct *task, struct pt_regs *reextern void dump_backtrace (struct pt_regs *regs,struct task_struct *task,const char *loglvl); -static inline bool on_thread_stack (void ) +/* +* IRQ stack accessors +*/ +static inline struct stack_info stackinfo_get_irq (void ) +{ +unsigned long low = (unsigned long )raw_cpu_read (irq_stack_ptr);
From https://sashiko.dev/#/patchset/20260629072713.3273743-1-wanghan%40linux.alibaba.com
Will this cause a link error when CONFIG_IRQ_STACKS is disabled? Looking at arch/riscv/kernel/irq.c, irq_stack_ptr is defined inside an #ifdef CONFIG_IRQ_STACKS block. Since stackinfo_get_irq() unconditionally references it here, it seems this might result in an undefined reference to irq_stack_ptr during linking.
Should this accessor be conditionally compiled, or should it provide a fallback when CONFIG_IRQ_STACKS is not set?
Thanks.
Shuai
_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
PATCH RESEND v4 5/7 riscv: stacktrace: switch to frame-pointer based unwinder
In reply to Wang HanReplace the open-coded frame-pointer walker in arch_stack_walk() with a robust kunwind state machine, modelled on arch/arm64/kernel/stacktrace.c and retargeted to the RISC-V {fp, ra} frame record convention. The new walker tracks stack bounds, consumes frame records monotonically, understands the metadata pt_regs records added in the previous frame record metadata patch, and recovers return addresses replaced by function graph tracing and kretprobes.
This commit introduces arch_stack_walk_reliable() but does not yet select HAVE_RELIABLE_STACKTRACE; that is done in a follow-up Kconfig patch so this commit can be reviewed and bisected as a pure unwinder replacement. Until that Kconfig change lands, livepatch is not yet enabled and arch_stack_walk_reliable() has no in-tree caller.
Three related callers are updated to keep the same frame-record assumptions everywhere:
* Function graph tracing: the old RISC-V unwinder matched function graph return-stack entries by the saved return-address slot. That was consistent with the static mcount path, but not with the dynamic ftrace path where the parent slot is ftrace_regs::ra. Use the architectural frame pointer as the function graph return-address cookie, matching the kunwind walker.
* Perf callchains: route kernel callchain collection through arch_stack_walk() so perf sees the same frame-pointer unwind behaviour as dump_stack() and the upcoming livepatch path.
* dump_backtrace() / __get_wchan() / show_stack(): these now go through arch_stack_walk(); the explicit "Call Trace:" header is moved into dump_backtrace() to preserve the original output.
The non-frame-pointer fallback walker is kept untouched for !CONFIG_FRAME_POINTER builds.
--- arch/riscv/kernel/ftrace.c | 6 +- arch/riscv/kernel/perf_callchain.c | 2 +- arch/riscv/kernel/stacktrace.c | 559 ++++++++++++++++++++++++----- 3 files changed, 471 insertions(+), 96 deletions(-)
arch/riscv/kernel/ftrace.c +4 -2
@@ -242,7 +242,8 @@ void prepare_ftrace_return(unsigned long *parent, unsigned long self_addr,*/ old = *parent; -if (!function_graph_enter (old, self_addr, frame_pointer, parent)) +if (!function_graph_enter (old, self_addr, frame_pointer, + (void *)frame_pointer)) *parent = return_hooker; }
@@ -264,7 +265,8 @@ void ftrace_graph_func(unsigned long ip, unsigned long parent_ip,*/ old = *parent; -if (!function_graph_enter_regs (old, ip, frame_pointer, parent, fregs)) +if (!function_graph_enter_regs (old, ip, frame_pointer, + (void *)frame_pointer, fregs)) *parent = return_hooker; }#endif /* CONFIG_DYNAMIC_FTRACE */
arch/riscv/kernel/perf_callchain.c +1 -1
@@ -44,5 +44,5 @@ void perf_callchain_kernel(struct perf_callchain_entry_ctx *entry,return ; } -walk_stackframe (NULL , regs, fill_callchain, entry); +arch_stack_walk (fill_callchain, entry,NULL , regs); }
arch/riscv/kernel/stacktrace.c +466 -93
600 lines not on this page · raw message
Signature · 1 line
2.43.0
Re: [PATCH v4 RESEND 5/7] riscv: stacktrace: switch to frame-pointer based unwinder
Quoted 675 lines · Wang Han
Replace the open-coded frame-pointer walker in
arch_stack_walk()with a robust kunwind state machine, modelled onarch/arm64/kernel/stacktrace.cand retargeted to the RISC-V {fp, ra} frame record convention. The new walker tracks stack bounds, consumes frame records monotonically, understands the metadatapt_regsrecords added in the previous frame record metadata patch, and recovers return addresses replaced by function graph tracing and kretprobes.This commit introduces
arch_stack_walk_reliable()but does not yet selectHAVE_RELIABLE_STACKTRACE; that is done in a follow-upKconfigpatch so this commit can be reviewed and bisected as a pure unwinder replacement. Until thatKconfigchange lands, livepatch is not yet enabled andarch_stack_walk_reliable()has no in-tree caller.Three related callers are updated to keep the same frame-record assumptions everywhere:
* Function graph tracing: the old RISC-V unwinder matched function graph return-stack entries by the saved return-address slot. That was consistent with the static mcount path, but not with the dynamic ftrace path where the parent slot is
ftrace_regs::ra. Use the architectural frame pointer as the function graph return-address cookie, matching the kunwind walker.* Perf callchains: route kernel callchain collection through
arch_stack_walk()so perf sees the same frame-pointer unwind behaviour asdump_stack()and the upcoming livepatch path.*
dump_backtrace()/__get_wchan()/show_stack(): these now go througharch_stack_walk(); the explicit "Call Trace:" header is moved intodump_backtrace()to preserve the original output.The non-frame-pointer fallback walker is kept untouched for !CONFIG_FRAME_POINTER builds.
Signed-off-by Wang Han <[email protected]>--- arch/riscv/kernel/ftrace.c | 6 +- arch/riscv/kernel/perf_callchain.c | 2 +- arch/riscv/kernel/stacktrace.c | 559 ++++++++++++++++++++++++----- 3 files changed, 471 insertions(+), 96 deletions(-)arch/riscv/kernel/ftrace.c +4
-2@@ -242,7 +242,8 @@ void prepare_ftrace_return(unsigned long *parent, unsigned long self_addr,*/ old = *parent; -if (!function_graph_enter (old, self_addr, frame_pointer, parent)) +if (!function_graph_enter (old, self_addr, frame_pointer, + (void *)frame_pointer)) *parent = return_hooker; }@@ -264,7 +265,8 @@ void ftrace_graph_func(unsigned long ip, unsigned long parent_ip,*/ old = *parent; -if (!function_graph_enter_regs (old, ip, frame_pointer, parent, fregs)) +if (!function_graph_enter_regs (old, ip, frame_pointer, + (void *)frame_pointer, fregs)) *parent = return_hooker; }#endif /* CONFIG_DYNAMIC_FTRACE */ arch/riscv/kernel/perf_callchain.c +1
-1@@ -44,5 +44,5 @@ void perf_callchain_kernel(struct perf_callchain_entry_ctx *entry,return ; } -walk_stackframe (NULL , regs, fill_callchain, entry); +arch_stack_walk (fill_callchain, entry,NULL , regs); }arch/riscv/kernel/stacktrace.c +459
-93@@ -11,98 +11,16 @@#include <linux/sched/task_stack.h> #include <linux/stacktrace.h> #include <linux/ftrace.h> +#include <linux/kprobes.h> +#include <linux/llist.h> #include <asm/stacktrace.h> -#ifdef CONFIG_FRAME_POINTER -/* -* This disables KASAN checking when reading a value from another task's stack, -* since the other task could be running on another CPU and could have poisoned -* the stack in the meantime. +* Non-frame-pointer fallback unwinder. +* Only compiled when CONFIG_FRAME_POINTER is not enabled. */ -#define READ_ONCE_TASK_STACK (task, x) \ -({ \ - unsigned long val; \ - unsigned long addr = x; \ - if ((task) == current) \ - val = READ_ONCE(addr); \ - else \ - val = READ_ONCE_NOCHECK(addr); \ - val; \ -}) - -extern asmlinkage void handle_exception(void); -extern unsigned long ret_from_exception_end; - -static inline int fp_is_valid (unsigned long fp,unsigned long sp) -{ -unsigned long low, high; - - low = sp +sizeof (struct stackframe ); - high =ALIGN (sp,THREAD_SIZE ); - -return !(fp < low || fp > high || fp &0x07 ); -} - -void notrace walk_stackframe(struct task_struct *task, struct pt_regs *regs, -bool (*fn)(void *,unsigned long ),void *arg) -{ -unsigned long fp, sp, pc; -int graph_idx =0 ; -int level =0 ; - -if (regs) { - fp =frame_pointer (regs); - sp =user_stack_pointer (regs); - pc =instruction_pointer (regs); - }else if (task ==NULL || task == current) { - fp = (unsigned long )__builtin_frame_address (0 ); - sp = current_stack_pointer; - pc = (unsigned long )walk_stackframe; - level =-1 ; - }else { -/* task blocked in __switch_to */ - fp = task->thread.s[0 ]; - sp = task->thread.sp; - pc = task->thread.ra; - } - -for (;;) { -struct stackframe *frame; - -if (unlikely (!__kernel_text_address (pc) || (level++ >=0 && !fn (arg, pc)))) -break ; - -if (unlikely (!fp_is_valid (fp, sp))) -break ; - -/* Unwind stack frame */ - frame = (struct stackframe *)fp -1 ; - sp = fp; -if (regs && (regs->epc == pc) &&fp_is_valid (frame->ra, sp)) { -/* We hit function where ra is not saved on the stack */ - fp = frame->ra; - pc = regs->ra; - }else { - fp =READ_ONCE_TASK_STACK (task, frame->fp); - pc =READ_ONCE_TASK_STACK (task, frame->ra); - pc =ftrace_graph_ret_addr (task, &graph_idx, pc, - &frame->ra); -if (pc >= (unsigned long )handle_exception && - pc < (unsigned long )&ret_from_exception_end) { -if (unlikely (!fn (arg, pc))) -break ; - - pc = ((struct pt_regs *)sp)->epc; - fp = ((struct pt_regs *)sp)->s0; - } - } - - } -} - -#else /* !CONFIG_FRAME_POINTER */ +#ifndef CONFIG_FRAME_POINTER void notrace walk_stackframe(struct task_struct *task,struct pt_regs *regs,bool (*fn)(void *,unsigned long ),void *arg)@@ -133,7 +51,12 @@ void notrace walk_stackframe(struct task_struct *task, } } -#endif /* CONFIG_FRAME_POINTER */ +#endif /* !CONFIG_FRAME_POINTER */ + +/* +* Common trace helpers. +* These are used by both the FP (kunwind) and non-FP (walk_stackframe) paths. +*/ static bool print_trace_address (void *arg,unsigned long pc) {@@ -146,12 +69,12 @@ static bool print_trace_address(void *arg, unsigned long pc) noinline void dump_backtrace(struct pt_regs *regs, struct task_struct *task,const char *loglvl) { -walk_stackframe (task, regs, print_trace_address, (void *)loglvl); +printk ("%sCall Trace:\n" , loglvl); +arch_stack_walk (print_trace_address, (void *)loglvl, task, regs); }void show_stack (struct task_struct *task,unsigned long *sp,const char *loglvl) { -pr_cont ("%sCall Trace:\n" , loglvl);dump_backtrace (NULL , task, loglvl); }@@ -171,17 +94,467 @@ unsigned long __get_wchan(struct task_struct *task)if (!try_get_task_stack (task))return 0 ; -walk_stackframe (task,NULL , save_wchan, &pc); +arch_stack_walk (save_wchan, &pc, task,NULL );put_task_stack (task);return pc; } -noinline noinstr void arch_stack_walk(stack_trace_consume_fn consume_entry, void *cookie, -struct task_struct *task,struct pt_regs *regs) +/* +* Frame-pointer-based kernel unwind infrastructure. +* Only compiled when CONFIG_FRAME_POINTER is enabled. +* +* See: arch/arm64/kernel/stacktrace.c for the reference implementation. +*/ +#ifdef CONFIG_FRAME_POINTER + +/* +* Per-cpu stacks are only accessible when unwinding the current task in a +* non-preemptible context. +*/ +#define STACKINFO_CPU (task, name) \ + ({ \ + (((task) == current) && !preemptible()) \ + ? stackinfo_get_##name() \ + : stackinfo_get_unknown(); \ + }) + +enum kunwind_source { +KUNWIND_SOURCE_UNKNOWN , +KUNWIND_SOURCE_FRAME , +KUNWIND_SOURCE_CALLER , +KUNWIND_SOURCE_TASK , +KUNWIND_SOURCE_REGS_PC , +}; + +union unwind_flags { +unsigned long all; +struct { +unsigned long fgraph :1 , + kretprobe :1 ; + }; +}; + +/* +* Kernel unwind state +* +* @common: Common unwind state. +* @task: The task being unwound. +* @graph_idx: Used by ftrace_graph_ret_addr() for optimized stack unwinding. +* @kr_cur: When KRETPROBES is selected, holds the kretprobe instance +* associated with the most recently encountered replacement ra +* value. +*/ +struct kunwind_state { +struct unwind_state common; +struct task_struct *task; +int graph_idx; +#ifdef CONFIG_KRETPROBES +struct llist_node *kr_cur; +#endif +enum kunwind_source source; +union unwind_flags flags; +struct pt_regs *regs; +}; + +static __always_inline void +kunwind_init (struct kunwind_state *state, +struct task_struct *task) +{ +unwind_init_common (&state->common); + state->task = task; + state->source =KUNWIND_SOURCE_UNKNOWN ; + state->flags.all =0 ; + state->regs =NULL ; +} + +/* +* Start an unwind from a pt_regs. +* +* The unwind will begin at the PC within the regs. +* +* The regs must be on a stack currently owned by the calling task. +*/ +static __always_inline void +kunwind_init_from_regs (struct kunwind_state *state, +struct pt_regs *regs) +{ +kunwind_init (state, current); + + state->regs = regs; + state->common.fp =frame_pointer (regs); + state->common.pc =instruction_pointer (regs); + state->source =KUNWIND_SOURCE_REGS_PC ; +} + +/* +* Start an unwind from a caller. +* +* The unwind will begin at the caller of whichever function this is inlined +* into. +* +* The function which invokes this must be noinline. +*/ +static __always_inline void +kunwind_init_from_caller (struct kunwind_state *state) +{ +unsigned long fp = (unsigned long )__builtin_frame_address (0 ); +struct frame_record *record = (struct frame_record *)fp -1 ; + +kunwind_init (state, current); + + state->common.fp =READ_ONCE (record->fp); + state->common.pc =READ_ONCE (record->ra); + state->source =KUNWIND_SOURCE_CALLER ; +} + +/* +* Start an unwind from a blocked task. +* +* The unwind will begin at the blocked task's saved PC (i.e. the caller of +* __switch_to). +* +* The caller should ensure the task is blocked in __switch_to for the +* duration of the unwind, or the unwind will be bogus. It is never valid to +* call this for the current task. +*/ +static __always_inline void +kunwind_init_from_task (struct kunwind_state *state, +struct task_struct *task) +{ +kunwind_init (state, task); + + state->common.fp = task->thread.s[0 ]; + state->common.pc = task->thread.ra; + state->source =KUNWIND_SOURCE_TASK ; +} + +static __always_inline int +kunwind_recover_return_address (struct kunwind_state *state) +{ +#ifdef CONFIG_FUNCTION_GRAPH_TRACER +if (state->task->ret_stack && + state->common.pc == (unsigned long )return_to_handler) { +unsigned long orig_pc; + + orig_pc =ftrace_graph_ret_addr (state->task, &state->graph_idx, + state->common.pc, + (void *)state->common.fp); +if (state->common.pc == orig_pc) { +WARN_ON_ONCE (state->task == current); +return -EINVAL ; + } + state->common.pc = orig_pc; + state->flags.fgraph =1 ; + } +#endif /* CONFIG_FUNCTION_GRAPH_TRACER */ + +#ifdef CONFIG_KRETPROBES +if (is_kretprobe_trampoline (state->common.pc)) { +unsigned long orig_pc; + + orig_pc =kretprobe_find_ret_addr (state->task, + (void *)state->common.fp, + &state->kr_cur); +if (!orig_pc) +return -EINVAL ; + state->common.pc = orig_pc; + state->flags.kretprobe =1 ; + } +#endif /* CONFIG_KRETPROBES */ + +return 0 ; +} + +/* +* When we reach an exception boundary marked by a metadata frame record, +* extract pt_regs from the stack and continue unwinding from the saved +* context (epc and s0/fp). +* +* On RISC-V, fp points above the metadata record, so the record's +* frame_record portion is at fp - sizeof(struct frame_record). +*/ +static __always_inline int +kunwind_next_regs_pc (struct kunwind_state *state) +{ +struct stack_info *info; +unsigned long fp = state->common.fp; +struct pt_regs *regs; + + regs =container_of ((unsigned long *)(fp -sizeof (struct frame_record )), + struct pt_regs, stackframe.record.fp); + + info =unwind_find_stack (&state->common, (unsigned long )regs, +sizeof (*regs)); +if (!info) +return -EINVAL ; + +unwind_consume_stack (&state->common, info, (unsigned long )regs, +sizeof (*regs)); + + state->regs = regs; + state->common.pc = regs->epc; + state->common.fp =frame_pointer (regs); + state->source =KUNWIND_SOURCE_REGS_PC ; +return 0 ; +} + +/* +* Handle a metadata frame record embedded in pt_regs. +* +* On RISC-V, fp points above the record (fp = metadata + 16), so the +* frame_record_meta starts at fp - sizeof(struct frame_record). +* +* FRAME_META_TYPE_FINAL: This is the outermost exception entry +* (user -> kernel). Unwinding terminates successfully. +* FRAME_META_TYPE_PT_REGS: This is a nested exception entry +* (kernel -> kernel). Continue unwinding from the saved context. +*/ +static __always_inline int +kunwind_next_frame_record_meta (struct kunwind_state *state) +{ +struct task_struct *tsk = state->task; +unsigned long fp = state->common.fp; +unsigned long meta_base = fp -sizeof (struct frame_record ); +struct frame_record_meta *meta; +struct stack_info *info; + + info =unwind_find_stack (&state->common, meta_base,sizeof (*meta)); +if (!info) +return -EINVAL ; + + meta = (struct frame_record_meta *)meta_base; +switch (READ_ONCE (meta->type)) { +case FRAME_META_TYPE_FINAL : +if (meta == &task_pt_regs (tsk)->stackframe) +return -ENOENT ; +WARN_ON_ONCE (tsk == current); +return -EINVAL ; +case FRAME_META_TYPE_PT_REGS : +return kunwind_next_regs_pc (state); +default : +WARN_ON_ONCE (tsk == current); +return -EINVAL ; + } +} + +/* +* Unwind from one frame record to the next. +* +* On RISC-V, the frame record sits at fp - sizeof(struct frame_record), +* immediately below the address pointed to by fp/s0. This applies to both +* normal frame records and metadata frame records (embedded in pt_regs). +* +* A metadata record is identified by both fp and ra being zero in the +* frame_record portion, with a type value following at fp + 16. +*/ +static __always_inline int +kunwind_next_frame_record (struct kunwind_state *state) +{ +unsigned long fp = state->common.fp; +struct frame_record *record; +struct stack_info *info; +unsigned long new_fp, new_pc; +unsigned long record_base; + +if (fp &0x7 ) +return -EINVAL ; + + record_base = fp -sizeof (*record); + + info =unwind_find_stack (&state->common, record_base,sizeof (*record)); +if (!info) +return -EINVAL ; + + record = (struct frame_record *)record_base; + new_fp =READ_ONCE (record->fp); + new_pc =READ_ONCE (record->ra); + +if (!new_fp && !new_pc) +return kunwind_next_frame_record_meta (state); + +unwind_consume_stack (&state->common, info, record_base, +sizeof (*record)); + + state->common.fp = new_fp; + state->common.pc = new_pc; + state->source =KUNWIND_SOURCE_FRAME ; + +return 0 ; +} + +/* +* Unwind from one frame record (A) to the next frame record (B). +* +* We terminate early if the location of B indicates a malformed chain of frame +* records (e.g. a cycle), determined based on the location and fp value of A +* and the location (but not the fp value) of B. +*/ +static __always_inline int +kunwind_next (struct kunwind_state *state) +{ +int err; + + state->flags.all =0 ; + +switch (state->source) { +case KUNWIND_SOURCE_FRAME : +case KUNWIND_SOURCE_CALLER : +case KUNWIND_SOURCE_TASK : +case KUNWIND_SOURCE_REGS_PC : + err =kunwind_next_frame_record (state); +break ; +default : + err = -EINVAL ; + } + +if (err) +return err; + +return kunwind_recover_return_address (state); +} + +typedef bool (*kunwind_consume_fn )(const struct kunwind_state *state,void *cookie); + +static __always_inline int +do_kunwind (struct kunwind_state *state,kunwind_consume_fn consume_state, +void *cookie) +{ +int ret; + + ret =kunwind_recover_return_address (state); +if (ret) +return ret; + +while (1 ) { +if (!consume_state (state, cookie)) +return -EINVAL ; + ret =kunwind_next (state); +if (ret == -ENOENT ) +return 0 ; +if (ret <0 ) +return ret; + } +} + +static __always_inline int +kunwind_stack_walk (kunwind_consume_fn consume_state, +void *cookie,struct task_struct *task, +struct pt_regs *regs) +{ +struct task_struct *tsk = task ?: current; +struct stack_info stacks[] = { +stackinfo_get_task (tsk), +STACKINFO_CPU (tsk, irq), +#ifdef CONFIG_VMAP_STACK +STACKINFO_CPU (tsk, overflow), +#endif + }; +struct kunwind_state state = { + .common = { + .stacks = stacks, + .nr_stacks =ARRAY_SIZE (stacks), + }, + }; + +if (regs) { +if (tsk != current) +return -EINVAL ; +kunwind_init_from_regs (&state, regs); + }else if (tsk == current) { +kunwind_init_from_caller (&state); + }else { +kunwind_init_from_task (&state, tsk); + } + +return do_kunwind (&state, consume_state, cookie); +} + +struct kunwind_consume_entry_data { +stack_trace_consume_fn consume_entry; +void *cookie; +}; + +static __always_inline bool +arch_kunwind_consume_entry (const struct kunwind_state *state,void *cookie) +{ +struct kunwind_consume_entry_data *data = cookie; + +return data->consume_entry (data->cookie, state->common.pc); +} + +static __always_inline bool +arch_reliable_kunwind_consume_entry (const struct kunwind_state *state,void *cookie) +{ +/* +* At an exception boundary we can reliably consume the saved PC. We do +* not know whether ra was live when the exception was taken, and +* so we cannot perform the next unwind step reliably. +* +* All that matters is whether the *entire* unwind is reliable, so give +* up as soon as we hit an exception boundary. +*/ +if (state->source ==KUNWIND_SOURCE_REGS_PC ) +return false; + +return arch_kunwind_consume_entry (state, cookie); +} + +#endif /* CONFIG_FRAME_POINTER */ + +/* +* arch_stack_walk - dual implementation. +* +* When CONFIG_FRAME_POINTER is enabled, uses the kunwind infrastructure for +* robust frame-pointer-based unwinding, consistent with arch_stack_walk_reliable. +* +* When CONFIG_FRAME_POINTER is disabled, falls back to the simple stack scan +* in walk_stackframe(). +*/ +#ifdef CONFIG_FRAME_POINTER + +noinline noinstr void arch_stack_walk(stack_trace_consume_fn consume_entry, +void *cookie,struct task_struct *task, +struct pt_regs *regs) +{ +struct kunwind_consume_entry_data data = { + .consume_entry = consume_entry, + .cookie = cookie, + }; + +kunwind_stack_walk (arch_kunwind_consume_entry, &data, task, regs); +} + +#else + +noinline noinstr void arch_stack_walk(stack_trace_consume_fn consume_entry, +void *cookie,struct task_struct *task, +struct pt_regs *regs) {walk_stackframe (task, regs, consume_entry, cookie); } +#endif /* CONFIG_FRAME_POINTER */ + +/* +* Reliable stack walk for livepatch (CONFIG_FRAME_POINTER only). +*/ +#ifdef CONFIG_FRAME_POINTER + +noinline noinstr int arch_stack_walk_reliable(stack_trace_consume_fn consume_entry, +void *cookie, +struct task_struct *task) +{ +struct kunwind_consume_entry_data data = { + .consume_entry = consume_entry, + .cookie = cookie, + };
arch/riscv/kernel/stacktrace.c unsigned long __get_wchan(struct task_struct *task) · line 560
Quoted 10 lines
arch/riscv/kernel/stacktrace.c +7
-0+ +return kunwind_stack_walk (arch_reliable_kunwind_consume_entry, &data, + task, NULL); +} + +#endif /* CONFIG_FRAME_POINTER */ +/* * Get the return address for a single stackframe and return a pointer to the * next frame tail.
LGTM.
Thanks.
Shuai
PATCH RESEND v4 6/7 riscv: Kconfig: enable HAVE_RELIABLE_STACKTRACE and HAVE_LIVEPATCH
In reply to Wang HanNow that the metadata frame records, the kunwind state machine and arch_stack_walk_reliable() are all in place, advertise the capability to the rest of the kernel:
* select HAVE_RELIABLE_STACKTRACE under FRAME_POINTER && 64BIT, so only the configurations with the tested metadata records and FP-based reliable walker enable it.
* select HAVE_LIVEPATCH under the same condition and source kernel/livepatch/Kconfig so the livepatch menu is reachable from the RISC-V configuration.
The 64BIT dependency is conservative scoping rather than a hard technical requirement: the metadata frame record, kunwind state machine and arch_stack_walk_reliable() also build on RV32, and the IRQ-stack frame-record adjustment fixes a latent RV32 issue. However, the syscall livepatch selftest and module relocation path have only been exercised on RV64 QEMU virt so far. The 64BIT gate can be relaxed in a follow-up once RV32 has equivalent coverage.
This is split out from the unwinder change so the policy decision and the implementation can be reviewed and reverted independently.
--- arch/riscv/Kconfig | 4 ++++ 1 file changed, 4 insertions(+)
arch/riscv/Kconfig +4 -0
@@ -186,6 +186,7 @@ config RISCV select HAVE_KRETPROBES # https://github.com/ClangBuiltLinux/linux/issues/1881 select HAVE_LD_DEAD_CODE_DATA_ELIMINATION if !LD_IS_LLD + select HAVE_LIVEPATCH if FRAME_POINTER && 64BIT select HAVE_MOVE_PMD select HAVE_MOVE_PUD select HAVE_PAGE_SIZE_4KB
@@ -196,6 +197,7 @@ config RISCV select HAVE_POSIX_CPU_TIMERS_TASK_WORK select HAVE_PREEMPT_DYNAMIC_KEY select HAVE_REGS_AND_STACK_ACCESS_API + select HAVE_RELIABLE_STACKTRACE if FRAME_POINTER && 64BIT select HAVE_RETHOOK select HAVE_RSEQ select HAVE_RUST if RUSTC_SUPPORTS_RISCV && CC_IS_CLANG
@@ -1392,3 +1394,5 @@ endmenu # "CPU Power Management" source "arch/riscv/kvm/Kconfig" source "drivers/acpi/Kconfig" + +source "kernel/livepatch/Kconfig"
Signature · 6 lines
2.43.0 _______________________________________________ linux-riscv mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-riscv
PATCH RESEND v4 7/7 selftests/livepatch: Add RISC-V syscall wrapper prefix
In reply to Wang HanThe syscall livepatch selftest resolves and patches a syscall wrapper symbol. To use that test for RISC-V livepatch validation, add the RISC-V FN_PREFIX definition for ARCH_HAS_SYSCALL_WRAPPER.
Without this macro, the syscall livepatch selftest cannot resolve the RISC-V target symbol, and the syscall-related livepatch test fails on RISC-V.
--- .../testing/selftests/livepatch/test_modules/test_klp_syscall.c | 2 ++ 1 file changed, 2 insertions(+)
tools/testing/selftests/livepatch/test_modules/test_klp_syscall.c +2 -0
@@ -24,6 +24,8 @@#define FN_PREFIX __s390x_#elif defined(__aarch64__)#define FN_PREFIX __arm64_ +#elif defined(__riscv) +#define FN_PREFIX __riscv_#elif defined(__powerpc__)#define FN_PREFIX #else
Signature · 1 line
2.43.0