Skip to content

riscv: bpf: Add BPF private stack support

Chen Pei bpf linux-kselftest linux-riscv lkml 3 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v3 Latest3 patches 1 replyIn review
  2. v23 patches 5 repliesSuperseded
  3. v13 patches 1 replySuperseded

Changes in v3 from the cover letter

- Patch 1: PRIV_STACK_GUARD_VAL moved to bpf_jit.h.
- Patch 3: The test redone as two asm programs in progs/verifier_stack_arg.c.

Patches in v3 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/3riscv: bpf: Add BPF private stack supportarch/riscv/net/bpf_jit.h arch/riscv/net/bpf_jit_comp64.c arch/riscv/net/bpf_jit_core.c+114 -120 replies
2/3selftests/bpf: Enable private stack tests for riscv64tools/testing/selftests/bpf/prog_tests/struct_ops_private_stack.c tools/testing/selftests/bpf/progs/verifier_private_stack.c+42 -20 replies
3/3selftests/bpf: Test a private stack together with stack argumentstools/testing/selftests/bpf/progs/verifier_stack_arg.c+49 -01 reply

Reviews trailers per patch

All 3 patches have a Reviewed-by or Acked-by.

PatchPu Lehui
1/3R-b
2/3R-b
3/3R-b

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v3 thread

Cover letter [PATCH v3 bpf-next 0/3] riscv: bpf: Add BPF private stack support

This series adds private-stack support to the RV64 BPF JIT.

The verifier rejects a struct_ops program that requests a private stack with -EACCES when the JIT does not report bpf_jit_supports_private_stack(), and sched_ext requests one for its dispatch callback, so no sched_ext scheduler can load on riscv64 today.

Allocation, the guard value and the lifetime handling follow arm64, x86-64 and powerpc64; the 16 byte two-copy guard layout follows arm64 and powerpc64, since x86-64 uses an 8 byte guard with a single copy. What is specific to riscv64 is where the pointers end up:

- Every BPF stack access goes through BPF_REG_FP (S5), so pointing S5 at the top of the per-CPU area and leaving SP on the kernel stack is all the JIT has to do. S5 is callee-saved and each subprog derives its own, so bpf2bpf calls need no extra save and restore.
- The outgoing stack argument area stays in the kernel frame: callers write it relative to SP while callees read it relative to RV_REG_FP. Unlike x86-64, no register here is shared with the private-stack frame pointer, so no verifier-side exclusion is needed and stack arguments keep working together with a private stack. arm64 already does the same, and patch 3 tests that combination.
- The per-CPU lookup is emitted after the tail call entry point. A program using a private stack can still be entered as a tail call target, at RV_TAILCALL_OFFSET, so it has to set up its own S5 there; and since that offset is a constant added to the target's bpf_func, everything in front of it must be the same size in every program.

The base address is materialized with absolute moves, as on arm64, x86-64 and powerpc64, which reach the per-CPU offset through gs:, TPIDR or r13; riscv64 has no such register, so the address of __per_cpu_offset is loaded too. Patch 1 has the detail, including why emit_imm() is a choice there and why the prologue length stays fixed across passes.

RV32 does not declare support and keeps the existing behaviour; the prologue signature is unchanged, so bpf_jit_comp32.c needs no edit.

Changes since v2:
- Patch 1: PRIV_STACK_GUARD_VAL moved to bpf_jit.h.
- Patch 3: The test redone as two asm programs in progs/verifier_stack_arg.c.

Changes since v1:
- Patch 1 commit message: guard layout credited to arm64 and powerpc64, value alone to x86-64; __per_cpu_offset described as a choice.
- Patch 3: test_private_stack_runtime() renamed to serial_test_private_stack_runtime(), and the stack argument skip prints a reason.

Testing
=======
QEMU virt rv64, an 8 vCPU SMP kernel and a 1 vCPU UP kernel, both built on the base commit below, source tree clean at the head of this series, selftests built from these patches. There are no private-stack related failures, and the private-stack suites logged no BUG, WARNING, unknown opcode or guard corruption. On UP, kfunc_call/subprog, kfunc_call/subprog_lskel and scx_simple are not run: all three fail on the unpatched base as well.

- verifier_private_stack, struct_ops_private_stack.
- tailcalls, stack_arg, subprogs, fentry_fexit, kfunc_call, percpu_alloc, async_stack_depth, arena_atomics, arena_ldsx.
- verifier_stack_arg, stack_arg_precision.
- may_goto_priv_stack, may_goto_far, verifier_may_goto_1, verifier_may_goto_2.
- scx_simple.
- sched_ext runner sweep on SMP: 40 cases, 39 pass. enable_cmask fails because its ops.enable() takes a struct scx_cmask __arena pointer and riscv64 does not implement bpf_jit_supports_arena_args().
- patch 3's test again on x86-64 and arm64, so all three architectures it is gated on have run it.

Chen Pei (3):
riscv: bpf: Add BPF private stack support selftests/bpf: Enable private stack tests for riscv64 selftests/bpf: Test a private stack together with stack arguments

 arch/riscv/net/bpf_jit.h                      |  2 +
 arch/riscv/net/bpf_jit_comp64.c               | 42 +++++++++-
 arch/riscv/net/bpf_jit_core.c                 | 82 +++++++++++++++++--
 .../bpf/prog_tests/struct_ops_private_stack.c |  3 +-
 .../bpf/progs/verifier_private_stack.c        | 41 +++++++++-
 .../selftests/bpf/progs/verifier_stack_arg.c  | 49 +++++++++++
 6 files changed, 205 insertions(+), 14 deletions(-)

base-commit: a3c8813409f13779fec225468aba7d4067a66468

Signature · 1 line
2.50.1