LoongArch: BPF: per-CPU address resolution and timed may_goto
Subsystems (3)
v3 cover LoongArch: BPF: per-CPU address resolution and timed may_goto
These are the first two patches of the LoongArch BPF JIT feature work, sent on their own to make review easier. The remaining pieces (per-program private stacks, exceptions/bpf_throw, sign-extending loads and atomics on arena pointers, and the matching selftests) will follow in a later series.
Both patches are independent of each other. They apply on top of Chenguang Zhao's "LoongArch bpf kptr xchg inline support" v4 series:
https://lore.kernel.org/all/[email protected]/
Patch 1 adds the internal-only BPF MOV that resolves a per-CPU offset to the current CPU's address, advertised through bpf_jit_supports_percpu_insn(). LoongArch keeps the current CPU's per-CPU base in $r21 (__my_cpu_offset), so the resolution is a single add. Exercised by the cpumask and percpu_alloc selftests.
Patch 2 implements arch_bpf_timed_may_goto() and advertises it, so the verifier lowers may_goto to its timed variant, which bounds the loop by a wall-clock timeout kept in a per-loop stack slot. It uses a custom calling
convention: the count/timestamp stack offset is passed in BPF_REG_AX and returned there, so the JIT call path skips the usual "BPF_REG_0 = return value" move for this helper. Exercised by the iters selftests.
Selftest results (all PASS):
Feature Test(s)
--------------------------------------------------------------
1 internal-only MOV (percpu) cpumask, percpu_alloc 2 timed may_goto iters
$ sudo ./test_progs -t cpumask ... Summary: 1/35 PASSED, 0 SKIPPED, 0 FAILED
$ sudo ./test_progs -t percpu_alloc ... Summary: 1/18 PASSED, 0 SKIPPED, 0 FAILED
$ sudo ./test_progs -t iters ... Summary: 1/93 PASSED, 0 SKIPPED, 0 FAILED
Based on loongarch-next:
https://git.kernel.org/pub/scm/linux/kernel/git/chenhuacai/linux-loongson.git/log/?h=loongarch-next
These two patches are split from the full v2 series (11 patches):
https://lore.kernel.org/all/[email protected]/
George Guo (2):
LoongArch: BPF: Support internal-only MOV to resolve per-CPU addrs
LoongArch: BPF: Add timed may_goto support
arch/loongarch/include/asm/inst.h | 1 + arch/loongarch/net/Makefile | 2 +- arch/loongarch/net/bpf_jit.c | 27 +++++++++++++- arch/loongarch/net/bpf_timed_may_goto.S | 47 +++++++++++++++++++++++++ 4 files changed, 75 insertions(+), 2 deletions(-) create mode 100644 arch/loongarch/net/bpf_timed_may_goto.S
Signature · 1 line
2.53.0
PATCH v3 1/2 LoongArch: BPF: Support internal-only MOV to resolve per-CPU addrs
From: George Guo <[email protected]>
Support the internal-only BPF_MOV instruction that resolves the absolute address of per-CPU data from its per-CPU offset. This instruction is used only for internal inlining optimizations between the BPF verifier and the JITs (e.g. inlining bpf_get_smp_processor_id() and per-CPU map lookups).
LoongArch keeps the per-CPU offset of the current CPU in $r21 (__my_cpu_offset), so resolving a per-CPU address only requires adding $r21 to the source register holding the per-CPU offset. Advertise the capability via bpf_jit_supports_percpu_insn().
--- arch/loongarch/include/asm/inst.h | 1 + arch/loongarch/net/bpf_jit.c | 14 ++++++++++++++ 2 files changed, 15 insertions(+)
arch/loongarch/include/asm/inst.h +1 -0
@@ -411,6 +411,7 @@ enum loongarch_gpr {LOONGARCH_GPR_T6 ,LOONGARCH_GPR_T7 ,LOONGARCH_GPR_T8 , +LOONGARCH_GPR_U0 =21 ,/* Kernel per-CPU base register ($r21) */ LOONGARCH_GPR_FP =22 ,LOONGARCH_GPR_S0 =23 , LOONGARCH_GPR_S1,
arch/loongarch/net/bpf_jit.c +14 -0
@@ -722,6 +722,15 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool extmove_reg (ctx, dst, t1);break ; } +if (insn_is_mov_percpu_addr (insn)) { +if (dst != src) +move_reg (ctx, dst, src); +#ifdef CONFIG_SMP +/* dst += __my_cpu_offset, held in $r21 */ +emit_insn (ctx, addd, dst, dst,LOONGARCH_GPR_U0 ); +#endif +break ; + }switch (off) {case 0 :move_reg (ctx, dst, src);
@@ -2376,6 +2385,11 @@ bool bpf_jit_supports_ptr_xchg(void)return true; } +bool bpf_jit_supports_percpu_insn (void ) +{ +return true; +} +/* Indicate the JIT backend supports mixing bpf2bpf and tailcalls. */ bool bpf_jit_supports_subprog_tailcalls (void ) {
Signature · 1 line
2.53.0
PATCH v3 2/2 LoongArch: BPF: Add timed may_goto support
In reply to George GuoFrom: George Guo <[email protected]>
Implement arch_bpf_timed_may_goto() and advertise it through bpf_jit_supports_timed_may_goto() so the verifier lowers may_goto into the timed variant: instead of a fixed iteration counter, the loop is bounded by a wall-clock timeout maintained in a per-loop stack slot.
arch_bpf_timed_may_goto() uses a custom calling convention: the verifier passes the count/timestamp stack offset in BPF_REG_AX and expects the updated count back in the same register. The JIT call path therefore skips the usual 'BPF_REG_0 = C return value' move for this helper.
--- arch/loongarch/net/Makefile | 2 +- arch/loongarch/net/bpf_jit.c | 13 ++++++- arch/loongarch/net/bpf_timed_may_goto.S | 47 +++++++++++++++++++++++++ 3 files changed, 60 insertions(+), 2 deletions(-) create mode 100644 arch/loongarch/net/bpf_timed_may_goto.S
arch/loongarch/net/Makefile +1 -1
@@ -4,4 +4,4 @@ # # Copyright (C) 2022 Loongson Technology Corporation Limited # -obj-$(CONFIG_BPF_JIT) += bpf_jit.o +obj-$(CONFIG_BPF_JIT) += bpf_jit.o bpf_timed_may_goto.o
arch/loongarch/net/bpf_jit.c +12 -1
@@ -1192,7 +1192,13 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool extmove_addr (ctx, t1, func_addr);emit_insn (ctx, jirl,LOONGARCH_GPR_RA , t1,0 ); -if (insn->src_reg !=BPF_PSEUDO_CALL ) +/* +* Call to arch_bpf_timed_may_goto() uses a custom calling +* convention with the argument and return value in BPF_REG_AX, +* so skip moving the C return value into BPF_REG_0. +*/ +if (insn->src_reg !=BPF_PSEUDO_CALL && + func_addr != (u64 )arch_bpf_timed_may_goto)move_reg (ctx, regmap[BPF_REG_0 ],LOONGARCH_GPR_A0 );break ;
@@ -2390,6 +2396,11 @@ bool bpf_jit_supports_percpu_insn(void)return true; } +bool bpf_jit_supports_timed_may_goto (void ) +{ +return true; +} +/* Indicate the JIT backend supports mixing bpf2bpf and tailcalls. */ bool bpf_jit_supports_subprog_tailcalls (void ) {
arch/loongarch/net/bpf_timed_may_goto.S new file +47 -0
47 lines not on this page · raw message
Signature · 1 line
2.53.0
Huacai Chen
maintainerHi, George,
In V1 it is independent,
In V2 it is part of
https://lore.kernel.org/loongarch/[email protected]/T/#t
In V3 it is independent again.
What do you want? You can ask Tiezhu and Hengqi to review your patches, but don't do such confusing things.
Huacai
Quoted 64 lines · George Guo
These are the first two patches of the LoongArch BPF JIT feature work, sent on their own to make review easier. The remaining pieces (per-program private stacks, exceptions/bpf_throw, sign-extending loads and atomics on arena pointers, and the matching selftests) will follow in a later series.
Both patches are independent of each other. They apply on top of Chenguang Zhao's "LoongArch bpf kptr xchg inline support" v4 series:
https://lore.kernel.org/all/[email protected]/
Patch 1 adds the internal-only BPF MOV that resolves a per-CPU offset to the current CPU's address, advertised through
bpf_jit_supports_percpu_insn(). LoongArch keeps the current CPU's per-CPU base in $r21 (__my_cpu_offset), so the resolution is a single add. Exercised by the cpumask andpercpu_allocselftests.Patch 2 implements
arch_bpf_timed_may_goto()and advertises it, so the verifier lowersmay_gototo its timed variant, which bounds the loop by a wall-clock timeout kept in a per-loop stack slot. It uses a custom calling
convention: the count/timestamp stack offset is passed inBPF_REG_AXand returned there, so the JIT call path skips the usual "BPF_REG_0= return value" move for this helper. Exercised by the iters selftests.Selftest results (all PASS):
Feature Test(s)
--------------------------------------------------------------1 internal-only MOV (percpu) cpumask, percpu_alloc 2 timed may_goto iters$ sudo ./test_progs -t cpumask ... Summary: 1/35 PASSED, 0 SKIPPED, 0 FAILED$ sudo ./test_progs -t percpu_alloc ... Summary: 1/18 PASSED, 0 SKIPPED, 0 FAILED$ sudo ./test_progs -t iters ... Summary: 1/93 PASSED, 0 SKIPPED, 0 FAILEDBased on loongarch-next:
https://git.kernel.org/pub/scm/linux/kernel/git/chenhuacai/linux-loongson.git/log/?h=loongarch-nextThese two patches are split from the full v2 series (11 patches):
https://lore.kernel.org/all/[email protected]/George Guo (2):
LoongArch: BPF: Support internal-only MOV to resolve per-CPU addrs
LoongArch: BPF: Add timedmay_gotosupportarch/loongarch/include/asm/inst.h | 1 + arch/loongarch/net/Makefile | 2 +- arch/loongarch/net/bpf_jit.c | 27 +++++++++++++- arch/loongarch/net/bpf_timed_may_goto.S | 47 +++++++++++++++++++++++++ 4 files changed, 75 insertions(+), 2 deletions(-) create mode 100644 arch/loongarch/net/bpf_timed_may_goto.S--
2.53.0
Hi Huacai,
Quoted 5 lines · Huacai Chen
In V1 it is independent,
In V2 it is part of [...] the 11-patch series
In V3 it is independent again.What do you want?
You are right that the v1 -> v2 -> v3 shape looks inconsistent. The plan has always been incremental: Tiezhu suggested sending this work as small, stacked series -- A first, then B rebased on A, then C on B -- so each round stays small enough to review carefully. My v2 bundled all 11 patches to show the full picture, but that was too large to review effectively, so v3 goes back to the incremental approach.
Concretely, this 2-patch series (per-CPU MOV + timed may_goto) is A. The remaining pieces -- private stack, exceptions/bpf_throw, arena LDSX and atomics, and the matching selftests -- will follow as separate series stacked on top of it. The structure stays incremental from here.
Quoted 2 lines
You can ask Tiezhu and Hengqi to review your patches, but don't do such confusing things.
Understood. I will keep the split stable and ask Tiezhu and Hengqi to review this batch.
Thanks,
George
Quoted 5 lines · George Guo
These are the first two patches of the LoongArch BPF JIT feature work, sent on their own to make review easier. The remaining pieces (per-program private stacks, exceptions/bpf_throw, sign-extending loads and atomics on arena pointers, and the matching selftests) will follow in a later series.
The patches apply cleanly on top of Chenguang's series.
The following selftests passed on LoongArch:
# ./test_progs -t cpumask # ./test_progs -t percpu_alloc # ./test_progs -t iters
Thanks,
Tiezhu
Quoted 10 lines · Tiezhu Yang
Quoted 5 lines · George Guo
These are the first two patches of the LoongArch BPF JIT feature work, sent on their own to make review easier. The remaining pieces (per-program private stacks, exceptions/bpf_throw, sign-extending loads and atomics on arena pointers, and the matching selftests) will follow in a later series.
The patches apply cleanly on top of Chenguang's series.
Quoted 8 lines
The following selftests passed on LoongArch:
# ./test_progs -t cpumask # ./test_progs -t percpu_alloc # ./test_progs -t itersAcked-by Tiezhu Yang <[email protected]>Tested-by Tiezhu Yang <[email protected]>
Applied, thanks.
Huacai
Quoted 4 lines
Thanks,
Tiezhu