Skip to contentSkip to conversation

LoongArch: BPF: per-CPU address resolution and timed may_goto

PATCH Applied George Guo 3 participants 7 messages Aug 4 – Aug 9

Subsystems (3)
lore
Share & tools
lore

George Guo

author
#0 lore raw

v3 cover LoongArch: BPF: per-CPU address resolution and timed may_goto

These are the first two patches of the LoongArch BPF JIT feature work, sent on their own to make review easier. The remaining pieces (per-program private stacks, exceptions/bpf_throw, sign-extending loads and atomics on arena pointers, and the matching selftests) will follow in a later series.

Both patches are independent of each other. They apply on top of Chenguang Zhao's "LoongArch bpf kptr xchg inline support" v4 series:

https://lore.kernel.org/all/[email protected]/

Patch 1 adds the internal-only BPF MOV that resolves a per-CPU offset to the current CPU's address, advertised through bpf_jit_supports_percpu_insn(). LoongArch keeps the current CPU's per-CPU base in $r21 (__my_cpu_offset), so the resolution is a single add. Exercised by the cpumask and percpu_alloc selftests.

Patch 2 implements arch_bpf_timed_may_goto() and advertises it, so the verifier lowers may_goto to its timed variant, which bounds the loop by a wall-clock timeout kept in a per-loop stack slot. It uses a custom calling
convention: the count/timestamp stack offset is passed in BPF_REG_AX and returned there, so the JIT call path skips the usual "BPF_REG_0 = return value" move for this helper. Exercised by the iters selftests.

Selftest results (all PASS):

Feature Test(s)
--------------------------------------------------------------

  1  internal-only MOV (percpu)     cpumask, percpu_alloc
  2  timed may_goto                 iters
$ sudo ./test_progs -t cpumask
...
Summary: 1/35 PASSED, 0 SKIPPED, 0 FAILED
$ sudo ./test_progs -t percpu_alloc
...
Summary: 1/18 PASSED, 0 SKIPPED, 0 FAILED
$ sudo ./test_progs -t iters
...
Summary: 1/93 PASSED, 0 SKIPPED, 0 FAILED

Based on loongarch-next:
https://git.kernel.org/pub/scm/linux/kernel/git/chenhuacai/linux-loongson.git/log/?h=loongarch-next

These two patches are split from the full v2 series (11 patches):
https://lore.kernel.org/all/[email protected]/

George Guo (2):
LoongArch: BPF: Support internal-only MOV to resolve per-CPU addrs
LoongArch: BPF: Add timed may_goto support

 arch/loongarch/include/asm/inst.h       |  1 +
 arch/loongarch/net/Makefile             |  2 +-
 arch/loongarch/net/bpf_jit.c            | 27 +++++++++++++-
 arch/loongarch/net/bpf_timed_may_goto.S | 47 +++++++++++++++++++++++++
 4 files changed, 75 insertions(+), 2 deletions(-)
 create mode 100644 arch/loongarch/net/bpf_timed_may_goto.S
Signature · 1 line
2.53.0
author
#1 lore raw

PATCH v3 1/2 LoongArch: BPF: Support internal-only MOV to resolve per-CPU addrs

From: George Guo <[email protected]>

Support the internal-only BPF_MOV instruction that resolves the absolute address of per-CPU data from its per-CPU offset. This instruction is used only for internal inlining optimizations between the BPF verifier and the JITs (e.g. inlining bpf_get_smp_processor_id() and per-CPU map lookups).

LoongArch keeps the per-CPU offset of the current CPU in $r21 (__my_cpu_offset), so resolving a per-CPU address only requires adding $r21 to the source register holding the per-CPU offset. Advertise the capability via bpf_jit_supports_percpu_insn().

---
 arch/loongarch/include/asm/inst.h |  1 +
 arch/loongarch/net/bpf_jit.c      | 14 ++++++++++++++
 2 files changed, 15 insertions(+)
arch/loongarch/include/asm/inst.h +1 -0
index 54f35bcd0f66..585667e361c2 100644
@@ -411,6 +411,7 @@ enum loongarch_gpr {
 	LOONGARCH_GPR_T6,
 	LOONGARCH_GPR_T7,
 	LOONGARCH_GPR_T8,
+	LOONGARCH_GPR_U0 = 21,	/* Kernel per-CPU base register ($r21) */
 	LOONGARCH_GPR_FP = 22,
 	LOONGARCH_GPR_S0 = 23,
 	LOONGARCH_GPR_S1,
arch/loongarch/net/bpf_jit.c +14 -0
index aa88f904a5ca..b8dd956d46bd 100644
@@ -722,6 +722,15 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext
 			move_reg(ctx, dst, t1);
 			break;
 		}
+		if (insn_is_mov_percpu_addr(insn)) {
+			if (dst != src)
+				move_reg(ctx, dst, src);
+#ifdef CONFIG_SMP
+			/* dst += __my_cpu_offset, held in $r21 */
+			emit_insn(ctx, addd, dst, dst, LOONGARCH_GPR_U0);
+#endif
+			break;
+		}
 		switch (off) {
 		case 0:
 			move_reg(ctx, dst, src);
@@ -2376,6 +2385,11 @@ bool bpf_jit_supports_ptr_xchg(void)
 	return true;
 }
 
+bool bpf_jit_supports_percpu_insn(void)
+{
+	return true;
+}
+
 /* Indicate the JIT backend supports mixing bpf2bpf and tailcalls. */
 bool bpf_jit_supports_subprog_tailcalls(void)
 {
Signature · 1 line
2.53.0
author
#2 lore raw

PATCH v3 2/2 LoongArch: BPF: Add timed may_goto support

In reply to George Guo

From: George Guo <[email protected]>

Implement arch_bpf_timed_may_goto() and advertise it through bpf_jit_supports_timed_may_goto() so the verifier lowers may_goto into the timed variant: instead of a fixed iteration counter, the loop is bounded by a wall-clock timeout maintained in a per-loop stack slot.

arch_bpf_timed_may_goto() uses a custom calling convention: the verifier passes the count/timestamp stack offset in BPF_REG_AX and expects the updated count back in the same register. The JIT call path therefore skips the usual 'BPF_REG_0 = C return value' move for this helper.

---
 arch/loongarch/net/Makefile             |  2 +-
 arch/loongarch/net/bpf_jit.c            | 13 ++++++-
 arch/loongarch/net/bpf_timed_may_goto.S | 47 +++++++++++++++++++++++++
 3 files changed, 60 insertions(+), 2 deletions(-)
 create mode 100644 arch/loongarch/net/bpf_timed_may_goto.S
arch/loongarch/net/Makefile +1 -1
index 1ec12a0c324a..8d9ddb48f9ea 100644
@@ -4,4 +4,4 @@
 #
 # Copyright (C) 2022 Loongson Technology Corporation Limited
 #
-obj-$(CONFIG_BPF_JIT) += bpf_jit.o
+obj-$(CONFIG_BPF_JIT) += bpf_jit.o bpf_timed_may_goto.o
arch/loongarch/net/bpf_jit.c +12 -1
index b8dd956d46bd..004a139e0f49 100644
@@ -1192,7 +1192,13 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext
 		move_addr(ctx, t1, func_addr);
 		emit_insn(ctx, jirl, LOONGARCH_GPR_RA, t1, 0);
 
-		if (insn->src_reg != BPF_PSEUDO_CALL)
+		/*
+		 * Call to arch_bpf_timed_may_goto() uses a custom calling
+		 * convention with the argument and return value in BPF_REG_AX,
+		 * so skip moving the C return value into BPF_REG_0.
+		 */
+		if (insn->src_reg != BPF_PSEUDO_CALL &&
+		    func_addr != (u64)arch_bpf_timed_may_goto)
 			move_reg(ctx, regmap[BPF_REG_0], LOONGARCH_GPR_A0);
 
 		break;
@@ -2390,6 +2396,11 @@ bool bpf_jit_supports_percpu_insn(void)
 	return true;
 }
 
+bool bpf_jit_supports_timed_may_goto(void)
+{
+	return true;
+}
+
 /* Indicate the JIT backend supports mixing bpf2bpf and tailcalls. */
 bool bpf_jit_supports_subprog_tailcalls(void)
 {
arch/loongarch/net/bpf_timed_may_goto.S new file +47 -0

47 lines not on this page · raw message

Signature · 1 line
2.53.0
maintainer
#3 lore raw
In reply to George Guo

Hi, George,

In V1 it is independent,

In V2 it is part of
https://lore.kernel.org/loongarch/[email protected]/T/#t

In V3 it is independent again.

What do you want? You can ask Tiezhu and Hengqi to review your patches, but don't do such confusing things.

Huacai

Quoted 64 lines · George Guo

These are the first two patches of the LoongArch BPF JIT feature work, sent on their own to make review easier. The remaining pieces (per-program private stacks, exceptions/bpf_throw, sign-extending loads and atomics on arena pointers, and the matching selftests) will follow in a later series.

Both patches are independent of each other. They apply on top of Chenguang Zhao's "LoongArch bpf kptr xchg inline support" v4 series:

https://lore.kernel.org/all/[email protected]/

Patch 1 adds the internal-only BPF MOV that resolves a per-CPU offset to the current CPU's address, advertised through bpf_jit_supports_percpu_insn(). LoongArch keeps the current CPU's per-CPU base in $r21 (__my_cpu_offset), so the resolution is a single add. Exercised by the cpumask and percpu_alloc selftests.

Patch 2 implements arch_bpf_timed_may_goto() and advertises it, so the verifier lowers may_goto to its timed variant, which bounds the loop by a wall-clock timeout kept in a per-loop stack slot. It uses a custom calling
convention: the count/timestamp stack offset is passed in BPF_REG_AX and returned there, so the JIT call path skips the usual "BPF_REG_0 = return value" move for this helper. Exercised by the iters selftests.

Selftest results (all PASS):

Feature Test(s)
--------------------------------------------------------------

  1  internal-only MOV (percpu)     cpumask, percpu_alloc
  2  timed may_goto                 iters
$ sudo ./test_progs -t cpumask
...
Summary: 1/35 PASSED, 0 SKIPPED, 0 FAILED
$ sudo ./test_progs -t percpu_alloc
...
Summary: 1/18 PASSED, 0 SKIPPED, 0 FAILED
$ sudo ./test_progs -t iters
...
Summary: 1/93 PASSED, 0 SKIPPED, 0 FAILED

Based on loongarch-next:
https://git.kernel.org/pub/scm/linux/kernel/git/chenhuacai/linux-loongson.git/log/?h=loongarch-next

These two patches are split from the full v2 series (11 patches):
https://lore.kernel.org/all/[email protected]/

George Guo (2):
LoongArch: BPF: Support internal-only MOV to resolve per-CPU addrs
LoongArch: BPF: Add timed may_goto support

 arch/loongarch/include/asm/inst.h       |  1 +
 arch/loongarch/net/Makefile             |  2 +-
 arch/loongarch/net/bpf_jit.c            | 27 +++++++++++++-
 arch/loongarch/net/bpf_timed_may_goto.S | 47 +++++++++++++++++++++++++
 4 files changed, 75 insertions(+), 2 deletions(-)
 create mode 100644 arch/loongarch/net/bpf_timed_may_goto.S

--
2.53.0

author
#4 lore raw

Hi Huacai,

Quoted 5 lines · Huacai Chen

In V1 it is independent,
In V2 it is part of [...] the 11-patch series
In V3 it is independent again.

What do you want?

You are right that the v1 -> v2 -> v3 shape looks inconsistent. The plan has always been incremental: Tiezhu suggested sending this work as small, stacked series -- A first, then B rebased on A, then C on B -- so each round stays small enough to review carefully. My v2 bundled all 11 patches to show the full picture, but that was too large to review effectively, so v3 goes back to the incremental approach.

Concretely, this 2-patch series (per-CPU MOV + timed may_goto) is A. The remaining pieces -- private stack, exceptions/bpf_throw, arena LDSX and atomics, and the matching selftests -- will follow as separate series stacked on top of it. The structure stays incremental from here.

Quoted 2 lines

You can ask Tiezhu and Hengqi to review your patches, but don't do such confusing things.

Understood. I will keep the split stable and ask Tiezhu and Hengqi to review this batch.

Thanks,
George

maintainer Acked-by Tested-by on the v3 series
#5 lore raw
In reply to George Guo
Quoted 5 lines · George Guo

These are the first two patches of the LoongArch BPF JIT feature work, sent on their own to make review easier. The remaining pieces (per-program private stacks, exceptions/bpf_throw, sign-extending loads and atomics on arena pointers, and the matching selftests) will follow in a later series.

The patches apply cleanly on top of Chenguang's series.

The following selftests passed on LoongArch:

  # ./test_progs -t cpumask
  # ./test_progs -t percpu_alloc
  # ./test_progs -t iters

Thanks,
Tiezhu

maintainer Applied
#6 lore raw
Quoted 10 lines · Tiezhu Yang
Quoted 5 lines · George Guo

These are the first two patches of the LoongArch BPF JIT feature work, sent on their own to make review easier. The remaining pieces (per-program private stacks, exceptions/bpf_throw, sign-extending loads and atomics on arena pointers, and the matching selftests) will follow in a later series.

The patches apply cleanly on top of Chenguang's series.

Quoted 8 lines

The following selftests passed on LoongArch:

   # ./test_progs -t cpumask
   # ./test_progs -t percpu_alloc
   # ./test_progs -t iters

Applied, thanks.

Huacai

Quoted 4 lines

Thanks,
Tiezhu