Skip to content

LoongArch bpf kptr xchg inline support

Chenguang Zhao bpf linux-kselftest loongarch 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v4 Latest4 patches 6 repliesIn review

Changes in v4 from the cover letter

- Resend patches against new kernel.

Patches in v4 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/4LoongArch: bpf: Fix memory ordering for value-returning atomicsarch/loongarch/include/asm/inst.h arch/loongarch/net/bpf_jit.c+36 -141 reply
2/4LoongArch: bpf: Advertise JIT support for kptr xchg inlinearch/loongarch/net/bpf_jit.c+5 -01 reply
3/4selftests/bpf: Enable kptr_xchg_inline test on LoongArchtools/testing/selftests/bpf/prog_tests/kptr_xchg_inline.c+2 -11 reply
4/4selftests/bpf: Add kptr-xchg benchmarktools/testing/selftests/bpf/Makefile tools/testing/selftests/bpf/bench.c tools/testing/selftests/bpf/benchs/bench_kptr_xchg.c +1 more+149 -01 reply

Reviews trailers per patch

3 of 4 patches have a Reviewed-by or Acked-by of their own; the series has one for all of them.

PatchHengqi ChenTiezhu Yang
SeriesA-b T-b
1/4A-b
2/4A-b
3/4A-b
4/4

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v4 thread

Cover letter [PATCH v4 0/4] LoongArch bpf kptr xchg inline support

From: Chenguang Zhao <[email protected]>

This series enables the BPF verifier to inline bpf_kptr_xchg() into BPF_XCHG on LoongArch64, and fixes the underlying JIT atomic ordering that makes such inlining safe.

The BPF verifier can lower bpf_kptr_xchg() to a single BPF_XCHG atomic when the JIT advertises ptr xchg support via bpf_jit_supports_ptr_xchg(). This removes helper-call overhead from the kptr exchange fast path. Inlining is only correct when the JITed exchange provides the same sequentially consistent ordering as the bpf_kptr_xchg() helper.

Patch 1 fixes memory ordering for all value-returning BPF atomic RMW operations emitted by the LoongArch bpf JIT. Per LKMM, value-returning atomic RMW must provide sequentially consistent ordering. Plain AMO instructions and bare ll/sc loops on LoongArch do not satisfy this by themselves:
- BPF_FETCH (ADD/AND/OR/XOR): switch to am*_db.{b,h,w,d}
- BPF_XCHG: switch to amswap_db.{b,h,w,d}
- BPF_CMPXCHG: emit dbar 0x700 after the ll/sc loop, matching __WEAK_LLSC_MB in cmpxchg.h
Non-value-returning RMW ops (plain BPF_ADD, BPF_AND, etc.) remain weakly ordered, consistent with LKMM. This fix is independent of kptr inlining and benefits all BPF programs using value-returning atomics on LoongArch.

Patch 2 implements bpf_jit_supports_ptr_xchg() so the verifier may inline bpf_kptr_xchg() on LoongArch64.

Patches 3 and 4 extend bpf selftests: functional coverage via kptr_xchg_inline, and an optional kptr-xchg benchmark to compare helper vs inlined paths.

Chenguang Zhao (4):
LoongArch: bpf: Fix memory ordering for value-returning atomics
LoongArch: bpf: Advertise JIT support for kptr xchg inline selftests/bpf: Enable kptr_xchg_inline test on LoongArch selftests/bpf: Add kptr-xchg benchmark

 arch/loongarch/include/asm/inst.h             | 18 ++++
 arch/loongarch/net/bpf_jit.c                  | 37 ++++---
 tools/testing/selftests/bpf/Makefile          |  2 +
 tools/testing/selftests/bpf/bench.c           |  2 +
 .../selftests/bpf/benchs/bench_kptr_xchg.c    | 96 +++++++++++++++++++
 .../bpf/prog_tests/kptr_xchg_inline.c         |  3 +-
 .../selftests/bpf/progs/kptr_xchg_bench.c     | 49 ++++++++++
 7 files changed, 192 insertions(+), 15 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/benchs/bench_kptr_xchg.c
 create mode 100644 tools/testing/selftests/bpf/progs/kptr_xchg_bench.c

---
v4:
- Resend patches against new kernel.

v3:
- https://lore.kernel.org/all/[email protected]/

v2:
- https://lore.kernel.org/all/[email protected]/

v1:
- https://lore.kernel.org/all/[email protected]/

Signature · 1 line
2.25.1