Skip to content

RFC Initial support for allocating BPF JITs in vmalloc for x86

Rick Edgecombe netdev 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 RFC Latest4 patches 4 repliesIn review

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/4bpf, x64: Implement BPF call retpolinearch/x86/include/asm/nospec-branch.h+43 -20 replies
2/4bpf, x64: Increase distance for bpf callsarch/x86/net/bpf_jit_comp.c+94 -230 replies
3/4bpf: Charge bpf jit limit in bpf_jit_alloc_execinclude/linux/filter.h kernel/bpf/core.c+14 -90 replies
4/4bpf, x64: Enable unprivlidged jit in vmallocarch/x86/net/bpf_jit_comp.c+32 -00 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.

Read the v1 thread

Cover letter [RFC PATCH 0/4] Initial support for allocating BPF JITs in vmalloc for x86

This introduces a new capability for BPF program JIT's to be located in vmalloc space on x86_64. This can serve as a backup area for CONFIG_BPF_JIT_ALWAYS_ON in case an unprivileged app uses all of the module space allowed by bpf_jit_limit.

In order to allow for calls from the increased distance of vmalloc from kernel/module space, relative calls are emitted as full indirect calls if the maximum relative call distance is exceeded. So the resulting performance of call BPF instructions in this case is similar to the BPF interpreter.

Below is the results for the BPF unit test "BPF_MAXINSNS: Call heavy transformations":

Config Duration 1 Duration 2
--------------------------------------------------

JIT Modules             13304           14224
JIT Vmalloc             33711           26877
Interpreter             31931           28180 
Retpoline JIT Modules   14517           13375
Retpoline JIT Vmalloc   65739           60281
Retpoline Interpreter   63097           61141

Since this benchmark is made up of only calls to the kernel text, it suggests real world performance shouldn't ever be worse than the interpreter.

The logic for when to use vmalloc, is that we use module space unless it is full, or, we are not CAP_SYS_ADMIN and bpf_jit_limit is exceeded. So vmalloc is only used when either the insertion would fail, or BPF would fallback to the interpreter. So there should be no run time regression when used in these scenarios.

Todo:
- This passes all the unit tests, but could use further verification that the compiler will still always converge.
- Fix ARM after "bpf: Charge bpf jit limit in bpf_jit_alloc_exec" change
- Update documentation for bpf_jit_limit

Possible future optimizations:
- Look at re-mapping some text in vmalloc so that calls can be in relative jump range. For example, a BPF library program could maybe be re-mapped multiple times so that a copy is always near the caller and so we could use the faster calls.

Rick Edgecombe (4):
bpf, x64: Implement BPF call retpoline
bpf, x64: Increase distance for bpf calls
bpf: Charge bpf jit limit in bpf_jit_alloc_exec
bpf, x64: Enable unprivlidged jit in vmalloc

 arch/x86/include/asm/nospec-branch.h |  45 +++++++-
 arch/x86/net/bpf_jit_comp.c          | 149 ++++++++++++++++++++++-----
 include/linux/filter.h               |   3 +
 kernel/bpf/core.c                    |  20 ++--
 4 files changed, 183 insertions(+), 34 deletions(-)
Signature · 1 line
2.17.1