Skip to content

bpf: magic kernel functions

Ihor Solodrai bpf dwarves llvm oe-kbuild-all 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 Latest8 patches 22 repliesIn review

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/8bpf: Add BTF_ID_LIST_END and BTF_ID_LIST_SIZE macrosinclude/linux/btf_ids.h+10 -02 replies
2/8bpf: Refactor btf_kfunc_id_set_containsinclude/linux/btf.h kernel/bpf/btf.c kernel/bpf/verifier.c+58 -201 reply
3/8bpf: Support for kfuncs with KF_MAGIC_ARGSinclude/linux/btf.h kernel/bpf/verifier.c+120 -46 replies
4/8bpf: Support __magic prog_aux arguments for kfuncskernel/bpf/verifier.c+11 -10 replies
5/8bpf: Re-define bpf_wq_set_callback as magic kfunckernel/bpf/helpers.c kernel/bpf/verifier.c tools/testing/selftests/bpf/bpf_experimental.h +2 more+20 -221 reply
6/8bpf,docs: Document KF_MAGIC_ARGS flag and __magic annotationDocumentation/bpf/kfuncs.rst+48 -11 reply
7/8bpf: Re-define bpf_task_work_schedule_* kfuncs as magickernel/bpf/helpers.c kernel/bpf/verifier.c tools/testing/selftests/bpf/progs/file_reader.c +4 more+30 -200 replies
8/8bpf: Re-define bpf_stream_vprintk as a magic kfunckernel/bpf/helpers.c kernel/bpf/stream.c kernel/bpf/verifier.c +2 more+16 -100 replies

Reviews trailers per patch

1 of 8 patches has a Reviewed-by or Acked-by.

PatchEduard Zingerman
1/8
2/8R-b
3/8
4/8
5/8
6/8
7/8
8/8

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v1 thread

Cover letter [PATCH bpf-next v1 0/8] bpf: magic kernel functions

This series develops the idea of implicit prog_aux argument for kfuncs
[1] into a generic "magic kfuncs" feature.

A mechanism is created for kfuncs to have arguments that are not visible to the BPF programs, and are provided to the kernel function implementation by the verifier.

This mechanism is then used in kfuncs that have an argument with __prog annotation [2], which is the current way of passing struct bpf_prog_aux pointer to kfuncs.

==== "Magic" ???

The usage of term "magic" is up for debate of course, I am open to suggestions. I used it as a placeholder first and now it weirdly makes sense. After all, "bpf" by itself doesn't mean anything either.

The feature effectively produces two variants of a kfunc signature/prototype: one with the full argument list, and another with particular arguments omitted.

There are many terms that in other contexts are used to describe similar properties: implicit, optional, virtual, overloaded, polymorphic, interface etc. None of them is quite right for this use case in BPF, and may trigger incorrect intuition if used.

An accurate term could be something like "verifier provided arguments" and "kfuncs with verifier provided arguments", but that's too long for usage in the identifiers. "Magic" on the other hand is a short and ambiguous adjective, which hopefully will prompt people to check the documentation.

==== Implementation

pahole's BTF encoding is changed [3] to detect magic kfuncs and emit two BTF functions from a single kfunc declaration:
* kfunc_impl() with the arguments matching kernel declaration
* kfunc() with __magic arguments omitted

BPF programs then can use both variants of the kfunc (to preserve backwards compatibility for BPF programs that include vmlinux.h), although non-_impl variant would be a preferred API.

To achieve this a few pieces of information must be possible to decode in pahole from kernel side kfunc declaration:
* which kfuncs are magic
* what arguments should be omitted

A simple way to mark magic kfuncs is with a KF_MAGIC_ARGS flag.

As for the arguments, I considered a couple of options:
* look at argument types and hardcode a list of types in pahole
* have multiple KF_MAGIC_ARGS_<N> flags, which would indicate the number of arguments omitted
* use a special arg name suffix (annotation), i.e. "__magic"

Of the three, the last one seems to be the simplest and most flexible.

It is also necessary for pahole to use conventional names for the emitted kfunc pair for the verifier to be able to recognize them.

These changes in BTF create discrepancies in the verifier:
* kfunc() now has an incorrect BTF function prototype
* kfunc_impl() doesn't have a corresponding ksym and BTF flags

In order to handle them correctly, it's necessary to be able to lookup kfunc() <-> kfunc_impl() pairs efficiently. Naive string lookup in BTF is possible, but it is slow, which may negatively impact verification performance for programs using relevant kfuncs.

Since the magic kfuncs are constant within a kernel, and their names in BTF are conventional, we can define a constant table of magic kfuncs using existing BTF_ID_LIST mechanism. A magic_kfuncs BTF ids table is therefore defined and used for efficient lookups.

An inconvenience of the implementation described above is that the writers of a magic kfunc have to do a couple of things:
* mark the kfunc with KF_MAGIC_ARGS
* mark the args with __magic annotation
* add kfunc to magic_kfuncs table

Another one is that for special kfuncs the relevant checks in the verifier must test for both original and _impl funcs. See changes to bpf_wq_set_callback_impl for an example.

==== Testing

A number of selftests are already using aux__prog -> magic kfuncs.

Successful BPF CI run with modified pahole:
https://github.com/kernel-patches/bpf/actions/runs/18918350607

[1] https://lore.kernel.org/bpf/[email protected]/
[2] https://docs.kernel.org/bpf/kfuncs.html#prog-annotation
[3] https://github.com/theihor/dwarves/tree/magic-args.draft

Ihor Solodrai (8):
bpf: Add BTF_ID_LIST_END and BTF_ID_LIST_SIZE macros
bpf: Refactor btf_kfunc_id_set_contains
bpf: Support for kfuncs with KF_MAGIC_ARGS
bpf: Support __magic prog_aux arguments for kfuncs
bpf: Re-define bpf_wq_set_callback as magic kfunc bpf,docs: Document KF_MAGIC_ARGS flag and __magic annotation
bpf: Re-define bpf_task_work_schedule_* kfuncs as magic
bpf: Re-define bpf_stream_vprintk as a magic kfunc

 Documentation/bpf/kfuncs.rst                  |  49 ++++-
 include/linux/btf.h                           |   7 +-
 include/linux/btf_ids.h                       |  10 ++
 kernel/bpf/btf.c                              |  70 ++++++--
 kernel/bpf/helpers.c                          |  31 ++--
 kernel/bpf/stream.c                           |   9 +-
 kernel/bpf/verifier.c                         | 167 ++++++++++++++++--
 tools/lib/bpf/bpf_helpers.h                   |   7 +-
 .../testing/selftests/bpf/bpf_experimental.h  |   5 -
 .../testing/selftests/bpf/progs/file_reader.c |   2 +-
 .../testing/selftests/bpf/progs/stream_fail.c |   6 +-
 tools/testing/selftests/bpf/progs/task_work.c |   6 +-
 .../selftests/bpf/progs/task_work_fail.c      |   8 +-
 .../selftests/bpf/progs/task_work_stress.c    |   2 +-
 .../bpf/progs/verifier_async_cb_context.c     |   4 +-
 tools/testing/selftests/bpf/progs/wq.c        |   2 +-
 .../testing/selftests/bpf/progs/wq_failures.c |   4 +-
 17 files changed, 312 insertions(+), 77 deletions(-)
Signature · 1 line
2.51.1