Add diagnostics for kfunc registration failures and jit subtest skips
Subsystems (4)
cover Add diagnostics for kfunc registration failures and jit subtest skips
While testing the unwind_goto series on arm64 I ran into two places where a failure is reported without any information about what went wrong, and each took a while to track down. These three patches add the diagnostics that would have saved that time.
Patch 1: when a kfunc in a registered set has no proper FUNC entry in the BTF of the registering object, register_btf_kfunc_id_set() returns -EINVAL and nothing is printed. With bpf_testmod built by pahole 1.30 the module fails to load with -22 and dmesg stays empty; the only hint is a resolve_btfids warning buried in the build log. Log the offending id, the owner of the BTF and the error code.
Patch 2: bpf_testmod_init() chains twelve registrations with "ret = ret ?:", so when one of them fails there is no way to tell which call it was without instrumenting the module. Print the failing expression.
Patch 3: when test_progs is built without LLVM development libraries, every subtest that checks jited output skips with a message that is only visible under -v and does not say what to do about it. Point at the fix, and add a missing newline in the verbose-only fallback message.
One observation from building both architectures: with CONFIG_WERROR=y (the x86_64 defconfig default) resolve_btfids runs with --fatal_warnings, so the stale-pahole case fails the module build outright instead of producing a module that fails at load time. Patches 1 and 2 only help the WERROR=n path, which is what we run in our arm64 CI.
Verified on arm64 (KVM guest) and x86_64 (TCG guest) against bpf-next master b71326388a: with the patched kernel a pahole-1.30-built bpf_testmod still fails to load, but dmesg now says which kfunc and which registration failed; with pahole 1.32 it loads cleanly, and the unwind selftests still pass 42/42.
Wang Yibo (3):
bpf: report kfunc sets that fail BTF sanity check selftests/bpf: name the failing registration in bpf_testmod_init selftests/bpf: say how to fix the missing LLVM skip for jited checks
kernel/bpf/btf.c | 6 ++- .../selftests/bpf/jit_disasm_helpers.c | 2 +- .../selftests/bpf/test_kmods/bpf_testmod.c | 41 +++++++++++-------- tools/testing/selftests/bpf/test_loader.c | 2 +- 4 files changed, 32 insertions(+), 19 deletions(-)
Signature · 1 line
2.51.0
PATCH 1/3 bpf: report kfunc sets that fail BTF sanity check
register_btf_kfunc_id_set() returns -EINVAL when a kfunc in the set has no valid FUNC/FUNC_PROTO pair in the BTF of the registering object, but nothing is logged: the caller (typically a module init function) propagates the error, insmod fails with -EINVAL and dmesg stays silent. The only hint is a warning from resolve_btfids at build time, which is easy to miss.
This happened with bpf_testmod built by pahole 1.30: that version does not encode __int128, so the three i128 test kfuncs had no BTF func entries, their ids in the set stayed 0 and loading the module failed with -22, with nothing printed to dmesg.
Log the offending kfunc id, the object that owns the BTF and the error code when the sanity check fails.
--- kernel/bpf/btf.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
kernel/bpf/btf.c +5 -1
@@ -10128,8 +10128,12 @@ static int __register_btf_kfunc_id_set(enum btf_kfunc_hook hook,for (i =0 ; i < kset->set->cnt; i++) { ret =btf_check_kfunc_protos (btf,btf_relocate_id (btf, kset->set->pairs[i].id), kset->set->pairs[i].flags); -if (ret) +if (ret) { +pr_err ("register_btf_kfunc_id_set: kfunc id %u in BTF of %s failed sanity check, ret %d\n" , + kset->set->pairs[i].id, + kset->owner ? kset->owner->name :"vmlinux" , ret); goto err_out; + } } ret =btf_populate_kfunc_set (btf, hook, kset);
Signature · 1 line
2.51.0
[email protected]
AI reviewRe: [PATCH bpf-next 1/3] bpf: report kfunc sets that fail BTF sanity check
TL;DR: Commit message only: it says "three i128 test kfuncs" but bpf_testmod appears to register about six; suggests fixing the count or dropping it. No code issues raised.
Quoted 1 line
bpf: report kfunc sets that fail BTF sanity check
TL;DR: Commit message only: it says "three i128 test kfuncs" but bpf_testmod appears to register about six; suggests fixing the count or dropping it. No code issues raised.
Quoted 14 lines
bpf: report kfunc sets that fail BTF sanity check
register_btf_kfunc_id_set()returns-EINVALwhen a kfunc in the set has no valid FUNC/FUNC_PROTO pair in the BTF of the registering object, but nothing is logged: the caller (typically a module init function) propagates the error, insmod fails with-EINVALand dmesg stays silent. The only hint is a warning fromresolve_btfidsat build time, which is easy to miss.This happened with
bpf_testmodbuilt by pahole 1.30: that version does not encode__int128, so the three i128 test kfuncs had no BTF func entries, their ids in the set stayed 0 and loading the module failed with -22, with nothing printed to dmesg.
Quoted 4 lines
Log the offending kfunc id, the object that owns the BTF and the error code when the sanity check fails.
Signed-off-by Wang Yibo <[email protected]>
The commit message says "the three i128 test kfuncs", but is that count accurate?
bpf_testmod registers six i128 test kfuncs in its kfunc set:
tools/testing/selftests/bpf/test_kmods/bpf_testmod.c:
BTF_ID_FLAGS(func, bpf_kfunc_call_test_i128)
...
BTF_ID_FLAGS(func, bpf_kfunc_call_test_i128_arg)
BTF_ID_FLAGS(func, bpf_kfunc_call_test_i128_arg_odd)
BTF_ID_FLAGS(func, bpf_kfunc_call_test_i128_arg_odd_many)
BTF_ID_FLAGS(func, bpf_kfunc_call_test_i128_arg_stack_hole)
BTF_ID_FLAGS(func, bpf_kfunc_call_test_i128_arg_pad)At least five of them take or return a bare __int128, and bpf_kfunc_call_test_i128_arg_stack_hole passes it through struct prog_test_i128_arg.
Could the message either state the correct number or say "the i128 test kfuncs" instead?
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/39699006936
PATCH 2/3 selftests/bpf: name the failing registration in bpf_testmod_init
In reply to Wang Yibobpf_testmod_init() chains its registrations with ret = ret ?: ...; when one of them fails the module only reports the final error code, and there is no way to tell which call failed without instrumenting the code. Wrap the calls in a macro that prints the failing expression and its return value instead.
--- .../selftests/bpf/test_kmods/bpf_testmod.c | 41 +++++++++++-------- 1 file changed, 25 insertions(+), 16 deletions(-)
tools/testing/selftests/bpf/test_kmods/bpf_testmod.c +25 -16
@@ -2593,22 +2593,31 @@ static int bpf_testmod_init(void)void **tramp;int ret; - ret =register_btf_kfunc_id_set (BPF_PROG_TYPE_UNSPEC , &bpf_testmod_common_kfunc_set); - ret = ret ?:register_btf_kfunc_id_set (BPF_PROG_TYPE_SCHED_CLS , &bpf_testmod_kfunc_set); - ret = ret ?:register_btf_kfunc_id_set (BPF_PROG_TYPE_TRACING , &bpf_testmod_kfunc_set); - ret = ret ?:register_btf_kfunc_id_set (BPF_PROG_TYPE_SYSCALL , &bpf_testmod_kfunc_set); - ret = ret ?:register_btf_kfunc_id_set (BPF_PROG_TYPE_STRUCT_OPS , &bpf_testmod_kfunc_set); - ret = ret ?:register_btf_fmodret_id_set (&bpf_testmod_trampoline_count_fmodret_set); - ret = ret ?:register_bpf_struct_ops (&bpf_bpf_testmod_ops, bpf_testmod_ops); - ret = ret ?:register_bpf_struct_ops (&bpf_testmod_ops2, bpf_testmod_ops2); - ret = ret ?:register_bpf_struct_ops (&bpf_testmod_ops3, bpf_testmod_ops3); - ret = ret ?:register_bpf_struct_ops (&testmod_st_ops, bpf_testmod_st_ops); - ret = ret ?:register_bpf_struct_ops (&testmod_multi_st_ops, bpf_testmod_multi_st_ops); - ret = ret ?:register_btf_id_dtor_kfuncs (bpf_testmod_dtors, -ARRAY_SIZE (bpf_testmod_dtors), -THIS_MODULE ); -if (ret <0 ) -return ret; +#define TRY (expr) \ + do { \ + ret = (expr); \ + if (ret < 0) { \ + pr_err("%s: %s failed, ret %d\n", \ + __func__, #expr, ret); \ + return ret; \ + } \ + } while (0) + +TRY (register_btf_kfunc_id_set (BPF_PROG_TYPE_UNSPEC , &bpf_testmod_common_kfunc_set)); +TRY (register_btf_kfunc_id_set (BPF_PROG_TYPE_SCHED_CLS , &bpf_testmod_kfunc_set)); +TRY (register_btf_kfunc_id_set (BPF_PROG_TYPE_TRACING , &bpf_testmod_kfunc_set)); +TRY (register_btf_kfunc_id_set (BPF_PROG_TYPE_SYSCALL , &bpf_testmod_kfunc_set)); +TRY (register_btf_kfunc_id_set (BPF_PROG_TYPE_STRUCT_OPS , &bpf_testmod_kfunc_set)); +TRY (register_btf_fmodret_id_set (&bpf_testmod_trampoline_count_fmodret_set)); +TRY (register_bpf_struct_ops (&bpf_bpf_testmod_ops, bpf_testmod_ops)); +TRY (register_bpf_struct_ops (&bpf_testmod_ops2, bpf_testmod_ops2)); +TRY (register_bpf_struct_ops (&bpf_testmod_ops3, bpf_testmod_ops3)); +TRY (register_bpf_struct_ops (&testmod_st_ops, bpf_testmod_st_ops)); +TRY (register_bpf_struct_ops (&testmod_multi_st_ops, bpf_testmod_multi_st_ops)); +TRY (register_btf_id_dtor_kfuncs (bpf_testmod_dtors, +ARRAY_SIZE (bpf_testmod_dtors), +THIS_MODULE )); +#undef TRYif (bpf_fentry_test1 (0 ) <0 )return -EINVAL ; sock =NULL ;
Signature · 1 line
2.51.0
Re: [PATCH bpf-next 2/3] selftests/bpf: name the failing registration in bpf_testmod_init
tools/testing/selftests/bpf/test_kmods/bpf_testmod.c static int bpf_testmod_init(void) · line 2604
Quoted 9 lines · Wang Yibo
tools/testing/selftests/bpf/test_kmods/bpf_testmod.c +9
-0+#define TRY (expr) \ + do { \ + ret = (expr); \ + if (ret < 0) { \ + pr_err("%s: %s failed, ret %d\n", \ + __func__, #expr, ret); \ + return ret; \ + } \ + } while (0)
pls no macros with hidden 'return'.
I also don't feel that the whole thing (patch 1 and the rest) is necessary.
pw-bot: cr
PATCH 3/3 selftests/bpf: say how to fix the missing LLVM skip for jited checks
In reply to Wang YiboWhen test_progs is built without LLVM development libraries, every subtest that checks jited output is skipped with only "tests are built w/o LLVM development libs" visible under -v. The message does not say that installing the libraries and rebuilding is all that is needed, and the verbose-only message in jit_disasm_helpers.c is missing a trailing newline, so it runs together with the following output.
Mention the fix in the skip message and add the newline.
--- tools/testing/selftests/bpf/jit_disasm_helpers.c | 2 +- tools/testing/selftests/bpf/test_loader.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
tools/testing/selftests/bpf/jit_disasm_helpers.c +1 -1
@@ -247,7 +247,7 @@ int get_jited_program_text(int fd, char *text, size_t text_sz)int get_jited_program_text (int fd,char *text,size_t text_sz) {if (env.verbosity >=VERBOSE_VERY ) -printf ("compiled w/o llvm development libraries, can't dis-assembly binary code" ); +printf ("compiled w/o llvm development libraries, can't disassemble binary code\n" );return -EOPNOTSUPP ; }
tools/testing/selftests/bpf/test_loader.c +1 -1
@@ -1506,7 +1506,7 @@ void run_subtest(struct test_loader *tester, tester->log_buf, tester->log_buf_sz);if (err == -EOPNOTSUPP ) {printf ("%s:SKIP: jited programs disassembly is not supported,\n" , __func__); -printf ("%s:SKIP: tests are built w/o LLVM development libs\n" , __func__); +printf ("%s:SKIP: rebuild with LLVM development libs installed\n" , __func__);test__skip (); goto tobj_cleanup; }
Signature · 1 line
2.51.0
[email protected]
AI reviewRe: [PATCH bpf-next 3/3] selftests/bpf: say how to fix the missing LLVM skip for jited checks
TL;DR: The new skip message says installing LLVM libs and rebuilding suffices, but a stale stub jit_disasm_helpers.o may be kept, so the skip could persist; a clean rebuild or SKIP_LLVM may need…
Quoted 1 line
selftests/bpf: say how to fix the missing LLVM skip for jited checks
TL;DR: The new skip message says installing LLVM libs and rebuilding suffices, but a stale stub jit_disasm_helpers.o may be kept, so the skip could persist; a clean rebuild or SKIP_LLVM may need mentioning.
tools/testing/selftests/bpf/test_loader.c void run_subtest(struct test_loader *tester, · line 1509
Quoted 10 lines
tools/testing/selftests/bpf/test_loader.c +1
-1@@ -1506,7 +1506,7 @@ void run_subtest(struct test_loader *tester, tester->log_buf, tester->log_buf_sz);if (err == -EOPNOTSUPP ) {printf ("%s:SKIP: jited programs disassembly is not supported,\n" , __func__); -printf ("%s:SKIP: tests are built w/o LLVM development libs\n" , __func__); +printf ("%s:SKIP: rebuild with LLVM development libs installed\n" , __func__);
Is "rebuild with LLVM development libs installed" enough to get the jited checks running?
The commit message frames the fix the same way, saying the old message "does not say that installing the libraries and rebuilding is all that is needed".
The stub get_jited_program_text() is chosen at compile time by -DHAVE_LLVM_SUPPORT, which the Makefile adds only through a target-specific variable:
$(OUTPUT)/jit_disasm_helpers.o: CFLAGS += $(LLVM_CFLAGS)
jit_disasm_helpers.o is built by the generic rule:
$(OUTPUT)/%.o: %.c
and that rule has only the .c file as a prerequisite. Nothing ties the object to the LLVM feature detection result or to the CFLAGS used.
If a user sees this message, installs the LLVM development packages and runs make again, would the existing jit_disasm_helpers.o with the stub be kept, so that the same -EOPNOTSUPP skip comes back?
A build with SKIP_LLVM=1 would also need that variable dropped, which the message does not mention.
Could the text say that a clean rebuild may be needed (removing jit_disasm_helpers.o or running make clean), or name SKIP_LLVM, instead of implying that installing the libraries and rebuilding is sufficient?
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/39699006936