selftests/bpf: unblock bpf-gcc test coverage
Subsystems (3)
cover selftests/bpf: unblock bpf-gcc test coverage
When doing some unrelated testing, stumbled upon this. A lot of tests are skipped run under BPF_GCC due to clang only gate. On top, the test harness declares them passing/OK.
Patch 1 renames a few placeholders a consistent dummy_test name [NFC].
A sample test with BPF_GCC used to say:
| #587/1 verifier_bswap/cpuv4 is not supported by compiler or jit, use a dummy test:OK
| #587 verifier_bswap:OK
Patch 2 adds a __skip("reason") annotation so a compile-time gated file reports SKIP rather than OK, making these gaps visible in test_progs output.
Now the output changes to
| #587/1 verifier_bswap/cpuv4 is not supported by compiler or jit, use a dummy test:SKIP
| #587 verifier_bswap:SKIP
Patch 3 augments the clang only gate with a per-feature gate supported by both the compilers. FWIW the clang half has to stay: clang only defines __BPF_FEATURE_MOVSX and friends at -mcpu=v4 but assembles the inline asm at -mcpu=v3 too, so dropping it would remove these tests from the default test_progs flavour where they run today.
And finally the output now is
| 587/1 verifier_bswap/BSWAP, 16:OK | #587/2 verifier_bswap/BSWAP, 16 @unpriv:SKIP | #587/3 verifier_bswap/BSWAP, 32:OK | #587/4 verifier_bswap/BSWAP, 32 @unpriv:SKIP | #587/5 verifier_bswap/BSWAP, 64:OK | #587/6 verifier_bswap/BSWAP, 64 @unpriv:SKIP | #587/7 verifier_bswap/bswap16_range:OK | #587/8 verifier_bswap/bswap32_range:OK | #587/9 verifier_bswap/bswap64_range:OK | #587/10 verifier_bswap/be16_range:OK | #587/11 verifier_bswap/be32_range:OK | #587/12 verifier_bswap/be64_range:OK | #587/13 verifier_bswap/le16_range:OK | #587/14 verifier_bswap/le32_range:OK | #587/15 verifier_bswap/le64_range:OK | #587/16 verifier_bswap/BSWAP, reset reg id:OK | #587 verifier_bswap:OK (SKIP: 3/16)
Programs per object recovered under BPF_GCC, 124 in total:
verifier_sdiv 1 -> 80 verifier_movsx 1 -> 17 verifier_ldsx 1 -> 14 verifier_bswap 1 -> 13
compute_live_registers 17 -> 19
verifier_gotol 1 -> 2
verifier_iterating_callbacks 34 -> 35
Two things stay clang-only on purpose: the arena tests using addr_space_cast, and verifier_load_acquire/verifier_store_release. gas implements neither.
The increased coverage bore fruits right away as I stumbled into a gas
bug: PR gas/3455. Fix is posted however workaround is needed for the time being.
Built for x86_64 with both clang and bpf-gcc; the per-object counts above were read back from the generated objects.
Vineet Gupta (3):
selftests/bpf: name the remaining placeholder programs dummy_test
[NFC]
selftests/bpf: report placeholder tests as SKIP, not OK selftests/bpf: Enable some of the blocked cpuv4 instruction tests for
bpf-gcc
.../testing/selftests/bpf/progs/arena_kfunc.c | 3 ++- tools/testing/selftests/bpf/progs/bpf_misc.h | 10 ++++++++-- .../bpf/progs/compute_live_registers.c | 7 +++++++ .../selftests/bpf/progs/stack_arg_fail.c | 3 ++- .../selftests/bpf/progs/stack_arg_precision.c | 1 + .../selftests/bpf/progs/verifier_bswap.c | 3 ++- .../selftests/bpf/progs/verifier_gotol.c | 1 + .../selftests/bpf/progs/verifier_ldsx.c | 19 ++++++++++++------- .../bpf/progs/verifier_load_acquire.c | 1 + .../selftests/bpf/progs/verifier_movsx.c | 3 ++- .../bpf/progs/verifier_private_stack.c | 1 + .../selftests/bpf/progs/verifier_sdiv.c | 3 ++- .../selftests/bpf/progs/verifier_stack_arg.c | 1 + .../bpf/progs/verifier_stack_arg_order.c | 1 + .../bpf/progs/verifier_store_release.c | 1 + tools/testing/selftests/bpf/test_loader.c | 9 +++++++++ 16 files changed, 53 insertions(+), 14 deletions(-)
Signature · 1 line
2.53.0-Meta
PATCH 1/3 selftests/bpf: name the remaining placeholder programs dummy_test [NFC]
Eleven of the thirteen feature-gated placeholder programs are called dummy_test. Two are not: hence this patch.
This is purely cosmetic as nothing depends on those names. The non-complaint setups use RUN_TESTS(skel) which does thru all programs normally. This is just more consistent and grep friendly.
--- tools/testing/selftests/bpf/progs/arena_kfunc.c | 2 +- tools/testing/selftests/bpf/progs/stack_arg_fail.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
tools/testing/selftests/bpf/progs/arena_kfunc.c +1 -1
@@ -234,7 +234,7 @@ __arch_x86_64 __arch_arm64 __description("arena_arg_stack: not supported, dummy test")__success -int arena_arg_stack(void *ctx) +int dummy_test(void *ctx) {return 0 ; }
tools/testing/selftests/bpf/progs/stack_arg_fail.c +1 -1
@@ -104,7 +104,7 @@ __naked void r11_store_zero_off(void) SEC("tc") __description("stack_arg_fail: not supported, dummy test") __success -int test_stack_arg_big (struct __sk_buff *skb) +int dummy_test (struct __sk_buff *skb) {return 0 ; }
Signature · 1 line
2.53.0-Meta
PATCH 2/3 selftests/bpf: report placeholder tests as SKIP, not OK
In reply to Vineet GuptaSeveral verifier test files wrap their contents in a feature gate and provide a one-line placeholder program in the #else arm, so the file still yields something when the toolchain or JIT cannot support the tested instructions.
However the placeholder is annotated __success, so it reports as a pass, whereas this is really a SKIP.
| #318/1 verifier_ldsx/cpuv4 is not supported by compiler or jit, ...:OK | #318 verifier_ldsx:OK
This causes skipped coverage to be overlooked: see the next patch where BPF_GCC builds miss a whole bunch of test due to clang only gate.
Add a __skip("reason") annotation for a compile-time gate, and use it for the placeholders.
After this change, above tests reported as
| #318/1 verifier_ldsx/cpuv4 is not supported by compiler or jit, ...:SKIP | #318 verifier_ldsx:SKIP
--- tools/testing/selftests/bpf/progs/arena_kfunc.c | 1 + tools/testing/selftests/bpf/progs/bpf_misc.h | 6 ++++++ tools/testing/selftests/bpf/progs/stack_arg_fail.c | 1 + tools/testing/selftests/bpf/progs/stack_arg_precision.c | 1 + tools/testing/selftests/bpf/progs/verifier_bswap.c | 1 + tools/testing/selftests/bpf/progs/verifier_gotol.c | 1 + tools/testing/selftests/bpf/progs/verifier_ldsx.c | 1 + .../testing/selftests/bpf/progs/verifier_load_acquire.c | 1 + tools/testing/selftests/bpf/progs/verifier_movsx.c | 1 + .../testing/selftests/bpf/progs/verifier_private_stack.c | 1 + tools/testing/selftests/bpf/progs/verifier_sdiv.c | 1 + tools/testing/selftests/bpf/progs/verifier_stack_arg.c | 1 + .../selftests/bpf/progs/verifier_stack_arg_order.c | 1 + .../testing/selftests/bpf/progs/verifier_store_release.c | 1 + tools/testing/selftests/bpf/test_loader.c | 9 +++++++++ 15 files changed, 28 insertions(+)
tools/testing/selftests/bpf/progs/arena_kfunc.c +1 -0
@@ -233,6 +233,7 @@ SEC("syscall")__arch_x86_64 __arch_arm64 __description("arena_arg_stack: not supported, dummy test") +__skip("arena_arg_stack: not supported") __successint dummy_test (void *ctx) {
tools/testing/selftests/bpf/progs/bpf_misc.h +6 -0
@@ -106,6 +106,11 @@* __description Text to be used for display and as an additional filter * alias, while the original program name stays matchable. * +* __skip Report the test as SKIP with the given reason instead of +* running it. For placeholder programs that stand in for a +* feature the toolchain or JIT cannot provide, so a run does +* not report OK for coverage it never executed. +* * __log_level Log level to use for the program, numeric value expected. * * __flag Adds one flag use for the program, the following values are valid:
@@ -139,6 +144,7 @@#define __failure __test_tag("test_expect_failure")#define __success __test_tag("test_expect_success")#define __description (desc) __test_tag("test_description=" desc) +#define __skip (reason) __test_tag("test_skip=" reason)#define __msg_unpriv (msg) __test_tag("test_expect_msg_unpriv=" msg)#define __not_msg_unpriv (msg) __test_tag("test_expect_not_msg_unpriv=" msg)#define __xlated_unpriv (msg) __test_tag("test_expect_xlated_unpriv=" msg)
tools/testing/selftests/bpf/progs/stack_arg_fail.c +1 -0
@@ -103,6 +103,7 @@ __naked void r11_store_zero_off(void) SEC("tc") __description("stack_arg_fail: not supported, dummy test") +__skip("stack_arg_fail: not supported") __successint dummy_test (struct __sk_buff *skb) {
tools/testing/selftests/bpf/progs/stack_arg_precision.c +1 -0
@@ -124,6 +124,7 @@ __naked void stack_arg_precision_bpf2bpf(void) SEC("socket") __description("stack_arg_precision: not supported, dummy test") +__skip("stack_arg_precision: not supported") __successint dummy_test (void ) {
tools/testing/selftests/bpf/progs/verifier_bswap.c +1 -0
@@ -117,6 +117,7 @@ l0_%=: \ SEC("socket") __description("cpuv4 is not supported by compiler or jit, use a dummy test") +__skip("cpuv4 is not supported by compiler or jit") __successint dummy_test (void ) {
tools/testing/selftests/bpf/progs/verifier_gotol.c +1 -0
@@ -52,6 +52,7 @@ __naked void gotol_large_imm(void) SEC("socket") __description("cpuv4 is not supported by compiler or jit, use a dummy test") +__skip("cpuv4 is not supported by compiler or jit") __successint dummy_test (void ) {
tools/testing/selftests/bpf/progs/verifier_ldsx.c +1 -0
@@ -436,6 +436,7 @@ void kfunc_root(void) SEC("socket") __description("cpuv4 is not supported by compiler or jit, use a dummy test") +__skip("cpuv4 is not supported by compiler or jit") __successint dummy_test (void ) {
tools/testing/selftests/bpf/progs/verifier_load_acquire.c +1 -0
@@ -267,6 +267,7 @@ __naked void load_acquire_with_invalid_reg(void) SEC("socket") __description("Clang version < 18, ENABLE_ATOMICS_TESTS not defined, and/or JIT doesn't support load-acquire, use a dummy test") +__skip("Clang version < 18, ENABLE_ATOMICS_TESTS not defined, and/or JIT doesn't support load-acquire") __successint dummy_test (void ) {
tools/testing/selftests/bpf/progs/verifier_movsx.c +1 -0
@@ -343,6 +343,7 @@ label_%=: \ SEC("socket") __description("cpuv4 is not supported by compiler or jit, use a dummy test") +__skip("cpuv4 is not supported by compiler or jit") __successint dummy_test (void ) {
tools/testing/selftests/bpf/progs/verifier_private_stack.c +1 -0
@@ -377,6 +377,7 @@ int private_stack_max_depth(void) SEC("kprobe") __description("private stack is not supported, use a dummy test") +__skip("private stack is not supported") __successint dummy_test (void ) {
tools/testing/selftests/bpf/progs/verifier_sdiv.c +1 -0
@@ -1271,6 +1271,7 @@ __naked void smod32_int_min_mod_neg2_imm(void) SEC("socket") __description("cpuv4 is not supported by compiler or jit, use a dummy test") +__skip("cpuv4 is not supported by compiler or jit") __successint dummy_test (void ) {
tools/testing/selftests/bpf/progs/verifier_stack_arg.c +1 -0
@@ -436,6 +436,7 @@ __naked void stack_arg_sequential_calls(void) SEC("socket") __description("stack_arg is not supported by compiler or jit, use a dummy test") +__skip("stack_arg is not supported by compiler or jit") __successint dummy_test (void ) {
tools/testing/selftests/bpf/progs/verifier_stack_arg_order.c +1 -0
@@ -174,6 +174,7 @@ __naked void stack_arg_read_without_write_2(void) SEC("socket") __description("stack_arg order is not supported by compiler or jit, use a dummy test") +__skip("stack_arg order is not supported by compiler or jit") __successint dummy_test (void ) {
tools/testing/selftests/bpf/progs/verifier_store_release.c +1 -0
@@ -290,6 +290,7 @@ __naked void store_release_with_invalid_reg(void) SEC("socket") __description("Clang version < 18, ENABLE_ATOMICS_TESTS not defined, and/or JIT doesn't support store-release, use a dummy test") +__skip("Clang version < 18, ENABLE_ATOMICS_TESTS not defined, and/or JIT doesn't support store-release") __successint dummy_test (void ) {
tools/testing/selftests/bpf/test_loader.c +9 -0
@@ -70,6 +70,7 @@ struct test_spec {int arch_mask;int load_mask;int linear_sz; +const char *skip_reason;bool auxiliary;bool valid; };
@@ -456,6 +457,8 @@ static int parse_test_spec(struct test_loader *tester,continue ;if ((val =str_has_pfx (s,"test_description=" ))) { description = val; + }else if ((val =str_has_pfx (s,"test_skip=" ))) { + spec->skip_reason = val; }else if (strcmp (s,"test_expect_failure" ) ==0 ) { spec->priv.expect_failure = true; spec->mode_mask |=PRIV ;
@@ -1327,6 +1330,12 @@ void run_subtest(struct test_loader *tester,if (!test__start_subtest_with_desc (subspec->name, subspec->description))return ; +if (spec->skip_reason) { +printf ("%s:SKIP: %s\n" , __func__, spec->skip_reason); +test__skip (); +return ; + } +if ((get_current_arch () & spec->arch_mask) ==0 ) {test__skip ();return ;
Signature · 1 line
2.53.0-Meta
PATCH 3/3 selftests/bpf: Enable some of the blocked cpuv4 instruction tests for bpf-gcc
In reply to Vineet GuptaA bunch of tests are gated behind clang only __clang_major__ and thus fail to build under BPF_GCC. What's worse is, before the prev patch they were also declared as passing/OK.
This need not be because (a) bpf-gcc does support the feature in codegen and (b) also advertises correctly with feature support preprocessor macros:
__BPF_FEATURE_BSWAP, __BPF_FEATURE_GOTOL, __BPF_FEATURE_LDSX, __BPF_FEATURE_MOVSX, __BPF_FEATURE_SDIV_SMOD, __BPF_FEATURE_ST
So loosen up the gates to also add specific feature check. clang >=18 has to stay, despite it being true for so long: clang only defines __BPF_FEATURE_MOVSX and friends at -mcpu=v4, but assembles the inline asm at -mcpu=v3 too, and dropping it would remove these tests from the default test_progs flavour where they run today.
Since bpf-gcc doesn't support addr_space_cast, wrap the corresponding tests under __BPF_FEATURE_ADDR_SPACE_CAST to skip under it. And also move the map under that guard so that a bpf-gcc object does not declare an arena map that no program in it references.
Recovered under BPF_GCC (programs per object, was 1 dummy each):
verifier_sdiv 80 verifier_movsx 17 verifier_ldsx 14 (20 under clang; 5 arena tests,
kfunc_root and the arena map stay clang-only)
verifier_bswap 13
CAN_USE_GOTOL and CAN_USE_BPF_ST only gate parts of a file, so those were not dummies to begin with:
compute_live_registers 17 -> 19 (21 under clang)
verifier_gotol 1 -> 2
verifier_iterating_callbacks 34 -> 35
The clang builds are unchanged: verifier_ldsx still emits 20 programs under the cpuv4 flavour.
One thing left alone: verifier_load_acquire.c and verifier_store_release.c stay clang-only, and their gates are correct as gas doesn't implement those (experimental -mxbpf not implemented at all). FWIW LLVM groups those under -mcpu=v4 and defines __BPF_FEATURE_LOAD_ACQ_STORE_REL there.
--- tools/testing/selftests/bpf/progs/bpf_misc.h | 4 ++-- .../bpf/progs/compute_live_registers.c | 7 +++++++ .../selftests/bpf/progs/verifier_bswap.c | 2 +- .../selftests/bpf/progs/verifier_ldsx.c | 18 +++++++++++------- .../selftests/bpf/progs/verifier_movsx.c | 2 +- .../selftests/bpf/progs/verifier_sdiv.c | 2 +- 6 files changed, 23 insertions(+), 12 deletions(-)
tools/testing/selftests/bpf/progs/bpf_misc.h +2 -2
@@ -261,11 +261,11 @@ (defined (__TARGET_ARCH_riscv) && __riscv_xlen ==64 ) || \defined (__TARGET_ARCH_arm) ||defined (__TARGET_ARCH_s390) || \ defined(__TARGET_ARCH_loongarch)) && \ - __clang_major__ >= 18 + (__clang_major__ >= 18 || defined(__BPF_FEATURE_GOTOL))#define CAN_USE_GOTOL #endif -#if __clang_major__ >=18 +#if __clang_major__ >=18 || defined(__BPF_FEATURE_ST)#define CAN_USE_BPF_ST #endif
tools/testing/selftests/bpf/progs/compute_live_registers.c +7 -0
@@ -299,7 +299,14 @@ __naked void gotol(void) "r3 = 24;" "if r1 > 0x7 goto +2;" "r0 = r2;" +#ifdef __clang__ "gotol +1;" +#else +/* gas mis-parses 'gotol +1' as 'goto l+1', same encoding +* without the sign (binutils PR gas/34558). +*/ +"gotol 1;" +#endif"r0 = r3;" "exit;" :
tools/testing/selftests/bpf/progs/verifier_bswap.c +1 -1
@@ -8,7 +8,7 @@ (defined (__TARGET_ARCH_riscv) && __riscv_xlen ==64 ) || \defined (__TARGET_ARCH_arm) ||defined (__TARGET_ARCH_s390) || \ defined(__TARGET_ARCH_loongarch)) && \ - __clang_major__ >= 18 + (__clang_major__ >= 18 || defined(__BPF_FEATURE_BSWAP)) SEC("socket")__description ("BSWAP, 16" )
tools/testing/selftests/bpf/progs/verifier_ldsx.c +11 -7
@@ -9,13 +9,7 @@ (defined (__TARGET_ARCH_riscv) && __riscv_xlen ==64 ) || \defined (__TARGET_ARCH_arm) ||defined (__TARGET_ARCH_s390) || \ defined(__TARGET_ARCH_loongarch)) && \ - __clang_major__ >= 18 - -struct { - __uint(type, BPF_MAP_TYPE_ARENA); - __uint(map_flags, BPF_F_MMAPABLE); - __uint(max_entries, 1); -} arena SEC(".maps"); + (__clang_major__ >= 18 || defined(__BPF_FEATURE_LDSX)) SEC("socket")__description ("LDSX, S8" )
@@ -263,6 +257,14 @@ __naked void ldsx_ctx_8(void) : __clobber_all); } +#ifdef __BPF_FEATURE_ADDR_SPACE_CAST + +struct { + __uint(type, BPF_MAP_TYPE_ARENA); + __uint(map_flags, BPF_F_MMAPABLE); + __uint(max_entries, 1); +} arena SEC(".maps"); + SEC("syscall")__description ("Arena LDSX Disasm" ) __success
@@ -432,6 +434,8 @@ void kfunc_root(void) bpf_arena_alloc_pages(0, 0, 0, 0, 0); } +#endif /* __BPF_FEATURE_ADDR_SPACE_CAST */ +#else SEC ("socket" )
tools/testing/selftests/bpf/progs/verifier_movsx.c +1 -1
@@ -8,7 +8,7 @@ (defined (__TARGET_ARCH_riscv) && __riscv_xlen ==64 ) || \defined (__TARGET_ARCH_arm) ||defined (__TARGET_ARCH_s390) || \ defined(__TARGET_ARCH_loongarch)) && \ - __clang_major__ >= 18 + (__clang_major__ >= 18 || defined(__BPF_FEATURE_MOVSX)) SEC("socket")__description ("MOV32SX, S8" )
tools/testing/selftests/bpf/progs/verifier_sdiv.c +1 -1
@@ -9,7 +9,7 @@ (defined (__TARGET_ARCH_riscv) && __riscv_xlen ==64 ) || \defined (__TARGET_ARCH_arm) ||defined (__TARGET_ARCH_s390) || \ defined(__TARGET_ARCH_loongarch)) && \ - __clang_major__ >= 18 + (__clang_major__ >= 18 || defined(__BPF_FEATURE_SDIV_SMOD)) SEC("socket")__description ("SDIV32, non-zero imm divisor, check 1" )
Signature · 1 line
2.53.0-Meta
[email protected]
AI reviewRe: [PATCH bpf-next 3/3] selftests/bpf: Enable some of the blocked cpuv4 instruction tests for bpf-gcc
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Low] The newly added multi-line comment for the gas bug workaround does not follow the required BPF…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] The newly added multi-line comment for the gas bug workaround does not follow the required BPF subsystem comment style. --
--- Patch [3]: [PATCH bpf-next 3/3] selftests/bpf: Enable some of the blocked cpuv4 instruction tests for bpf-gcc --- commit 55694625f0a27b85eb77860cd1b4f07749dec04c
Author: Vineet Gupta <[email protected]>
selftests/bpf: Enable some of the blocked cpuv4 instruction tests for bpf-gcc
This commit enables several blocked BPF CPUv4 instruction tests for bpf-gcc by checking specific feature macros instead of relying solely on clang checks. It also skips address space cast tests for bpf-gcc as the feature is not currently supported, while retaining the existing tests for clang.
Quoted 5 lines
tools/testing/selftests/bpf/progs/compute_live_registers.c +0
-0
tools/testing/selftests/bpf/progs/compute_live_registers.c __naked void gotol(void) · line 307
Quoted 9 lines
tools/testing/selftests/bpf/progs/compute_live_registers.c +5
-0@@ -299,7 +299,14 @@ __naked void gotol(void)"r3 = 24;" "if r1 > 0x7 goto +2;" "r0 = r2;" +#ifdef __clang__"gotol +1;" +#else +/* gas mis-parses 'gotol +1' as 'goto l+1', same encoding +* without the sign (binutils PR gas/34558). +*/
[Severity: Low]
This isn't a bug, but does this multi-line comment follow the preferred BPF subsystem comment style?
The BPF subsystem requires the opening /* to be on its own line:
/* * gas mis-parses 'gotol +1' as 'goto l+1', same encoding * without the sign (binutils PR gas/34558). */
Quoted 5 lines
tools/testing/selftests/bpf/progs/compute_live_registers.c +2
-0+"gotol 1;" +#endif"r0 = r3;" "exit;" :
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=3
[email protected]
AI reviewRe: [PATCH bpf-next 2/3] selftests/bpf: report placeholder tests as SKIP, not OK
In reply to Vineet GuptaThe commit converts 13 placeholder programs to use the new __skip() annotation, but two feature-gated placeholders with the identical pattern were not converted. Were these intentionally excluded?
Quoted 22 lines
tools/testing/selftests/bpf/progs/bpf_misc.h +6
-0@@ -106,6 +106,11 @@* __description Text to be used for display and as an additional filter * alias, while the original program name stays matchable. * +* __skip Report the test as SKIP with the given reason instead of +* running it. For placeholder programs that stand in for a +* feature the toolchain or JIT cannot provide, so a run does +* not report OK for coverage it never executed. +* * __log_level Log level to use for the program, numeric value expected. * * __flag Adds one flag use for the program, the following values are valid: @@ -139,6 +144,7 @@#define __failure __test_tag("test_expect_failure")#define __success __test_tag("test_expect_success")#define __description (desc) __test_tag("test_description=" desc) +#define __skip (reason) __test_tag("test_skip=" reason)#define __msg_unpriv (msg) __test_tag("test_expect_msg_unpriv=" msg)
[ ... ]
Quoted 12 lines
tools/testing/selftests/bpf/progs/verifier_ldsx.c +1
-0@@ -436,6 +436,7 @@ void kfunc_root(void) SEC("socket") __description("cpuv4 is not supported by compiler or jit, use a dummy test") +__skip("cpuv4 is not supported by compiler or jit") __successint dummy_test (void ) {
[ ... ]
Quoted 24 lines
tools/testing/selftests/bpf/test_loader.c +3
-0@@ -70,6 +70,7 @@ struct test_spec {int arch_mask;int load_mask;int linear_sz; +const char *skip_reason;bool auxiliary;bool valid; };@@ -456,6 +457,8 @@ static int parse_test_spec(struct test_loader *tester,continue ;if ((val =str_has_pfx (s,"test_description=" ))) { description = val; + }else if ((val =str_has_pfx (s,"test_skip=" ))) { + spec->skip_reason = val; }else if (strcmp (s,"test_expect_failure" ) ==0 ) { spec->priv.expect_failure = true; spec->mode_mask |=PRIV ;@@ -1327,6 +1330,12 @@ void run_subtest(struct test_loader *tester,if (!test__start_subtest_with_desc (subspec->name, subspec->description))return ;
tools/testing/selftests/bpf/test_loader.c void run_subtest(struct test_loader *tester, · line 1341
Quoted 10 lines
tools/testing/selftests/bpf/test_loader.c +6
-0+if (spec->skip_reason) { +printf ("%s:SKIP: %s\n" , __func__, spec->skip_reason); +test__skip (); +return ; + } +if ((get_current_arch () & spec->arch_mask) ==0 ) {test__skip ();return ;
The commit converts 13 placeholder programs to use the new __skip() annotation, but two feature-gated placeholders with the identical pattern were not converted. Were these intentionally excluded?
The preceding commit b1831d82fe2c states "Eleven of the thirteen feature-gated placeholder programs are called dummy_test", but grep finds fifteen such programs.
tools/testing/selftests/bpf/progs/verifier_aggregate_ret.c has a compile-time gate at line 8:
#if defined(__clang_major__) && __clang_major__ >= 23
with its #else arm at lines 166-172 supplying the same __success placeholder pattern that this commit converts everywhere else:
#else
SEC("socket")
__description("verifier_aggregate_ret: needs LLVM 23, dummy test") __success
int dummy_test(void)
Similarly, tools/testing/selftests/bpf/progs/verifier_percpu_addr.c has a gate at line 7:
#if defined(__TARGET_ARCH_x86)
with its #else placeholder at lines 60-66:
#else
SEC("raw_tp")
__description("percpu addr dummy")
__success
int dummy_test(void)
Both reach the modified test_loader.c code path via prog_tests/verifier.c (test_verifier_aggregate_ret and test_verifier_percpu_addr), so __skip() would work for them.
The gates fire today, so the placeholders are what actually gets built and reported:
- verifier_aggregate_ret.c: any toolchain older than clang 23
- verifier_percpu_addr.c: every non-x86 build (arm64, s390x, riscv64, loongarch)
On those builds these two tests still report OK for coverage that never executed.
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/32523960074
Re: [PATCH bpf-next 3/3] selftests/bpf: Enable some of the blocked cpuv4 instruction tests for bpf-gcc
In reply to [email protected]Quoted 33 lines · On 8/21/26 1:19 PM, [email protected] wrote:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] The newly added multi-line comment for the gas bug workaround does not follow the required BPF subsystem comment style. ----- Patch [3]: [PATCH bpf-next 3/3] selftests/bpf: Enable some of the blocked cpuv4 instruction tests for bpf-gcc --- commit
55694625f0a27b85eb77860cd1b4f07749dec04c
Author: Vineet Gupta <[email protected]>selftests/bpf: Enable some of the blocked cpuv4 instruction tests for bpf-gcc
This commit enables several blocked BPF CPUv4 instruction tests for bpf-gcc by checking specific feature macros instead of relying solely on clang checks. It also skips address space cast tests for bpf-gcc as the feature is not currently supported, while retaining the existing tests for clang.
Quoted 5 lines
tools/testing/selftests/bpf/progs/compute_live_registers.c +0
-0tools/testing/selftests/bpf/progs/compute_live_registers.c __naked void gotol(void) · line 307
Quoted 9 lines
tools/testing/selftests/bpf/progs/compute_live_registers.c +5
-0@@ -299,7 +299,14 @@ __naked void gotol(void)"r3 = 24;" "if r1 > 0x7 goto +2;" "r0 = r2;" +#ifdef __clang__"gotol +1;" +#else +/* gas mis-parses 'gotol +1' as 'goto l+1', same encoding +* without the sign (binutils PR gas/34558). +*/ [Severity: Low]
This isn't a bug, but does this multi-line comment follow the preferred BPF subsystem comment style?
Quoted 6 lines
The BPF subsystem requires the opening /* to be on its own line:
/* * gas mis-parses 'gotol +1' as 'goto l+1', same encoding * without the sign (binutils PR gas/34558). */
Right, I'll fix this in v2.
Thx,
-Vineet
Re: [PATCH bpf-next 2/3] selftests/bpf: report placeholder tests as SKIP, not OK
In reply to [email protected]Quoted 113 lines · On 8/21/26 2:00 PM, [email protected] wrote:
Quoted 22 lines
tools/testing/selftests/bpf/progs/bpf_misc.h +6
-0@@ -106,6 +106,11 @@* __description Text to be used for display and as an additional filter * alias, while the original program name stays matchable. * +* __skip Report the test as SKIP with the given reason instead of +* running it. For placeholder programs that stand in for a +* feature the toolchain or JIT cannot provide, so a run does +* not report OK for coverage it never executed. +* * __log_level Log level to use for the program, numeric value expected. * * __flag Adds one flag use for the program, the following values are valid: @@ -139,6 +144,7 @@#define __failure __test_tag("test_expect_failure")#define __success __test_tag("test_expect_success")#define __description (desc) __test_tag("test_description=" desc) +#define __skip (reason) __test_tag("test_skip=" reason)#define __msg_unpriv (msg) __test_tag("test_expect_msg_unpriv=" msg)[ ... ]
Quoted 12 lines
tools/testing/selftests/bpf/progs/verifier_ldsx.c +1
-0@@ -436,6 +436,7 @@ void kfunc_root(void) SEC("socket") __description("cpuv4 is not supported by compiler or jit, use a dummy test") +__skip("cpuv4 is not supported by compiler or jit") __successint dummy_test (void ) {[ ... ]
Quoted 24 lines
tools/testing/selftests/bpf/test_loader.c +3
-0@@ -70,6 +70,7 @@ struct test_spec {int arch_mask;int load_mask;int linear_sz; +const char *skip_reason;bool auxiliary;bool valid; };@@ -456,6 +457,8 @@ static int parse_test_spec(struct test_loader *tester,continue ;if ((val =str_has_pfx (s,"test_description=" ))) { description = val; + }else if ((val =str_has_pfx (s,"test_skip=" ))) { + spec->skip_reason = val; }else if (strcmp (s,"test_expect_failure" ) ==0 ) { spec->priv.expect_failure = true; spec->mode_mask |=PRIV ;@@ -1327,6 +1330,12 @@ void run_subtest(struct test_loader *tester,if (!test__start_subtest_with_desc (subspec->name, subspec->description))return ;tools/testing/selftests/bpf/test_loader.c void run_subtest(struct test_loader *tester, · line 1341
Quoted 10 lines
tools/testing/selftests/bpf/test_loader.c +6
-0+if (spec->skip_reason) { +printf ("%s:SKIP: %s\n" , __func__, spec->skip_reason); +test__skip (); +return ; + } +if ((get_current_arch () & spec->arch_mask) ==0 ) {test__skip ();return ;The commit converts 13 placeholder programs to use the new
__skip()annotation, but two feature-gated placeholders with the identical pattern were not converted. Were these intentionally excluded?The preceding commit
b1831d82fe2cstates "Eleven of the thirteen feature-gated placeholder programs are calleddummy_test", but grep finds fifteen such programs.
tools/testing/selftests/bpf/progs/verifier_aggregate_ret.chas a compile-time gate at line 8:#if defined(__clang_major__) && __clang_major__ >= 23with its #else arm at lines 166-172 supplying the same
__successplaceholder pattern that this commit converts everywhere else:#else
SEC("socket")
__description("verifier_aggregate_ret: needs LLVM 23, dummy test")__success
intdummy_test(void)Similarly,
tools/testing/selftests/bpf/progs/verifier_percpu_addr.chas a gate at line 7:#if defined(__TARGET_ARCH_x86)with its #else placeholder at lines 60-66:
#else
SEC("raw_tp")
__description("percpu addr dummy")
__success
intdummy_test(void)Both reach the modified
test_loader.ccode path viaprog_tests/verifier.c(test_verifier_aggregate_retandtest_verifier_percpu_addr), so__skip()would work for them.
Quoted 8 lines
The gates fire today, so the placeholders are what actually gets built and reported:
-verifier_aggregate_ret.c: any toolchain older than clang 23
-verifier_percpu_addr.c: every non-x86 build (arm64, s390x, riscv64, loongarch)On those builds these two tests still report OK for coverage that never executed.
Bot is correct. The reason is the older baseline I first worked against had the aforementioned numbers. After rebase I did remember to rerun the numbers but obviously there was some snafu. I'll fix this in v2.
Thx,
-Vineet
Re: [PATCH bpf-next 1/3] selftests/bpf: name the remaining placeholder programs dummy_test [NFC]
In reply to Vineet GuptaQuoted 9 lines · Vineet Gupta
Eleven of the thirteen feature-gated placeholder programs are called
dummy_test. Two are not: hence this patch.This is purely cosmetic as nothing depends on those names. The non-complaint setups use
RUN_TESTS(skel)which does thru all programs normally. This is just more consistent and grep friendly.Signed-off-by Vineet Gupta <[email protected]>---
Re: [PATCH bpf-next 2/3] selftests/bpf: report placeholder tests as SKIP, not OK
In reply to Vineet GuptaQuoted 16 lines · Vineet Gupta
Several verifier test files wrap their contents in a feature gate and provide a one-line placeholder program in the #else arm, so the file still yields something when the toolchain or JIT cannot support the tested instructions.
However the placeholder is annotated
__success, so it reports as a pass, whereas this is really a SKIP.Quoted 2 lines
#318/1 verifier_ldsx/cpuv4 is not supported by compiler or jit, ...:OK
#318 verifier_ldsx:OKThis causes skipped coverage to be overlooked: see the next patch where
BPF_GCCbuilds miss a whole bunch of test due to clang only gate.Add a
__skip("reason")annotation for a compile-time gate, and use it for the placeholders.
Quoted 7 lines
After this change, above tests reported as
Quoted 2 lines
#318/1 verifier_ldsx/cpuv4 is not supported by compiler or jit, ...:SKIP
#318 verifier_ldsx:SKIPSigned-off-by Vineet Gupta <[email protected]>---
Neat feature, thanks!
...
Re: [PATCH bpf-next 3/3] selftests/bpf: Enable some of the blocked cpuv4 instruction tests for bpf-gcc
In reply to Vineet GuptaQuoted 7 lines · Vineet Gupta
A bunch of tests are gated behind clang only
__clang_major__and thus fail to build underBPF_GCC. What's worse is, before the prev patch they were also declared as passing/OK.This need not be because (a) bpf-gcc does support the feature in codegen and (b) also advertises correctly with feature support preprocessor macros:
Quoted 8 lines
__BPF_FEATURE_BSWAP,__BPF_FEATURE_GOTOL,__BPF_FEATURE_LDSX,__BPF_FEATURE_MOVSX,__BPF_FEATURE_SDIV_SMOD,__BPF_FEATURE_STSo loosen up the gates to also add specific feature check. clang >=18 has to stay, despite it being true for so long: clang only defines
__BPF_FEATURE_MOVSXand friends at -mcpu=v4, but assembles the inline asm at -mcpu=v3 too, and dropping it would remove these tests from the defaulttest_progsflavour where they run today.
Huh, that's unfortunate.
Quoted 24 lines
Since bpf-gcc doesn't support
addr_space_cast, wrap the corresponding tests under__BPF_FEATURE_ADDR_SPACE_CASTto skip under it. And also move the map under that guard so that a bpf-gcc object does not declare an arena map that no program in it references.Recovered under
BPF_GCC(programs per object, was 1 dummy each):verifier_sdiv 80 verifier_movsx 17 verifier_ldsx 14 (20 under clang; 5 arena tests,
kfunc_rootand the arena map stay clang-only)
verifier_bswap13
CAN_USE_GOTOLandCAN_USE_BPF_STonly gate parts of a file, so those were not dummies to begin with:
compute_live_registers17 -> 19 (21 under clang)
verifier_gotol1 -> 2
verifier_iterating_callbacks34 -> 35The clang builds are unchanged:
verifier_ldsxstill emits 20 programs under the cpuv4 flavour.
Quoted 7 lines
One thing left alone:
verifier_load_acquire.candverifier_store_release.cstay clang-only, and their gates are correct as gas doesn't implement those (experimental -mxbpf not implemented at all). FWIW LLVM groups those under -mcpu=v4 and defines__BPF_FEATURE_LOAD_ACQ_STORE_RELthere.Signed-off-by Vineet Gupta <[email protected]>---
I'll wait for v2 and apply.
...