Skip to contentSkip to conversation

selftests/bpf: unblock bpf-gcc test coverage

PATCH Superseded Vineet Gupta 4 participants 11 messages Aug 21 – Aug 24

Subsystems (3)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover selftests/bpf: unblock bpf-gcc test coverage

When doing some unrelated testing, stumbled upon this. A lot of tests are skipped run under BPF_GCC due to clang only gate. On top, the test harness declares them passing/OK.

Patch 1 renames a few placeholders a consistent dummy_test name [NFC].

A sample test with BPF_GCC used to say:

| #587/1 verifier_bswap/cpuv4 is not supported by compiler or jit, use a dummy test:OK
| #587 verifier_bswap:OK

Patch 2 adds a __skip("reason") annotation so a compile-time gated file reports SKIP rather than OK, making these gaps visible in test_progs output.

Now the output changes to

| #587/1 verifier_bswap/cpuv4 is not supported by compiler or jit, use a dummy test:SKIP
| #587 verifier_bswap:SKIP

Patch 3 augments the clang only gate with a per-feature gate supported by both the compilers. FWIW the clang half has to stay: clang only defines __BPF_FEATURE_MOVSX and friends at -mcpu=v4 but assembles the inline asm at -mcpu=v3 too, so dropping it would remove these tests from the default test_progs flavour where they run today.

And finally the output now is

| 587/1	 verifier_bswap/BSWAP, 16:OK
| #587/2 verifier_bswap/BSWAP, 16 @unpriv:SKIP
| #587/3 verifier_bswap/BSWAP, 32:OK
| #587/4 verifier_bswap/BSWAP, 32 @unpriv:SKIP
| #587/5 verifier_bswap/BSWAP, 64:OK
| #587/6 verifier_bswap/BSWAP, 64 @unpriv:SKIP
| #587/7 verifier_bswap/bswap16_range:OK
| #587/8 verifier_bswap/bswap32_range:OK
| #587/9 verifier_bswap/bswap64_range:OK
| #587/10 verifier_bswap/be16_range:OK
| #587/11 verifier_bswap/be32_range:OK
| #587/12 verifier_bswap/be64_range:OK
| #587/13 verifier_bswap/le16_range:OK
| #587/14 verifier_bswap/le32_range:OK
| #587/15 verifier_bswap/le64_range:OK
| #587/16 verifier_bswap/BSWAP, reset reg id:OK
| #587	 verifier_bswap:OK (SKIP: 3/16)

Programs per object recovered under BPF_GCC, 124 in total:

  verifier_sdiv                  1 -> 80
  verifier_movsx                 1 -> 17
  verifier_ldsx                  1 -> 14
  verifier_bswap                 1 -> 13

compute_live_registers 17 -> 19
verifier_gotol 1 -> 2
verifier_iterating_callbacks 34 -> 35

Two things stay clang-only on purpose: the arena tests using addr_space_cast, and verifier_load_acquire/verifier_store_release. gas implements neither.

The increased coverage bore fruits right away as I stumbled into a gas
bug: PR gas/3455. Fix is posted however workaround is needed for the time being.

Built for x86_64 with both clang and bpf-gcc; the per-object counts above were read back from the generated objects.

Vineet Gupta (3):
selftests/bpf: name the remaining placeholder programs dummy_test

    [NFC]

selftests/bpf: report placeholder tests as SKIP, not OK selftests/bpf: Enable some of the blocked cpuv4 instruction tests for
bpf-gcc

 .../testing/selftests/bpf/progs/arena_kfunc.c |  3 ++-
 tools/testing/selftests/bpf/progs/bpf_misc.h  | 10 ++++++++--
 .../bpf/progs/compute_live_registers.c        |  7 +++++++
 .../selftests/bpf/progs/stack_arg_fail.c      |  3 ++-
 .../selftests/bpf/progs/stack_arg_precision.c |  1 +
 .../selftests/bpf/progs/verifier_bswap.c      |  3 ++-
 .../selftests/bpf/progs/verifier_gotol.c      |  1 +
 .../selftests/bpf/progs/verifier_ldsx.c       | 19 ++++++++++++-------
 .../bpf/progs/verifier_load_acquire.c         |  1 +
 .../selftests/bpf/progs/verifier_movsx.c      |  3 ++-
 .../bpf/progs/verifier_private_stack.c        |  1 +
 .../selftests/bpf/progs/verifier_sdiv.c       |  3 ++-
 .../selftests/bpf/progs/verifier_stack_arg.c  |  1 +
 .../bpf/progs/verifier_stack_arg_order.c      |  1 +
 .../bpf/progs/verifier_store_release.c        |  1 +
 tools/testing/selftests/bpf/test_loader.c     |  9 +++++++++
 16 files changed, 53 insertions(+), 14 deletions(-)
Signature · 1 line
2.53.0-Meta
#1 lore raw

PATCH 1/3 selftests/bpf: name the remaining placeholder programs dummy_test [NFC]

Eleven of the thirteen feature-gated placeholder programs are called dummy_test. Two are not: hence this patch.

This is purely cosmetic as nothing depends on those names. The non-complaint setups use RUN_TESTS(skel) which does thru all programs normally. This is just more consistent and grep friendly.

---
 tools/testing/selftests/bpf/progs/arena_kfunc.c    | 2 +-
 tools/testing/selftests/bpf/progs/stack_arg_fail.c | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
tools/testing/selftests/bpf/progs/arena_kfunc.c +1 -1
index bf0d304e0e59..47bed5c4c488 100644
@@ -234,7 +234,7 @@ __arch_x86_64
 __arch_arm64
 __description("arena_arg_stack: not supported, dummy test")
 __success
-int arena_arg_stack(void *ctx)
+int dummy_test(void *ctx)
 {
 	return 0;
 }
tools/testing/selftests/bpf/progs/stack_arg_fail.c +1 -1
index ad9d4bfe15dc..6f6f59306d0e 100644
@@ -104,7 +104,7 @@ __naked void r11_store_zero_off(void)
 SEC("tc")
 __description("stack_arg_fail: not supported, dummy test")
 __success
-int test_stack_arg_big(struct __sk_buff *skb)
+int dummy_test(struct __sk_buff *skb)
 {
 	return 0;
 }
Signature · 1 line
2.53.0-Meta
#3 lore raw

PATCH 2/3 selftests/bpf: report placeholder tests as SKIP, not OK

In reply to Vineet Gupta

Several verifier test files wrap their contents in a feature gate and provide a one-line placeholder program in the #else arm, so the file still yields something when the toolchain or JIT cannot support the tested instructions.

However the placeholder is annotated __success, so it reports as a pass, whereas this is really a SKIP.

|  #318/1   verifier_ldsx/cpuv4 is not supported by compiler or jit, ...:OK
|  #318     verifier_ldsx:OK

This causes skipped coverage to be overlooked: see the next patch where BPF_GCC builds miss a whole bunch of test due to clang only gate.

Add a __skip("reason") annotation for a compile-time gate, and use it for the placeholders.

After this change, above tests reported as

|  #318/1   verifier_ldsx/cpuv4 is not supported by compiler or jit, ...:SKIP
|  #318     verifier_ldsx:SKIP
---
 tools/testing/selftests/bpf/progs/arena_kfunc.c          | 1 +
 tools/testing/selftests/bpf/progs/bpf_misc.h             | 6 ++++++
 tools/testing/selftests/bpf/progs/stack_arg_fail.c       | 1 +
 tools/testing/selftests/bpf/progs/stack_arg_precision.c  | 1 +
 tools/testing/selftests/bpf/progs/verifier_bswap.c       | 1 +
 tools/testing/selftests/bpf/progs/verifier_gotol.c       | 1 +
 tools/testing/selftests/bpf/progs/verifier_ldsx.c        | 1 +
 .../testing/selftests/bpf/progs/verifier_load_acquire.c  | 1 +
 tools/testing/selftests/bpf/progs/verifier_movsx.c       | 1 +
 .../testing/selftests/bpf/progs/verifier_private_stack.c | 1 +
 tools/testing/selftests/bpf/progs/verifier_sdiv.c        | 1 +
 tools/testing/selftests/bpf/progs/verifier_stack_arg.c   | 1 +
 .../selftests/bpf/progs/verifier_stack_arg_order.c       | 1 +
 .../testing/selftests/bpf/progs/verifier_store_release.c | 1 +
 tools/testing/selftests/bpf/test_loader.c                | 9 +++++++++
 15 files changed, 28 insertions(+)
tools/testing/selftests/bpf/progs/arena_kfunc.c +1 -0
index 47bed5c4c488..50609f3b0564 100644
@@ -233,6 +233,7 @@ SEC("syscall")
 __arch_x86_64
 __arch_arm64
 __description("arena_arg_stack: not supported, dummy test")
+__skip("arena_arg_stack: not supported")
 __success
 int dummy_test(void *ctx)
 {
tools/testing/selftests/bpf/progs/bpf_misc.h +6 -0
index 5eacf1b43252..88b0bfba83bb 100644
@@ -106,6 +106,11 @@
  * __description     Text to be used for display and as an additional filter
  *                   alias, while the original program name stays matchable.
  *
+ * __skip            Report the test as SKIP with the given reason instead of
+ *                   running it. For placeholder programs that stand in for a
+ *                   feature the toolchain or JIT cannot provide, so a run does
+ *                   not report OK for coverage it never executed.
+ *
  * __log_level       Log level to use for the program, numeric value expected.
  *
  * __flag            Adds one flag use for the program, the following values are valid:
@@ -139,6 +144,7 @@
 #define __failure		__test_tag("test_expect_failure")
 #define __success		__test_tag("test_expect_success")
 #define __description(desc)	__test_tag("test_description=" desc)
+#define __skip(reason)		__test_tag("test_skip=" reason)
 #define __msg_unpriv(msg)	__test_tag("test_expect_msg_unpriv=" msg)
 #define __not_msg_unpriv(msg)	__test_tag("test_expect_not_msg_unpriv=" msg)
 #define __xlated_unpriv(msg)	__test_tag("test_expect_xlated_unpriv=" msg)
tools/testing/selftests/bpf/progs/stack_arg_fail.c +1 -0
index 6f6f59306d0e..eed97d582515 100644
@@ -103,6 +103,7 @@ __naked void r11_store_zero_off(void)
 
 SEC("tc")
 __description("stack_arg_fail: not supported, dummy test")
+__skip("stack_arg_fail: not supported")
 __success
 int dummy_test(struct __sk_buff *skb)
 {
tools/testing/selftests/bpf/progs/stack_arg_precision.c +1 -0
index bee2eeec021d..ce0301a41fa9 100644
@@ -124,6 +124,7 @@ __naked void stack_arg_precision_bpf2bpf(void)
 
 SEC("socket")
 __description("stack_arg_precision: not supported, dummy test")
+__skip("stack_arg_precision: not supported")
 __success
 int dummy_test(void)
 {
tools/testing/selftests/bpf/progs/verifier_bswap.c +1 -0
index cffaf36192bc..074faef5480e 100644
@@ -117,6 +117,7 @@ l0_%=:							\
 
 SEC("socket")
 __description("cpuv4 is not supported by compiler or jit, use a dummy test")
+__skip("cpuv4 is not supported by compiler or jit")
 __success
 int dummy_test(void)
 {
tools/testing/selftests/bpf/progs/verifier_gotol.c +1 -0
index d5d8f24df394..c0fd30a3a4c1 100644
@@ -52,6 +52,7 @@ __naked void gotol_large_imm(void)
 
 SEC("socket")
 __description("cpuv4 is not supported by compiler or jit, use a dummy test")
+__skip("cpuv4 is not supported by compiler or jit")
 __success
 int dummy_test(void)
 {
tools/testing/selftests/bpf/progs/verifier_ldsx.c +1 -0
index 41340877dc9d..e58ea69e3854 100644
@@ -436,6 +436,7 @@ void kfunc_root(void)
 
 SEC("socket")
 __description("cpuv4 is not supported by compiler or jit, use a dummy test")
+__skip("cpuv4 is not supported by compiler or jit")
 __success
 int dummy_test(void)
 {
tools/testing/selftests/bpf/progs/verifier_load_acquire.c +1 -0
index d17026d7480d..bcc34e798c63 100644
@@ -267,6 +267,7 @@ __naked void load_acquire_with_invalid_reg(void)
 
 SEC("socket")
 __description("Clang version < 18, ENABLE_ATOMICS_TESTS not defined, and/or JIT doesn't support load-acquire, use a dummy test")
+__skip("Clang version < 18, ENABLE_ATOMICS_TESTS not defined, and/or JIT doesn't support load-acquire")
 __success
 int dummy_test(void)
 {
tools/testing/selftests/bpf/progs/verifier_movsx.c +1 -0
index a4d8814eb5ed..5559c0583dd1 100644
@@ -343,6 +343,7 @@ label_%=: 	                                        \
 
 SEC("socket")
 __description("cpuv4 is not supported by compiler or jit, use a dummy test")
+__skip("cpuv4 is not supported by compiler or jit")
 __success
 int dummy_test(void)
 {
tools/testing/selftests/bpf/progs/verifier_private_stack.c +1 -0
index ea0a7e73331d..3d9994a52464 100644
@@ -377,6 +377,7 @@ int private_stack_max_depth(void)
 
 SEC("kprobe")
 __description("private stack is not supported, use a dummy test")
+__skip("private stack is not supported")
 __success
 int dummy_test(void)
 {
tools/testing/selftests/bpf/progs/verifier_sdiv.c +1 -0
index 95f3239ce228..515644ace516 100644
@@ -1271,6 +1271,7 @@ __naked void smod32_int_min_mod_neg2_imm(void)
 
 SEC("socket")
 __description("cpuv4 is not supported by compiler or jit, use a dummy test")
+__skip("cpuv4 is not supported by compiler or jit")
 __success
 int dummy_test(void)
 {
tools/testing/selftests/bpf/progs/verifier_stack_arg.c +1 -0
index 7e0ce5db28a0..51d22faf4559 100644
@@ -436,6 +436,7 @@ __naked void stack_arg_sequential_calls(void)
 
 SEC("socket")
 __description("stack_arg is not supported by compiler or jit, use a dummy test")
+__skip("stack_arg is not supported by compiler or jit")
 __success
 int dummy_test(void)
 {
tools/testing/selftests/bpf/progs/verifier_stack_arg_order.c +1 -0
index c9fe4857da3f..8e4325273b3d 100644
@@ -174,6 +174,7 @@ __naked void stack_arg_read_without_write_2(void)
 
 SEC("socket")
 __description("stack_arg order is not supported by compiler or jit, use a dummy test")
+__skip("stack_arg order is not supported by compiler or jit")
 __success
 int dummy_test(void)
 {
tools/testing/selftests/bpf/progs/verifier_store_release.c +1 -0
index 72f1eb006074..0abbee5ce109 100644
@@ -290,6 +290,7 @@ __naked void store_release_with_invalid_reg(void)
 
 SEC("socket")
 __description("Clang version < 18, ENABLE_ATOMICS_TESTS not defined, and/or JIT doesn't support store-release, use a dummy test")
+__skip("Clang version < 18, ENABLE_ATOMICS_TESTS not defined, and/or JIT doesn't support store-release")
 __success
 int dummy_test(void)
 {
tools/testing/selftests/bpf/test_loader.c +9 -0
index 07807757b518..794a7dfb0579 100644
@@ -70,6 +70,7 @@ struct test_spec {
 	int arch_mask;
 	int load_mask;
 	int linear_sz;
+	const char *skip_reason;
 	bool auxiliary;
 	bool valid;
 };
@@ -456,6 +457,8 @@ static int parse_test_spec(struct test_loader *tester,
 			continue;
 		if ((val = str_has_pfx(s, "test_description="))) {
 			description = val;
+		} else if ((val = str_has_pfx(s, "test_skip="))) {
+			spec->skip_reason = val;
 		} else if (strcmp(s, "test_expect_failure") == 0) {
 			spec->priv.expect_failure = true;
 			spec->mode_mask |= PRIV;
@@ -1327,6 +1330,12 @@ void run_subtest(struct test_loader *tester,
 	if (!test__start_subtest_with_desc(subspec->name, subspec->description))
 		return;
 
+	if (spec->skip_reason) {
+		printf("%s:SKIP: %s\n", __func__, spec->skip_reason);
+		test__skip();
+		return;
+	}
+
 	if ((get_current_arch() & spec->arch_mask) == 0) {
 		test__skip();
 		return;
Signature · 1 line
2.53.0-Meta
#7 lore raw

PATCH 3/3 selftests/bpf: Enable some of the blocked cpuv4 instruction tests for bpf-gcc

In reply to Vineet Gupta

A bunch of tests are gated behind clang only __clang_major__ and thus fail to build under BPF_GCC. What's worse is, before the prev patch they were also declared as passing/OK.

This need not be because (a) bpf-gcc does support the feature in codegen and (b) also advertises correctly with feature support preprocessor macros:

__BPF_FEATURE_BSWAP, __BPF_FEATURE_GOTOL, __BPF_FEATURE_LDSX, __BPF_FEATURE_MOVSX, __BPF_FEATURE_SDIV_SMOD, __BPF_FEATURE_ST

So loosen up the gates to also add specific feature check. clang >=18 has to stay, despite it being true for so long: clang only defines __BPF_FEATURE_MOVSX and friends at -mcpu=v4, but assembles the inline asm at -mcpu=v3 too, and dropping it would remove these tests from the default test_progs flavour where they run today.

Since bpf-gcc doesn't support addr_space_cast, wrap the corresponding tests under __BPF_FEATURE_ADDR_SPACE_CAST to skip under it. And also move the map under that guard so that a bpf-gcc object does not declare an arena map that no program in it references.

Recovered under BPF_GCC (programs per object, was 1 dummy each):

  verifier_sdiv    80
  verifier_movsx   17
  verifier_ldsx    14   (20 under clang; 5 arena tests,

kfunc_root and the arena map stay clang-only)
verifier_bswap 13

CAN_USE_GOTOL and CAN_USE_BPF_ST only gate parts of a file, so those were not dummies to begin with:

compute_live_registers 17 -> 19 (21 under clang)
verifier_gotol 1 -> 2
verifier_iterating_callbacks 34 -> 35

The clang builds are unchanged: verifier_ldsx still emits 20 programs under the cpuv4 flavour.

One thing left alone: verifier_load_acquire.c and verifier_store_release.c stay clang-only, and their gates are correct as gas doesn't implement those (experimental -mxbpf not implemented at all). FWIW LLVM groups those under -mcpu=v4 and defines __BPF_FEATURE_LOAD_ACQ_STORE_REL there.

---
 tools/testing/selftests/bpf/progs/bpf_misc.h   |  4 ++--
 .../bpf/progs/compute_live_registers.c         |  7 +++++++
 .../selftests/bpf/progs/verifier_bswap.c       |  2 +-
 .../selftests/bpf/progs/verifier_ldsx.c        | 18 +++++++++++-------
 .../selftests/bpf/progs/verifier_movsx.c       |  2 +-
 .../selftests/bpf/progs/verifier_sdiv.c        |  2 +-
 6 files changed, 23 insertions(+), 12 deletions(-)
tools/testing/selftests/bpf/progs/bpf_misc.h +2 -2
index 88b0bfba83bb..eb88d9ce6c34 100644
@@ -261,11 +261,11 @@
      (defined(__TARGET_ARCH_riscv) && __riscv_xlen == 64) ||		\
      defined(__TARGET_ARCH_arm) || defined(__TARGET_ARCH_s390) ||	\
      defined(__TARGET_ARCH_loongarch)) &&				\
-	__clang_major__ >= 18
+	(__clang_major__ >= 18 || defined(__BPF_FEATURE_GOTOL))
 #define CAN_USE_GOTOL
 #endif
 
-#if __clang_major__ >= 18
+#if __clang_major__ >= 18 || defined(__BPF_FEATURE_ST)
 #define CAN_USE_BPF_ST
 #endif
 
tools/testing/selftests/bpf/progs/compute_live_registers.c +7 -0
index 338e67cd8856..c7d5a0abff2c 100644
@@ -299,7 +299,14 @@ __naked void gotol(void)
 		"r3 = 24;"
 		"if r1 > 0x7 goto +2;"
 		"r0 = r2;"
+#ifdef __clang__
 		"gotol +1;"
+#else
+		/* gas mis-parses 'gotol +1' as 'goto l+1', same encoding
+		 * without the sign (binutils PR gas/34558).
+		 */
+		"gotol 1;"
+#endif
 		"r0 = r3;"
 		"exit;"
 		:
tools/testing/selftests/bpf/progs/verifier_bswap.c +1 -1
index 074faef5480e..48ffb5b242d2 100644
@@ -8,7 +8,7 @@
 	(defined(__TARGET_ARCH_riscv) && __riscv_xlen == 64) || \
 	defined(__TARGET_ARCH_arm) || defined(__TARGET_ARCH_s390) || \
 	defined(__TARGET_ARCH_loongarch)) && \
-	__clang_major__ >= 18
+	(__clang_major__ >= 18 || defined(__BPF_FEATURE_BSWAP))
 
 SEC("socket")
 __description("BSWAP, 16")
tools/testing/selftests/bpf/progs/verifier_ldsx.c +11 -7
index e58ea69e3854..75762e11b443 100644
@@ -9,13 +9,7 @@
 	(defined(__TARGET_ARCH_riscv) && __riscv_xlen == 64) || \
 	defined(__TARGET_ARCH_arm) || defined(__TARGET_ARCH_s390) || \
 	defined(__TARGET_ARCH_loongarch)) && \
-	__clang_major__ >= 18
-
-struct {
-	__uint(type, BPF_MAP_TYPE_ARENA);
-	__uint(map_flags, BPF_F_MMAPABLE);
-	__uint(max_entries, 1);
-} arena SEC(".maps");
+	(__clang_major__ >= 18 || defined(__BPF_FEATURE_LDSX))
 
 SEC("socket")
 __description("LDSX, S8")
@@ -263,6 +257,14 @@ __naked void ldsx_ctx_8(void)
 	: __clobber_all);
 }
 
+#ifdef __BPF_FEATURE_ADDR_SPACE_CAST
+
+struct {
+	__uint(type, BPF_MAP_TYPE_ARENA);
+	__uint(map_flags, BPF_F_MMAPABLE);
+	__uint(max_entries, 1);
+} arena SEC(".maps");
+
 SEC("syscall")
 __description("Arena LDSX Disasm")
 __success
@@ -432,6 +434,8 @@ void kfunc_root(void)
 	bpf_arena_alloc_pages(0, 0, 0, 0, 0);
 }
 
+#endif /* __BPF_FEATURE_ADDR_SPACE_CAST */
+
 #else
 
 SEC("socket")
tools/testing/selftests/bpf/progs/verifier_movsx.c +1 -1
index 5559c0583dd1..195b27a51224 100644
@@ -8,7 +8,7 @@
 	(defined(__TARGET_ARCH_riscv) && __riscv_xlen == 64) || \
 	defined(__TARGET_ARCH_arm) || defined(__TARGET_ARCH_s390) || \
 	defined(__TARGET_ARCH_loongarch)) && \
-	__clang_major__ >= 18
+	(__clang_major__ >= 18 || defined(__BPF_FEATURE_MOVSX))
 
 SEC("socket")
 __description("MOV32SX, S8")
tools/testing/selftests/bpf/progs/verifier_sdiv.c +1 -1
index 515644ace516..d6cbe1e27fd6 100644
@@ -9,7 +9,7 @@
 	(defined(__TARGET_ARCH_riscv) && __riscv_xlen == 64) || \
 	defined(__TARGET_ARCH_arm) || defined(__TARGET_ARCH_s390) || \
 	defined(__TARGET_ARCH_loongarch)) && \
-	__clang_major__ >= 18
+	(__clang_major__ >= 18 || defined(__BPF_FEATURE_SDIV_SMOD))
 
 SEC("socket")
 __description("SDIV32, non-zero imm divisor, check 1")
Signature · 1 line
2.53.0-Meta
#8 lore raw

Re: [PATCH bpf-next 3/3] selftests/bpf: Enable some of the blocked cpuv4 instruction tests for bpf-gcc

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Low] The newly added multi-line comment for the gas bug workaround does not follow the required BPF…

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] The newly added multi-line comment for the gas bug workaround does not follow the required BPF subsystem comment style. --

--- Patch [3]: [PATCH bpf-next 3/3] selftests/bpf: Enable some of the blocked cpuv4 instruction tests for bpf-gcc --- commit 55694625f0a27b85eb77860cd1b4f07749dec04c
Author: Vineet Gupta <[email protected]>

selftests/bpf: Enable some of the blocked cpuv4 instruction tests for bpf-gcc

This commit enables several blocked BPF CPUv4 instruction tests for bpf-gcc by checking specific feature macros instead of relying solely on clang checks. It also skips address space cast tests for bpf-gcc as the feature is not currently supported, while retaining the existing tests for clang.

Quoted 5 lines
tools/testing/selftests/bpf/progs/compute_live_registers.c +0 -0
index 338e67cd8856a..c7d5a0abff2c3 100644

tools/testing/selftests/bpf/progs/compute_live_registers.c __naked void gotol(void) · line 307

Quoted 9 lines
tools/testing/selftests/bpf/progs/compute_live_registers.c +5 -0
@@ -299,7 +299,14 @@ __naked void gotol(void)
 		"r3 = 24;"
 		"if r1 > 0x7 goto +2;"
 		"r0 = r2;"
+#ifdef __clang__
 		"gotol +1;"
+#else
+		/* gas mis-parses 'gotol +1' as 'goto l+1', same encoding
+		 * without the sign (binutils PR gas/34558).
+		 */

[Severity: Low]
This isn't a bug, but does this multi-line comment follow the preferred BPF subsystem comment style?

The BPF subsystem requires the opening /* to be on its own line:

		/*
		 * gas mis-parses 'gotol +1' as 'goto l+1', same encoding
		 * without the sign (binutils PR gas/34558).
		 */
Quoted 5 lines
tools/testing/selftests/bpf/progs/compute_live_registers.c +2 -0
+		"gotol 1;"
+#endif
 		"r0 = r3;"
 		"exit;"
 		:
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=3
#4 lore raw

Re: [PATCH bpf-next 2/3] selftests/bpf: report placeholder tests as SKIP, not OK

In reply to Vineet Gupta
The commit converts 13 placeholder programs to use the new __skip() annotation, but two feature-gated placeholders with the identical pattern were not converted. Were these intentionally excluded?
Quoted 22 lines
tools/testing/selftests/bpf/progs/bpf_misc.h +6 -0
index 5eacf1b43252..88b0bfba83bb 100644
@@ -106,6 +106,11 @@
  * __description     Text to be used for display and as an additional filter
  *                   alias, while the original program name stays matchable.
  *
+ * __skip            Report the test as SKIP with the given reason instead of
+ *                   running it. For placeholder programs that stand in for a
+ *                   feature the toolchain or JIT cannot provide, so a run does
+ *                   not report OK for coverage it never executed.
+ *
  * __log_level       Log level to use for the program, numeric value expected.
  *
  * __flag            Adds one flag use for the program, the following values are valid:
@@ -139,6 +144,7 @@
 #define __failure		__test_tag("test_expect_failure")
 #define __success		__test_tag("test_expect_success")
 #define __description(desc)	__test_tag("test_description=" desc)
+#define __skip(reason)		__test_tag("test_skip=" reason)
 #define __msg_unpriv(msg)	__test_tag("test_expect_msg_unpriv=" msg)

[ ... ]

Quoted 12 lines
tools/testing/selftests/bpf/progs/verifier_ldsx.c +1 -0
index 41340877dc9d..e58ea69e3854 100644
@@ -436,6 +436,7 @@ void kfunc_root(void)
 
 SEC("socket")
 __description("cpuv4 is not supported by compiler or jit, use a dummy test")
+__skip("cpuv4 is not supported by compiler or jit")
 __success
 int dummy_test(void)
 {

[ ... ]

Quoted 24 lines
tools/testing/selftests/bpf/test_loader.c +3 -0
index 07807757b518..794a7dfb0579 100644
@@ -70,6 +70,7 @@ struct test_spec {
 	int arch_mask;
 	int load_mask;
 	int linear_sz;
+	const char *skip_reason;
 	bool auxiliary;
 	bool valid;
 };
@@ -456,6 +457,8 @@ static int parse_test_spec(struct test_loader *tester,
 			continue;
 		if ((val = str_has_pfx(s, "test_description="))) {
 			description = val;
+		} else if ((val = str_has_pfx(s, "test_skip="))) {
+			spec->skip_reason = val;
 		} else if (strcmp(s, "test_expect_failure") == 0) {
 			spec->priv.expect_failure = true;
 			spec->mode_mask |= PRIV;
@@ -1327,6 +1330,12 @@ void run_subtest(struct test_loader *tester,
 	if (!test__start_subtest_with_desc(subspec->name, subspec->description))
 		return;

tools/testing/selftests/bpf/test_loader.c void run_subtest(struct test_loader *tester, · line 1341

Quoted 10 lines
tools/testing/selftests/bpf/test_loader.c +6 -0
 
+	if (spec->skip_reason) {
+		printf("%s:SKIP: %s\n", __func__, spec->skip_reason);
+		test__skip();
+		return;
+	}
+
 	if ((get_current_arch() & spec->arch_mask) == 0) {
 		test__skip();
 		return;

The commit converts 13 placeholder programs to use the new __skip() annotation, but two feature-gated placeholders with the identical pattern were not converted. Were these intentionally excluded?

The preceding commit b1831d82fe2c states "Eleven of the thirteen feature-gated placeholder programs are called dummy_test", but grep finds fifteen such programs.

tools/testing/selftests/bpf/progs/verifier_aggregate_ret.c has a compile-time gate at line 8:

  #if defined(__clang_major__) && __clang_major__ >= 23

with its #else arm at lines 166-172 supplying the same __success placeholder pattern that this commit converts everywhere else:

  #else

SEC("socket")
__description("verifier_aggregate_ret: needs LLVM 23, dummy test") __success
int dummy_test(void)

Similarly, tools/testing/selftests/bpf/progs/verifier_percpu_addr.c has a gate at line 7:

  #if defined(__TARGET_ARCH_x86)

with its #else placeholder at lines 60-66:

  #else

SEC("raw_tp")
__description("percpu addr dummy")
__success
int dummy_test(void)

Both reach the modified test_loader.c code path via prog_tests/verifier.c (test_verifier_aggregate_ret and test_verifier_percpu_addr), so __skip() would work for them.

The gates fire today, so the placeholders are what actually gets built and reported:
- verifier_aggregate_ret.c: any toolchain older than clang 23
- verifier_percpu_addr.c: every non-x86 build (arm64, s390x, riscv64, loongarch)

On those builds these two tests still report OK for coverage that never executed.

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/32523960074

#9 lore raw

Re: [PATCH bpf-next 3/3] selftests/bpf: Enable some of the blocked cpuv4 instruction tests for bpf-gcc

In reply to [email protected]
Quoted 33 lines · On 8/21/26 1:19 PM, [email protected] wrote:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] The newly added multi-line comment for the gas bug workaround does not follow the required BPF subsystem comment style. --

--- Patch [3]: [PATCH bpf-next 3/3] selftests/bpf: Enable some of the blocked cpuv4 instruction tests for bpf-gcc --- commit 55694625f0a27b85eb77860cd1b4f07749dec04c
Author: Vineet Gupta <[email protected]>

selftests/bpf: Enable some of the blocked cpuv4 instruction tests for bpf-gcc

This commit enables several blocked BPF CPUv4 instruction tests for bpf-gcc by checking specific feature macros instead of relying solely on clang checks. It also skips address space cast tests for bpf-gcc as the feature is not currently supported, while retaining the existing tests for clang.

Quoted 5 lines
tools/testing/selftests/bpf/progs/compute_live_registers.c +0 -0
index 338e67cd8856a..c7d5a0abff2c3 100644

tools/testing/selftests/bpf/progs/compute_live_registers.c __naked void gotol(void) · line 307

Quoted 9 lines
tools/testing/selftests/bpf/progs/compute_live_registers.c +5 -0
@@ -299,7 +299,14 @@ __naked void gotol(void)
  		"r3 = 24;"
  		"if r1 > 0x7 goto +2;"
  		"r0 = r2;"
+#ifdef __clang__
  		"gotol +1;"
+#else
+		/* gas mis-parses 'gotol +1' as 'goto l+1', same encoding
+		 * without the sign (binutils PR gas/34558).
+		 */

[Severity: Low]
This isn't a bug, but does this multi-line comment follow the preferred BPF subsystem comment style?

Quoted 6 lines

The BPF subsystem requires the opening /* to be on its own line:

		/*
		 * gas mis-parses 'gotol +1' as 'goto l+1', same encoding
		 * without the sign (binutils PR gas/34558).
		 */

Right, I'll fix this in v2.

Thx,
-Vineet

#5 lore raw

Re: [PATCH bpf-next 2/3] selftests/bpf: report placeholder tests as SKIP, not OK

In reply to [email protected]
Quoted 113 lines · On 8/21/26 2:00 PM, [email protected] wrote:
Quoted 22 lines
tools/testing/selftests/bpf/progs/bpf_misc.h +6 -0
index 5eacf1b43252..88b0bfba83bb 100644
@@ -106,6 +106,11 @@
   * __description     Text to be used for display and as an additional filter
   *                   alias, while the original program name stays matchable.
   *
+ * __skip            Report the test as SKIP with the given reason instead of
+ *                   running it. For placeholder programs that stand in for a
+ *                   feature the toolchain or JIT cannot provide, so a run does
+ *                   not report OK for coverage it never executed.
+ *
   * __log_level       Log level to use for the program, numeric value expected.
   *
   * __flag            Adds one flag use for the program, the following values are valid:
@@ -139,6 +144,7 @@
  #define __failure		__test_tag("test_expect_failure")
  #define __success		__test_tag("test_expect_success")
  #define __description(desc)	__test_tag("test_description=" desc)
+#define __skip(reason)		__test_tag("test_skip=" reason)
  #define __msg_unpriv(msg)	__test_tag("test_expect_msg_unpriv=" msg)

[ ... ]

Quoted 12 lines
tools/testing/selftests/bpf/progs/verifier_ldsx.c +1 -0
index 41340877dc9d..e58ea69e3854 100644
@@ -436,6 +436,7 @@ void kfunc_root(void)
 
  SEC("socket")
  __description("cpuv4 is not supported by compiler or jit, use a dummy test")
+__skip("cpuv4 is not supported by compiler or jit")
  __success
  int dummy_test(void)
  {

[ ... ]

Quoted 24 lines
tools/testing/selftests/bpf/test_loader.c +3 -0
index 07807757b518..794a7dfb0579 100644
@@ -70,6 +70,7 @@ struct test_spec {
  	int arch_mask;
  	int load_mask;
  	int linear_sz;
+	const char *skip_reason;
  	bool auxiliary;
  	bool valid;
  };
@@ -456,6 +457,8 @@ static int parse_test_spec(struct test_loader *tester,
  			continue;
  		if ((val = str_has_pfx(s, "test_description="))) {
  			description = val;
+		} else if ((val = str_has_pfx(s, "test_skip="))) {
+			spec->skip_reason = val;
  		} else if (strcmp(s, "test_expect_failure") == 0) {
  			spec->priv.expect_failure = true;
  			spec->mode_mask |= PRIV;
@@ -1327,6 +1330,12 @@ void run_subtest(struct test_loader *tester,
  	if (!test__start_subtest_with_desc(subspec->name, subspec->description))
  		return;

tools/testing/selftests/bpf/test_loader.c void run_subtest(struct test_loader *tester, · line 1341

Quoted 10 lines
tools/testing/selftests/bpf/test_loader.c +6 -0
 
+	if (spec->skip_reason) {
+		printf("%s:SKIP: %s\n", __func__, spec->skip_reason);
+		test__skip();
+		return;
+	}
+
  	if ((get_current_arch() & spec->arch_mask) == 0) {
  		test__skip();
  		return;

The commit converts 13 placeholder programs to use the new __skip() annotation, but two feature-gated placeholders with the identical pattern were not converted. Were these intentionally excluded?

The preceding commit b1831d82fe2c states "Eleven of the thirteen feature-gated placeholder programs are called dummy_test", but grep finds fifteen such programs.

tools/testing/selftests/bpf/progs/verifier_aggregate_ret.c has a compile-time gate at line 8:

   #if defined(__clang_major__) && __clang_major__ >= 23

with its #else arm at lines 166-172 supplying the same __success placeholder pattern that this commit converts everywhere else:

   #else

SEC("socket")
__description("verifier_aggregate_ret: needs LLVM 23, dummy test") __success
int dummy_test(void)

Similarly, tools/testing/selftests/bpf/progs/verifier_percpu_addr.c has a gate at line 7:

   #if defined(__TARGET_ARCH_x86)

with its #else placeholder at lines 60-66:

   #else

SEC("raw_tp")
__description("percpu addr dummy")
__success
int dummy_test(void)

Both reach the modified test_loader.c code path via prog_tests/verifier.c (test_verifier_aggregate_ret and test_verifier_percpu_addr), so __skip() would work for them.

Quoted 8 lines

The gates fire today, so the placeholders are what actually gets built and reported:
- verifier_aggregate_ret.c: any toolchain older than clang 23
- verifier_percpu_addr.c: every non-x86 build (arm64, s390x, riscv64, loongarch)

On those builds these two tests still report OK for coverage that never executed.

Bot is correct. The reason is the older baseline I first worked against had the aforementioned numbers. After rebase I did remember to rerun the numbers but obviously there was some snafu. I'll fix this in v2.

Thx,
-Vineet

maintainer Acked-by on v1 1/3
#2 lore raw

Re: [PATCH bpf-next 1/3] selftests/bpf: name the remaining placeholder programs dummy_test [NFC]

In reply to Vineet Gupta
Quoted 9 lines · Vineet Gupta

Eleven of the thirteen feature-gated placeholder programs are called dummy_test. Two are not: hence this patch.

This is purely cosmetic as nothing depends on those names. The non-complaint setups use RUN_TESTS(skel) which does thru all programs normally. This is just more consistent and grep friendly.

---

maintainer Acked-by on v1 2/3
#6 lore raw

Re: [PATCH bpf-next 2/3] selftests/bpf: report placeholder tests as SKIP, not OK

In reply to Vineet Gupta
Quoted 16 lines · Vineet Gupta

Several verifier test files wrap their contents in a feature gate and provide a one-line placeholder program in the #else arm, so the file still yields something when the toolchain or JIT cannot support the tested instructions.

However the placeholder is annotated __success, so it reports as a pass, whereas this is really a SKIP.

Quoted 2 lines

#318/1 verifier_ldsx/cpuv4 is not supported by compiler or jit, ...:OK
#318 verifier_ldsx:OK

This causes skipped coverage to be overlooked: see the next patch where BPF_GCC builds miss a whole bunch of test due to clang only gate.

Add a __skip("reason") annotation for a compile-time gate, and use it for the placeholders.

Quoted 7 lines

After this change, above tests reported as

Quoted 2 lines

#318/1 verifier_ldsx/cpuv4 is not supported by compiler or jit, ...:SKIP
#318 verifier_ldsx:SKIP

---

Neat feature, thanks!

...

maintainer Acked-by on v1 3/3
#10 lore raw

Re: [PATCH bpf-next 3/3] selftests/bpf: Enable some of the blocked cpuv4 instruction tests for bpf-gcc

In reply to Vineet Gupta
Quoted 7 lines · Vineet Gupta

A bunch of tests are gated behind clang only __clang_major__ and thus fail to build under BPF_GCC. What's worse is, before the prev patch they were also declared as passing/OK.

This need not be because (a) bpf-gcc does support the feature in codegen and (b) also advertises correctly with feature support preprocessor macros:

Quoted 8 lines

__BPF_FEATURE_BSWAP, __BPF_FEATURE_GOTOL, __BPF_FEATURE_LDSX, __BPF_FEATURE_MOVSX, __BPF_FEATURE_SDIV_SMOD, __BPF_FEATURE_ST

So loosen up the gates to also add specific feature check. clang >=18 has to stay, despite it being true for so long: clang only defines __BPF_FEATURE_MOVSX and friends at -mcpu=v4, but assembles the inline asm at -mcpu=v3 too, and dropping it would remove these tests from the default test_progs flavour where they run today.

Huh, that's unfortunate.

Quoted 24 lines

Since bpf-gcc doesn't support addr_space_cast, wrap the corresponding tests under __BPF_FEATURE_ADDR_SPACE_CAST to skip under it. And also move the map under that guard so that a bpf-gcc object does not declare an arena map that no program in it references.

Recovered under BPF_GCC (programs per object, was 1 dummy each):

  verifier_sdiv    80
  verifier_movsx   17
  verifier_ldsx    14   (20 under clang; 5 arena tests,

kfunc_root and the arena map stay clang-only)
verifier_bswap 13

CAN_USE_GOTOL and CAN_USE_BPF_ST only gate parts of a file, so those were not dummies to begin with:

compute_live_registers 17 -> 19 (21 under clang)
verifier_gotol 1 -> 2
verifier_iterating_callbacks 34 -> 35

The clang builds are unchanged: verifier_ldsx still emits 20 programs under the cpuv4 flavour.

Quoted 7 lines

One thing left alone: verifier_load_acquire.c and verifier_store_release.c stay clang-only, and their gates are correct as gas doesn't implement those (experimental -mxbpf not implemented at all). FWIW LLVM groups those under -mcpu=v4 and defines __BPF_FEATURE_LOAD_ACQ_STORE_REL there.

---

I'll wait for v2 and apply.

...