selftests/bpf: Fix tests for llvm23 true signature
Subsystems (4)
Yonghong Song
reviewercover selftests/bpf: Fix tests for llvm23 true signature
LLVM23 ([1]) records the 'true' function signature in BTF, i.e. the signature inferred after optimization rather than the one written in C. This caused two kinds of selftest failures (see below).
Case 1: keep int return type for tailcall subprogs (patches 1 and 2)
The verifier requires any subprog that issues a bpf_tail_call to return an 'int' (see check_btf_func() in kernel/bpf/check_btf.c, which rejects it with "tail_call is only allowed in functions that return 'int'").
Several tailcall subprogs do 'return 0' (or another constant) whose result no caller uses. With llvm23 the compiler folds the constant and, since the return value is dead, optimizes the subprog to effectively return 'void' and records 'void' in BTF, so the program fails to load.
Modifying bpf_tail_call_static() (in bpf_helpers.h) to return 'int' and adding a trailing 'return 0' does not help: the compiler still sees a constant result and can change the caller's return type to 'void'.
Instead, use barrier_var() so the returned value cannot be folded into a constant, and make the value live by storing it into an array map or a global variable (where the caller does not already consume it).
Case 2: adjust tracing prog ctx layout for the true signature (patch 3)
test_pkt_access_subprog2() has an unused argument that llvm optimizes away. Before llvm23 the BTF signature did not match the optimized assembly, so the verifier fell back to MAX_BPF_FUNC_REG_ARGS (5) u64 arguments and the fexit return value sat after args[5]. With llvm23 the true signature has a single argument, so the return value moves to the slot after args[1]. Select the matching ctx struct based on __clang_major__ so the test works with both old and new llvm.
[1] https://github.com/llvm/llvm-project/pull/198426
Yonghong Song (3):
selftests/bpf: Keep int return type for tailcall subprogs with maps selftests/bpf: Keep int return type for tailcall subprogs selftests/bpf: Adjust fexit_bpf2bpf ctx layout for llvm23 true
signature
.../selftests/bpf/progs/fexit_bpf2bpf.c | 13 ++++++++++-- .../selftests/bpf/progs/tailcall_bpf2bpf2.c | 5 ++++- .../bpf/progs/tailcall_bpf2bpf_hierarchy1.c | 21 +++++++++++++++---- .../bpf/progs/tailcall_bpf2bpf_hierarchy2.c | 18 +++++++++++----- .../bpf/progs/tailcall_bpf2bpf_hierarchy3.c | 11 +++++++--- .../progs/tailcall_bpf2bpf_hierarchy_fentry.c | 20 +++++++++++++++--- .../selftests/bpf/progs/verifier_sock.c | 9 ++++++-- 7 files changed, 77 insertions(+), 20 deletions(-)
Signature · 1 line
2.53.0-Meta
Yonghong Song
reviewerPATCH 1/3 selftests/bpf: Keep int return type for tailcall subprogs with maps
LLVM23 ([1]) supports 'true' function signature in BTF. The return type of the caller of a tailcall must be an 'int'. Otherwise, verification will fail (see check_btf_func() in check_btf.c). So with llvm23, it is possible that the compiler may change the caller's return type from 'int' to 'void'. To prevent this, barrier_var() is used to avoid returning a constant prone to be optimized.
Use Array map to store the subprog return values. Otherwise, due to
data_map = bpf_object__find_map_by_name(fentry_obj, ".bss");
if (!ASSERT_FALSE(!data_map || !bpf_map__is_internal(data_map),
"find data_map"))
goto out;in prog_tests/tailcalls.c test_tailcall_hierarchy_count(), the test result will be different from the original one.
[1] https://github.com/llvm/llvm-project/pull/198426
--- .../bpf/progs/tailcall_bpf2bpf_hierarchy1.c | 21 +++++++++++++++---- .../progs/tailcall_bpf2bpf_hierarchy_fentry.c | 20 +++++++++++++++--- 2 files changed, 34 insertions(+), 7 deletions(-)
tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_hierarchy1.c +17 -4
@@ -11,25 +11,38 @@ struct {__uint (value_size,sizeof (__u32)); } jmp_table SEC(".maps"); +struct { + __uint(type, BPF_MAP_TYPE_ARRAY); + __uint(max_entries, 2); + __type(key, int); + __type(value, __u64); +} array SEC(".maps"); +int count =0 ;static __noinline int subprog_tail(struct __sk_buff *skb) { +int ret =0 ; +bpf_tail_call_static (skb, &jmp_table,0 ); -return 0 ; +barrier_var (ret); +return ret; } SEC("tc")int entry (struct __sk_buff *skb) { -int ret =1 ; +int ret =1 , ret1, ret2, key1 =0 , key2 =1 ;clobber_regs_stack (); count++; -subprog_tail (skb); -subprog_tail (skb); + ret1 =subprog_tail (skb); + ret2 =subprog_tail (skb); + +bpf_map_update_elem (&array, &key1, &ret1,0 ); +bpf_map_update_elem (&array, &key2, &ret2,0 );return ret; }
tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_hierarchy_fentry.c +17 -3
@@ -13,23 +13,37 @@ struct {__uint (value_size,sizeof (__u32)); } jmp_table SEC(".maps"); +struct { + __uint(type, BPF_MAP_TYPE_ARRAY); + __uint(max_entries, 2); + __type(key, int); + __type(value, __u64); +} array SEC(".maps"); +int count =0 ;static __noinline int subprog_tail(void *ctx) { +int ret =0 ; +bpf_tail_call_static (ctx, &jmp_table,0 ); -return 0 ; +barrier_var (ret); +return ret; } SEC("fentry/dummy")int BPF_PROG (fentry ,struct sk_buff *skb) { +int key1 =0 , key2 =1 , ret1, ret2; +clobber_regs_stack (); count++; -subprog_tail (ctx); -subprog_tail (ctx); + ret1 =subprog_tail (ctx); + ret2 =subprog_tail (ctx); +bpf_map_update_elem (&array, &key1, &ret1,0 ); +bpf_map_update_elem (&array, &key2, &ret2,0 );return 0 ; }
Signature · 1 line
2.53.0-Meta
[email protected]
AI reviewRe: [PATCH bpf-next 1/3] selftests/bpf: Keep int return type for tailcall subprogs with maps
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] The BPF array map `array` defines its value type as `__u64` (8 bytes), but the code passes…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] The BPF array map array defines its value type as __u64 (8 bytes), but the code passes pointers to local int variables (4 bytes) as the map value to bpf_map_update_elem(). This causes an out-of-bounds stack read by the BPF helper, leading to either BPF verifier rejection or incorrect map state. --
commit 638a9e186d9da40d2e394848eda6a546f07a86fd
Author: Yonghong Song <[email protected]>
selftests/bpf: Keep int return type for tailcall subprogs with maps
This commit prevents LLVM23 from changing the caller's return type in BPF tailcalls by using a barrier_var. It also switches to using an Array map to store subprog return values to preserve test results.
Quoted 3 lines
tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_hierarchy1.c +0
-0
[ ... ]
Quoted 6 lines
tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_hierarchy1.c +6
-0+struct { + __uint(type, BPF_MAP_TYPE_ARRAY); + __uint(max_entries, 2); + __type(key, int); + __type(value, __u64); +} array SEC(".maps");
[ ... ]
Quoted 6 lines
SEC("tc")
intentry(struct __sk_buff *skb)
{tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_hierarchy1.c +1
-1-int ret =1 ; +int ret =1 , ret1, ret2, key1 =0 , key2 =1 ;
tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_hierarchy1.c struct { · line 45
Quoted 10 lines
tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_hierarchy1.c +5
-2clobber_regs_stack (); count++; -subprog_tail (skb); -subprog_tail (skb); + ret1 =subprog_tail (skb); + ret2 =subprog_tail (skb); + +bpf_map_update_elem (&array, &key1, &ret1,0 ); +bpf_map_update_elem (&array, &key2, &ret2,0 );
[Severity: Medium]
Will the BPF verifier reject these bpf_map_update_elem() calls, or will they read out-of-bounds on the stack?
The array map defines its value size as __u64 (8 bytes), but ret1 and ret2 are declared as 32-bit int variables (4 bytes). When passed into bpf_map_update_elem(), the helper will read 8 bytes starting from the pointers to the 4-byte integers, potentially reading uninitialized adjacent stack memory.
Quoted 7 lines
tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_hierarchy1.c +0
-0return ret; }tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_hierarchy_fentry.c +0
-0
[ ... ]
Quoted 6 lines
tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_hierarchy_fentry.c +6
-0+struct { + __uint(type, BPF_MAP_TYPE_ARRAY); + __uint(max_entries, 2); + __type(key, int); + __type(value, __u64); +} array SEC(".maps");
[ ... ]
Quoted 4 lines
SEC("fentry/dummy")
intBPF_PROG(fentry, struct sk_buff *skb)
{tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_hierarchy_fentry.c +1
-0+int key1 =0 , key2 =1 , ret1, ret2;
tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_hierarchy_fentry.c struct { · line 46
Quoted 10 lines
tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_hierarchy_fentry.c +5
-2+clobber_regs_stack (); count++; -subprog_tail (ctx); -subprog_tail (ctx); + ret1 =subprog_tail (ctx); + ret2 =subprog_tail (ctx); +bpf_map_update_elem (&array, &key1, &ret1,0 ); +bpf_map_update_elem (&array, &key2, &ret2,0 );
[Severity: Medium]
Are ret1 and ret2 causing the same 4-byte stack read for an 8-byte map value here in the fentry program?
Quoted 3 lines
return 0;
}
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Yonghong Song
reviewerRe: [PATCH bpf-next 1/3] selftests/bpf: Keep int return type for tailcall subprogs with maps
Quoted 49 lines · On 6/9/26 9:48 AM, [email protected] wrote:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] The BPF array maparraydefines its value type as__u64(8 bytes), but the code passes pointers to localintvariables (4 bytes) as the map value tobpf_map_update_elem(). This causes an out-of-bounds stack read by the BPF helper, leading to either BPF verifier rejection or incorrect map state. --commit
638a9e186d9da40d2e394848eda6a546f07a86fd
Author: Yonghong Song <[email protected]>selftests/bpf: Keep int return type for tailcall subprogs with maps
This commit prevents LLVM23 from changing the caller's return type in BPF tailcalls by using a
barrier_var. It also switches to using an Array map to store subprog return values to preserve test results.Quoted 3 lines
tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_hierarchy1.c +0
-0[ ... ]
Quoted 6 lines
tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_hierarchy1.c +6
-0+struct { + __uint(type, BPF_MAP_TYPE_ARRAY); + __uint(max_entries, 2); + __type(key, int); + __type(value, __u64); +} array SEC(".maps");[ ... ]
Quoted 6 lines
SEC("tc") int entry(struct __sk_buff *skb) {tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_hierarchy1.c +1
-1-int ret =1 ; +int ret =1 , ret1, ret2, key1 =0 , key2 =1 ;tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_hierarchy1.c struct { · line 45
Quoted 10 lines
tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_hierarchy1.c +5
-2clobber_regs_stack (); count++; -subprog_tail (skb); -subprog_tail (skb); + ret1 =subprog_tail (skb); + ret2 =subprog_tail (skb); + +bpf_map_update_elem (&array, &key1, &ret1,0 ); +bpf_map_update_elem (&array, &key2, &ret2,0 );[Severity: Medium]
Will the BPF verifier reject thesebpf_map_update_elem()calls, or will they read out-of-bounds on the stack?
Quoted 5 lines
The array map defines its value size as
__u64(8 bytes), but ret1 and ret2 are declared as 32-bit int variables (4 bytes). When passed intobpf_map_update_elem(), the helper will read 8 bytes starting from the pointers to the 4-byte integers, potentially reading uninitialized adjacent stack memory.
Good point. I should do __type(value, int) instead of __type(value, __u64). Will fix in the next revision.
Quoted 24 lines
Quoted 7 lines
return ret; }tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_hierarchy_fentry.c +0
-0[ ... ]
Quoted 6 lines
tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_hierarchy_fentry.c +6
-0+struct { + __uint(type, BPF_MAP_TYPE_ARRAY); + __uint(max_entries, 2); + __type(key, int); + __type(value, __u64); +} array SEC(".maps");[ ... ]
Quoted 7 lines
SEC("fentry/dummy") int BPF_PROG(fentry, struct sk_buff *skb) {tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_hierarchy_fentry.c +2
-0+int key1 =0 , key2 =1 , ret1, ret2; +clobber_regs_stack ();
Quoted 10 lines
tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_hierarchy_fentry.c struct { · line 46
Quoted 7 lines
count++;
tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_hierarchy_fentry.c +4
-2-subprog_tail (ctx); -subprog_tail (ctx); + ret1 =subprog_tail (ctx); + ret2 =subprog_tail (ctx); +bpf_map_update_elem (&array, &key1, &ret1,0 ); +bpf_map_update_elem (&array, &key2, &ret2,0 );[Severity: Medium]
Are ret1 and ret2 causing the same 4-byte stack read for an 8-byte map value here in the fentry program?
The same as above. The value type in the map should be just 'int'.
Quoted 4 lines
Quoted 4 lines
return 0; }
Re: [PATCH bpf-next 1/3] selftests/bpf: Keep int return type for tailcall subprogs with maps
In reply to Yonghong SongQuoted 41 lines · Yonghong Song
LLVM23 ([1]) supports 'true' function signature in BTF. The return type of the caller of a tailcall must be an 'int'. Otherwise, verification will fail (see
check_btf_func()incheck_btf.c). So with llvm23, it is possible that the compiler may change the caller's return type from 'int' to 'void'. To prevent this,barrier_var()is used to avoid returning a constant prone to be optimized.Use Array map to store the subprog return values. Otherwise, due to
data_map = bpf_object__find_map_by_name(fentry_obj, ".bss"); if (!ASSERT_FALSE(!data_map || !bpf_map__is_internal(data_map), "find data_map")) goto out;in
prog_tests/tailcalls.ctest_tailcall_hierarchy_count(), the test result will be different from the original one.[1] https://github.com/llvm/llvm-project/pull/198426
Signed-off-by Yonghong Song <[email protected]>--- .../bpf/progs/tailcall_bpf2bpf_hierarchy1.c | 21 +++++++++++++++---- .../progs/tailcall_bpf2bpf_hierarchy_fentry.c | 20 +++++++++++++++--- 2 files changed, 34 insertions(+), 7 deletions(-)tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_hierarchy1.c +7
-0@@ -11,25 +11,38 @@ struct {__uint (value_size,sizeof (__u32)); } jmp_table SEC(".maps"); +struct { + __uint(type, BPF_MAP_TYPE_ARRAY); + __uint(max_entries, 2); + __type(key, int); + __type(value, __u64); +} array SEC(".maps"); +int count =0 ;
tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_hierarchy1.c struct { · line 30
Quoted 10 lines
tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_hierarchy1.c +4
-1static __noinline int subprog_tail(struct __sk_buff *skb) { +int ret =0 ; +bpf_tail_call_static (skb, &jmp_table,0 ); -return 0 ; +barrier_var (ret); +return ret;
I think it's missing the actual fix. bpf_tail_call_static() should return 0; instead of being void. And here, instead of barrier_var() tricks,
it can pass the return from bpf_tail_call_static() further out.
Quoted 8 lines
tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_hierarchy1.c +1
-1} SEC("tc")int entry (struct __sk_buff *skb) { -int ret =1 ; +int ret =1 , ret1, ret2, key1 =0 , key2 =1 ;
tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_hierarchy1.c struct { · line 45
Quoted 10 lines
tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_hierarchy1.c +5
-2clobber_regs_stack (); count++; -subprog_tail (skb); -subprog_tail (skb); + ret1 =subprog_tail (skb); + ret2 =subprog_tail (skb); + +bpf_map_update_elem (&array, &key1, &ret1,0 ); +bpf_map_update_elem (&array, &key2, &ret2,0 );
I'm sure there is a way to do this without introducing maps or global volatile variables.
pw-bot: cr
Yonghong Song
reviewerRe: [PATCH bpf-next 1/3] selftests/bpf: Keep int return type for tailcall subprogs with maps
Quoted 43 lines · Alexei Starovoitov
Quoted 42 lines · Yonghong Song
LLVM23 ([1]) supports 'true' function signature in BTF. The return type of the caller of a tailcall must be an 'int'. Otherwise, verification will fail (see
check_btf_func()incheck_btf.c). So with llvm23, it is possible that the compiler may change the caller's return type from 'int' to 'void'. To prevent this,barrier_var()is used to avoid returning a constant prone to be optimized.Use Array map to store the subprog return values. Otherwise, due to
data_map = bpf_object__find_map_by_name(fentry_obj, ".bss"); if (!ASSERT_FALSE(!data_map || !bpf_map__is_internal(data_map), "find data_map")) goto out;in
prog_tests/tailcalls.ctest_tailcall_hierarchy_count(), the test result will be different from the original one.[1] https://github.com/llvm/llvm-project/pull/198426Signed-off-by Yonghong Song <[email protected]>--- .../bpf/progs/tailcall_bpf2bpf_hierarchy1.c | 21 +++++++++++++++---- .../progs/tailcall_bpf2bpf_hierarchy_fentry.c | 20 +++++++++++++++--- 2 files changed, 34 insertions(+), 7 deletions(-)tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_hierarchy1.c +7
-0@@ -11,25 +11,38 @@ struct {__uint (value_size,sizeof (__u32)); } jmp_table SEC(".maps"); +struct { + __uint(type, BPF_MAP_TYPE_ARRAY); + __uint(max_entries, 2); + __type(key, int); + __type(value, __u64); +} array SEC(".maps"); +int count =0 ;
Quoted 13 lines
tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_hierarchy1.c struct { · line 30
Quoted 9 lines
tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_hierarchy1.c +4
-1static __noinline int subprog_tail(struct __sk_buff *skb) { +int ret =0 ; +bpf_tail_call_static (skb, &jmp_table,0 ); -return 0 ; +barrier_var (ret); +return ret;I think it's missing the actual fix.
bpf_tail_call_static()should return 0; instead of being void. And here, instead ofbarrier_var()tricks,
it can pass the return frombpf_tail_call_static()further out.
The following should work:
tools/lib/bpf/bpf_helpers.h +3 -1
@@ -147,7 +147,7 @@*/ #if (defined(__clang__) && __clang_major__ >= 8) || (!defined(__clang__) && __GNUC__ > 12) #if defined(__bpf__) -static __always_inline void +static __always_inline int bpf_tail_call_static(void *ctx, const void *map, const __u32 slot) { if (!__builtin_constant_p(slot))
@@ -172,6 +172,8 @@ bpf_tail_call_static(void *ctx, const void *map, const __u32 slot) "call 12" :: [ctx]"r"(ctx), [map]"r"(map), [slot]"i"(slot) : "r0", "r1", "r2", "r3", "r4", "r5"); + + return 0; } #endif #endif
tools/testing/selftests/bpf/progs/verifier_sock.c +8 -3
@@ -1120,8 +1120,11 @@ int tail_call(struct __sk_buff *sk)static __noinline int static_tail_call(struct __sk_buff *sk) { -bpf_tail_call_static (sk, &jmp_table,0 ); -return 0 ; +int ret; + + ret =bpf_tail_call_static (sk, &jmp_table,0 ); +barrier_var (ret); +return ret; }/* Tail calls in sub-programs invalidate packet pointers. */
@@ -1144,10 +1147,12 @@ __failure __msg("invalid mem access")int invalidate_pkt_pointers_by_static_tail_call (struct __sk_buff *sk) {int *p = (void *)(long )sk->data; +int ret;if ((void *)(p +1 ) > (void *)(long )sk->data_end)return TCX_DROP ; -static_tail_call (sk); + ret =static_tail_call (sk); +__sink (ret); *p =42 ;/* this is unsafe */ return TCX_PASS ; }
Note that for above static_tail_call(), we cannot do
int static_tail_call(struct __sk_buff *sk)
{
return bpf_tail_call_static(sk, &jmp_table, 0);
}The following is some llvm optimization dump for
return bpf_tail_call_static(sk, &jmp_table, 0):
; Function Attrs: noinline nounwind
define internal fastcc void @static_tail_call(ptr noundef %0) unnamed_addr #2 !dbg !34454 {
#dbg_value(ptr %0, !34457, !DIExpression(), !34458)
#dbg_value(ptr %0, !34388, !DIExpression(), !34459)
#dbg_value(ptr @jmp_table, !34395, !DIExpression(), !34459)
#dbg_value(i32 0, !34396, !DIExpression(), !34459) tail call void asm sideeffect "r1 = $0\0A\09r2 = $1\0A\09r3 = $2\0A\09call 12", "r,r,i,~{r0},~{r1},~{r2},~{r3},~{r4},~{r5}"(ptr %0, ptr nonnull @jmp_table, i32 0) #7, !dbg !34461, !srcloc !34400 ret void, !dbg !34462
}
define dso_local range(i32 0, 3) i32 @invalidate_pkt_pointers_by_static_tail_call(ptr noundef %0) #3 section "tc" !dbg !34424 {
...
15: ; preds = %1
tail call fastcc void @static_tail_call(ptr noundef %0), !dbg !34445
store i32 0, ptr %2, align 4, !dbg !34446, !DIAssignID !34447
#dbg_assign(i32 0, !34428, !DIExpression(), !34447, ptr %2, !DIExpression(), !34433)
call void asm sideeffect "", "=*imr,0"(ptr nonnull elementtype(i32) %2, i32 0) #7, !dbg !34448, !srcloc !34449 store i32 42, ptr %7, align 4, !dbg !34450
br label %16, !dbg !34451
...
}
For static_tail_call(), compiler knows the return value 0. So it changed the signature to 'void static_tail_call(...)'. The value '0' is used in invalidate_pkt_pointers_by_static_tail_call().
Could you check whether the above change make sense?
BTW, for the helper, it would be great if we can return 'r0'.
static __always_inline intbpf_tail_call_static(void *ctx, const void *map, const __u32 slot)
{
if (!__builtin_constant_p(slot))
__bpf_unreachable();
/*
* Provide a hard guarantee that LLVM won't optimize setting r2 (map
* pointer) and r3 (constant map index) from _different paths_ ending
* up at the _same_ call insn as otherwise we won't be able to use the
* jmpq/nopl retpoline-free patching by the x86-64 JIT in the kernel
* given they mismatch. See also d2e4c1e6c294 ("bpf: Constant map key
* tracking for prog array pokes") for details on verifier tracking.
*
* Note on clobber list: we need to stay in-line with BPF calling
* convention, so even if we don't end up using r0, r4, r5, we need
* to mark them as clobber so that LLVM doesn't end up using them
* before / after the call.
*/
asm volatile("r1 = %[ctx]\n\t"
"r2 = %[map]\n\t"
"r3 = %[slot]\n\t"
"call 12"
:: [ctx]"r"(ctx), [map]"r"(map), [slot]"i"(slot)
: "r0", "r1", "r2", "r3", "r4", "r5");
return 'r0';
}But we have
const struct bpf_func_proto bpf_tail_call_proto = { /* func is unused for tail_call, we set it to pass the * get_helper_proto check */ .func = BPF_PTR_POISON, .gpl_only = false, .ret_type = RET_VOID, .arg1_type = ARG_PTR_TO_CTX, .arg2_type = ARG_CONST_MAP_PTR, .arg3_type = ARG_ANYTHING, }; and for RET_VOID, we have below in verifier.c: switch (base_type(ret_type)) { case RET_INTEGER: /* sets type to SCALAR_VALUE */ mark_reg_unknown(env, regs, BPF_REG_0); break; case RET_VOID: regs[BPF_REG_0].type = NOT_INIT; break; so we cannot return 'r0'.
Quoted 11 lines
Quoted 11 lines
tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_hierarchy1.c +1
-1} SEC("tc")int entry (struct __sk_buff *skb) { -int ret =1 ; +int ret =1 , ret1, ret2, key1 =0 , key2 =1 ;clobber_regs_stack ();
Quoted 8 lines
tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_hierarchy1.c struct { · line 45
Quoted 8 lines
tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_hierarchy1.c +5
-2count++; -subprog_tail (skb); -subprog_tail (skb); + ret1 =subprog_tail (skb); + ret2 =subprog_tail (skb); + +bpf_map_update_elem (&array, &key1, &ret1,0 ); +bpf_map_update_elem (&array, &key2, &ret2,0 );
__sink(ret1)/__sink(ret2) should work here.
Quoted 3 lines
I'm sure there is a way to do this without introducing maps or global volatile variables.
pw-bot: cr
Re: [PATCH bpf-next 1/3] selftests/bpf: Keep int return type for tailcall subprogs with maps
Quoted 13 lines · Yonghong Song
const struct
bpf_func_protobpf_tail_call_proto= { /* func is unused fortail_call, we set it to pass the *get_helper_protocheck */ .func =BPF_PTR_POISON, .gpl_only = false, .ret_type =RET_VOID, .arg1_type =ARG_PTR_TO_CTX, .arg2_type =ARG_CONST_MAP_PTR, .arg3_type =ARG_ANYTHING, }; and forRET_VOID, we have below inverifier.c: switch (base_type(ret_type)) { caseRET_INTEGER: /* sets type toSCALAR_VALUE*/mark_reg_unknown(env, regs, BPF_REG_0); break; caseRET_VOID:
regs[BPF_REG_0].type =NOT_INIT; break; so we cannot
return 'r0'.
Ahh, so bpf_tail_call() is a void return.
Then, I'll take it back. bpf_tail_call_static() is correct with void return. I guess we have to teach the verifier to accept void subprogs asap.
In the meantime let's do shortest possible hack for tailcall_bpf2bpf_hierarchy1.c to pass with llvm 23.
Yonghong Song
reviewerRe: [PATCH bpf-next 1/3] selftests/bpf: Keep int return type for tailcall subprogs with maps
Quoted 13 lines · Alexei Starovoitov
Quoted 4 lines · Yonghong Song
const struct
bpf_func_protobpf_tail_call_proto= { /* func is unused fortail_call, we set it to pass the *get_helper_protocheck */ .func =BPF_PTR_POISON, .gpl_only = false, .ret_type =RET_VOID,Quoted 8 lines
.arg1_type =
ARG_PTR_TO_CTX, .arg2_type =ARG_CONST_MAP_PTR, .arg3_type =ARG_ANYTHING, }; and forRET_VOID, we have below inverifier.c: switch (base_type(ret_type)) { caseRET_INTEGER: /* sets type toSCALAR_VALUE*/mark_reg_unknown(env, regs, BPF_REG_0); break; caseRET_VOID:
regs[BPF_REG_0].type =NOT_INIT; break; so we cannot
return 'r0'.
Quoted 6 lines
Ahh, so
bpf_tail_call()is a void return.
Then, I'll take it back.bpf_tail_call_static()is correct with void return. I guess we have to teach the verifier to accept void subprogs asap.In the meantime let's do shortest possible hack for
tailcall_bpf2bpf_hierarchy1.cto pass with llvm 23.
Sounds good. Will do.
Yonghong Song
reviewerPATCH 2/3 selftests/bpf: Keep int return type for tailcall subprogs
In reply to Yonghong SongSimilar to the previous patch, barrier_var() is used to prevent the LLVM23 compiler from optimizing away the return value of subprogs that issue a bpf_tail_call. Where the caller does not already consume the return value, store it into a global variable as well, so the compiler cannot change the subprog return type from 'int' to 'void'.
--- .../selftests/bpf/progs/tailcall_bpf2bpf2.c | 5 ++++- .../bpf/progs/tailcall_bpf2bpf_hierarchy2.c | 18 +++++++++++++----- .../bpf/progs/tailcall_bpf2bpf_hierarchy3.c | 11 ++++++++--- .../selftests/bpf/progs/verifier_sock.c | 9 +++++++-- 4 files changed, 32 insertions(+), 11 deletions(-)
tools/testing/selftests/bpf/progs/tailcall_bpf2bpf2.c +4 -1
@@ -13,11 +13,14 @@ struct {static __noinline int subprog_tail(struct __sk_buff *skb) { +int ret =1 ; +if (load_byte (skb,0 ))bpf_tail_call_static (skb, &jmp_table,1 );else bpf_tail_call_static (skb, &jmp_table,0 ); -return 1 ; +barrier_var (ret); +return ret; }int count =0 ;
tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_hierarchy2.c +13 -5
@@ -25,8 +25,11 @@ int count1 = 0;static __noinline int subprog_tail0(struct __sk_buff *skb) { +int ret =0 ; +bpf_tail_call_static (skb, &jmp_table,0 ); -return 0 ; +barrier_var (ret); +return ret; } __auxiliary
@@ -41,16 +44,21 @@ int classifier_0(struct __sk_buff *skb)static __noinline int subprog_tail1(struct __sk_buff *skb) { +int ret =0 ; +bpf_tail_call_static (skb, &jmp_table,1 ); -return 0 ; +barrier_var (ret); +return ret; } +int ret1, ret2; + __auxiliary SEC("tc")int classifier_1 (struct __sk_buff *skb) { count1++; -subprog_tail1 (skb); + ret1 =subprog_tail1 (skb);return 0 ; }
@@ -63,8 +71,8 @@ int tailcall_bpf2bpf_hierarchy_2(struct __sk_buff *skb)clobber_regs_stack (); -subprog_tail0 (skb); -subprog_tail1 (skb); + ret1 =subprog_tail0 (skb); + ret2 =subprog_tail1 (skb);__sink (ret);return (count1 <<16 ) | count0;
tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_hierarchy3.c +8 -3
@@ -33,17 +33,22 @@ int count = 0;static __noinline int subprog_tail(struct __sk_buff *skb, void *jmp_table) { +int ret =0 ; +bpf_tail_call_static (skb, jmp_table,0 ); -return 0 ; +barrier_var (ret); +return ret; } +int ret1, ret2; + __auxiliary SEC("tc")int classifier_0 (struct __sk_buff *skb) { count++; -subprog_tail (skb, &jmp_table0); -subprog_tail (skb, &jmp_table1); + ret1 =subprog_tail (skb, &jmp_table0); + ret2 =subprog_tail (skb, &jmp_table1);return count; }
tools/testing/selftests/bpf/progs/verifier_sock.c +7 -2
@@ -1120,8 +1120,11 @@ int tail_call(struct __sk_buff *sk)static __noinline int static_tail_call(struct __sk_buff *sk) { +int ret =0 ; +bpf_tail_call_static (sk, &jmp_table,0 ); -return 0 ; +barrier_var (ret); +return ret; }/* Tail calls in sub-programs invalidate packet pointers. */
@@ -1138,6 +1141,8 @@ int invalidate_pkt_pointers_by_global_tail_call(struct __sk_buff *sk)return TCX_PASS ; } +int ret1; +/* Tail calls in static sub-programs invalidate packet pointers. */ SEC("tc") __failure __msg("invalid mem access")
@@ -1147,7 +1152,7 @@ int invalidate_pkt_pointers_by_static_tail_call(struct __sk_buff *sk)if ((void *)(p +1 ) > (void *)(long )sk->data_end)return TCX_DROP ; -static_tail_call (sk); + ret1 =static_tail_call (sk); *p =42 ;/* this is unsafe */ return TCX_PASS ; }
Signature · 1 line
2.53.0-Meta
[email protected]
AI reviewRe: [PATCH bpf-next 2/3] selftests/bpf: Keep int return type for tailcall subprogs
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] Incomplete fix for compiler optimization: `classifier_0` does not consume `subprog_tail0` return…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] Incomplete fix for compiler optimization: classifier_0 does not consume subprog_tail0 return value. --
commit 889c5150d158afd20cd2f2219687b45dbae685e2
Author: Yonghong Song <[email protected]>
selftests/bpf: Keep int return type for tailcall subprogs
This commit updates selftests to prevent the LLVM23 compiler from optimizing away the return value of subprogs that issue a bpf_tail_call. It does this by storing the return value into a global variable so the compiler cannot change the subprog return type to void.
Quoted 4 lines
tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_hierarchy2.c +0
-0
[ ... ]
tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_hierarchy2.c int classifier_0(struct __sk_buff *skb)
Quoted 1 line
tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_hierarchy2.c +0
-0@@ -41,16 +44,21 @@ int classifier_0(struct __sk_buff *skb)
[Severity: Medium]
Does this miss an update to classifier_0()?
The commit message notes that "Where the caller does not already consume the return value, store it into a global variable as well".
In this file, classifier_1() and tailcall_bpf2bpf_hierarchy_2() were updated to store the return values into ret1 and ret2.
However, classifier_0() still calls subprog_tail0(skb) without consuming its return value:
int classifier_0(struct __sk_buff *skb)
{
count0++;
subprog_tail0(skb);
return 0;
}Could this allow the compiler to still optimize away the return type of subprog_tail0?
Quoted 22 lines
tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_hierarchy2.c +7
-2static __noinline int subprog_tail1(struct __sk_buff *skb) { +int ret =0 ; +bpf_tail_call_static (skb, &jmp_table,1 ); -return 0 ; +barrier_var (ret); +return ret; } +int ret1, ret2; + __auxiliary SEC("tc")int classifier_1 (struct __sk_buff *skb) { count1++; -subprog_tail1 (skb); + ret1 =subprog_tail1 (skb);return 0 ; }
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
Yonghong Song
reviewerPATCH 3/3 selftests/bpf: Adjust fexit_bpf2bpf ctx layout for llvm23 true signature
In reply to Yonghong Songtest_pkt_access_subprog2() is defined in C as
int test_pkt_access_subprog2(int val, volatile struct __sk_buff *skb)
but llvm optimizes away the unused 'int val' argument. Before llvm23 the BTF signature did not match the optimized assembly, so the verifier set attach_func_proto to NULL and fell back to MAX_BPF_FUNC_REG_ARGS (5) u64 arguments (see btf_ctx_access()). The fexit ctx struct therefore placed the return value after args[5].
With llvm23 the 'true' signature
int test_pkt_access_subprog2(volatile struct __sk_buff *skb)
is recorded in BTF, so nr_args becomes 1 and the return value moves to the slot right after args[1]. Select the matching args_subprog2 layout based on __clang_major__ so the test works with both old and new llvm.
--- tools/testing/selftests/bpf/progs/fexit_bpf2bpf.c | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-)
tools/testing/selftests/bpf/progs/fexit_bpf2bpf.c +11 -2
@@ -53,14 +53,23 @@ int BPF_PROG(test_subprog1, struct sk_buff *skb, int ret)* r0 = *(u32 *)(r1 + 0) * w0 <<= 1 * exit -* In such case the verifier falls back to conservative and +* Before llvm23, in such case the verifier falls back to conservative and * tracing program can access arguments and return value as u64 -* instead of accurate types. +* instead of accurate types. With llvm23, the true signature +* int test_pkt_access_subprog2(volatile struct __sk_buff *skb) +* is available in btf. */ +#if __clang_major__ >=23 +struct args_subprog2 { +__u64 args[1 ]; +__u64 ret; +}; +#else struct args_subprog2 {__u64 args[5 ];__u64 ret; }; +#endif __u64 test_result_subprog2 =0 ; SEC("fexit/test_pkt_access_subprog2")int test_subprog2 (struct args_subprog2 *ctx)
Signature · 1 line
2.53.0-Meta
[email protected]
AI reviewRe: [PATCH bpf-next 3/3] selftests/bpf: Adjust fexit_bpf2bpf ctx layout for llvm23 true signature
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Pre-existing issues:
- [Low] The modified multi-line comment retains text on the same line as the opening /*, violating the BPF subsystem comment style guidelines. --
commit 895b9c1828c7212301bcd7381a175a6b61a7df58
Author: Yonghong Song <[email protected]>
selftests/bpf: Adjust fexit_bpf2bpf ctx layout for llvm23 true signature
This commit adjusts the fexit context layout for test_pkt_access_subprog2() to accommodate differences in how llvm23 records the function signature in BTF. It selects the matching arguments struct based on the compiler version to ensure compatibility.
Quoted 5 lines
tools/testing/selftests/bpf/progs/fexit_bpf2bpf.c +0
-0@@ -53,14 +53,23 @@ int BPF_PROG(test_subprog1, struct sk_buff *skb, int ret)* r0 = *(u32 *)(r1 + 0)
tools/testing/selftests/bpf/progs/fexit_bpf2bpf.c int BPF_PROG(test_subprog1, struct sk_buff *skb, int ret) · line 61
Quoted 10 lines
tools/testing/selftests/bpf/progs/fexit_bpf2bpf.c +4
-2* w0 <<= 1 * exit -* In such case the verifier falls back to conservative and +* Before llvm23, in such case the verifier falls back to conservative and * tracing program can access arguments and return value as u64 -* instead of accurate types. +* instead of accurate types. With llvm23, the true signature +* int test_pkt_access_subprog2(volatile struct __sk_buff *skb) +* is available in btf. */
[Severity: Low]
This is a pre-existing issue, but since this multi-line comment block is being modified, could we update its opening to comply with the current BPF subsystem style rules?
The guidelines prefer the opening /* to be on its own line, rather than keeping text on the same line.
[ ... ]
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=3