Skip to contentSkip to conversation

bpf: do not increment tailcall count when prog is NULL

PATCH Superseded Hari Bathini 3 participants 6 messages Feb 20 – Mar 3

Subsystems (2)

v1 of 4 Superseded 1 of 2 applied Latest is v4 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/2 bpf: do not increment tailcall count when prog is NULL

Currently, tailcall count is incremented in the interpreter even when tailcall fails due to non-existent prog. Fix this by holding off on the tailcall count increment until after NULL check on the prog.

Fixes 04fd61ab36ec ("bpf: allow bpf programs to tail-call other bpf programs")
---
 kernel/bpf/core.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
kernel/bpf/core.c +2 -2
index 1b9b18e5b03c..3f6b2263c8e0 100644
@@ -2057,12 +2057,12 @@ static u64 ___bpf_prog_run(u64 *regs, const struct bpf_insn *insn)
 		if (unlikely(tail_call_cnt >= MAX_TAIL_CALL_CNT))
 			goto out;
 
-		tail_call_cnt++;
-
 		prog = READ_ONCE(array->ptrs[index]);
 		if (!prog)
 			goto out;
 
+		tail_call_cnt++;
+
 		/* ARG1 at this point is guaranteed to point to CTX from
 		 * the verifier side due to the fact that the tail call is
 		 * handled like a helper, that is, bpf_tail_call_proto,
Signature · 1 line
2.53.0
#1 lore raw

PATCH 2/2 bpf: zero extend function arguments while calling kernel function

For kernel functions, even if a function parameter is not of u64 size, it may still be cast to a data type of u64 size. In fact, page_cnt, a u32 argument in bpf_arena_reserve_pages() kfunc is implicitly getting typecasted to long. Without zero extension for that argument, the verifier_arena/reserve_invalid_region selftest fails on ppc64le. So, be more conservative and do zero extension for all arguments. JIT compile code can eliminate unnecessary zero extension code-gen.

Fixes e6ac2450d6de ("bpf: Support bpf program calling kernel function")
---
 kernel/bpf/verifier.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
kernel/bpf/verifier.c +5 -2
index f0ca69f888fa..3f6e0e1c71e4 100644
@@ -12018,8 +12018,11 @@ static void __mark_btf_func_reg_size(struct bpf_verifier_env *env, struct bpf_re
 		/* Function return value */
 		reg->subreg_def = reg_size == sizeof(u64) ?
 			DEF_NOT_SUBREG : env->insn_idx + 1;
-	} else if (reg_size == sizeof(u64)) {
-		/* Function argument */
+	} else {
+		/*
+		 * Be more conservative and do zero extension for all arguments,
+		 * as how these arguments get used in kernel is anybody's guess.
+		 */
 		mark_insn_zext(env, reg);
 	}
 }
Signature · 1 line
2.53.0
#2 lore raw

Re: [PATCH 2/2] bpf: zero extend function arguments while calling kernel function

Quoted 22 lines · Hari Bathini

For kernel functions, even if a function parameter is not of u64 size, it may still be cast to a data type of u64 size. In fact, page_cnt, a u32 argument in bpf_arena_reserve_pages() kfunc is implicitly getting typecasted to long. Without zero extension for that argument, the verifier_arena/reserve_invalid_region selftest fails on ppc64le. So, be more conservative and do zero extension for all arguments. JIT compile code can eliminate unnecessary zero extension code-gen.

Fixes e6ac2450d6de ("bpf: Support bpf program calling kernel function")
---
 kernel/bpf/verifier.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
kernel/bpf/verifier.c +0 -0
index f0ca69f888fa..3f6e0e1c71e4 100644
@@ -12018,8 +12018,11 @@ static void __mark_btf_func_reg_size(struct bpf_verifier_env *env, struct bpf_re
                /* Function return value */

kernel/bpf/verifier.c static void __mark_btf_func_reg_size(struct bpf_verifier_env *env, struct bpf_re · line 12026

Quoted 10 lines
kernel/bpf/verifier.c +5 -2
                reg->subreg_def = reg_size == sizeof(u64) ?
                        DEF_NOT_SUBREG : env->insn_idx + 1;
-       } else if (reg_size == sizeof(u64)) {
-               /* Function argument */
+       } else {
+               /*
+                * Be more conservative and do zero extension for all arguments,
+                * as how these arguments get used in kernel is anybody's guess.
+                */
                mark_insn_zext(env, reg);

This needs a selftest and better explanation of what's going on. So far the patch doesn't make sense to me at all.
zext marking is only to simplify JIT's life for extensions from 32-bit to 64-bit. All <= 32 extensions are done by LLVM when it generates bpf code on the caller side.
If native callee messed it up, it's a bug in JIT that doesn't do correct job for kfunc calling and translation from bpf ABI to native.

#3 lore raw

Re: [PATCH 2/2] bpf: zero extend function arguments while calling kernel function

Quoted 34 lines · Alexei Starovoitov
Quoted 22 lines · Hari Bathini

For kernel functions, even if a function parameter is not of u64 size, it may still be cast to a data type of u64 size. In fact, page_cnt, a u32 argument in bpf_arena_reserve_pages() kfunc is implicitly getting typecasted to long. Without zero extension for that argument, the verifier_arena/reserve_invalid_region selftest fails on ppc64le. So, be more conservative and do zero extension for all arguments. JIT compile code can eliminate unnecessary zero extension code-gen.

Fixes e6ac2450d6de ("bpf: Support bpf program calling kernel function")
---
  kernel/bpf/verifier.c | 7 +++++--
  1 file changed, 5 insertions(+), 2 deletions(-)
kernel/bpf/verifier.c +0 -0
index f0ca69f888fa..3f6e0e1c71e4 100644
@@ -12018,8 +12018,11 @@ static void __mark_btf_func_reg_size(struct bpf_verifier_env *env, struct bpf_re
                 /* Function return value */

kernel/bpf/verifier.c static void __mark_btf_func_reg_size(struct bpf_verifier_env *env, struct bpf_re · line 12026

Quoted 10 lines
kernel/bpf/verifier.c +5 -2
                 reg->subreg_def = reg_size == sizeof(u64) ?
                         DEF_NOT_SUBREG : env->insn_idx + 1;
-       } else if (reg_size == sizeof(u64)) {
-               /* Function argument */
+       } else {
+               /*
+                * Be more conservative and do zero extension for all arguments,
+                * as how these arguments get used in kernel is anybody's guess.
+                */
                 mark_insn_zext(env, reg);
Quoted 8 lines

This needs a selftest and better explanation of what's going on. So far the patch doesn't make sense to me at all.
zext marking is only to simplify JIT's life for extensions from 32-bit to 64-bit. All <= 32 extensions are done by LLVM when it generates bpf code on the caller side.
If native callee messed it up, it's a bug in JIT that doesn't do correct job for kfunc calling and translation from bpf ABI to native.

Hi Alexei,

Thanks for taking a look.

What we’re seeing on ppc64le is that the powerpc JIT implementation was implicitly relying on the verifier’s zext marking for kfunc arguments. In other words, the JIT wasn’t fully taking care of the required ABI- level zero extension when translating from the BPF calling convention to the native one. So as the verifier only marked sizeof(u64) arguments, the powerpc JIT ended up passing non–zero-extended values in cases where the native ABI expects them to be properly extended.

From the verifier’s perspective, that assumption is indeed backwards — the verifier shouldn’t need to compensate for arch-specific ABI handling in JITs. The responsibility to satisfy the native calling convention for kfunc calls lies with the arch JIT.

I’ve prepared a fix for the powerpc JIT to explicitly zero-extend arguments as required by the ABI during kfunc call translation, instead of depending on verifier zext state. I have posted the powerpc JIT fix here:

https://lore.kernel.org/all/[email protected]/

and proposed a test case in the selftests for the same here:

https://lore.kernel.org/bpf/[email protected]/

Thanks again for the review and for calling this out.

- Hari

#4 lore raw
In reply to Hari Bathini
Quoted 6 lines · Hari Bathini

Currently, tailcall count is incremented in the interpreter even when tailcall fails due to non-existent prog. Fix this by holding off on the tailcall count increment until after NULL check on the prog.

Fixes 04fd61ab36ec ("bpf: allow bpf programs to tail-call other bpf programs")

fixes tag isn't necessary here. This is not a bug. Applied to bpf-next to make it consistent with JITs.

#5 lore raw
In reply to Hari Bathini
This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:

Hello:

This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:

Quoted 9 lines · On Fri, 20 Feb 2026 11:59:58 +0530 you wrote:

Currently, tailcall count is incremented in the interpreter even when tailcall fails due to non-existent prog. Fix this by holding off on the tailcall count increment until after NULL check on the prog.

Fixes 04fd61ab36ec ("bpf: allow bpf programs to tail-call other bpf programs")

[...]

Here is the summary with links:
- [1/2] bpf: do not increment tailcall count when prog is NULL

    https://git.kernel.org/bpf/bpf-next/c/3733f4be2870

- [2/2] bpf: zero extend function arguments while calling kernel function (no matching commit)

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html