bpf: do not increment tailcall count when prog is NULL
PATCH 1/2 bpf: do not increment tailcall count when prog is NULL
Currently, tailcall count is incremented in the interpreter even when tailcall fails due to non-existent prog. Fix this by holding off on the tailcall count increment until after NULL check on the prog.
--- kernel/bpf/core.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
kernel/bpf/core.c +2 -2
@@ -2057,12 +2057,12 @@ static u64 ___bpf_prog_run(u64 *regs, const struct bpf_insn *insn)if (unlikely (tail_call_cnt >=MAX_TAIL_CALL_CNT )) goto out; - tail_call_cnt++; - prog =READ_ONCE (array->ptrs[index]);if (!prog) goto out; + tail_call_cnt++; +/* ARG1 at this point is guaranteed to point to CTX from * the verifier side due to the fact that the tail call is * handled like a helper, that is, bpf_tail_call_proto,
Signature · 1 line
2.53.0
PATCH 2/2 bpf: zero extend function arguments while calling kernel function
For kernel functions, even if a function parameter is not of u64 size, it may still be cast to a data type of u64 size. In fact, page_cnt, a u32 argument in bpf_arena_reserve_pages() kfunc is implicitly getting typecasted to long. Without zero extension for that argument, the verifier_arena/reserve_invalid_region selftest fails on ppc64le. So, be more conservative and do zero extension for all arguments. JIT compile code can eliminate unnecessary zero extension code-gen.
--- kernel/bpf/verifier.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
kernel/bpf/verifier.c +5 -2
@@ -12018,8 +12018,11 @@ static void __mark_btf_func_reg_size(struct bpf_verifier_env *env, struct bpf_re/* Function return value */ reg->subreg_def = reg_size ==sizeof (u64) ?DEF_NOT_SUBREG : env->insn_idx +1 ; - }else if (reg_size ==sizeof (u64)) { -/* Function argument */ + }else { +/* +* Be more conservative and do zero extension for all arguments, +* as how these arguments get used in kernel is anybody's guess. +*/ mark_insn_zext (env, reg); } }
Signature · 1 line
2.53.0
Re: [PATCH 2/2] bpf: zero extend function arguments while calling kernel function
Quoted 22 lines · Hari Bathini
For kernel functions, even if a function parameter is not of u64 size, it may still be cast to a data type of u64 size. In fact,
page_cnt, a u32 argument inbpf_arena_reserve_pages()kfunc is implicitly getting typecasted to long. Without zero extension for that argument, the verifier_arena/reserve_invalid_region selftest fails on ppc64le. So, be more conservative and do zero extension for all arguments. JIT compile code can eliminate unnecessary zero extension code-gen.Fixes e6ac2450d6de ("bpf: Support bpf program calling kernel function")Reported-by Saket Kumar Bhaskar <[email protected]>Signed-off-by Hari Bathini <[email protected]>--- kernel/bpf/verifier.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)kernel/bpf/verifier.c +0
-0@@ -12018,8 +12018,11 @@ static void __mark_btf_func_reg_size(struct bpf_verifier_env *env, struct bpf_re/* Function return value */
kernel/bpf/verifier.c static void __mark_btf_func_reg_size(struct bpf_verifier_env *env, struct bpf_re · line 12026
Quoted 10 lines
kernel/bpf/verifier.c +5
-2reg->subreg_def = reg_size ==sizeof (u64) ?DEF_NOT_SUBREG : env->insn_idx +1 ; - }else if (reg_size ==sizeof (u64)) { -/* Function argument */ + }else { +/* +* Be more conservative and do zero extension for all arguments, +* as how these arguments get used in kernel is anybody's guess. +*/ mark_insn_zext (env, reg);
This needs a selftest and better explanation of what's going on. So far the patch doesn't make sense to me at all.
zext marking is only to simplify JIT's life for extensions from 32-bit to 64-bit. All <= 32 extensions are done by LLVM when it generates bpf code on the caller side.
If native callee messed it up, it's a bug in JIT that doesn't do correct job for kfunc calling and translation from bpf ABI to native.
Re: [PATCH 2/2] bpf: zero extend function arguments while calling kernel function
Quoted 34 lines · Alexei Starovoitov
Quoted 22 lines · Hari Bathini
For kernel functions, even if a function parameter is not of u64 size, it may still be cast to a data type of u64 size. In fact,
page_cnt, a u32 argument inbpf_arena_reserve_pages()kfunc is implicitly getting typecasted to long. Without zero extension for that argument, the verifier_arena/reserve_invalid_region selftest fails on ppc64le. So, be more conservative and do zero extension for all arguments. JIT compile code can eliminate unnecessary zero extension code-gen.Fixes e6ac2450d6de ("bpf: Support bpf program calling kernel function")Reported-by Saket Kumar Bhaskar <[email protected]>Signed-off-by Hari Bathini <[email protected]>--- kernel/bpf/verifier.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)kernel/bpf/verifier.c +0
-0@@ -12018,8 +12018,11 @@ static void __mark_btf_func_reg_size(struct bpf_verifier_env *env, struct bpf_re/* Function return value */ kernel/bpf/verifier.c static void __mark_btf_func_reg_size(struct bpf_verifier_env *env, struct bpf_re · line 12026
Quoted 10 lines
kernel/bpf/verifier.c +5
-2reg->subreg_def = reg_size ==sizeof (u64) ?DEF_NOT_SUBREG : env->insn_idx +1 ; - }else if (reg_size ==sizeof (u64)) { -/* Function argument */ + }else { +/* +* Be more conservative and do zero extension for all arguments, +* as how these arguments get used in kernel is anybody's guess. +*/ mark_insn_zext (env, reg);
Quoted 8 lines
This needs a selftest and better explanation of what's going on. So far the patch doesn't make sense to me at all.
zext marking is only to simplify JIT's life for extensions from 32-bit to 64-bit. All <= 32 extensions are done by LLVM when it generates bpf code on the caller side.
If native callee messed it up, it's a bug in JIT that doesn't do correct job for kfunc calling and translation from bpf ABI to native.
Hi Alexei,
Thanks for taking a look.
What we’re seeing on ppc64le is that the powerpc JIT implementation was implicitly relying on the verifier’s zext marking for kfunc arguments. In other words, the JIT wasn’t fully taking care of the required ABI- level zero extension when translating from the BPF calling convention to the native one. So as the verifier only marked sizeof(u64) arguments, the powerpc JIT ended up passing non–zero-extended values in cases where the native ABI expects them to be properly extended.
From the verifier’s perspective, that assumption is indeed backwards — the verifier shouldn’t need to compensate for arch-specific ABI handling in JITs. The responsibility to satisfy the native calling convention for kfunc calls lies with the arch JIT.
I’ve prepared a fix for the powerpc JIT to explicitly zero-extend arguments as required by the ABI during kfunc call translation, instead of depending on verifier zext state. I have posted the powerpc JIT fix here:
https://lore.kernel.org/all/[email protected]/
and proposed a test case in the selftests for the same here:
https://lore.kernel.org/bpf/[email protected]/
Thanks again for the review and for calling this out.
- Hari
Quoted 6 lines · Hari Bathini
Currently, tailcall count is incremented in the interpreter even when tailcall fails due to non-existent prog. Fix this by holding off on the tailcall count increment until after NULL check on the prog.
Fixes 04fd61ab36ec ("bpf: allow bpf programs to tail-call other bpf programs")
fixes tag isn't necessary here. This is not a bug. Applied to bpf-next to make it consistent with JITs.
This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:
Hello:
This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:
Quoted 9 lines · On Fri, 20 Feb 2026 11:59:58 +0530 you wrote:
Currently, tailcall count is incremented in the interpreter even when tailcall fails due to non-existent prog. Fix this by holding off on the tailcall count increment until after NULL check on the prog.
Fixes 04fd61ab36ec ("bpf: allow bpf programs to tail-call other bpf programs")Suggested-by Ilya Leoshkevich <[email protected]>Signed-off-by Hari Bathini <[email protected]>[...]
Here is the summary with links:
- [1/2] bpf: do not increment tailcall count when prog is NULL
https://git.kernel.org/bpf/bpf-next/c/3733f4be2870
- [2/2] bpf: zero extend function arguments while calling kernel function (no matching commit)
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html