bpf: Keep target extended until its last freplace link detaches
Subsystems (4)
v2 cover bpf: Keep target extended until its last freplace link detaches
From: Yuan Chen <[email protected]>
is_extended is a plain boolean, but one target prog can carry several freplace links at the same time, one on its entry and one on a global subprogram. __bpf_trampoline_unlink_prog() cleared it whenever *any* of them detached, so with two links on one target, detaching one of them re-armed the unbounded tail call loop closed by commit d6083f040d5d ("bpf: Prevent tailcall infinite loop caused by freplace"):
a single BPF_PROG_TEST_RUN on the tail-called target then panics the kernel, reproduced on both bpf-next and master.
Patch 1 replaces the boolean with a count of the freplace links attached to each target prog, so the target stays extended until its last link detaches. Patch 2 adds a selftest covering the sequence.
Changes since v1:
- freplace_link_cnt is u16 instead of u64, per Alexei's review: it is bounded by BPF_MAX_SUBPROGS, which counts the entry function too, so up to 256 links, one past u8, and it sits in the padding next to stack_arg_sp_adjust, so bpf_prog_aux does not grow
- dropped the separate is_extended boolean, deriving the extended state from the count itself
- renamed bpf_freplace_check_tgt_prog() to bpf_freplace_link_tgt_prog(), it reserves the target on success
- dropped the WARN_ON_ONCE() on detach, per Leon's review: the count is paired by the link lifetime and the sibling prog_array_member_cnt decrement is not guarded either
Yuan Chen (2):
bpf: Keep target extended until its last freplace link detaches selftests/bpf: Verify is_extended with multiple freplace links
include/linux/bpf.h | 4 +- kernel/bpf/arraymap.c | 2 +- kernel/bpf/trampoline.c | 8 +- .../selftests/bpf/prog_tests/tailcalls.c | 74 +++++++++++++++++++ .../bpf/progs/tailcall_freplace_multi.c | 27 +++++++ 5 files changed, 108 insertions(+), 7 deletions(-) create mode 100644 tools/testing/selftests/bpf/progs/tailcall_freplace_multi.c
Signature · 1 line
2.54.0
PATCH v2 1/2 bpf: Keep target extended until its last freplace link detaches
From: Yuan Chen <[email protected]>
The is_extended / prog_array_member_cnt protocol introduced by commit d6083f040d5d ("bpf: Prevent tailcall infinite loop caused by freplace") keeps a prog extended by a freplace program out of prog_array maps, and vice versa: once a tail call re-enters an extended subprogram, its tail_call_cnt resets on every execution and the loop never terminates.
But is_extended is a plain boolean, while one target prog can carry several freplace links at the same time, one on its entry and one on a global subprogram. __bpf_trampoline_unlink_prog() cleared is_extended whenever *any* freplace link detached, so detaching one of two links re-armed the unbounded loop through the remaining one.
Replace the is_extended boolean with a count of the freplace links attached to each target prog, so the target stays extended until its last link detaches. Also rename bpf_freplace_check_tgt_prog() to bpf_freplace_link_tgt_prog(), as the helper has never been a pure
check: it reserves the target prog on success.
--- include/linux/bpf.h | 4 ++-- kernel/bpf/arraymap.c | 2 +- kernel/bpf/trampoline.c | 8 ++++---- 3 files changed, 7 insertions(+), 7 deletions(-)
include/linux/bpf.h +2 -2
@@ -1773,7 +1773,6 @@ struct bpf_prog_aux {bool xdp_has_frags;bool exception_cb;bool exception_boundary; -bool is_extended;/* true if extended by freplace program */ bool jits_use_priv_stack;bool priv_stack_requested;bool changes_pkt_data;
@@ -1785,7 +1784,7 @@ struct bpf_prog_aux {u8 verdict; } sig;u64 prog_array_member_cnt;/* counts how many times as member of prog_array */ -struct mutex ext_mutex;/* mutex for is_extended and prog_array_member_cnt */ +struct mutex ext_mutex;/* mutex for freplace_link_cnt and prog_array_member_cnt */ struct bpf_arena *arena;void (*recursion_detected)(struct bpf_prog *prog);/* callback if recursion is detected */ /* BTF_KIND_FUNC_PROTO for valid attach_btf_id */
@@ -1817,6 +1816,7 @@ struct bpf_prog_aux {char name[BPF_OBJ_NAME_LEN ];u64 (*bpf_exception_cb)(u64 cookie,u64 sp,u64 bp,u64 ,u64 );u16 stack_arg_sp_adjust; +u16 freplace_link_cnt;/* counts freplace links extending this prog */ #ifdef CONFIG_SECURITY void *security;#endif
kernel/bpf/arraymap.c +1 -1
@@ -974,7 +974,7 @@ static void *prog_fd_array_get_ptr(struct bpf_map *map, }mutex_lock (&prog->aux->ext_mutex); - is_extended = prog->aux->is_extended; + is_extended = prog->aux->freplace_link_cnt >0 ;if (!is_extended) prog->aux->prog_array_member_cnt++;mutex_unlock (&prog->aux->ext_mutex);
kernel/bpf/trampoline.c +4 -4
@@ -813,7 +813,7 @@ static enum bpf_tramp_prog_type bpf_attach_type_to_tramp(struct bpf_prog *prog) } } -static int bpf_freplace_check_tgt_prog (struct bpf_prog *tgt_prog) +static int bpf_freplace_link_tgt_prog (struct bpf_prog *tgt_prog) {struct bpf_prog_aux *aux = tgt_prog->aux;
@@ -827,7 +827,7 @@ static int bpf_freplace_check_tgt_prog(struct bpf_prog *tgt_prog)*/ return -EBUSY ; - aux->is_extended = true; + aux->freplace_link_cnt++;return 0 ; }
@@ -933,7 +933,7 @@ static int __bpf_trampoline_link_prog(struct bpf_tramp_node *node,/* Cannot attach extension if fentry/fexit are in use. */ if (cnt)return -EBUSY ; - err =bpf_freplace_check_tgt_prog (tgt_prog); + err =bpf_freplace_link_tgt_prog (tgt_prog);if (err)return err; tr->extension_prog = node->link->prog;
@@ -979,7 +979,7 @@ static int __bpf_trampoline_unlink_prog(struct bpf_tramp_node *node, tr->extension_prog->bpf_func, NULL); tr->extension_prog =NULL ;guard (mutex)(&tgt_prog->aux->ext_mutex); - tgt_prog->aux->is_extended = false; + tgt_prog->aux->freplace_link_cnt--;return err; }bpf_trampoline_remove_prog (tr, node);
Signature · 1 line
2.54.0
Quoted 8 lines · On 24/9/26 10:37, [email protected] wrote:
From: Yuan Chen <[email protected]>
The
is_extended/prog_array_member_cntprotocol introduced by commitd6083f040d5d("bpf: Prevent tailcall infinite loop caused by freplace") keeps a prog extended by a freplace program out ofprog_arraymaps, and vice versa: once a tail call re-enters an extended subprogram, itstail_call_cntresets on every execution and the loop never terminates.
Quoted 5 lines
But
is_extendedis a plain boolean, while one target prog can carry several freplace links at the same time, one on its entry and one on a global subprogram.__bpf_trampoline_unlink_prog()clearedis_extendedwhenever *any* freplace link detached, so detaching one of two links re-armed the unbounded loop through the remaining one.
A prog may have multiple global subprogs. And each of the subprogs can be attached with freplace prog. When all the subprogs are attached with freplace progs then detach one of the freplace prog, the *is_extended* becomes *false*, which relaxes the restriction between tailcall and freplace introduced by the commit d6083f040d5d ("bpf: Prevent tailcall infinite loop caused by freplace").
Quoted 10 lines
Replace the
is_extendedboolean with a count of the freplace links attached to each target prog, so the target stays extended until its last link detaches. Also renamebpf_freplace_check_tgt_prog()tobpf_freplace_link_tgt_prog(), as the helper has never been a pure
check: it reserves the target prog on success.Fixes d6083f040d5d ("bpf: Prevent tailcall infinite loop caused by freplace")Signed-off-by Yuan Chen <[email protected]>
[...]
kernel/bpf/trampoline.c static int __bpf_trampoline_link_prog(struct bpf_tramp_node *node, · line 939
Quoted 9 lines
kernel/bpf/trampoline.c +1
-1@@ -933,7 +933,7 @@ static int __bpf_trampoline_link_prog(struct bpf_tramp_node *node,/* Cannot attach extension if fentry/fexit are in use. */ if (cnt)return -EBUSY ; - err =bpf_freplace_check_tgt_prog (tgt_prog); + err =bpf_freplace_link_tgt_prog (tgt_prog);if (err)return err; tr->extension_prog = node->link->prog;
return bpf_arch_text_poke(tr->func.addr, BPF_MOD_NOP,
BPF_MOD_JUMP, NULL,
node->link->prog->bpf_func);I think there are existing issues here: if bpf_arch_text_poke() returns error, the tr->extension_prog and aux->freplace_link_cnt should be rollbacked.
Thanks,
Leon
Quoted 9 lines
kernel/bpf/trampoline.c +1
-1@@ -979,7 +979,7 @@ static int __bpf_trampoline_unlink_prog(struct bpf_tramp_node *node, tr->extension_prog->bpf_func, NULL); tr->extension_prog =NULL ;guard (mutex)(&tgt_prog->aux->ext_mutex); - tgt_prog->aux->is_extended = false; + tgt_prog->aux->freplace_link_cnt--;return err; }bpf_trampoline_remove_prog (tr, node);
Quoted 50 lines · Leon Hwang
Quoted 8 lines · On 24/9/26 10:37, [email protected] wrote:
From: Yuan Chen <[email protected]>
The
is_extended/prog_array_member_cntprotocol introduced by commitd6083f040d5d("bpf: Prevent tailcall infinite loop caused by freplace") keeps a prog extended by a freplace program out ofprog_arraymaps, and vice versa: once a tail call re-enters an extended subprogram, itstail_call_cntresets on every execution and the loop never terminates.Quoted 5 lines
But
is_extendedis a plain boolean, while one target prog can carry several freplace links at the same time, one on its entry and one on a global subprogram.__bpf_trampoline_unlink_prog()clearedis_extendedwhenever *any* freplace link detached, so detaching one of two links re-armed the unbounded loop through the remaining one.A prog may have multiple global subprogs. And each of the subprogs can be attached with freplace prog. When all the subprogs are attached with freplace progs then detach one of the freplace prog, the *is_extended* becomes *false*, which relaxes the restriction between tailcall and freplace introduced by the commit
d6083f040d5d("bpf: Prevent tailcall infinite loop caused by freplace").Quoted 10 lines
Replace the
is_extendedboolean with a count of the freplace links attached to each target prog, so the target stays extended until its last link detaches. Also renamebpf_freplace_check_tgt_prog()tobpf_freplace_link_tgt_prog(), as the helper has never been a pure
check: it reserves the target prog on success.Fixes d6083f040d5d ("bpf: Prevent tailcall infinite loop caused by freplace")Signed-off-by Yuan Chen <[email protected]>[...]
kernel/bpf/trampoline.c static int __bpf_trampoline_link_prog(struct bpf_tramp_node *node, · line 939
Quoted 9 lines
kernel/bpf/trampoline.c +1
-1@@ -933,7 +933,7 @@ static int __bpf_trampoline_link_prog(struct bpf_tramp_node *node,/* Cannot attach extension if fentry/fexit are in use. */ if (cnt)return -EBUSY ; - err =bpf_freplace_check_tgt_prog (tgt_prog); + err =bpf_freplace_link_tgt_prog (tgt_prog);if (err)return err; tr->extension_prog = node->link->prog;return bpf_arch_text_poke(tr->func.addr, BPF_MOD_NOP, BPF_MOD_JUMP, NULL, node->link->prog->bpf_func);
Quoted 3 lines
I think there are existing issues here: if
bpf_arch_text_poke()returns error, thetr->extension_progandaux->freplace_link_cntshould be rollbacked.
+1, let's set tr->extension_prog and tgt_prog->aux->freplace_link_cnt++ only if bpf_arch_text_poke succeeds
jirka
Quoted 15 lines
Thanks,
LeonQuoted 9 lines
kernel/bpf/trampoline.c +1
-1@@ -979,7 +979,7 @@ static int __bpf_trampoline_unlink_prog(struct bpf_tramp_node *node, tr->extension_prog->bpf_func, NULL); tr->extension_prog =NULL ;guard (mutex)(&tgt_prog->aux->ext_mutex); - tgt_prog->aux->is_extended = false; + tgt_prog->aux->freplace_link_cnt--;return err; }bpf_trampoline_remove_prog (tr, node);
Thanks for the review. This is a pre-existing bug in the link path, not introduced by this patch. I'll send a separate fix for the rollback later. This patch stays focused on the is_extended counting.
Quoted 66 lines · Leon Hwang
Quoted 8 lines · On 24/9/26 10:37, [email protected] wrote:
From: Yuan Chen <[email protected]>
The
is_extended/prog_array_member_cntprotocol introduced by commitd6083f040d5d("bpf: Prevent tailcall infinite loop caused by freplace") keeps a prog extended by a freplace program out ofprog_arraymaps, and vice versa: once a tail call re-enters an extended subprogram, itstail_call_cntresets on every execution and the loop never terminates.Quoted 5 lines
But
is_extendedis a plain boolean, while one target prog can carry several freplace links at the same time, one on its entry and one on a global subprogram.__bpf_trampoline_unlink_prog()clearedis_extendedwhenever *any* freplace link detached, so detaching one of two links re-armed the unbounded loop through the remaining one.A prog may have multiple global subprogs. And each of the subprogs can be attached with freplace prog. When all the subprogs are attached with freplace progs then detach one of the freplace prog, the *is_extended* becomes *false*, which relaxes the restriction between tailcall and freplace introduced by the commit
d6083f040d5d("bpf: Prevent tailcall infinite loop caused by freplace").Quoted 10 lines
Replace the
is_extendedboolean with a count of the freplace links attached to each target prog, so the target stays extended until its last link detaches. Also renamebpf_freplace_check_tgt_prog()tobpf_freplace_link_tgt_prog(), as the helper has never been a pure
check: it reserves the target prog on success.Fixes d6083f040d5d ("bpf: Prevent tailcall infinite loop caused by freplace")Signed-off-by Yuan Chen <[email protected]>[...]
kernel/bpf/trampoline.c static int __bpf_trampoline_link_prog(struct bpf_tramp_node *node, · line 939
Quoted 9 lines
kernel/bpf/trampoline.c +1
-1@@ -933,7 +933,7 @@ static int __bpf_trampoline_link_prog(struct bpf_tramp_node *node,/* Cannot attach extension if fentry/fexit are in use. */ if (cnt)return -EBUSY ; - err =bpf_freplace_check_tgt_prog (tgt_prog); + err =bpf_freplace_link_tgt_prog (tgt_prog);if (err)return err; tr->extension_prog = node->link->prog;return bpf_arch_text_poke(tr->func.addr, BPF_MOD_NOP, BPF_MOD_JUMP, NULL, node->link->prog->bpf_func);I think there are existing issues here: if
bpf_arch_text_poke()returns error, thetr->extension_progandaux->freplace_link_cntshould be rollbacked.Thanks,
LeonQuoted 9 lines
kernel/bpf/trampoline.c +1
-1@@ -979,7 +979,7 @@ static int __bpf_trampoline_unlink_prog(struct bpf_tramp_node *node, tr->extension_prog->bpf_func, NULL); tr->extension_prog =NULL ;guard (mutex)(&tgt_prog->aux->ext_mutex); - tgt_prog->aux->is_extended = false; + tgt_prog->aux->freplace_link_cnt--;return err; }bpf_trampoline_remove_prog (tr, node);
PATCH 1/1 bpf: Roll back freplace link state when bpf_arch_text_poke() fails
From: Yuan Chen <[email protected]>
A freplace attach claims the target prog by bumping tgt_prog->aux->freplace_link_cnt and setting tr->extension_prog. bpf_arch_text_poke() then makes the extension take effect. If the poke fails, the claims are never released: the attach unwinds through bpf_link_cleanup(), which clears link->prog, so bpf_trampoline_unlink_prog() never runs.
Drop the link count under ext_mutex on the error path, and set tr->extension_prog only after the poke succeeded. The count is still bumped before the poke: it blocks prog_array updates while the entry is patched.
--- kernel/bpf/trampoline.c | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-)
kernel/bpf/trampoline.c +10 -3
@@ -973,10 +973,17 @@ static int __bpf_trampoline_link_prog(struct bpf_tramp_node *node, err =bpf_freplace_link_tgt_prog (tgt_prog);if (err)return err; + err =bpf_arch_text_poke (tr->func.addr,BPF_MOD_NOP , +BPF_MOD_JUMP ,NULL , + node->link->prog->bpf_func); +if (err) { +/* Undo the claim from bpf_freplace_link_tgt_prog(). */ +guard (mutex)(&tgt_prog->aux->ext_mutex); + tgt_prog->aux->freplace_link_cnt--; +return err; + } tr->extension_prog = node->link->prog; -return bpf_arch_text_poke (tr->func.addr,BPF_MOD_NOP , -BPF_MOD_JUMP ,NULL , - node->link->prog->bpf_func); +return 0 ; } err =bpf_trampoline_add_prog (tr, node, cnt);if (err)
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCH bpf-next] bpf: Roll back freplace link state when bpf_arch_text_poke() fails
The Fixes: tag points to d6083f040d5d, but should this be commit c43c348a8446 instead?
Quoted 9 lines
bpf: Roll back freplace link state when
bpf_arch_text_poke()failsA freplace attach claims the target prog by bumping
tgt_prog->aux->freplace_link_cntand settingtr->extension_prog.bpf_arch_text_poke()then makes the extension take effect. If the poke fails, the claims are never released: the attach unwinds throughbpf_link_cleanup(), which clearslink->prog, sobpf_trampoline_unlink_prog()never runs.
Quoted 8 lines
Drop the link count under
ext_mutexon the error path, and settr->extension_progonly after the poke succeeded. The count is still bumped before the poke: it blocksprog_arrayupdates while the entry is patched.Fixes d6083f040d5d ("bpf: Prevent tailcall infinite loop caused by freplace")Suggested-by Leon Hwang <[email protected]>Signed-off-by Yuan Chen <[email protected]>
The Fixes: tag points to d6083f040d5d, but should this be commit c43c348a8446 instead?
Looking at the commit history, c43c348a8446 ("bpf: Keep target extended until its last freplace link detaches") introduced the freplace_link_cnt field and the bpf_freplace_link_tgt_prog() function that increments it.
The bug being fixed here is that when bpf_freplace_link_tgt_prog() succeeds but bpf_arch_text_poke() subsequently fails, the incremented count is not rolled back. This exact code pattern was introduced by c43c348a8446, not d6083f040d5d.
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/36096074687
Re: [PATCH bpf-next] bpf: Roll back freplace link state when bpf_arch_text_poke() fails
In reply to [email protected]Quoted 9 lines · On 25/9/26 12:35, [email protected] wrote:
From: Yuan Chen <[email protected]>
A freplace attach claims the target prog by bumping
tgt_prog->aux->freplace_link_cntand settingtr->extension_prog.bpf_arch_text_poke()then makes the extension take effect. If the poke fails, the claims are never released: the attach unwinds throughbpf_link_cleanup(), which clearslink->prog, sobpf_trampoline_unlink_prog()never runs.
Quoted 6 lines
Drop the link count under
ext_mutexon the error path, and settr->extension_progonly after the poke succeeded. The count is still bumped before the poke: it blocksprog_arrayupdates while the entry is patched.Fixes d6083f040d5d ("bpf: Prevent tailcall infinite loop caused by freplace")
Your commit msg also says the issue about tr->extension_prog.
An extra Fixes tag for tr->extension_prog should be added.
The changes below lgtm:
Thanks,
Leon
Quoted 31 lines
Suggested-by Leon Hwang <[email protected]>Signed-off-by Yuan Chen <[email protected]>--- kernel/bpf/trampoline.c | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-)kernel/bpf/trampoline.c +10
-3@@ -973,10 +973,17 @@ static int __bpf_trampoline_link_prog(struct bpf_tramp_node *node, err =bpf_freplace_link_tgt_prog (tgt_prog);if (err)return err; + err =bpf_arch_text_poke (tr->func.addr,BPF_MOD_NOP , +BPF_MOD_JUMP ,NULL , + node->link->prog->bpf_func); +if (err) { +/* Undo the claim from bpf_freplace_link_tgt_prog(). */ +guard (mutex)(&tgt_prog->aux->ext_mutex); + tgt_prog->aux->freplace_link_cnt--; +return err; + } tr->extension_prog = node->link->prog; -return bpf_arch_text_poke (tr->func.addr,BPF_MOD_NOP , -BPF_MOD_JUMP ,NULL , - node->link->prog->bpf_func); +return 0 ; } err =bpf_trampoline_add_prog (tr, node, cnt);if (err)
Re:Re: [PATCH bpf-next] bpf: Roll back freplace link state when bpf_arch_text_poke() fails
Thanks for the review. v2 adds the be8704ff07d2 Fixes tag and collects your Ack.
Quoted 61 lines · Leon Hwang
Quoted 9 lines · On 25/9/26 12:35, [email protected] wrote:
From: Yuan Chen <[email protected]>
A freplace attach claims the target prog by bumping
tgt_prog->aux->freplace_link_cntand settingtr->extension_prog.bpf_arch_text_poke()then makes the extension take effect. If the poke fails, the claims are never released: the attach unwinds throughbpf_link_cleanup(), which clearslink->prog, sobpf_trampoline_unlink_prog()never runs.Quoted 6 lines
Drop the link count under
ext_mutexon the error path, and settr->extension_progonly after the poke succeeded. The count is still bumped before the poke: it blocksprog_arrayupdates while the entry is patched.Fixes d6083f040d5d ("bpf: Prevent tailcall infinite loop caused by freplace")Your commit msg also says the issue about
tr->extension_prog.An extra Fixes tag for
tr->extension_progshould be added.Fixes be8704ff07d2 ("bpf: Introduce dynamic program extensions")The changes below lgtm:
Acked-by Leon Hwang <[email protected]>Thanks,
LeonQuoted 31 lines
Suggested-by Leon Hwang <[email protected]>Signed-off-by Yuan Chen <[email protected]>--- kernel/bpf/trampoline.c | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-)kernel/bpf/trampoline.c +10
-3@@ -973,10 +973,17 @@ static int __bpf_trampoline_link_prog(struct bpf_tramp_node *node, err =bpf_freplace_link_tgt_prog (tgt_prog);if (err)return err; + err =bpf_arch_text_poke (tr->func.addr,BPF_MOD_NOP , +BPF_MOD_JUMP ,NULL , + node->link->prog->bpf_func); +if (err) { +/* Undo the claim from bpf_freplace_link_tgt_prog(). */ +guard (mutex)(&tgt_prog->aux->ext_mutex); + tgt_prog->aux->freplace_link_cnt--; +return err; + } tr->extension_prog = node->link->prog; -return bpf_arch_text_poke (tr->func.addr,BPF_MOD_NOP , -BPF_MOD_JUMP ,NULL , - node->link->prog->bpf_func); +return 0 ; } err =bpf_trampoline_add_prog (tr, node, cnt);if (err)
PATCH v2 2/2 selftests/bpf: Verify is_extended with multiple freplace links
In reply to [email protected]From: Yuan Chen <[email protected]>
Extend tc_bpf2bpf with two freplace links, one on entry_tc and one on subprog_tc, and detach the one on the entry: while subprog_tc is still extended, updating entry_tc into a prog_array map must keep failing with -EBUSY, and succeed again once the last link detaches. The test asserts the rejection instead of running the prog, as the update succeeds and the prog loops unbounded on an unfixed kernel.
--- .../selftests/bpf/prog_tests/tailcalls.c | 74 +++++++++++++++++++ .../bpf/progs/tailcall_freplace_multi.c | 27 +++++++ 2 files changed, 101 insertions(+) create mode 100644 tools/testing/selftests/bpf/progs/tailcall_freplace_multi.c
tools/testing/selftests/bpf/prog_tests/tailcalls.c +74 -0
@@ -6,6 +6,7 @@#include "tailcall_bpf2bpf_hierarchy2.skel.h" #include "tailcall_bpf2bpf_hierarchy3.skel.h" #include "tailcall_freplace.skel.h" +#include "tailcall_freplace_multi.skel.h" #include "tc_bpf2bpf.skel.h" #include "tailcall_fail.skel.h" #include "tailcall_cgrp_storage_owner.skel.h"
@@ -1655,6 +1656,77 @@ static void test_tailcall_bpf2bpf_freplace(void)tc_bpf2bpf__destroy (tc_skel); } +static void test_tailcall_freplace_multi (void ) +{ +struct tailcall_freplace_multi *freplace_skel =NULL ; +struct bpf_link *link_subprog =NULL , *link_entry =NULL ; +struct tc_bpf2bpf *tc_skel =NULL ; +int tc_prog_fd, map_fd, key =0 , err; + + tc_skel =tc_bpf2bpf__open_and_load (); +if (!ASSERT_OK_PTR (tc_skel,"tc_bpf2bpf__open_and_load" )) +return ; + + tc_prog_fd =bpf_program__fd (tc_skel->progs.entry_tc); + + freplace_skel =tailcall_freplace_multi__open (); +if (!ASSERT_OK_PTR (freplace_skel,"tailcall_freplace_multi__open" )) + goto out; + + err =bpf_program__set_attach_target (freplace_skel->progs.subprog_freplace, + tc_prog_fd,"subprog_tc" ); +if (!ASSERT_OK (err,"set_attach_target subprog_tc" )) + goto out; + + err =bpf_program__set_attach_target (freplace_skel->progs.entry_freplace, + tc_prog_fd,"entry_tc" ); +if (!ASSERT_OK (err,"set_attach_target entry_tc" )) + goto out; + + err =tailcall_freplace_multi__load (freplace_skel); +if (!ASSERT_OK (err,"tailcall_freplace_multi__load" )) + goto out; + + map_fd =bpf_map__fd (freplace_skel->maps.jmp_table); + + link_subprog =bpf_program__attach_freplace (freplace_skel->progs.subprog_freplace, + tc_prog_fd,"subprog_tc" ); +if (!ASSERT_OK_PTR (link_subprog,"attach_freplace subprog_tc" )) + goto out; + + link_entry =bpf_program__attach_freplace (freplace_skel->progs.entry_freplace, + tc_prog_fd,"entry_tc" ); +if (!ASSERT_OK_PTR (link_entry,"attach_freplace entry_tc" )) + goto out; + + err =bpf_map_update_elem (map_fd, &key, &tc_prog_fd,BPF_ANY ); +if (!ASSERT_ERR (err,"update jmp_table with extended prog" )) + goto out; + + err =bpf_link__destroy (link_entry); + link_entry =NULL ; +if (!ASSERT_OK (err,"destroy entry link" )) + goto out; + + err =bpf_map_update_elem (map_fd, &key, &tc_prog_fd,BPF_ANY ); +if (!ASSERT_ERR (err,"update jmp_table with still extended prog" )) + goto out; + + err =bpf_link__destroy (link_subprog); + link_subprog =NULL ; +if (!ASSERT_OK (err,"destroy subprog link" )) + goto out; + + err =bpf_map_update_elem (map_fd, &key, &tc_prog_fd,BPF_ANY ); +ASSERT_OK (err,"update jmp_table" ); + +out: +bpf_link__destroy (link_subprog); +bpf_link__destroy (link_entry); +tailcall_freplace_multi__destroy (freplace_skel); +tc_bpf2bpf__destroy (tc_skel); +} +static void test_tailcall_failure () {RUN_TESTS (tailcall_fail);
@@ -2005,6 +2077,8 @@ void test_tailcalls(void)test_tailcall_freplace ();if (test__start_subtest ("tailcall_bpf2bpf_freplace" ))test_tailcall_bpf2bpf_freplace (); +if (test__start_subtest ("tailcall_freplace_multi" )) +test_tailcall_freplace_multi ();if (test__start_subtest ("tailcall_failure" ))test_tailcall_failure ();if (test__start_subtest ("tailcall_sleepable" ))
tools/testing/selftests/bpf/progs/tailcall_freplace_multi.c new file +27 -0
@@ -0,0 +1,27 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2026 KylinSoft Corporation. */ + +#include <linux/bpf.h> +#include <bpf/bpf_helpers.h> + +struct { + __uint(type, BPF_MAP_TYPE_PROG_ARRAY); + __uint(max_entries, 1); + __uint(key_size, sizeof(__u32)); + __uint(value_size, sizeof(__u32)); +} jmp_table SEC(".maps"); + +SEC("freplace") +int subprog_freplace (struct __sk_buff *skb) +{ +bpf_tail_call_static (skb, &jmp_table,0 ); +return 0 ; +} + +SEC("freplace") +int entry_freplace (struct __sk_buff *skb) +{ +return 0 ; +} + +char __license[] SEC("license") = "GPL";
Signature · 1 line
2.54.0
Re: [PATCH bpf-next v2 2/2] selftests/bpf: Verify is_extended with multiple freplace links
Quoted 29 lines · On 24/9/26 10:37, [email protected] wrote:
From: Yuan Chen <[email protected]>
Extend
tc_bpf2bpfwith two freplace links, one onentry_tcand one onsubprog_tc, and detach the one on the entry: whilesubprog_tcis still extended, updatingentry_tcinto aprog_arraymap must keep failing with-EBUSY, and succeed again once the last link detaches. The test asserts the rejection instead of running the prog, as the update succeeds and the prog loops unbounded on an unfixed kernel.Signed-off-by Yuan Chen <[email protected]>--- .../selftests/bpf/prog_tests/tailcalls.c | 74 +++++++++++++++++++ .../bpf/progs/tailcall_freplace_multi.c | 27 +++++++ 2 files changed, 101 insertions(+) create mode 100644 tools/testing/selftests/bpf/progs/tailcall_freplace_multi.ctools/testing/selftests/bpf/prog_tests/tailcalls.c +1
-0@@ -6,6 +6,7 @@#include "tailcall_bpf2bpf_hierarchy2.skel.h" #include "tailcall_bpf2bpf_hierarchy3.skel.h" #include "tailcall_freplace.skel.h" +#include "tailcall_freplace_multi.skel.h" #include "tc_bpf2bpf.skel.h" #include "tailcall_fail.skel.h" #include "tailcall_cgrp_storage_owner.skel.h"
tools/testing/selftests/bpf/prog_tests/tailcalls.c static void test_tailcall_bpf2bpf_freplace(void) · line 1664
Quoted 9 lines
tools/testing/selftests/bpf/prog_tests/tailcalls.c +6
-0@@ -1655,6 +1656,77 @@ static void test_tailcall_bpf2bpf_freplace(void)tc_bpf2bpf__destroy (tc_skel); } +static void test_tailcall_freplace_multi (void ) +{ +struct tailcall_freplace_multi *freplace_skel =NULL ; +struct bpf_link *link_subprog =NULL , *link_entry =NULL ; +struct tc_bpf2bpf *tc_skel =NULL ; +int tc_prog_fd, map_fd, key =0 , err;
Pls keep the above lines with inverted Christmas tree style.
And, drop '= NULL' for tc_skel, because tc_skel is the first variable to be initialized.
Thanks,
Leon
Quoted 5 lines
tools/testing/selftests/bpf/prog_tests/tailcalls.c +4
-0+ + tc_skel =tc_bpf2bpf__open_and_load (); +if (!ASSERT_OK_PTR (tc_skel,"tc_bpf2bpf__open_and_load" )) +return ;
Quoted 29 lines · On Thu, Sep 24, 2026 at 10:37:35AM +0800, [email protected] wrote:
From: Yuan Chen <[email protected]>
is_extendedis a plain boolean, but one target prog can carry several freplace links at the same time, one on its entry and one on a global subprogram.__bpf_trampoline_unlink_prog()cleared it whenever *any* of them detached, so with two links on one target, detaching one of them re-armed the unbounded tail call loop closed by commitd6083f040d5d("bpf: Prevent tailcall infinite loop caused by freplace"):
a singleBPF_PROG_TEST_RUNon the tail-called target then panics the kernel, reproduced on both bpf-next and master.Patch 1 replaces the boolean with a count of the freplace links attached to each target prog, so the target stays extended until its last link detaches. Patch 2 adds a selftest covering the sequence.
Changes since v1:
-
freplace_link_cntis u16 instead of u64, per Alexei's review: it is bounded byBPF_MAX_SUBPROGS, which counts the entry function too, so up to 256 links, one past u8, and it sits in the padding next tostack_arg_sp_adjust, sobpf_prog_auxdoes not grow
- dropped the separateis_extendedboolean, deriving the extended state from the count itself
- renamedbpf_freplace_check_tgt_prog()tobpf_freplace_link_tgt_prog(), it reserves the target on success
- dropped theWARN_ON_ONCE()on detach, per Leon's review: the count is paired by the link lifetime and the siblingprog_array_member_cntdecrement is not guarded either
Quoted 3 lines
Yuan Chen (2):
bpf: Keep target extended until its last freplace link detaches selftests/bpf: Verifyis_extendedwith multiple freplace links
jirka
Quoted 13 lines
include/linux/bpf.h | 4 +- kernel/bpf/arraymap.c | 2 +- kernel/bpf/trampoline.c | 8 +- .../selftests/bpf/prog_tests/tailcalls.c | 74 +++++++++++++++++++ .../bpf/progs/tailcall_freplace_multi.c | 27 +++++++ 5 files changed, 108 insertions(+), 7 deletions(-) create mode 100644 tools/testing/selftests/bpf/progs/tailcall_freplace_multi.cSignature · 1 line
2.54.0
This series was applied to bpf/bpf-next.git (master) by Kumar Kartikeya Dwivedi <[email protected]>:
Hello:
This series was applied to bpf/bpf-next.git (master) by Kumar Kartikeya Dwivedi <[email protected]>:
Quoted 11 lines · On Thu, 24 Sep 2026 10:37:35 +0800 you wrote:
From: Yuan Chen <[email protected]>
is_extendedis a plain boolean, but one target prog can carry several freplace links at the same time, one on its entry and one on a global subprogram.__bpf_trampoline_unlink_prog()cleared it whenever *any* of them detached, so with two links on one target, detaching one of them re-armed the unbounded tail call loop closed by commitd6083f040d5d("bpf: Prevent tailcall infinite loop caused by freplace"):
a singleBPF_PROG_TEST_RUNon the tail-called target then panics the kernel, reproduced on both bpf-next and master.
Quoted 1 line
[...]
Here is the summary with links:
- [bpf-next,v2,1/2] bpf: Keep target extended until its last freplace link detaches
https://git.kernel.org/bpf/bpf-next/c/c43c348a8446 - [bpf-next,v2,2/2] selftests/bpf: Verify is_extended with multiple freplace links https://git.kernel.org/bpf/bpf-next/c/01d37848bf5b
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html