add mptcp_subflow bpf_iter
Subsystems (5)
Geliang Tang
reviewerv5 cover add mptcp_subflow bpf_iter
From: Geliang Tang <[email protected]>
v5:
- update patch 1 as Andrii suggested: if msk is NULL, initializekit->msk to NULL in _new() and check it in _next().
v4:
- squash patch 1/5 and 3/5 in v3 together as Matt suggested.
- a new squash-to patch to drop mptcp_subflow_active declaration in bpf_burst.
v3:
- drop bpf_iter__mptcp_subflow, __diag_push, __diag_pop and __diag_ignore_all
- drop declarations for bpf kfuncs
v2:
- update patch 1 as Martin and Andrii suggested.
- fix warnings and errors reported by MPTCP CI.
This patch set adds a mptcp_subflow type bpf_iter, and self tests.
Geliang Tang (5):
bpf: Add mptcp_subflow bpf_iter
selftests/bpf: Add mptcp_subflow bpf_iter test prog selftests/bpf: Add mptcp_subflow bpf_iter subtest Squash to "mptcp: add sched_data helpers"
Squash to "selftests/bpf: Add bpf_burst scheduler & test"
net/mptcp/bpf.c | 57 +++++++++++-- net/mptcp/protocol.h | 2 - .../testing/selftests/bpf/bpf_experimental.h | 7 ++ .../testing/selftests/bpf/prog_tests/mptcp.c | 85 +++++++++++++++++++ tools/testing/selftests/bpf/progs/mptcp_bpf.h | 4 + .../selftests/bpf/progs/mptcp_bpf_burst.c | 1 - .../selftests/bpf/progs/mptcp_bpf_iter.c | 38 +++++++++ 7 files changed, 186 insertions(+), 8 deletions(-) create mode 100644 tools/testing/selftests/bpf/progs/mptcp_bpf_iter.c
Signature · 1 line
2.43.0
Geliang Tang
reviewerPATCH v5 1/5 bpf: Add mptcp_subflow bpf_iter
From: Geliang Tang <[email protected]>
It's necessary to traverse all subflows on the conn_list of an MPTCP socket and then call kfunc to modify the fields of each subflow. In kernel space, mptcp_for_each_subflow() helper is used for this:
mptcp_for_each_subflow(msk, subflow) kfunc(subflow);
But in the MPTCP BPF program, this has not yet been implemented. As Martin suggested recently, this conn_list walking + modify-by-kfunc usage fits the bpf_iter use case. So this patch adds a new bpf_iter type named "mptcp_subflow" to do this and implements its helpers bpf_iter_mptcp_subflow_new()/_next()/_destroy().
Since these bpf_iter mptcp_subflow helpers are invoked in its selftest in a ftrace hook for mptcp_sched_get_send(), it's necessary to register them into a BPF_PROG_TYPE_TRACING type kfunc set together with other two used kfuncs mptcp_subflow_active() and mptcp_subflow_set_scheduled().
Then bpf_for_each() for mptcp_subflow can be used in BPF program like this:
i = 0;
bpf_rcu_read_lock();
bpf_for_each(mptcp_subflow, subflow, msk) {
if (i++ >= MPTCP_SUBFLOWS_MAX)
break;
kfunc(subflow);
}
bpf_rcu_read_unlock();v2: remove msk->pm.lock in _new() and _destroy() (Martin)
drop DEFINE_BPF_ITER_FUNC, change opaque[3] to opaque[2] (Andrii)
v3: drop bpf_iter__mptcp_subflow
v4: if msk is NULL, initialize kit->msk to NULL in _new() and check it in
_next() (Andrii)
--- net/mptcp/bpf.c | 57 ++++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 52 insertions(+), 5 deletions(-)
net/mptcp/bpf.c +52 -5
@@ -201,9 +201,51 @@ static const struct btf_kfunc_id_set bpf_mptcp_fmodret_set = { .set = &bpf_mptcp_fmodret_ids, }; -__diag_push (); -__diag_ignore_all ("-Wmissing-prototypes" , -"kfuncs which will be used in BPF programs" ); +struct bpf_iter_mptcp_subflow { +__u64 __opaque[2 ]; +} __attribute__((aligned (8 ))); + +struct bpf_iter_mptcp_subflow_kern { +struct mptcp_sock *msk; +struct list_head *pos; +} __attribute__((aligned (8 ))); + +__bpf_kfunc_start_defs (); + +__bpf_kfunc int bpf_iter_mptcp_subflow_new(struct bpf_iter_mptcp_subflow *it, +struct mptcp_sock *msk) +{ +struct bpf_iter_mptcp_subflow_kern *kit = (void *)it; + + kit->msk = msk; +if (!msk) +return -EINVAL ; + + kit->pos = &msk->conn_list; +return 0 ; +} + +__bpf_kfunc struct mptcp_subflow_context * +bpf_iter_mptcp_subflow_next (struct bpf_iter_mptcp_subflow *it) +{ +struct bpf_iter_mptcp_subflow_kern *kit = (void *)it; +struct mptcp_subflow_context *subflow; +struct mptcp_sock *msk = kit->msk; + +if (!msk) +return NULL ; + + subflow = list_entry(kit->pos->next, struct mptcp_subflow_context, node); +if (!subflow ||list_entry_is_head (subflow, &msk->conn_list, node)) +return NULL ; + + kit->pos = &subflow->node; +return subflow; +} + +__bpf_kfunc void bpf_iter_mptcp_subflow_destroy(struct bpf_iter_mptcp_subflow *it) +{ +} __bpf_kfunc struct mptcp_subflow_context *bpf_mptcp_subflow_ctx_by_pos (const struct mptcp_sched_data *data,unsigned int pos)
@@ -218,12 +260,15 @@ __bpf_kfunc bool bpf_mptcp_subflow_queues_empty(struct sock *sk)return tcp_rtx_queue_empty (sk); } -__diag_pop (); +__bpf_kfunc_end_defs (); BTF_KFUNCS_START(bpf_mptcp_sched_kfunc_ids) +BTF_ID_FLAGS(func, bpf_iter_mptcp_subflow_new) +BTF_ID_FLAGS(func, bpf_iter_mptcp_subflow_next) +BTF_ID_FLAGS(func, bpf_iter_mptcp_subflow_destroy) +BTF_ID_FLAGS(func, mptcp_subflow_active) BTF_ID_FLAGS(func, mptcp_subflow_set_scheduled) BTF_ID_FLAGS(func, bpf_mptcp_subflow_ctx_by_pos) -BTF_ID_FLAGS(func, mptcp_subflow_active) BTF_ID_FLAGS(func, mptcp_set_timeout) BTF_ID_FLAGS(func, mptcp_wnd_end) BTF_ID_FLAGS(func, tcp_stream_memory_free)
@@ -241,6 +286,8 @@ static int __init bpf_mptcp_kfunc_init(void)int ret; ret =register_btf_fmodret_id_set (&bpf_mptcp_fmodret_set); + ret = ret ?:register_btf_kfunc_id_set (BPF_PROG_TYPE_TRACING , + &bpf_mptcp_sched_kfunc_set); ret = ret ?:register_btf_kfunc_id_set (BPF_PROG_TYPE_STRUCT_OPS , &bpf_mptcp_sched_kfunc_set);#ifdef CONFIG_BPF_JIT
Signature · 1 line
2.43.0
Re: [PATCH mptcp-next v5 1/5] bpf: Add mptcp_subflow bpf_iter
Quoted 37 lines · Geliang Tang
From: Geliang Tang <[email protected]>
It's necessary to traverse all subflows on the
conn_listof an MPTCP socket and then call kfunc to modify the fields of each subflow. In kernel space,mptcp_for_each_subflow()helper is used for this:mptcp_for_each_subflow(msk, subflow) kfunc(subflow);But in the MPTCP BPF program, this has not yet been implemented. As Martin suggested recently, this
conn_listwalking + modify-by-kfunc usage fits thebpf_iteruse case. So this patch adds a newbpf_itertype named "mptcp_subflow" to do this and implements its helpers bpf_iter_mptcp_subflow_new()/_next()/_destroy().Since these
bpf_itermptcp_subflowhelpers are invoked in its selftest in a ftrace hook formptcp_sched_get_send(), it's necessary to register them into aBPF_PROG_TYPE_TRACINGtype kfunc set together with other two used kfuncsmptcp_subflow_active()andmptcp_subflow_set_scheduled().Then
bpf_for_each()formptcp_subflowcan be used in BPF program like this:i = 0; bpf_rcu_read_lock(); bpf_for_each(mptcp_subflow, subflow, msk) { if (i++ >= MPTCP_SUBFLOWS_MAX) break; kfunc(subflow); } bpf_rcu_read_unlock();v2: remove
msk->pm.lockin_new()and_destroy()(Martin)
dropDEFINE_BPF_ITER_FUNC, change opaque[3] to opaque[2] (Andrii)
v3: dropbpf_iter__mptcp_subflow
v4: if msk is NULL, initializekit->mskto NULL in_new()and check it in
Quoted 8 lines
_next() (Andrii)Suggested-by Martin KaFai Lau <[email protected]>Signed-off-by Geliang Tang <[email protected]>--- net/mptcp/bpf.c | 57 ++++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 52 insertions(+), 5 deletions(-)
Looks ok from setting up open-coded iterator things, but I can't speak for other aspects I mentioned below.
Quoted 37 lines
net/mptcp/bpf.c +26
-3@@ -201,9 +201,51 @@ static const struct btf_kfunc_id_set bpf_mptcp_fmodret_set = { .set = &bpf_mptcp_fmodret_ids, }; -__diag_push (); -__diag_ignore_all ("-Wmissing-prototypes" , -"kfuncs which will be used in BPF programs" ); +struct bpf_iter_mptcp_subflow { +__u64 __opaque[2 ]; +} __attribute__((aligned (8 ))); + +struct bpf_iter_mptcp_subflow_kern { +struct mptcp_sock *msk; +struct list_head *pos; +} __attribute__((aligned (8 ))); + +__bpf_kfunc_start_defs (); + +__bpf_kfunc int bpf_iter_mptcp_subflow_new(struct bpf_iter_mptcp_subflow *it, +struct mptcp_sock *msk) +{ +struct bpf_iter_mptcp_subflow_kern *kit = (void *)it; + + kit->msk = msk; +if (!msk) +return -EINVAL ; + + kit->pos = &msk->conn_list; +return 0 ; +} + +__bpf_kfunc struct mptcp_subflow_context * +bpf_iter_mptcp_subflow_next (struct bpf_iter_mptcp_subflow *it)
net/mptcp/bpf.c static const struct btf_kfunc_id_set bpf_mptcp_fmodret_set = { · line 239
Quoted 10 lines
net/mptcp/bpf.c +10
-0+{ +struct bpf_iter_mptcp_subflow_kern *kit = (void *)it; +struct mptcp_subflow_context *subflow; +struct mptcp_sock *msk = kit->msk; + +if (!msk) +return NULL ; + + subflow = list_entry(kit->pos->next, struct mptcp_subflow_context, node); +if (!subflow ||list_entry_is_head (subflow, &msk->conn_list, node))
it's a bit weird that you need both !subflow and list_entry_is_head(). Can you have NULL subflow at all? But this is the question to msk->conn_list semantics.
Quoted 13 lines
net/mptcp/bpf.c +9
-0+return NULL ; + + kit->pos = &subflow->node; +return subflow; +} + +__bpf_kfunc void bpf_iter_mptcp_subflow_destroy(struct bpf_iter_mptcp_subflow *it) +{ +} __bpf_kfunc struct mptcp_subflow_context *bpf_mptcp_subflow_ctx_by_pos (const struct mptcp_sched_data *data,unsigned int pos)
net/mptcp/bpf.c __bpf_kfunc bool bpf_mptcp_subflow_queues_empty(struct sock *sk) · line 266
Quoted 8 lines
net/mptcp/bpf.c +2
-1@@ -218,12 +260,15 @@ __bpf_kfunc bool bpf_mptcp_subflow_queues_empty(struct sock *sk)return tcp_rtx_queue_empty (sk); } -__diag_pop (); +__bpf_kfunc_end_defs ();BTF_KFUNCS_START (bpf_mptcp_sched_kfunc_ids) +BTF_ID_FLAGS (func, bpf_iter_mptcp_subflow_new)
I'm not 100% sure, but I suspect you might need to specify KF_TRUSTED_ARGS here to ensure that struct mptcp_sock *msk is a valid owned kernel object. Other BPF folks might help to clarify this.
Quoted 22 lines
net/mptcp/bpf.c +5
-2+BTF_ID_FLAGS(func, bpf_iter_mptcp_subflow_next) +BTF_ID_FLAGS(func, bpf_iter_mptcp_subflow_destroy) +BTF_ID_FLAGS(func, mptcp_subflow_active) BTF_ID_FLAGS(func, mptcp_subflow_set_scheduled) BTF_ID_FLAGS(func, bpf_mptcp_subflow_ctx_by_pos) -BTF_ID_FLAGS(func, mptcp_subflow_active) BTF_ID_FLAGS(func, mptcp_set_timeout) BTF_ID_FLAGS(func, mptcp_wnd_end)BTF_ID_FLAGS (func, tcp_stream_memory_free)@@ -241,6 +286,8 @@ static int __init bpf_mptcp_kfunc_init(void)int ret; ret =register_btf_fmodret_id_set (&bpf_mptcp_fmodret_set); + ret = ret ?:register_btf_kfunc_id_set (BPF_PROG_TYPE_TRACING , + &bpf_mptcp_sched_kfunc_set); ret = ret ?:register_btf_kfunc_id_set (BPF_PROG_TYPE_STRUCT_OPS , &bpf_mptcp_sched_kfunc_set);#ifdef CONFIG_BPF_JIT --2.43.0
Geliang Tang
reviewerRe: [PATCH mptcp-next v5 1/5] bpf: Add mptcp_subflow bpf_iter
Hi Andrii,
Quoted 111 lines · Andrii Nakryiko
Quoted 47 lines · Geliang Tang
From: Geliang Tang <[email protected]>
It's necessary to traverse all subflows on the
conn_listof an MPTCP
socket and then call kfunc to modify the fields of each subflow. In kernel space,mptcp_for_each_subflow()helper is used for this:
mptcp_for_each_subflow(msk, subflow)
kfunc(subflow);But in the MPTCP BPF program, this has not yet been implemented. As Martin suggested recently, this
conn_listwalking + modify-by-kfunc usage fits thebpf_iteruse case. So this patch adds a newbpf_itertype named "mptcp_subflow" to do this and implements its helpers bpf_iter_mptcp_subflow_new()/_next()/_destroy().Since these
bpf_itermptcp_subflowhelpers are invoked in its selftest
in a ftrace hook formptcp_sched_get_send(), it's necessary to register
them into aBPF_PROG_TYPE_TRACINGtype kfunc set together with other
two used kfuncsmptcp_subflow_active()andmptcp_subflow_set_scheduled().Then
bpf_for_each()formptcp_subflowcan be used in BPF program like
this:i = 0;
bpf_rcu_read_lock();bpf_for_each(mptcp_subflow, subflow, msk){ if (i++ >=MPTCP_SUBFLOWS_MAX) break; kfunc(subflow);
}
bpf_rcu_read_unlock();v2: remove
msk->pm.lockin_new()and_destroy()(Martin) dropDEFINE_BPF_ITER_FUNC, change opaque[3] to opaque[2] (Andrii)
v3: dropbpf_iter__mptcp_subflow
v4: if msk is NULL, initializekit->mskto NULL in_new()and check it in
_next()(Andrii)Quoted 7 lines
Suggested-by Martin KaFai Lau <[email protected]>Signed-off-by Geliang Tang <[email protected]>---
net/mptcp/bpf.c | 57 ++++++++++++++++++++++++++++++++++++++++++++-
----1 file changed, 52 insertions(+), 5 deletions(-)Looks ok from setting up open-coded iterator things, but I can't speak
for other aspects I mentioned below.Quoted 50 lines
net/mptcp/bpf.c +36
-3@@ -201,9 +201,51 @@ static const struct btf_kfunc_id_set bpf_mptcp_fmodret_set = { .set = &bpf_mptcp_fmodret_ids, }; -__diag_push (); -__diag_ignore_all ("-Wmissing-prototypes" , - "kfuncs which will be used in BPF programs"); +struct bpf_iter_mptcp_subflow { + __u64 __opaque[2]; +} __attribute__((aligned (8 ))); + +struct bpf_iter_mptcp_subflow_kern { + struct mptcp_sock *msk; + struct list_head *pos; +} __attribute__((aligned (8 ))); + +__bpf_kfunc_start_defs (); + +__bpf_kfunc int bpf_iter_mptcp_subflow_new(structbpf_iter_mptcp_subflow *it, + struct mptcp_sock *msk) +{ + struct bpf_iter_mptcp_subflow_kern *kit = (void *)it; + + kit->msk = msk; + if (!msk) + return -EINVAL; + + kit->pos = &msk->conn_list; + return 0; +} + +__bpf_kfunc struct mptcp_subflow_context * +bpf_iter_mptcp_subflow_next(struct bpf_iter_mptcp_subflow *it) +{ + struct bpf_iter_mptcp_subflow_kern *kit = (void *)it; + struct mptcp_subflow_context *subflow; + struct mptcp_sock *msk = kit->msk; + + if (!msk) + return NULL; + + subflow = list_entry(kit->pos->next, struct mptcp_subflow_context, node); + if (!subflow || list_entry_is_head(subflow, &msk-
Quoted 6 lines
Quoted 1 line
Quoted 1 line
conn_list, node))it's a bit weird that you need both !subflow and
list_entry_is_head().
Can you have NULL subflow at all? But this is the question tomsk->conn_listsemantics.
Do you mean to return subflow regardless of whether subflow is NULL? If so, we need to use list_is_last() instead of list_entry_is_head():
{
struct bpf_iter_mptcp_subflow_kern *kit = (void *)it;
if (!kit->msk || list_is_last(kit->pos, &kit->msk->conn_list))
return NULL;
kit->pos = kit->pos->next;
return list_entry(kit->pos, struct mptcp_subflow_context, node);
}Hope this is a better version.
Quoted 20 lines
Quoted 20 lines
+ return NULL;
net/mptcp/bpf.c +6
-0+ + kit->pos = &subflow->node; + return subflow; +} + +__bpf_kfunc void bpf_iter_mptcp_subflow_destroy(struct
bpf_iter_mptcp_subflow*it)net/mptcp/bpf.c +2
-0+{ +}__bpf_kfunc struct
mptcp_subflow_context*bpf_mptcp_subflow_ctx_by_pos(const struct mptcp_sched_data *data, unsigned int pos)net/mptcp/bpf.c +0
-0@@ -218,12 +260,15 @@ __bpf_kfunc bool bpf_mptcp_subflow_queues_empty(struct sock *sk) return tcp_rtx_queue_empty(sk); }
Quoted 8 lines
net/mptcp/bpf.c __bpf_kfunc bool bpf_mptcp_subflow_queues_empty(struct sock *sk) · line 266
Quoted 5 lines
net/mptcp/bpf.c +2
-1-__diag_pop (); +__bpf_kfunc_end_defs (); BTF_KFUNCS_START(bpf_mptcp_sched_kfunc_ids) +BTF_ID_FLAGS (func, bpf_iter_mptcp_subflow_new)I'm not 100% sure, but I suspect you might need to specify
KF_TRUSTED_ARGShere to ensure thatstruct mptcp_sock *mskis a
KF_TRUSTED_ARGS breaks the selftest in patch 2 [1]:
'''
libbpf: prog 'trace_mptcp_sched_get_send': BPF program load failed:
Invalid argument
libbpf: prog 'trace_mptcp_sched_get_send': -- BEGIN PROG LOAD LOG -- 0: R1=ctx() R10=fp0
; int BPF_PROG(trace_mptcp_sched_get_send, struct mptcp_sock *msk) @ mptcp_bpf_iter.c:13
0: (79) r6 = *(u64 *)(r1 +0)
func 'mptcp_sched_get_send' arg0 has btf_id 28019 type STRUCT 'mptcp_sock'
1: R1=ctx() R6_w=ptr_mptcp_sock()
; if (bpf_get_current_pid_tgid() >> 32 != pid) @ mptcp_bpf_iter.c:17
1: (85) call bpf_get_current_pid_tgid#14 ; R0_w=scalar()
2: (77) r0 >>= 32 ;
R0_w=scalar(smin=0,smax=umax=0xffffffff,var_off=(0x0; 0xffffffff))
3: (18) r1 = 0xffffb766c1684004 ;
R1_w=map_value(map=mptcp_bp.bss,ks=4,vs=8,off=4)
5: (61) r1 = *(u32 *)(r1 +0) ;
R1_w=scalar(smin=0,smax=umax=0xffffffff,var_off=(0x0; 0xffffffff))
6: (67) r1 <<= 32 ;
R1_w=scalar(smax=0x7fffffff00000000,umax=0xffffffff00000000,smin32=0,sm ax32=umax32=0,var_off=(0x0; 0xffffffff00000000))
7: (c7) r1 s>>= 32 ;
R1_w=scalar(smin=0xffffffff80000000,smax=0x7fffffff)
8: (5d) if r0 != r1 goto pc+39 ;
R0_w=scalar(smin=smin32=0,smax=umax=umax32=0x7fffffff,var_off=(0x0; 0x7fffffff)) R1_w=scalar(smin=smin32=0,smax=umax=umax32=0x7fffffff,var_off=(0x0; 0x7fffffff))
; iter = 0; @ mptcp_bpf_iter.c:20
9: (18) r8 = 0xffffb766c1684000 ;
R8_w=map_value(map=mptcp_bp.bss,ks=4,vs=8)
11: (b4) w1 = 0 ; R1_w=0
12: (63) *(u32 *)(r8 +0) = r1 ; R1_w=0
R8_w=map_value(map=mptcp_bp.bss,ks=4,vs=8)
; bpf_rcu_read_lock(); @ mptcp_bpf_iter.c:22
13: (85) call bpf_rcu_read_lock#84967 ;
14: (bf) r7 = r10 ; R7_w=fp0 R10=fp0
; iter = 0; @ mptcp_bpf_iter.c:20
15: (07) r7 += -16 ; R7_w=fp-16
; bpf_for_each(mptcp_subflow, subflow, msk) { @ mptcp_bpf_iter.c:23
16: (bf) r1 = r7 ; R1_w=fp-16 R7_w=fp-16
17: (bf) r2 = r6 ; R2_w=ptr_mptcp_sock()
R6=ptr_mptcp_sock()
18: (85) call bpf_iter_mptcp_subflow_new#62694
R2 must be referenced or trusted
processed 17 insns (limit 1000000) max_states_per_insn 0 total_states 1 peak_states 1 mark_read 1
-- END PROG LOAD LOG --
libbpf: prog 'trace_mptcp_sched_get_send': failed to load: -22
libbpf: failed to load object 'mptcp_bpf_iter'
libbpf: failed to load BPF skeleton 'mptcp_bpf_iter': -22 test_bpf_iter:FAIL:skel_open_load: mptcp_iter unexpected error: -22
#169/4 mptcp/bpf_iter:FAIL
'''
I don't know how to fix it yet.
Quoted 6 lines
valid owned kernel object. Other BPF folks might help to clarify this.
net/mptcp/bpf.c __bpf_kfunc bool bpf_mptcp_subflow_queues_empty(struct sock *sk) · line 269
Quoted 3 lines
net/mptcp/bpf.c +3
-0+BTF_ID_FLAGS(func, bpf_iter_mptcp_subflow_next) +BTF_ID_FLAGS(func, bpf_iter_mptcp_subflow_destroy) +BTF_ID_FLAGS (func, mptcp_subflow_active)
But we do need to add KF_ITER_NEW/NEXT/DESTROY flags here:
BTF_ID_FLAGS(func, bpf_iter_mptcp_subflow_new, KF_ITER_NEW) BTF_ID_FLAGS(func, bpf_iter_mptcp_subflow_next, KF_ITER_NEXT | KF_RET_NULL)BTF_ID_FLAGS(func, bpf_iter_mptcp_subflow_destroy, KF_ITER_DESTROY)
With these flags, we can drop this additional test in loop:
if (i++ >= MPTCP_SUBFLOWS_MAX) break;
This problem has been bothering me for a while. I got "infinite loop detected" errors when walking the list with bpf_for_each(mptcp_subflow). So I had to use this additional test to break it.
With these KF_ITER_NEW/NEXT/DESTROY flags, no need to use this additional test anymore.
Thanks,
-Geliang
Quoted 23 lines
Quoted 23 lines
net/mptcp/bpf.c +0
-1BTF_ID_FLAGS(func, mptcp_subflow_set_scheduled) BTF_ID_FLAGS(func, bpf_mptcp_subflow_ctx_by_pos) -BTF_ID_FLAGS(func, mptcp_subflow_active) BTF_ID_FLAGS(func, mptcp_set_timeout) BTF_ID_FLAGS(func, mptcp_wnd_end)BTF_ID_FLAGS(func,
tcp_stream_memory_free)net/mptcp/bpf.c +2
-0@@ -241,6 +286,8 @@ static int __init bpf_mptcp_kfunc_init(void) int ret; ret = register_btf_fmodret_id_set(&bpf_mptcp_fmodret_set); + ret = ret ?:register_btf_kfunc_id_set (BPF_PROG_TYPE_TRACING , + &bpf_mptcp_sched_kfunc_set);ret = ret ?:
register_btf_kfunc_id_set(BPF_PROG_TYPE_STRUCT_OPS,&bpf_mptcp_sched_kfunc_set);
#ifdefCONFIG_BPF_JIT
--
2.43.0
Re: [PATCH mptcp-next v5 1/5] bpf: Add mptcp_subflow bpf_iter
Quoted 115 lines · Geliang Tang
Hi Andrii,
Quoted 111 lines · Andrii Nakryiko
Quoted 45 lines · Geliang Tang
From: Geliang Tang <[email protected]>
It's necessary to traverse all subflows on the
conn_listof an MPTCP
socket and then call kfunc to modify the fields of each subflow. In kernel space,mptcp_for_each_subflow()helper is used for this:mptcp_for_each_subflow(msk, subflow) kfunc(subflow);But in the MPTCP BPF program, this has not yet been implemented. As Martin suggested recently, this
conn_listwalking + modify-by-kfunc usage fits thebpf_iteruse case. So this patch adds a newbpf_itertype named "mptcp_subflow" to do this and implements its helpers bpf_iter_mptcp_subflow_new()/_next()/_destroy().Since these
bpf_itermptcp_subflowhelpers are invoked in its selftest
in a ftrace hook formptcp_sched_get_send(), it's necessary to register
them into aBPF_PROG_TYPE_TRACINGtype kfunc set together with other
two used kfuncsmptcp_subflow_active()andmptcp_subflow_set_scheduled().Then
bpf_for_each()formptcp_subflowcan be used in BPF program like
this:i = 0; bpf_rcu_read_lock(); bpf_for_each(mptcp_subflow, subflow, msk) { if (i++ >= MPTCP_SUBFLOWS_MAX) break; kfunc(subflow); } bpf_rcu_read_unlock();v2: remove
msk->pm.lockin_new()and_destroy()(Martin)
dropDEFINE_BPF_ITER_FUNC, change opaque[3] to opaque[2]
(Andrii)
v3: dropbpf_iter__mptcp_subflow
v4: if msk is NULL, initializekit->mskto NULL in_new()and check it inQuoted 9 lines
_next() (Andrii)Suggested-by Martin KaFai Lau <[email protected]>Signed-off-by Geliang Tang <[email protected]>--- net/mptcp/bpf.c | 57 ++++++++++++++++++++++++++++++++++++++++++++- ---- 1 file changed, 52 insertions(+), 5 deletions(-)Looks ok from setting up open-coded iterator things, but I can't speak
for other aspects I mentioned below.Quoted 50 lines
net/mptcp/bpf.c +36
-3@@ -201,9 +201,51 @@ static const struct btf_kfunc_id_set bpf_mptcp_fmodret_set = { .set = &bpf_mptcp_fmodret_ids, }; -__diag_push (); -__diag_ignore_all ("-Wmissing-prototypes" , -"kfuncs which will be used in BPF programs" ); +struct bpf_iter_mptcp_subflow { +__u64 __opaque[2 ]; +} __attribute__((aligned (8 ))); + +struct bpf_iter_mptcp_subflow_kern { +struct mptcp_sock *msk; +struct list_head *pos; +} __attribute__((aligned (8 ))); + +__bpf_kfunc_start_defs (); + +__bpf_kfunc int bpf_iter_mptcp_subflow_new(structbpf_iter_mptcp_subflow *it, +struct mptcp_sock *msk) +{ +struct bpf_iter_mptcp_subflow_kern *kit = (void *)it; + + kit->msk = msk; +if (!msk) +return -EINVAL ; + + kit->pos = &msk->conn_list; +return 0 ; +} + +__bpf_kfunc struct mptcp_subflow_context * +bpf_iter_mptcp_subflow_next(struct bpf_iter_mptcp_subflow *it) +{ + struct bpf_iter_mptcp_subflow_kern *kit = (void *)it; + struct mptcp_subflow_context *subflow; + struct mptcp_sock *msk = kit->msk; + + if (!msk) + return NULL; + + subflow = list_entry(kit->pos->next, struct mptcp_subflow_context, node); + if (!subflow || list_entry_is_head(subflow, &msk-
Quoted 8 lines
Quoted 6 lines
Quoted 1 line
Quoted 1 line
conn_list, node))it's a bit weird that you need both !subflow and
list_entry_is_head().
Can you have NULL subflow at all? But this is the question tomsk->conn_listsemantics.Do you mean to return subflow regardless of whether subflow is NULL? If
Can you have a NULL subflow in the middle of the list and some more items after that? If yes, then you should *skip* NULL subflow.
But if the NULL subflow is always last, then do you even need list_entry_is_head() or list_is_last()?
But ultimately, I have no idea what the data looks like, just double-check that the stopping condition is correct, that's all. Has nothing to do with BPF open-coded iterators.
Quoted 12 lines
so, we need to use
list_is_last()instead oflist_entry_is_head():{ struct bpf_iter_mptcp_subflow_kern *kit = (void *)it; if (!kit->msk || list_is_last(kit->pos, &kit->msk->conn_list)) return NULL; kit->pos = kit->pos->next; return list_entry(kit->pos, struct mptcp_subflow_context, node); }
Quoted 1 line
Hope this is a better version.
ok, works for me as well
Quoted 24 lines
Quoted 24 lines
Quoted 24 lines
net/mptcp/bpf.c +7
-0+return NULL ; + + kit->pos = &subflow->node; +return subflow; +} + +__bpf_kfunc void bpf_iter_mptcp_subflow_destroy(struct
bpf_iter_mptcp_subflow*it)net/mptcp/bpf.c +2
-0+{ +} __bpf_kfunc struct mptcp_subflow_context * bpf_mptcp_subflow_ctx_by_pos(const struct mptcp_sched_data *data,unsigned int pos)
net/mptcp/bpf.c +1
-1@@ -218,12 +260,15 @@ __bpf_kfunc bool bpf_mptcp_subflow_queues_empty(struct sock *sk) return tcp_rtx_queue_empty(sk); } -__diag_pop (); +__bpf_kfunc_end_defs ();
Quoted 7 lines
Quoted 5 lines
net/mptcp/bpf.c __bpf_kfunc bool bpf_mptcp_subflow_queues_empty(struct sock *sk) · line 266
Quoted 2 lines
net/mptcp/bpf.c +1
-0BTF_KFUNCS_START (bpf_mptcp_sched_kfunc_ids) +BTF_ID_FLAGS (func, bpf_iter_mptcp_subflow_new)I'm not 100% sure, but I suspect you might need to specify
KF_TRUSTED_ARGShere to ensure thatstruct mptcp_sock *mskis a
KF_TRUSTED_ARGSbreaks the selftest in patch 2 [1]:
Please, for the next revision, CC [email protected] on *all* patches in your series, so we don't have to search for the selftests in another mailing list.
Quoted 57 lines
'''
libbpf: prog 'trace_mptcp_sched_get_send': BPF program load failed:
Invalid argument
libbpf: prog 'trace_mptcp_sched_get_send': -- BEGIN PROG LOAD LOG -- 0: R1=ctx() R10=fp0
; intBPF_PROG(trace_mptcp_sched_get_send, struct mptcp_sock *msk)@ mptcp_bpf_iter.c:13
0: (79) r6 = *(u64 *)(r1 +0)
func 'mptcp_sched_get_send' arg0 hasbtf_id28019 type STRUCT 'mptcp_sock'
1: R1=ctx() R6_w=ptr_mptcp_sock()
; if (bpf_get_current_pid_tgid()>> 32 != pid) @ mptcp_bpf_iter.c:17
1: (85) call bpf_get_current_pid_tgid#14 ; R0_w=scalar()
2: (77) r0 >>= 32 ;
R0_w=scalar(smin=0,smax=umax=0xffffffff,var_off=(0x0;0xffffffff))
3: (18) r1 =0xffffb766c1684004;
R1_w=map_value(map=mptcp_bp.bss,ks=4,vs=8,off=4)
5: (61) r1 = *(u32 *)(r1 +0) ;
R1_w=scalar(smin=0,smax=umax=0xffffffff,var_off=(0x0;0xffffffff))
6: (67) r1 <<= 32 ;
R1_w=scalar(smax=0x7fffffff00000000,umax=0xffffffff00000000,smin32=0,sm ax32=umax32=0,var_off=(0x0;0xffffffff00000000))
7: (c7) r1 s>>= 32 ;
R1_w=scalar(smin=0xffffffff80000000,smax=0x7fffffff)
8: (5d) if r0 != r1 goto pc+39 ;
R0_w=scalar(smin=smin32=0,smax=umax=umax32=0x7fffffff,var_off=(0x0;0x7fffffff)) R1_w=scalar(smin=smin32=0,smax=umax=umax32=0x7fffffff,var_off=(0x0;0x7fffffff))
; iter = 0; @ mptcp_bpf_iter.c:20
9: (18) r8 =0xffffb766c1684000;
R8_w=map_value(map=mptcp_bp.bss,ks=4,vs=8)
11: (b4) w1 = 0 ; R1_w=0
12: (63) *(u32 *)(r8 +0) = r1 ; R1_w=0
R8_w=map_value(map=mptcp_bp.bss,ks=4,vs=8)
;bpf_rcu_read_lock(); @ mptcp_bpf_iter.c:22
13: (85) call bpf_rcu_read_lock#84967 ;
14: (bf) r7 = r10 ; R7_w=fp0 R10=fp0
; iter = 0; @ mptcp_bpf_iter.c:20
15: (07) r7 += -16 ; R7_w=fp-16
;bpf_for_each(mptcp_subflow, subflow, msk){ @ mptcp_bpf_iter.c:23
16: (bf) r1 = r7 ; R1_w=fp-16 R7_w=fp-16
17: (bf) r2 = r6 ; R2_w=ptr_mptcp_sock()
R6=ptr_mptcp_sock()
18: (85) call bpf_iter_mptcp_subflow_new#62694
R2 must be referenced or trusted
processed 17 insns (limit 1000000)max_states_per_insn0total_states1peak_states1mark_read1
-- END PROG LOAD LOG --
libbpf: prog 'trace_mptcp_sched_get_send': failed to load: -22
libbpf: failed to load object 'mptcp_bpf_iter'
libbpf: failed to load BPF skeleton 'mptcp_bpf_iter': -22 test_bpf_iter:FAIL:skel_open_load:mptcp_iterunexpected error: -22
#169/4 mptcp/bpf_iter:FAIL
'''
Quoted 1 line
I don't know how to fix it yet.
And the verifier is pointing out the real problem (selftest is at [0] for those interested).
struct mptcp_sock *msk argument for fentry/mptcp_sched_get_send can't be trusted to be valid and be properly protected from being freed, etc. You need to have kfuncs that will have KF_ACQUIRE semantics and will take refcount or whatnot (or KF_RCU for RCU-protection).
See some examples where we do the same with tasks or maybe sockets, I'm not very familiar with this area.
Quoted 21 lines
Quoted 7 lines
valid owned kernel object. Other BPF folks might help to clarify this.
net/mptcp/bpf.c __bpf_kfunc bool bpf_mptcp_subflow_queues_empty(struct sock *sk) · line 269
Quoted 3 lines
net/mptcp/bpf.c +3
-0+BTF_ID_FLAGS(func, bpf_iter_mptcp_subflow_next) +BTF_ID_FLAGS(func, bpf_iter_mptcp_subflow_destroy) +BTF_ID_FLAGS (func, mptcp_subflow_active)But we do need to add KF_ITER_NEW/NEXT/DESTROY flags here:
BTF_ID_FLAGS(func, bpf_iter_mptcp_subflow_new, KF_ITER_NEW)BTF_ID_FLAGS(func, bpf_iter_mptcp_subflow_next, KF_ITER_NEXT | KF_RET_NULL)BTF_ID_FLAGS(func, bpf_iter_mptcp_subflow_destroy, KF_ITER_DESTROY)With these flags, we can drop this additional test in loop:
if (i++ >= MPTCP_SUBFLOWS_MAX) break;
Quoted 8 lines
This problem has been bothering me for a while. I got "infinite loop detected" errors when walking the list with
bpf_for_each(mptcp_subflow). So I had to use this additional test to break it.With these KF_ITER_NEW/NEXT/DESTROY flags, no need to use this additional test anymore.
These flags is what makes those functions an open-coded iterator implementation, yes.
Quoted 30 lines
Thanks,
-GeliangQuoted 23 lines
Quoted 23 lines
net/mptcp/bpf.c +0
-1BTF_ID_FLAGS(func, mptcp_subflow_set_scheduled) BTF_ID_FLAGS(func, bpf_mptcp_subflow_ctx_by_pos) -BTF_ID_FLAGS(func, mptcp_subflow_active) BTF_ID_FLAGS(func, mptcp_set_timeout)BTF_ID_FLAGS (func, mptcp_wnd_end)
BTF_ID_FLAGS(func, tcp_stream_memory_free)net/mptcp/bpf.c +2
-0@@ -241,6 +286,8 @@ static int __init bpf_mptcp_kfunc_init(void)int ret; ret =register_btf_fmodret_id_set (&bpf_mptcp_fmodret_set); + ret = ret ?:register_btf_kfunc_id_set (BPF_PROG_TYPE_TRACING , + &bpf_mptcp_sched_kfunc_set); ret = ret ?:register_btf_kfunc_id_set(BPF_PROG_TYPE_STRUCT_OPS,
&bpf_mptcp_sched_kfunc_set);
#ifdefCONFIG_BPF_JIT
--
2.43.0
Martin KaFai Lau
reviewerRe: [PATCH mptcp-next v5 1/5] bpf: Add mptcp_subflow bpf_iter
Quoted 13 lines · Andrii Nakryiko
Quoted 13 lines
Quoted 13 lines
Quoted 13 lines
+__bpf_kfunc int bpf_iter_mptcp_subflow_new(struct
bpf_iter_mptcp_subflow*it,net/mptcp/bpf.c +11
-0+ struct mptcp_sock *msk) +{ +struct bpf_iter_mptcp_subflow_kern *kit = (void *)it; + + kit->msk = msk; +if (!msk) +return -EINVAL ; + + kit->pos = &msk->conn_list; +return 0 ; +}
[ ... ]
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 2 lines
BTF_KFUNCS_START(bpf_mptcp_sched_kfunc_ids)
+BTF_ID_FLAGS(func,bpf_iter_mptcp_subflow_new)I'm not 100% sure, but I suspect you might need to specify
KF_TRUSTED_ARGShere to ensure thatstruct mptcp_sock *mskis a
+1
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
net/mptcp/bpf.c static int __init bpf_mptcp_kfunc_init(void) · line 292
Quoted 8 lines
net/mptcp/bpf.c +2
-0@@ -241,6 +286,8 @@ static int __init bpf_mptcp_kfunc_init(void)int ret; ret =register_btf_fmodret_id_set (&bpf_mptcp_fmodret_set); + ret = ret ?:register_btf_kfunc_id_set (BPF_PROG_TYPE_TRACING , + &bpf_mptcp_sched_kfunc_set);
This cannot be used in tracing.
Going back to my earlier question in v1. How is the msk->conn_list protected?
Geliang Tang
reviewerRe: [PATCH mptcp-next v5 1/5] bpf: Add mptcp_subflow bpf_iter
Hi Martin, Andrii, Matt,
Quoted 18 lines · Martin KaFai Lau
Quoted 14 lines · Andrii Nakryiko
Quoted 14 lines
Quoted 14 lines
Quoted 14 lines
+__bpf_kfunc int bpf_iter_mptcp_subflow_new(struct
bpf_iter_mptcp_subflow*it,
+ structmptcp_sock*msk)net/mptcp/bpf.c +10
-0+{ + struct bpf_iter_mptcp_subflow_kern *kit = (void *)it; + + kit->msk = msk; + if (!msk) + return -EINVAL; + + kit->pos = &msk->conn_list; + return 0; +}[ ... ]
Quoted 8 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 2 lines
BTF_KFUNCS_START(bpf_mptcp_sched_kfunc_ids)
+BTF_ID_FLAGS(func,bpf_iter_mptcp_subflow_new)I'm not 100% sure, but I suspect you might need to specify
KF_TRUSTED_ARGShere to ensure thatstruct mptcp_sock *mskis a+1
So we must add KF_TRUSTED_ARGS flag, right?
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
net/mptcp/bpf.c +0
-0@@ -241,6 +286,8 @@ static int __initbpf_mptcp_kfunc_init (void ) int ret;
Quoted 8 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
ret =
register_btf_fmodret_id_set(&bpf_mptcp_fmodret_set); + ret = ret ?:
register_btf_kfunc_id_set(BPF_PROG_TYPE_TRACING, +&bpf_mptcp_sched_kfunc_set);This cannot be used in tracing.
Actually, we don’t need to use mptcp_subflow bpf_iter in tracing.
We plan to use it in MPTCP BPF packet schedulers, which are not tracing, but "struct_ops" types. And they work well with KF_TRUSTED_ARGS flag in bpf_iter_mptcp_subflow_new:
BTF_ID_FLAGS(func, bpf_iter_mptcp_subflow_new, KF_ITER_NEW | KF_TRUSTED_ARGS);
An example of the scheduler is:
SEC("struct_ops")
int BPF_PROG(bpf_first_get_subflow, struct mptcp_sock *msk,
struct mptcp_sched_data *data)
{
struct mptcp_subflow_context *subflow;
bpf_rcu_read_lock();
bpf_for_each(mptcp_subflow, subflow, msk) {
mptcp_subflow_set_scheduled(subflow, true);
break;
}
bpf_rcu_read_unlock();
return 0;
}SEC(".struct_ops")
struct mptcp_sched_ops first = {
.init = (void *)mptcp_sched_first_init,
.release = (void *)mptcp_sched_first_release,
.get_subflow = (void *)bpf_first_get_subflow,
.name = "bpf_first",};
But BPF mptcp_sched_ops code has not been merged into bpf-next yet, so I simply test this bpf_for_each(mptcp_subflow) in tracing since I noticed other bpf_iter selftests are using tracing too:
progs/iters_task.cSEC("fentry.s/" SYS_PREFIX "sys_getpgid")
progs/iters_css.cSEC("fentry.s/" SYS_PREFIX "sys_getpgid")
If this bpf_for_each(mptcp_subflow) can only be used in struct_ops, I will try to move the selftest into a struct_ops.
Quoted 4 lines
Going back to my earlier question in v1. How is the
msk->conn_listprotected?
msk->conn_list is protected by msk socket lock. (@Matt, am I right?) We use this in kernel code:
struct sock *sk = (struct sock *)msk; lock_sock(sk); kfunc(&msk->conn_list); release_sock(sk);
If so, should we also use lock_sock/release_sock in bpf_iter_mptcp_subflow_next()?
Thanks,
-Geliang
Geliang Tang
reviewerRe: [PATCH mptcp-next v5 1/5] bpf: Add mptcp_subflow bpf_iter
Quoted 107 lines · Geliang Tang
Hi Martin, Andrii, Matt,
Quoted 23 lines · Martin KaFai Lau
Quoted 16 lines · Andrii Nakryiko
Quoted 16 lines
Quoted 16 lines
Quoted 16 lines
+__bpf_kfunc int bpf_iter_mptcp_subflow_new(struct
bpf_iter_mptcp_subflow*it,
+ structmptcp_sock
*msk)
+{
+ structbpf_iter_mptcp_subflow_kern*kit = (void *)it;net/mptcp/bpf.c +8
-0+ + kit->msk = msk; + if (!msk) + return -EINVAL; + + kit->pos = &msk->conn_list; + return 0; +}[ ... ]
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 2 lines
BTF_KFUNCS_START(bpf_mptcp_sched_kfunc_ids)
+BTF_ID_FLAGS(func,bpf_iter_mptcp_subflow_new)Quoted 6 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
I'm not 100% sure, but I suspect you might need to specify
KF_TRUSTED_ARGShere to ensure thatstruct mptcp_sock *mskis
a+1
So we must add
KF_TRUSTED_ARGSflag, right?Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
net/mptcp/bpf.c +0
-0@@ -241,6 +286,8 @@ static int __initbpf_mptcp_kfunc_init (void ) int ret;Quoted 8 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
ret =
register_btf_fmodret_id_set(&bpf_mptcp_fmodret_set); + ret = ret ?:
register_btf_kfunc_id_set(BPF_PROG_TYPE_TRACING, +&bpf_mptcp_sched_kfunc_set);This cannot be used in tracing.
Actually, we don’t need to use
mptcp_subflowbpf_iterin tracing.We plan to use it in MPTCP BPF packet schedulers, which are not tracing, but "
struct_ops" types. And they work well withKF_TRUSTED_ARGSflag inbpf_iter_mptcp_subflow_new:
BTF_ID_FLAGS(func, bpf_iter_mptcp_subflow_new, KF_ITER_NEW | KF_TRUSTED_ARGS);An example of the scheduler is:
SEC("struct_ops")
intBPF_PROG(bpf_first_get_subflow, struct mptcp_sock *msk, struct mptcp_sched_data *data)
{
structmptcp_subflow_context*subflow;
bpf_rcu_read_lock();bpf_for_each(mptcp_subflow, subflow, msk){mptcp_subflow_set_scheduled(subflow, true); break;
}
bpf_rcu_read_unlock();return 0;
}
SEC(".struct_ops")
structmptcp_sched_opsfirst = {
.init = (void *)mptcp_sched_first_init, .release = (void *)mptcp_sched_first_release, .get_subflow = (void *)bpf_first_get_subflow, .name = "bpf_first",
};But BPF
mptcp_sched_opscode has not been merged into bpf-next yet, so
I simply test thisbpf_for_each(mptcp_subflow)in tracing since I noticed otherbpf_iterselftests are using tracing too:
progs/iters_task.cSEC("fentry.s/" SYS_PREFIX "sys_getpgid")
progs/iters_css.cSEC("fentry.s/" SYS_PREFIX "sys_getpgid")If this
bpf_for_each(mptcp_subflow)can only be used instruct_ops, I will try to move the selftest into astruct_ops.Quoted 4 lines
Going back to my earlier question in v1. How is the
msk->conn_listprotected?
msk->conn_listis protected by msk socket lock. (@Matt, am I right?) We
use this in kernel code:
Quoted 8 lines
struct sock *sk = (struct sock *)msk; lock_sock(sk); kfunc(&msk->conn_list); release_sock(sk);If so, should we also use lock_sock/release_sock in
bpf_iter_mptcp_subflow_next()?
bpf_for_each(mptcp_subflow) is expected to be used in the get_subflow() interface of mptcp_sched_ops to traverse all subflows and then pick one subflow to send data. This interface is invoked in mptcp_sched_get_send(), here the msk socket lock is hold already:
int mptcp_sched_get_send(struct mptcp_sock *msk)
{
struct mptcp_subflow_context *subflow;
struct mptcp_sched_data data;
msk_owned_by_me(msk);
... ...
mptcp_for_each_subflow(msk, subflow) {
if (READ_ONCE(subflow->scheduled))
return 0;
}
data.reinject = false;
if (msk->sched == &mptcp_sched_default || !msk->sched)
return mptcp_sched_default_get_subflow(msk, &data);
return msk->sched->get_subflow(msk, &data);
}So no need to hold msk socket lock again in BPF schedulers. This means we can walk the conn_list without any lock. bpf_rcu_read_lock() and bpf_rcu_read_unlock() can be dropped in BPF schedulers too. Am I right?
Thanks,
-Geliang
Martin KaFai Lau
reviewerRe: [PATCH mptcp-next v5 1/5] bpf: Add mptcp_subflow bpf_iter
Quoted 63 lines · Geliang Tang
Quoted 63 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
(quoted diff) +0
-0@@ -241,6 +286,8 @@ static int __init bpf_mptcp_kfunc_init(void) int ret;Quoted 8 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
(quoted diff) +2
-0ret = register_btf_fmodret_id_set(&bpf_mptcp_fmodret_set); + ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_TRACING, +
&bpf_mptcp_sched_kfunc_set);This cannot be used in tracing.
Actually, we don’t need to use
mptcp_subflowbpf_iterin tracing.We plan to use it in MPTCP BPF packet schedulers, which are not tracing, but "
struct_ops" types. And they work well withKF_TRUSTED_ARGSflag inbpf_iter_mptcp_subflow_new:
BTF_ID_FLAGS(func, bpf_iter_mptcp_subflow_new, KF_ITER_NEW | KF_TRUSTED_ARGS);An example of the scheduler is:
SEC("struct_ops")
int BPF_PROG(bpf_first_get_subflow, structmptcp_sock*msk,
structmptcp_sched_data*data)
{
structmptcp_subflow_context*subflow;
bpf_rcu_read_lock();bpf_for_each(mptcp_subflow, subflow, msk){mptcp_subflow_set_scheduled(subflow, true); break;
}
bpf_rcu_read_unlock();return 0;
}
SEC(".struct_ops")
structmptcp_sched_opsfirst = {
.init = (void *)mptcp_sched_first_init, .release = (void *)mptcp_sched_first_release, .get_subflow = (void *)bpf_first_get_subflow, .name = "bpf_first",
};But BPF
mptcp_sched_opscode has not been merged into bpf-next yet, so
I simply test thisbpf_for_each(mptcp_subflow)in tracing since I noticed otherbpf_iterselftests are using tracing too:
progs/iters_task.cSEC("fentry.s/" SYS_PREFIX "sys_getpgid")
progs/iters_css.cSEC("fentry.s/" SYS_PREFIX "sys_getpgid")If this
bpf_for_each(mptcp_subflow)can only be used instruct_ops, I will try to move the selftest into astruct_ops.Quoted 1 line
Quoted 6 lines
Quoted 6 lines
Quoted 3 lines
Going back to my earlier question in v1. How is the
msk->conn_listprotected?
msk->conn_listis protected by msk socket lock. (@Matt, am I right?) We use this in kernel code:
A KF_TRUSTED_ARGS msk does not mean its sock locked. That said, only the tp_btf should have trusted msk args but still it is hard to audit all existing and future tracepoints which may or may not have msk lock held.
If the subflow list is rcu-ify, it will be easier to reason for tracing use case. Regardless, the use case is not for tracing, so this discussion is probably moot now.
Quoted 35 lines
Quoted 9 lines
struct sock *sk = (struct sock *)msk; lock_sock(sk); kfunc(&msk->conn_list); release_sock(sk);If so, should we also use lock_sock/release_sock in
bpf_iter_mptcp_subflow_next()?
bpf_for_each(mptcp_subflow)is expected to be used in theget_subflow()interface ofmptcp_sched_opsto traverse all subflows and then pick one subflow to send data. This interface is invoked inmptcp_sched_get_send(), here the msk socket lock is hold already:int
mptcp_sched_get_send(struct mptcp_sock *msk){ struct mptcp_subflow_context *subflow; struct mptcp_sched_data data; msk_owned_by_me(msk); ... ... mptcp_for_each_subflow(msk, subflow) { if (READ_ONCE(subflow->scheduled)) return 0; } data.reinject = false; if (msk->sched == &mptcp_sched_default || !msk->sched) return mptcp_sched_default_get_subflow(msk, &data); return msk->sched->get_subflow(msk, &data); }
Quoted 3 lines
So no need to hold msk socket lock again in BPF schedulers. This means we can walk the
conn_listwithout any lock.bpf_rcu_read_lock()andbpf_rcu_read_unlock()can be dropped in BPF schedulers too. Am I right?
hmm... don't know how bpf_rcu_read_lock() comes into picture considering the msk lock has already been held in the struct_ops that you are planning to add.
Something you may need to consider on the subflow locking situation. Not sure exactly what the bpf prog needs to do on a subflow. e.g. If it needs to change some fields in the subflow, does it need to lock the subflow or the msk lock is good enough.
Geliang Tang
reviewerPATCH v5 2/5 selftests/bpf: Add mptcp_subflow bpf_iter test prog
In reply to Geliang TangFrom: Geliang Tang <[email protected]>
This patch adds a new mptcp bpf selftest program for the newly added mptcp_subflow bpf_iter.
Export bpf_iter_mptcp_subflow_new/_next/_destroy into bpf_experimental.h then use bpf_for_each(mptcp_subflow, subflow, msk) to walk the mptcp subflow list.
Add a ftrace hook for mptcp_sched_get_send() to do this test and invoke kfuncs mptcp_subflow_active() and mptcp_subflow_set_scheduled() in the loops.
--- .../testing/selftests/bpf/bpf_experimental.h | 7 ++++ tools/testing/selftests/bpf/progs/mptcp_bpf.h | 4 ++ .../selftests/bpf/progs/mptcp_bpf_iter.c | 38 +++++++++++++++++++ 3 files changed, 49 insertions(+) create mode 100644 tools/testing/selftests/bpf/progs/mptcp_bpf_iter.c
tools/testing/selftests/bpf/bpf_experimental.h +7 -0
@@ -549,6 +549,13 @@ extern int bpf_iter_css_new(struct bpf_iter_css *it,extern struct cgroup_subsys_state *bpf_iter_css_next (struct bpf_iter_css *it)__weak __ksym; extern void bpf_iter_css_destroy(struct bpf_iter_css *it) __weak __ksym; +struct bpf_iter_mptcp_subflow ; +extern int bpf_iter_mptcp_subflow_new(struct bpf_iter_mptcp_subflow *it, + struct mptcp_sock *msk) __weak __ksym; +extern struct mptcp_subflow_context * +bpf_iter_mptcp_subflow_next (struct bpf_iter_mptcp_subflow *it)__weak __ksym; +extern void bpf_iter_mptcp_subflow_destroy(struct bpf_iter_mptcp_subflow *it) __weak __ksym; + extern int bpf_wq_init(struct bpf_wq *wq, void *p__map, unsigned int flags) __weak __ksym; extern int bpf_wq_start(struct bpf_wq *wq, unsigned int flags) __weak __ksym; extern int bpf_wq_set_callback_impl(struct bpf_wq *wq,
tools/testing/selftests/bpf/progs/mptcp_bpf.h +4 -0
@@ -43,9 +43,13 @@ mptcp_subflow_tcp_sock(const struct mptcp_subflow_context *subflow) }/* ksym */ +extern bool mptcp_subflow_active(struct mptcp_subflow_context *subflow) __ksym; extern void mptcp_subflow_set_scheduled(struct mptcp_subflow_context *subflow, bool scheduled) __ksym; +extern void bpf_rcu_read_lock(void) __ksym; +extern void bpf_rcu_read_unlock(void) __ksym; +extern struct mptcp_subflow_context *bpf_mptcp_subflow_ctx_by_pos (const struct mptcp_sched_data *data,unsigned int pos) __ksym;
tools/testing/selftests/bpf/progs/mptcp_bpf_iter.c new file +38 -0
@@ -0,0 +1,38 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2024, Kylin Software */ + +/* vmlinux.h, bpf_helpers.h and other 'define' */ +#include "bpf_tracing_net.h" +#include "mptcp_bpf.h" + +char _license[] SEC("license") = "GPL"; +int iter =0 ; +int pid; + +SEC("fentry/mptcp_sched_get_send") +int BPF_PROG (trace_mptcp_sched_get_send ,struct mptcp_sock *msk) +{ +struct mptcp_subflow_context *subflow; +int i =0 ; + +if (bpf_get_current_pid_tgid () >>32 != pid) +return 0 ; + +bpf_rcu_read_lock (); +bpf_for_each (mptcp_subflow, subflow, msk) { +if (i++ >=MPTCP_SUBFLOWS_MAX ) +break ; + +if (subflow->token != msk->token) +break ; + +if (!mptcp_subflow_active (subflow)) +continue ; + +mptcp_subflow_set_scheduled (subflow, false); + } +bpf_rcu_read_unlock (); + + iter = i; +return 0 ; +}
Signature · 1 line
2.43.0
Geliang Tang
reviewerPATCH v5 3/5 selftests/bpf: Add mptcp_subflow bpf_iter subtest
In reply to Geliang TangFrom: Geliang Tang <[email protected]>
This patch adds a subtest named test_bpf_iter to load and verify the newly added mptcp_subflow type bpf_iter example in test_mptcp. Use endpoint_init() to add a new subflow endpoint. A new helper recv_send() is invoked to trigger the ftrace hook for mptcp_sched_get_send() after wait_for_new_subflows(). Check if skel->bss->iter equals 2 to verify whether the ftrace program loops twice as expected.
--- .../testing/selftests/bpf/prog_tests/mptcp.c | 85 +++++++++++++++++++ 1 file changed, 85 insertions(+)
tools/testing/selftests/bpf/prog_tests/mptcp.c +85 -0
@@ -11,6 +11,7 @@#include "mptcp_sock.skel.h" #include "mptcpify.skel.h" #include "mptcp_subflow.skel.h" +#include "mptcp_bpf_iter.skel.h" #include "mptcp_bpf_first.skel.h" #include "mptcp_bpf_bkup.skel.h" #include "mptcp_bpf_rr.skel.h"
@@ -259,6 +260,34 @@ static void send_byte(int fd)ASSERT_EQ (write (fd, &b,sizeof (b)),1 ,"send single byte" ); } +static int recv_send (int server_fd) +{ +char buf[1 ]; +int ret, fd; +ssize_t n; + + fd =accept (server_fd,NULL ,NULL ); +if (!ASSERT_OK_FD (fd,"accept" )) +return -1 ; + + n =recv (fd, buf,sizeof (buf),0 ); +if (!ASSERT_GT (n,0 ,"recv" )) { + ret =-1 ; + goto close; + } + + n =send (fd, buf, n,0 ); +if (!ASSERT_GT (n,0 ,"send" )) { + ret =-1 ; + goto close; + } + + ret =0 ; +close: +close (fd); +return ret; +} +static int verify_mptcpify (int server_fd,int client_fd) {struct __mptcp_info info;
@@ -471,6 +500,60 @@ static void test_subflow(void)close (cgroup_fd); } +static void run_bpf_iter (void ) +{ +int server_fd, client_fd; + + server_fd =start_mptcp_server (AF_INET ,ADDR_1 ,PORT_1 ,0 ); +if (!ASSERT_OK_FD (server_fd,"start_mptcp_server" )) +return ; + + client_fd =connect_to_fd (server_fd,0 ); +if (!ASSERT_OK_FD (client_fd,"connect_to_fd" )) + goto close_server; + +send_byte (client_fd); +wait_for_new_subflows (client_fd); +recv_send (server_fd); + +close (client_fd); +close_server: +close (server_fd); +} + +static void test_bpf_iter (void ) +{ +struct mptcp_bpf_iter *skel; +struct nstoken *nstoken; +int err; + + skel =mptcp_bpf_iter__open_and_load (); +if (!ASSERT_OK_PTR (skel,"skel_open_load: mptcp_iter" )) +return ; + + skel->bss->pid =getpid (); + + err =mptcp_bpf_iter__attach (skel); +if (!ASSERT_OK (err,"skel_attach: mptcp_iter" )) + goto skel_destroy; + + nstoken =create_netns (); +if (!ASSERT_OK_PTR (nstoken,"create_netns: mptcp_iter" )) + goto skel_destroy; + +if (endpoint_init ("subflow" ) <0 ) + goto close_netns; + +run_bpf_iter (); + +ASSERT_EQ (skel->bss->iter,2 ,"iter" ); + +close_netns: +cleanup_netns (nstoken); +skel_destroy: +mptcp_bpf_iter__destroy (skel); +} +static struct nstoken *sched_init (char *flags,char *sched) {struct nstoken *nstoken;
@@ -652,6 +735,8 @@ void test_mptcp(void)test_mptcpify ();if (test__start_subtest ("subflow" ))test_subflow (); +if (test__start_subtest ("bpf_iter" )) +test_bpf_iter ();if (test__start_subtest ("default" ))test_default ();if (test__start_subtest ("first" ))
Signature · 1 line
2.43.0
Geliang Tang
reviewerPATCH v5 4/5 Squash to "mptcp: add sched_data helpers"
In reply to Geliang TangFrom: Geliang Tang <[email protected]>
Drop bpf_mptcp_subflow_ctx_by_pos declaration, since "-Wmissing-declarations" is ignored in __bpf_kfunc_start_defs.
--- net/mptcp/protocol.h | 2 -- 1 file changed, 2 deletions(-)
net/mptcp/protocol.h +0 -2
@@ -722,8 +722,6 @@ void mptcp_sock_graft(struct sock *sk, struct socket *parent);u64 mptcp_wnd_end (const struct mptcp_sock *msk);void mptcp_set_timeout (struct sock *sk);bool bpf_mptcp_subflow_queues_empty (struct sock *sk); -struct mptcp_subflow_context * -bpf_mptcp_subflow_ctx_by_pos (const struct mptcp_sched_data *data,unsigned int pos);struct sock *__mptcp_nmpc_sk (struct mptcp_sock *msk);bool __mptcp_close (struct sock *sk,long timeout);void mptcp_cancel_work (struct sock *sk);
Signature · 1 line
2.43.0
Geliang Tang
reviewerPATCH v5 5/5 Squash to "selftests/bpf: Add bpf_burst scheduler & test"
In reply to Geliang TangFrom: Geliang Tang <[email protected]>
Drop mptcp_subflow_active declaration.
--- tools/testing/selftests/bpf/progs/mptcp_bpf_burst.c | 1 - 1 file changed, 1 deletion(-)
tools/testing/selftests/bpf/progs/mptcp_bpf_burst.c +0 -1
@@ -16,7 +16,6 @@ struct bpf_subflow_send_info {__u64 linger_time; }; -extern bool mptcp_subflow_active(struct mptcp_subflow_context *subflow) __ksym; extern void mptcp_set_timeout(struct sock *sk) __ksym; extern __u64 mptcp_wnd_end(const struct mptcp_sock *msk) __ksym; extern bool tcp_stream_memory_free(const struct sock *sk, int wake) __ksym;
Signature · 1 line
2.43.0
Hi Geliang,
Thank you for your modifications, that's great!
But sadly, our CI spotted some issues with it when trying to build it.
You can find more details there:
https://github.com/multipath-tcp/mptcp_net-next/actions/runs/10828370093
Status: failure
Initiator: Patchew Applier
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/8e5f73626f7f
Patchwork: https://patchwork.kernel.org/project/mptcp/list/?series=889689
Feel free to reply to this email if you cannot access logs, if you need some support to fix the error, if this doesn't seem to be caused by your modifications or if the error is a false positive one.
Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (NGI0 Core)
Matthieu Baerts
maintainerHi Geliang,
Quoted 9 lines · MPTCP CI
Hi Geliang,
Thank you for your modifications, that's great!
But sadly, our CI spotted some issues with it when trying to build it.
You can find more details there:
https://github.com/multipath-tcp/mptcp_net-next/actions/runs/10828370093
Please ignore, it is due to an issue upstream. I'm on it.
Cheers,
Matt
Signature · 1 line
Sponsored by the NGI0 Core fund.
Hi Geliang,
Thank you for your modifications, that's great!
Our CI did some validations and here is its report:
- KVM Validation: normal: Success! ✅
- KVM Validation: debug: Unstable: 1 failed test(s): mptcp_connect_mmap 🔴
- KVM Validation: btf (only bpftest_all): Success! ✅
- Task: https://github.com/multipath-tcp/mptcp_net-next/actions/runs/10828370091
Initiator: Patchew Applier
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/8e5f73626f7f
Patchwork: https://patchwork.kernel.org/project/mptcp/list/?series=889689
If there are some issues, you can reproduce them using the same environment as the one used by the CI thanks to a docker image, e.g.:
$ cd [kernel source code]
$ docker run -v "${PWD}:${PWD}:rw" -w "${PWD}" --privileged --rm -it \
--pull always mptcp/mptcp-upstream-virtme-docker:latest \
auto-normalFor more details:
https://github.com/multipath-tcp/mptcp-upstream-virtme-docker
Please note that despite all the efforts that have been already done to have a stable tests suite when executed on a public CI like here, it is possible some reported issues are not due to your modifications. Still, do not hesitate to help us improve that ;-)
Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (NGI0 Core)
Hi Geliang,
Thank you for your modifications, that's great!
But sadly, our CI spotted some issues with it when trying to build it.
You can find more details there:
https://github.com/multipath-tcp/mptcp_net-next/actions/runs/10829533095
Status: failure
Initiator: Patchew Applier
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/f12243725ecc
Patchwork: https://patchwork.kernel.org/project/mptcp/list/?series=889689
Feel free to reply to this email if you cannot access logs, if you need some support to fix the error, if this doesn't seem to be caused by your modifications or if the error is a false positive one.
Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (NGI0 Core)
Matthieu Baerts
maintainerQuoted 9 lines · MPTCP CI
Hi Geliang,
Thank you for your modifications, that's great!
But sadly, our CI spotted some issues with it when trying to build it.
You can find more details there:
https://github.com/multipath-tcp/mptcp_net-next/actions/runs/10829533095
FYI, this error is different from the previous one, no longer due to a modification upstream because I recently applied a fix for that in our tree.
Cheers,
Matt
Signature · 1 line
Sponsored by the NGI0 Core fund.
Hi Geliang,
Thank you for your modifications, that's great!
Our CI did some validations and here is its report:
- KVM Validation: normal: Success! ✅
- KVM Validation: debug: Success! ✅
- KVM Validation: btf (only bpftest_all): Success! ✅
- Task: https://github.com/multipath-tcp/mptcp_net-next/actions/runs/10829533086
Initiator: Patchew Applier
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/f12243725ecc
Patchwork: https://patchwork.kernel.org/project/mptcp/list/?series=889689
If there are some issues, you can reproduce them using the same environment as the one used by the CI thanks to a docker image, e.g.:
$ cd [kernel source code]
$ docker run -v "${PWD}:${PWD}:rw" -w "${PWD}" --privileged --rm -it \
--pull always mptcp/mptcp-upstream-virtme-docker:latest \
auto-normalFor more details:
https://github.com/multipath-tcp/mptcp-upstream-virtme-docker
Please note that despite all the efforts that have been already done to have a stable tests suite when executed on a public CI like here, it is possible some reported issues are not due to your modifications. Still, do not hesitate to help us improve that ;-)
Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (NGI0 Core)