Squash to "Add mptcp_subflow bpf_iter support"
Subsystems (5)
Geliang Tang
reviewercover Squash to "Add mptcp_subflow bpf_iter support"
From: Geliang Tang <[email protected]>
Address Martin's suggestions for v2.
Geliang Tang (3):
Squash to "bpf: Add mptcp_subflow bpf_iter"
Revert "bpf: Acquire and release mptcp socket" Squash to "selftests/bpf: Add mptcp_subflow bpf_iter subtest"
net/mptcp/bpf.c | 34 +++++-------------- .../testing/selftests/bpf/bpf_experimental.h | 2 +- tools/testing/selftests/bpf/progs/mptcp_bpf.h | 3 -- .../selftests/bpf/progs/mptcp_bpf_iters.c | 6 +--- 4 files changed, 11 insertions(+), 34 deletions(-)
Signature · 1 line
2.43.0
Geliang Tang
reviewerPATCH 1/3 Squash to "bpf: Add mptcp_subflow bpf_iter"
From: Geliang Tang <[email protected]>
Drop the NULL check as Martin suggested.
Use the "struct sock *sk" instead of "struct mptcp-sock *msk" as the argument in the bpf_iter_mptcp_subflow_new as Martin suggested.
Use msk_owned_by_me().
--- net/mptcp/bpf.c | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-)
net/mptcp/bpf.c +9 -6
@@ -234,24 +234,27 @@ bpf_mptcp_subflow_ctx(const struct sock *sk) __bpf_kfunc static intbpf_iter_mptcp_subflow_new (struct bpf_iter_mptcp_subflow *it, -struct mptcp_sock *msk) +struct sock *sk) {struct bpf_iter_mptcp_subflow_kern *kit = (void *)it; -struct sock *sk = (struct sock *)msk; +struct mptcp_sock *msk;BUILD_BUG_ON (sizeof (struct bpf_iter_mptcp_subflow_kern ) >sizeof (struct bpf_iter_mptcp_subflow ));BUILD_BUG_ON (__alignof__(struct bpf_iter_mptcp_subflow_kern ) != __alignof__(struct bpf_iter_mptcp_subflow )); - kit->msk = msk; -if (!msk) +if (unlikely (!sk || !sk_fullsock (sk)))return -EINVAL ; -if (!sock_owned_by_user_nocheck (sk) && - !spin_is_locked (&sk->sk_lock.slock)) +if (sk->sk_protocol !=IPPROTO_MPTCP )return -EINVAL ; + msk =mptcp_sk (sk); + kit->msk = msk; + +msk_owned_by_me (msk); + kit->pos = &msk->conn_list;return 0 ; }
Signature · 1 line
2.43.0
Matthieu Baerts
maintainerRe: [PATCH mptcp-next 1/3] Squash to "bpf: Add mptcp_subflow bpf_iter"
Hi Geliang,
(+cc Mat who reviewed the previous versions)
Quoted 8 lines · Geliang Tang
From: Geliang Tang <[email protected]>
Drop the NULL check as Martin suggested.
Use the "struct sock *sk" instead of "struct mptcp-sock *msk" as the argument in the
bpf_iter_mptcp_subflow_newas Martin suggested.Use
msk_owned_by_me().
Regarding this, do you mind checking my question on the thread initially sent to the BPF mailing please?
Quoted 5 lines
Quoted 4 lines
This set is only showing the cg sockopt bpf prog but missing the major
struct_opspiece. It is hard to comment. I assumed the lock situation is the same for thestruct_opswhere the lock will be held before calling thestruct_opsprog?
Quoted 7 lines
Can we restrict the use of this helper only for some specific callbacks? Or for only some specific
struct_ops?Last time I looked, I thought that you could only restrict it like that:
"this helper can be used by (any)struct_ops". But then does it mean this helper could be called from a TCP BPF CC ops or another unrelated one for example? How can we avoid that?
Because with the suggested modification, there is no more checks regarding the lock situation: it is probably fine now with [gs]etsockopt(), but what about later with struct_ops? Can we restrict this kfunc to be used only in some contexts? e.g. a part or all the MPTCP struct_ops, but not the other ones?
Cheers,
Matt
Signature · 1 line
Sponsored by the NGI0 Core fund.
Geliang Tang
reviewerRe: [PATCH mptcp-next 1/3] Squash to "bpf: Add mptcp_subflow bpf_iter"
Hi Matt,
Thanks for your review.
Quoted 38 lines · Matthieu Baerts
Hi Geliang,
(+cc Mat who reviewed the previous versions)
Quoted 9 lines · Geliang Tang
From: Geliang Tang <[email protected]>
Drop the NULL check as Martin suggested.
Use the "struct sock *sk" instead of "struct mptcp-sock *msk" as the
argument in thebpf_iter_mptcp_subflow_newas Martin suggested.Use
msk_owned_by_me().Regarding this, do you mind checking my question on the thread initially
sent to the BPF mailing please?Quoted 12 lines
Quoted 7 lines
This set is only showing the cg sockopt bpf prog but missing the major
struct_opspiece. It is hard to comment. I assumed the lock situation is
the same for thestruct_opswhere the lock will be held before calling
thestruct_opsprog?Can we restrict the use of this helper only for some specific callbacks?
Or for only some specificstruct_ops?Quoted 7 lines
Last time I looked, I thought that you could only restrict it like that:
"this helper can be used by (any)struct_ops". But then does it mean
this helper could be called from a TCP BPF CC ops or another unrelated
one for example? How can we avoid that?
Quoted 6 lines
Because with the suggested modification, there is no more checks regarding the lock situation: it is probably fine now with [gs]etsockopt(), but what about later with
struct_ops? Can we restrict
this kfunc to be used only in some contexts? e.g. a part or all the MPTCPstruct_ops, but not the other ones?
We use register_btf_kfunc_id_set() to allow these kfuncs to be used in struct_ops:
register_btf_kfunc_id_set(BPF_PROG_TYPE_STRUCT_OPS,
&bpf_mptcp_common_kfunc_set);
And we also check if the protocol of the socket is MPTCP in bpf_iter_mptcp_subflow_new(). So this mptcp_subflow bpf_iter is restricted to be used for MPTCP. Other struct_ops likes a TCP BPF CC ops can't use it.
Do you think this restriction is enough?
Another way to restrict kfuncs is to set get_func_proto of struct bpf_verifier_ops. But this way has not been applied to other BPF iters.
Thanks,
-Geliang
Quoted 3 lines
Cheers,
Matt
Matthieu Baerts
maintainerRe: [PATCH mptcp-next 1/3] Squash to "bpf: Add mptcp_subflow bpf_iter"
Hi Geliang,
Quoted 50 lines · Geliang Tang
Hi Matt,
Thanks for your review.
Quoted 38 lines · Matthieu Baerts
Hi Geliang,
(+cc Mat who reviewed the previous versions)
Quoted 9 lines · Geliang Tang
From: Geliang Tang <[email protected]>
Drop the NULL check as Martin suggested.
Use the "struct sock *sk" instead of "struct mptcp-sock *msk" as the
argument in thebpf_iter_mptcp_subflow_newas Martin suggested.Use
msk_owned_by_me().Regarding this, do you mind checking my question on the thread initially
sent to the BPF mailing please?Quoted 12 lines
Quoted 7 lines
This set is only showing the cg sockopt bpf prog but missing the major
struct_opspiece. It is hard to comment. I assumed the lock situation is
the same for thestruct_opswhere the lock will be held before calling
thestruct_opsprog?Can we restrict the use of this helper only for some specific callbacks?
Or for only some specificstruct_ops?Quoted 7 lines
Last time I looked, I thought that you could only restrict it like that:
"this helper can be used by (any)struct_ops". But then does it mean
this helper could be called from a TCP BPF CC ops or another unrelated
one for example? How can we avoid that?Quoted 6 lines
Because with the suggested modification, there is no more checks regarding the lock situation: it is probably fine now with [gs]etsockopt(), but what about later with
struct_ops? Can we restrict
this kfunc to be used only in some contexts? e.g. a part or all the MPTCPstruct_ops, but not the other ones?
Quoted 5 lines
We use
register_btf_kfunc_id_set()to allow these kfuncs to be used instruct_ops:register_btf_kfunc_id_set(BPF_PROG_TYPE_STRUCT_OPS,
&bpf_mptcp_common_kfunc_set);
OK so if I understand correctly, it means that the kfunc that are part of this 'mptcp_common' set can be used from any struct ops, right? Or in other words, the new MPTCP kfunc could be used from the TCP BPF CC struct ops for example, right?
If yes, it might be clearer to ask BPF maintainers how we should design the new kfunc that are supposed to be used only in the context of the new BPF packet scheduler, e.g. should we check who own the msk socket.
I can ask this question if you prefer.
Quoted 4 lines
And we also check if the protocol of the socket is MPTCP in
bpf_iter_mptcp_subflow_new(). So thismptcp_subflowbpf_iteris restricted to be used for MPTCP. Otherstruct_opslikes a TCP BPF CC ops can't use it.
I see. For the bpf_iter case, these restrictions should be fine. Maybe not for the kfunc needed for the scheduler part.
Quoted 4 lines
Do you think this restriction is enough?
Another way to restrict kfuncs is to set
get_func_protoof structbpf_verifier_ops. But this way has not been applied to other BPF iters.
No, I guess for bpf_iter, we don't need that. Probably best to see them as a "generic thing" and have the previous checks we already have.
Cheers,
Matt
Signature · 1 line
Sponsored by the NGI0 Core fund.
Mat Martineau
maintainerRe: [PATCH mptcp-next 1/3] Squash to "bpf: Add mptcp_subflow bpf_iter"
In reply to Geliang TangQuoted 44 lines · Geliang Tang
From: Geliang Tang <[email protected]>
Drop the NULL check as Martin suggested.
Use the "struct sock *sk" instead of "struct mptcp-sock *msk" as the argument in the
bpf_iter_mptcp_subflow_newas Martin suggested.Use
msk_owned_by_me().Signed-off-by Geliang Tang <[email protected]>---
net/mptcp/bpf.c| 15 +++++++++------1 file changed, 9 insertions(+), 6 deletions(-)net/mptcp/bpf.c +0
-0@@ -234,24 +234,27 @@ bpf_mptcp_subflow_ctx(const struct sock *sk)
__bpf_kfuncstatic int bpf_iter_mptcp_subflow_new(structbpf_iter_mptcp_subflow*it,net/mptcp/bpf.c +1
-1- struct mptcp_sock *msk) + struct sock *sk){ struct bpf_iter_mptcp_subflow_kern *kit = (void *)it;net/mptcp/bpf.c +1
-1-struct sock *sk = (struct sock *)msk; +struct mptcp_sock *msk;BUILD_BUG_ON(sizeof(struct bpf_iter_mptcp_subflow_kern) > sizeof(struct bpf_iter_mptcp_subflow)); BUILD_BUG_ON(__alignof__(struct bpf_iter_mptcp_subflow_kern) != __alignof__(struct bpf_iter_mptcp_subflow));net/mptcp/bpf.c +1
-2- kit->msk = msk; -if (!msk) +if (unlikely (!sk || !sk_fullsock (sk)))return -EINVAL;net/mptcp/bpf.c +1
-2-if (!sock_owned_by_user_nocheck (sk) && - !spin_is_locked (&sk->sk_lock.slock)) +if (sk->sk_protocol !=IPPROTO_MPTCP )return -EINVAL;
net/mptcp/bpf.c bpf_mptcp_subflow_ctx(const struct sock *sk) · line 256
Quoted 4 lines
net/mptcp/bpf.c +4
-0+ msk =mptcp_sk (sk); + kit->msk = msk; + +msk_owned_by_me (msk);
Hi Geliang -
The convention is to check the owner before assigning the msk.
I also noticed in the reply to patch 5 that Martin mentioned adding a "bpf_mptcp_sock_from_sock check" to bpf_iter_mptcp_subflow_new (https://lore.kernel.org/netdev/[email protected]/)
- Mat
Quoted 9 lines
+
kit->pos = &msk->conn_list; return 0; }Signature · 1 line
2.43.0
Geliang Tang
reviewerRe: [PATCH mptcp-next 1/3] Squash to "bpf: Add mptcp_subflow bpf_iter"
Hi Mat,
Thanks for your review.
Quoted 48 lines · Mat Martineau
Quoted 46 lines · Geliang Tang
From: Geliang Tang <[email protected]>
Drop the NULL check as Martin suggested.
Use the "struct sock *sk" instead of "struct mptcp-sock *msk" as the
argument in thebpf_iter_mptcp_subflow_newas Martin suggested.Use
msk_owned_by_me().Signed-off-by Geliang Tang <[email protected]>---
net/mptcp/bpf.c| 15 +++++++++------1 file changed, 9 insertions(+), 6 deletions(-)net/mptcp/bpf.c +0
-0@@ -234,24 +234,27 @@ bpf_mptcp_subflow_ctx(const struct sock *sk)
__bpf_kfuncstatic int bpf_iter_mptcp_subflow_new(structbpf_iter_mptcp_subflow*it,net/mptcp/bpf.c +1
-1- struct mptcp_sock *msk) + struct sock *sk){ struct bpf_iter_mptcp_subflow_kern *kit = (void *)it;net/mptcp/bpf.c +1
-1-struct sock *sk = (struct sock *)msk; +struct mptcp_sock *msk;BUILD_BUG_ON(sizeof(struct bpf_iter_mptcp_subflow_kern) > sizeof(struct bpf_iter_mptcp_subflow)); BUILD_BUG_ON(__alignof__(struct bpf_iter_mptcp_subflow_kern) != __alignof__(struct bpf_iter_mptcp_subflow));net/mptcp/bpf.c +1
-2- kit->msk = msk; -if (!msk) +if (unlikely (!sk || !sk_fullsock (sk)))return -EINVAL;net/mptcp/bpf.c +1
-2-if (!sock_owned_by_user_nocheck (sk) && - !spin_is_locked(&sk->sk_lock.slock)) +if (sk->sk_protocol !=IPPROTO_MPTCP )return -EINVAL;
Quoted 8 lines
net/mptcp/bpf.c bpf_mptcp_subflow_ctx(const struct sock *sk) · line 256
Quoted 4 lines
net/mptcp/bpf.c +4
-0+ msk =mptcp_sk (sk); + kit->msk = msk; + +msk_owned_by_me (msk);Hi Geliang -
The convention is to check the owner before assigning the msk.
Yes, indeed. Updated in v2.
Quoted 3 lines
I also noticed in the reply to patch 5 that Martin mentioned adding a "
bpf_mptcp_sock_from_sockcheck" tobpf_iter_mptcp_subflow_new
bpf_mptcp_sock_from_sock should be dropped, so open-code it in bpf_iter_mptcp_subflow_new.
Thanks,
-Geliang
Quoted 16 lines
(
https://lore.kernel.org/netdev/9b373a23-c093-42d8-b4ae-99f2e62e7681@li
nux.dev/)- Mat
Quoted 9 lines
+
kit->pos = &msk->conn_list; return 0; }Signature · 1 line
2.43.0
Geliang Tang
reviewerPATCH 2/3 Revert "bpf: Acquire and release mptcp socket"
In reply to Geliang TangFrom: Geliang Tang <[email protected]>
Drop this patch as Martin suggested.
--- net/mptcp/bpf.c | 19 ------------------- 1 file changed, 19 deletions(-)
net/mptcp/bpf.c +0 -19
@@ -276,23 +276,6 @@ bpf_iter_mptcp_subflow_destroy(struct bpf_iter_mptcp_subflow *it) { } -__bpf_kfunc static struct -mptcp_sock *bpf_mptcp_sock_acquire (struct mptcp_sock *msk) -{ -struct sock *sk = (struct sock *)msk; - -if (sk &&refcount_inc_not_zero (&sk->sk_refcnt)) -return msk; -return NULL ; -} - -__bpf_kfunc static void bpf_mptcp_sock_release(struct mptcp_sock *msk) -{ -struct sock *sk = (struct sock *)msk; - -WARN_ON_ONCE (!sk || !refcount_dec_not_one (&sk->sk_refcnt)); -} - __bpf_kfunc struct mptcp_subflow_context *bpf_mptcp_subflow_ctx_by_pos (const struct mptcp_sched_data *data,unsigned int pos) {
@@ -313,8 +296,6 @@ BTF_ID_FLAGS(func, bpf_mptcp_subflow_ctx, KF_RET_NULL) BTF_ID_FLAGS(func, bpf_iter_mptcp_subflow_new, KF_ITER_NEW | KF_TRUSTED_ARGS) BTF_ID_FLAGS(func, bpf_iter_mptcp_subflow_next, KF_ITER_NEXT | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_iter_mptcp_subflow_destroy, KF_ITER_DESTROY) -BTF_ID_FLAGS(func, bpf_mptcp_sock_acquire, KF_ACQUIRE | KF_RET_NULL) -BTF_ID_FLAGS(func, bpf_mptcp_sock_release, KF_RELEASE)BTF_KFUNCS_END (bpf_mptcp_common_kfunc_ids) static const struct btf_kfunc_id_set bpf_mptcp_common_kfunc_set = {
Signature · 1 line
2.43.0
Geliang Tang
reviewerPATCH 3/3 Squash to "selftests/bpf: Add mptcp_subflow bpf_iter subtest"
In reply to Geliang TangFrom: Geliang Tang <[email protected]>
Drop bpf_mptcp_sock_acquire/release.
--- tools/testing/selftests/bpf/bpf_experimental.h | 2 +- tools/testing/selftests/bpf/progs/mptcp_bpf.h | 3 --- tools/testing/selftests/bpf/progs/mptcp_bpf_iters.c | 6 +----- 3 files changed, 2 insertions(+), 9 deletions(-)
tools/testing/selftests/bpf/bpf_experimental.h +1 -1
@@ -577,7 +577,7 @@ extern void bpf_iter_css_destroy(struct bpf_iter_css *it) __weak __ksym;struct bpf_iter_mptcp_subflow ; extern int bpf_iter_mptcp_subflow_new(struct bpf_iter_mptcp_subflow *it, - struct mptcp_sock *msk) __weak __ksym; + struct sock *sk) __weak __ksym;extern struct mptcp_subflow_context *bpf_iter_mptcp_subflow_next (struct bpf_iter_mptcp_subflow *it)__weak __ksym; extern void
tools/testing/selftests/bpf/progs/mptcp_bpf.h +0 -3
@@ -43,9 +43,6 @@ mptcp_subflow_tcp_sock(const struct mptcp_subflow_context *subflow) }/* ksym */ -extern struct mptcp_sock *bpf_mptcp_sock_acquire (struct mptcp_sock *msk) __ksym; -extern void bpf_mptcp_sock_release(struct mptcp_sock *msk) __ksym; -extern struct mptcp_subflow_context *bpf_mptcp_subflow_ctx (const struct sock *sk) __ksym; extern struct sock *
tools/testing/selftests/bpf/progs/mptcp_bpf_iters.c +1 -5
@@ -28,10 +28,7 @@ int iters_subflow(struct bpf_sockopt *ctx)if (!msk || msk->pm.server_side || !msk->pm.subflows)return 1 ; - msk =bpf_mptcp_sock_acquire (msk); -if (!msk) -return 1 ; -bpf_for_each (mptcp_subflow, subflow, msk) { +bpf_for_each (mptcp_subflow, subflow, (struct sock *)sk) {/* Here MPTCP-specific packet scheduler kfunc can be called: * this test is not doing anything really useful, only to * verify the iteration works.
@@ -58,6 +55,5 @@ int iters_subflow(struct bpf_sockopt *ctx) ids = local_ids; out: -bpf_mptcp_sock_release (msk);return 1 ; }
Signature · 1 line
2.43.0
Matthieu Baerts
maintainerRe: [PATCH mptcp-next 3/3] Squash to "selftests/bpf: Add mptcp_subflow bpf_iter subtest"
Quoted 3 lines · Geliang Tang
From: Geliang Tang <[email protected]>
Drop bpf_mptcp_sock_acquire/release.
It might be good to share a link to the WIP commits/patches to Martin (and us) to better understand how it is going to be used.
Maybe a link to a git repo containing commits from the series "use bpf_iter in bpf schedulers" where the squash-to commits have been squashed?
Cheers,
Matt
Signature · 1 line
Sponsored by the NGI0 Core fund.
Geliang Tang
reviewerRe: [PATCH mptcp-next 3/3] Squash to "selftests/bpf: Add mptcp_subflow bpf_iter subtest"
Quoted 11 lines · Matthieu Baerts
Quoted 3 lines · Geliang Tang
From: Geliang Tang <[email protected]>
Drop bpf_mptcp_sock_acquire/release.
It might be good to share a link to the WIP commits/patches to Martin (and us) to better understand how it is going to be used.
Maybe a link to a git repo containing commits from the series "use
bpf_iterin bpf schedulers" where the squash-to commits have been squashed?
Sure, I'll do this.
I think it would be better to directly send these patches to BPF maintainers as RFC ones for more suggestions. Since the last time I sent the link to Martin, he didn't have time to read these patches.
Thanks,
-Geliang
Quoted 3 lines
Cheers,
Matt
Matthieu Baerts
maintainerRe: [PATCH mptcp-next 3/3] Squash to "selftests/bpf: Add mptcp_subflow bpf_iter subtest"
Hi Geliang,
(sorry, it looks like I missed this reply)
Quoted 13 lines · Geliang Tang
Quoted 11 lines · Matthieu Baerts
Quoted 3 lines · Geliang Tang
From: Geliang Tang <[email protected]>
Drop bpf_mptcp_sock_acquire/release.
It might be good to share a link to the WIP commits/patches to Martin (and us) to better understand how it is going to be used.
Maybe a link to a git repo containing commits from the series "use
bpf_iterin bpf schedulers" where the squash-to commits have been squashed?
Quoted 5 lines
Sure, I'll do this.
I think it would be better to directly send these patches to BPF maintainers as RFC ones for more suggestions. Since the last time I sent the link to Martin, he didn't have time to read these patches.
I think this time, it is OK with just a link, because Martin asked for it. Probably best to wait before sending these patches not to add more pressure to rework the scheduler API.
Cheers,
Matt
Signature · 1 line
Sponsored by the NGI0 Core fund.
Hi Geliang,
Thank you for your modifications, that's great!
Our CI did some validations and here is its report:
- KVM Validation: normal: Success! ✅
- KVM Validation: debug: Success! ✅
- KVM Validation: btf-normal (only bpftest_all): Success! ✅
- KVM Validation: btf-debug (only bpftest_all): Success! ✅
- Task: https://github.com/multipath-tcp/mptcp_net-next/actions/runs/12948765422
Initiator: Patchew Applier
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/927f2c1cee5c
Patchwork: https://patchwork.kernel.org/project/mptcp/list/?series=928101
If there are some issues, you can reproduce them using the same environment as the one used by the CI thanks to a docker image, e.g.:
$ cd [kernel source code]
$ docker run -v "${PWD}:${PWD}:rw" -w "${PWD}" --privileged --rm -it \
--pull always mptcp/mptcp-upstream-virtme-docker:latest \
auto-normalFor more details:
https://github.com/multipath-tcp/mptcp-upstream-virtme-docker
Please note that despite all the efforts that have been already done to have a stable tests suite when executed on a public CI like here, it is possible some reported issues are not due to your modifications. Still, do not hesitate to help us improve that ;-)
Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (NGI0 Core)