BPF packet scheduler
Subsystems (6)
v17 cover BPF packet scheduler
v17:
- drop "mptcp: register default scheduler"
- add back "mptcp: add last_snd write access"
- update "mptcp: add get_subflow wrappers" as Paolo suggested.
- fix the issue reported-by kernel test robot <[email protected]> in [1].
v16:
- add CONFIG_BPF_JIT check around the scheduler code inside bpf.c
- depends on export/20220426T102605 +
Squash to "bpf: add bpf_skc_to_mptcp_sock_proto" + Squash to "selftests: bpf: test bpf_skc_to_mptcp_sock" Squash to "selftests: bpf: add MPTCP test base"
v15:
- set mptcp_sched_id and mptcp_sched_type in bpf_mptcp_sched_init().
- add CONFIG_BPF_JIT in patch 6.
- drop '#include <sys/param.h>' in patch 8.
- depends on export/20220426T102605 +
Squash to "bpf: add bpf_skc_to_mptcp_sock_proto" + Squash to "selftests: bpf: test bpf_skc_to_mptcp_sock"
v14:
- add struct mptcp_sched_data.
base-commit: export/20220422T060337
v13:
- rename retrans to reinject
- drop "add last_snd write access" patch
- change %lu to %zu to fix the build break on i386.
base-commit: export/20220420T152103
v12:
- add call_me_again flag.
base-commit: export/20220419T055400
v11:
- add retrans argment for get_subflow()
base-commit: export/20220408T100826
v10:
- patch 5: keep msk->last_snd setting in get_subflow().
- patch 6: add bpf_mptcp_sched_btf_struct_access().
- patch 8: use MIN() in sys/param.h, instead of defining a new one.
- update commit logs.
base-commit: export/20220406T054706
v9:
- patch 2: add the missing mptcp_sched_init() invoking in mptcp_proto_init().
- patch 5: set last_snd after invoking get_subflow().
- patch 7: merge the squash-to patch.
v8:
- use global sched_list instead of pernet sched_list.
- drop synchronize_rcu() in mptcp_unregister_scheduler().
- update mptcp_init_sched and mptcp_release_sched as Mat and Florian suggested.
- fix the build break in patch 8.
- depends on: "add skc_to_mptcp_sock" v14.
- export/20220325T055307
v7:
- add bpf_try_module_get in mptcp_init_sched.
- add bpf_module_put in mptcp_release_sched.
- rename bpf_first to mptcp_bpf_first.
- update commit logs.
v6:
- still use pernet sched_list, use current->nsproxy->net_ns in BPF context instead of using init_net.
- patch 1:
- use rcu_read_lock instead of spin_lock in mptcp_sched_find as Florian suggested.
- drop synchronize_rcu in sched_exit_net as Florian suggested.
- keep synchronize_rcu in mptcp_unregister_scheduler, otherwise, got a workqueue lockup in my test.
- update Makefile as Mat suggested.
- patch 2:
- add mptcp_sched_data_init to register default sched, instead of registering it in init_net.
- patch 5:
- move mptcp_sched_get_subflow to protocol.h as Mat suggested.
- patch 6:
- use current->nsproxy->net_ns instead of init_net.
- patch 8:
- add send_data to send more data, instead of send_byte.
v5:
- patch 1: define per-namespace sched_list (but only used init_net namespace. It is difficult to get 'net' in bpf_mptcp_sched_reg and bpf_mptcp_sched_unreg. I need some suggestions here.)
- patch 2: skip mptcp_sched_default in mptcp_unregister_scheduler.
- patch 8: add tests into mptcp.c, instead of bpf_tcp_ca.c.
v4:
- set msk->sched to &mptcp_sched_default when the sched argument is NULL
in mptcp_init_sched().
v3:
- add mptcp_release_sched helper in patch 4.
- rename mptcp_set_sched to mptcp_init_sched in patch 4.
- add mptcp_sched_first_release in patch 7.
- do some cleanups.
v2:
- split into more small patches.
- change all parameters of mptcp_sched_ops from sk to msk:
void (*init)(struct mptcp_sock *msk);
void (*release)(struct mptcp_sock *msk);
struct sock * (*get_subflow)(struct mptcp_sock *msk); - add tests in bpf_tcp_ca.c, instead of adding a new one.
v1:
- Addressed to the commends in the RFC version.
Geliang Tang (8):
mptcp: add struct mptcp_sched_ops
mptcp: add a new sysctl scheduler
mptcp: add sched in mptcp_sock
mptcp: add get_subflow wrappers
mptcp: add bpf_mptcp_sched_ops
mptcp: add last_snd write access
selftests: bpf: add bpf_first scheduler
selftests: bpf: add bpf_first test
Documentation/networking/mptcp-sysctl.rst | 8 + include/net/mptcp.h | 19 ++ kernel/bpf/bpf_struct_ops_types.h | 4 + net/mptcp/Makefile | 2 +- net/mptcp/bpf.c | 163 ++++++++++++++++++ net/mptcp/ctrl.c | 14 ++ net/mptcp/protocol.c | 32 ++-- net/mptcp/protocol.h | 50 ++++++ net/mptcp/sched.c | 89 ++++++++++ .../testing/selftests/bpf/bpf_mptcp_helpers.h | 20 +++ .../testing/selftests/bpf/prog_tests/mptcp.c | 112 ++++++++++++ .../selftests/bpf/progs/mptcp_bpf_first.c | 35 ++++ 12 files changed, 528 insertions(+), 20 deletions(-) create mode 100644 net/mptcp/sched.c create mode 100644 tools/testing/selftests/bpf/progs/mptcp_bpf_first.c
Signature · 1 line
2.34.1
PATCH v17 1/8 mptcp: add struct mptcp_sched_ops
This patch defines struct mptcp_sched_ops, which has three struct members, name, owner and list, and three function pointers, init, release and get_subflow.
Add the scheduler registering, unregistering and finding functions to add, delete and find a packet scheduler on the global list mptcp_sched_list.
For supporting a "redundant" packet scheduler in the future, this patch adds a flag in struct mptcp_sched_data named call_again to indicate that get_subflow() function needs to be called again.
--- include/net/mptcp.h | 19 +++++++ net/mptcp/Makefile | 2 +- net/mptcp/protocol.h | 3 + net/mptcp/sched.c | 56 +++++++++++++++++++ .../testing/selftests/bpf/bpf_mptcp_helpers.h | 18 ++++++ 5 files changed, 97 insertions(+), 1 deletion(-) create mode 100644 net/mptcp/sched.c
include/net/mptcp.h +19 -0
@@ -95,6 +95,25 @@ struct mptcp_out_options {#endif }; +#define MPTCP_SCHED_NAME_MAX 16 + +struct mptcp_sched_data { +struct sock *sock; +bool call_again; +}; + +struct mptcp_sched_ops { +void (*get_subflow)(struct mptcp_sock *msk,bool reinject, +struct mptcp_sched_data *data); + +char name[MPTCP_SCHED_NAME_MAX ]; +struct module *owner; +struct list_head list; + +void (*init)(struct mptcp_sock *msk); +void (*release)(struct mptcp_sock *msk); +}____cacheline_aligned_in_smp ; +#ifdef CONFIG_MPTCP extern struct request_sock_ops mptcp_subflow_request_sock_ops;
net/mptcp/Makefile +1 -1
@@ -2,7 +2,7 @@ obj-$(CONFIG_MPTCP) += mptcp.o mptcp-y := protocol.o subflow.o options.o token.o crypto.o ctrl.o pm.o diag.o \ - mib.o pm_netlink.o sockopt.o pm_userspace.o + mib.o pm_netlink.o sockopt.o pm_userspace.o sched.o obj-$(CONFIG_SYN_COOKIES) += syncookies.o obj-$(CONFIG_INET_MPTCP_DIAG) += mptcp_diag.o
net/mptcp/protocol.h +3 -0
@@ -621,6 +621,9 @@ int mptcp_subflow_create_socket(struct sock *sk, struct socket **new_sock);void mptcp_info2sockaddr (const struct mptcp_addr_info *info,struct sockaddr_storage *addr,unsigned short family); +struct mptcp_sched_ops *mptcp_sched_find (const char *name); +int mptcp_register_scheduler (struct mptcp_sched_ops *sched); +void mptcp_unregister_scheduler (struct mptcp_sched_ops *sched);static inline bool __mptcp_subflow_active (struct mptcp_subflow_context *subflow) {
net/mptcp/sched.c new file +56 -0
56 lines not on this page · raw message
tools/testing/selftests/bpf/bpf_mptcp_helpers.h +18 -0
@@ -6,6 +6,24 @@#include "bpf_tcp_helpers.h" +#define MPTCP_SCHED_NAME_MAX 16 + +struct mptcp_sched_data { +struct sock *sock; +bool call_again; +}; + +struct mptcp_sched_ops { +char name[MPTCP_SCHED_NAME_MAX ]; + +void (*init)(struct mptcp_sock *msk); +void (*release)(struct mptcp_sock *msk); + +void (*get_subflow)(struct mptcp_sock *msk,bool reinject, +struct mptcp_sched_data *data); +void *owner; +}; +struct mptcp_sock {struct inet_connection_sock sk;
Signature · 1 line
2.34.1
PATCH v17 2/8 mptcp: add a new sysctl scheduler
In reply to Geliang TangThis patch adds a new sysctl, named scheduler, to support for selection of different schedulers. Export mptcp_get_scheduler helper to get this sysctl.
--- Documentation/networking/mptcp-sysctl.rst | 8 ++++++++ net/mptcp/ctrl.c | 14 ++++++++++++++ net/mptcp/protocol.h | 1 + 3 files changed, 23 insertions(+)
Documentation/networking/mptcp-sysctl.rst +8 -0
@@ -75,3 +75,11 @@ stale_loss_cnt - INTEGER This is a per-namespace sysctl. Default: 4 + +scheduler - STRING + Select the scheduler of your choice. + + Support for selection of different schedulers. This is a per-namespace + sysctl. + + Default: "default"
net/mptcp/ctrl.c +14 -0
@@ -32,6 +32,7 @@ struct mptcp_pernet {u8 checksum_enabled;u8 allow_join_initial_addr_port;u8 pm_type; +char scheduler[MPTCP_SCHED_NAME_MAX ]; };static struct mptcp_pernet *mptcp_get_pernet (const struct net *net)
@@ -69,6 +70,11 @@ int mptcp_get_pm_type(const struct net *net)return mptcp_get_pernet (net)->pm_type; } +const char *mptcp_get_scheduler (const struct net *net) +{ +return mptcp_get_pernet (net)->scheduler; +} +static void mptcp_pernet_set_defaults (struct mptcp_pernet *pernet) { pernet->mptcp_enabled =1 ;
@@ -77,6 +83,7 @@ static void mptcp_pernet_set_defaults(struct mptcp_pernet *pernet) pernet->allow_join_initial_addr_port =1 ; pernet->stale_loss_cnt =4 ; pernet->pm_type =MPTCP_PM_TYPE_KERNEL ; +strcpy (pernet->scheduler,"default" ); }#ifdef CONFIG_SYSCTL
@@ -128,6 +135,12 @@ static struct ctl_table mptcp_sysctl_table[] = { .extra1 = SYSCTL_ZERO, .extra2 = &mptcp_pm_type_max }, + { + .procname = "scheduler", + .maxlen = MPTCP_SCHED_NAME_MAX, + .mode = 0644, + .proc_handler = proc_dostring, + }, {} };
@@ -149,6 +162,7 @@ static int mptcp_pernet_new_table(struct net *net, struct mptcp_pernet *pernet) table[3 ].data = &pernet->allow_join_initial_addr_port; table[4 ].data = &pernet->stale_loss_cnt; table[5 ].data = &pernet->pm_type; + table[6 ].data = &pernet->scheduler; hdr =register_net_sysctl (net,MPTCP_SYSCTL_PATH , table);if (!hdr)
net/mptcp/protocol.h +1 -0
@@ -596,6 +596,7 @@ int mptcp_is_checksum_enabled(const struct net *net);int mptcp_allow_join_id0 (const struct net *net);unsigned int mptcp_stale_loss_cnt (const struct net *net);int mptcp_get_pm_type (const struct net *net); +const char *mptcp_get_scheduler (const struct net *net);void mptcp_subflow_fully_established (struct mptcp_subflow_context *subflow,struct mptcp_options_received *mp_opt);bool __mptcp_retransmit_pending_data (struct sock *sk);
Signature · 1 line
2.34.1
PATCH v17 3/8 mptcp: add sched in mptcp_sock
In reply to Geliang TangThis patch adds a new struct member sched in struct mptcp_sock. And two helpers mptcp_init_sched() and mptcp_release_sched() to init and release it.
Init it with the sysctl scheduler in mptcp_init_sock(), copy the scheduler from the parent in mptcp_sk_clone(), and release it in __mptcp_destroy_sock().
--- net/mptcp/protocol.c | 7 ++++ net/mptcp/protocol.h | 4 +++ net/mptcp/sched.c | 33 +++++++++++++++++++ .../testing/selftests/bpf/bpf_mptcp_helpers.h | 1 + 4 files changed, 45 insertions(+)
net/mptcp/protocol.c +7 -0
@@ -2665,6 +2665,11 @@ static int mptcp_init_sock(struct sock *sk)if (ret)return ret; + ret =mptcp_init_sched (mptcp_sk (sk), +mptcp_sched_find (mptcp_get_scheduler (net))); +if (ret) +return ret; +/* fetch the ca name; do it outside __mptcp_init_sock(), so that clone will * propagate the correct value */
@@ -2824,6 +2829,7 @@ static void __mptcp_destroy_sock(struct sock *sk)sk_stop_timer (sk, &sk->sk_timer);mptcp_data_unlock (sk); msk->pm.status =0 ; +mptcp_release_sched (msk);/* clears msk->subflow, allowing the following loop to close * even the initial subflow
@@ -3001,6 +3007,7 @@ struct sock *mptcp_sk_clone(const struct sock *sk, msk->snd_una = msk->write_seq; msk->wnd_end = msk->snd_nxt + req->rsk_rcv_wnd; msk->setsockopt_seq =mptcp_sk (sk)->setsockopt_seq; +mptcp_init_sched (msk,mptcp_sk (sk)->sched);if (mp_opt->suboptions &OPTIONS_MPTCP_MPC ) { msk->can_ack = true;
net/mptcp/protocol.h +4 -0
@@ -298,6 +298,7 @@ struct mptcp_sock {struct socket *subflow;/* outgoing connect/listener/!mp_capable */ struct sock *first;struct mptcp_pm_data pm; +struct mptcp_sched_ops *sched;struct {u32 space;/* bytes copied in last measurement window */ u32 copied;/* bytes copied in this measurement window */
@@ -625,6 +626,9 @@ void mptcp_info2sockaddr(const struct mptcp_addr_info *info,struct mptcp_sched_ops *mptcp_sched_find (const char *name);int mptcp_register_scheduler (struct mptcp_sched_ops *sched);void mptcp_unregister_scheduler (struct mptcp_sched_ops *sched); +int mptcp_init_sched (struct mptcp_sock *msk, +struct mptcp_sched_ops *sched); +void mptcp_release_sched (struct mptcp_sock *msk);static inline bool __mptcp_subflow_active (struct mptcp_subflow_context *subflow) {
net/mptcp/sched.c +33 -0
@@ -54,3 +54,36 @@ void mptcp_unregister_scheduler(struct mptcp_sched_ops *sched)list_del_rcu (&sched->list);spin_unlock (&mptcp_sched_list_lock); } + +int mptcp_init_sched (struct mptcp_sock *msk, +struct mptcp_sched_ops *sched) +{ +if (!sched) + goto out; + +if (!bpf_try_module_get (sched, sched->owner)) +return -EBUSY ; + + msk->sched = sched; +if (msk->sched->init) + msk->sched->init (msk); + +pr_debug ("sched=%s" , msk->sched->name); + +out: +return 0 ; +} + +void mptcp_release_sched (struct mptcp_sock *msk) +{ +struct mptcp_sched_ops *sched = msk->sched; + +if (!sched) +return ; + + msk->sched =NULL ; +if (sched->release) + sched->release (msk); + +bpf_module_put (sched, sched->owner); +}
tools/testing/selftests/bpf/bpf_mptcp_helpers.h +1 -0
@@ -29,6 +29,7 @@ struct mptcp_sock {__u32 token;struct sock *first; +struct mptcp_sched_ops *sched;char ca_name[TCP_CA_NAME_MAX ]; } __attribute__((preserve_access_index));
Signature · 1 line
2.34.1
PATCH v17 4/8 mptcp: add get_subflow wrappers
In reply to Geliang TangThis patch defines two new wrappers mptcp_sched_get_send() and mptcp_sched_get_retrans(), invoke get_subflow() of msk->sched in them. Use them instead of using mptcp_subflow_get_send() or mptcp_subflow_get_retrans() directly.
--- net/mptcp/protocol.c | 25 ++++++------------------- net/mptcp/protocol.h | 42 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 48 insertions(+), 19 deletions(-)
net/mptcp/protocol.c +6 -19
@@ -1427,7 +1427,7 @@ bool mptcp_subflow_active(struct mptcp_subflow_context *subflow)* returns the subflow that will transmit the next DSS * additionally updates the rtx timeout */ -static struct sock *mptcp_subflow_get_send (struct mptcp_sock *msk) +struct sock *mptcp_subflow_get_send (struct mptcp_sock *msk) {struct subflow_send_info send_info[SSK_MODE_MAX ];struct mptcp_subflow_context *subflow;
@@ -1438,14 +1438,6 @@ static struct sock *mptcp_subflow_get_send(struct mptcp_sock *msk)u64 linger_time;long tout =0 ; -sock_owned_by_me (sk); - -if (__mptcp_check_fallback (msk)) { -if (!msk->first) -return NULL ; -return sk_stream_memory_free (msk->first) ? msk->first :NULL ; - } -/* re-use last subflow, if the burst allow that */ if (msk->last_snd && msk->snd_burst > 0 && sk_stream_memory_free(msk->last_snd) &&
@@ -1575,7 +1567,7 @@ void __mptcp_push_pending(struct sock *sk, unsigned int flags)int ret =0 ; prev_ssk = ssk; - ssk =mptcp_subflow_get_send (msk); + ssk =mptcp_sched_get_send (msk);/* First check. If the ssk has changed since * the last round, release prev_ssk
@@ -1644,7 +1636,7 @@ static void __mptcp_subflow_push_pending(struct sock *sk, struct sock *ssk)* check for a different subflow usage only after * spooling the first chunk of data */ - xmit_ssk = first ? ssk :mptcp_subflow_get_send (mptcp_sk (sk)); + xmit_ssk = first ? ssk :mptcp_sched_get_send (mptcp_sk (sk));if (!xmit_ssk) goto out;if (xmit_ssk != ssk) {
@@ -2218,17 +2210,12 @@ static void mptcp_timeout_timer(struct timer_list *t)* * A backup subflow is returned only if that is the only kind available. */ -static struct sock *mptcp_subflow_get_retrans (struct mptcp_sock *msk) +struct sock *mptcp_subflow_get_retrans (struct mptcp_sock *msk) {struct sock *backup =NULL , *pick =NULL ;struct mptcp_subflow_context *subflow;int min_stale_count =INT_MAX ; -sock_owned_by_me ((const struct sock *)msk); - -if (__mptcp_check_fallback (msk)) -return NULL ; -mptcp_for_each_subflow (msk, subflow) {struct sock *ssk =mptcp_subflow_tcp_sock (subflow);
@@ -2481,7 +2468,7 @@ static void __mptcp_retrans(struct sock *sk)mptcp_clean_una_wakeup (sk);/* first check ssk: need to kick "stale" logic */ - ssk =mptcp_subflow_get_retrans (msk); + ssk =mptcp_sched_get_retrans (msk); dfrag =mptcp_rtx_head (sk);if (!dfrag) {if (mptcp_data_fin_enabled (msk)) {
@@ -3146,7 +3133,7 @@ void __mptcp_check_push(struct sock *sk, struct sock *ssk)return ;if (!sock_owned_by_user (sk)) { -struct sock *xmit_ssk =mptcp_subflow_get_send (mptcp_sk (sk)); +struct sock *xmit_ssk =mptcp_sched_get_send (mptcp_sk (sk));if (xmit_ssk == ssk)__mptcp_subflow_push_pending (sk, ssk);
net/mptcp/protocol.h +42 -0
46 lines not on this page · raw message
Signature · 1 line
2.34.1
PATCH v17 5/8 mptcp: add bpf_mptcp_sched_ops
In reply to Geliang TangThis patch implements a new struct bpf_struct_ops, bpf_mptcp_sched_ops. Register and unregister the bpf scheduler in .reg and .unreg.
This MPTCP BPF scheduler implementation is similar to BPF TCP CC. And net/ipv4/bpf_tcp_ca.c is a frame of reference for this patch.
--- kernel/bpf/bpf_struct_ops_types.h | 4 + net/mptcp/bpf.c | 154 ++++++++++++++++++++++++++++++ 2 files changed, 158 insertions(+)
kernel/bpf/bpf_struct_ops_types.h +4 -0
@@ -8,5 +8,9 @@ BPF_STRUCT_OPS_TYPE(bpf_dummy_ops)#ifdef CONFIG_INET #include <net/tcp.h> BPF_STRUCT_OPS_TYPE(tcp_congestion_ops) +#ifdef CONFIG_MPTCP +#include <net/mptcp.h> +BPF_STRUCT_OPS_TYPE(mptcp_sched_ops) +#endif #endif #endif
net/mptcp/bpf.c +154 -0
162 lines not on this page · raw message
Signature · 1 line
2.34.1
PATCH v17 6/8 mptcp: add last_snd write access
In reply to Geliang TangThis patch exports the member last_snd of struct mptcp_sock in bpf_mptcp_helpers.h, and adds BPF write access to it.
--- net/mptcp/bpf.c | 15 ++++++++++++--- tools/testing/selftests/bpf/bpf_mptcp_helpers.h | 1 + 2 files changed, 13 insertions(+), 3 deletions(-)
net/mptcp/bpf.c +12 -3
@@ -18,8 +18,8 @@#ifdef CONFIG_BPF_JIT extern struct bpf_struct_ops bpf_mptcp_sched_ops;extern struct btf *btf_vmlinux; -static const struct btf_type *mptcp_sched_type__read_mostly ; -static u32 mptcp_sched_id; +static const struct btf_type *mptcp_sock_type, *mptcp_sched_type__read_mostly ; +static u32 mptcp_sock_id, mptcp_sched_id;static u32 optional_ops[] = { offsetof(struct mptcp_sched_ops, init),
@@ -47,12 +47,15 @@ static int bpf_mptcp_sched_btf_struct_access(struct bpf_verifier_log *log,return btf_struct_access (log, btf, t, off, size, atype, next_btf_id, flag); -if (t != mptcp_sched_type) { +if (t != mptcp_sock_type && t != mptcp_sched_type) {bpf_log (log,"only access to mptcp_sched is supported\n" );return -EACCES ; }switch (off) { +case offsetof(struct mptcp_sock , last_snd): + end = offsetofend(struct mptcp_sock, last_snd); +break ;case offsetof(struct mptcp_sched_data , sock): end = offsetofend(struct mptcp_sched_data, sock);break ;
@@ -145,6 +148,12 @@ static int bpf_mptcp_sched_init(struct btf *btf) {s32 type_id; + type_id =btf_find_by_name_kind (btf,"mptcp_sock" ,BTF_KIND_STRUCT ); +if (type_id <0 ) +return -EINVAL ; + mptcp_sock_id = type_id; + mptcp_sock_type =btf_type_by_id (btf, mptcp_sock_id); + type_id =btf_find_by_name_kind (btf,"mptcp_sched_data" ,BTF_KIND_STRUCT );if (type_id <0 )
tools/testing/selftests/bpf/bpf_mptcp_helpers.h +1 -0
@@ -27,6 +27,7 @@ struct mptcp_sched_ops {struct mptcp_sock {struct inet_connection_sock sk; +struct sock *last_snd;__u32 token;struct sock *first;struct mptcp_sched_ops *sched;
Signature · 1 line
2.34.1
PATCH v17 7/8 selftests: bpf: add bpf_first scheduler
In reply to Geliang TangThis patch implements the simplest MPTCP scheduler, named bpf_first, which always picks the first subflow to send data. It's a sample of MPTCP BPF scheduler implementations.
--- .../selftests/bpf/progs/mptcp_bpf_first.c | 35 +++++++++++++++++++ 1 file changed, 35 insertions(+) create mode 100644 tools/testing/selftests/bpf/progs/mptcp_bpf_first.c
tools/testing/selftests/bpf/progs/mptcp_bpf_first.c new file +35 -0
@@ -0,0 +1,35 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2022, SUSE. */ + +#include <linux/bpf.h> +#include "bpf_mptcp_helpers.h" + +char _license[] SEC("license") = "GPL"; + +SEC("struct_ops/mptcp_sched_first_init") +void BPF_PROG (mptcp_sched_first_init ,struct mptcp_sock *msk) +{ +} + +SEC("struct_ops/mptcp_sched_first_release") +void BPF_PROG (mptcp_sched_first_release ,struct mptcp_sock *msk) +{ +} + +void BPF_STRUCT_OPS (bpf_first_get_subflow ,struct mptcp_sock *msk, +bool reinject,struct mptcp_sched_data *data) +{ +struct sock *ssk = msk->first; + + msk->last_snd = ssk; + data->sock = ssk; + data->call_again =0 ; +} + +SEC(".struct_ops") +struct mptcp_sched_ops first = { + .init = (void *)mptcp_sched_first_init, + .release = (void *)mptcp_sched_first_release, + .get_subflow = (void *)bpf_first_get_subflow, + .name ="bpf_first" , +};
Signature · 1 line
2.34.1
PATCH v17 8/8 selftests: bpf: add bpf_first test
In reply to Geliang TangThis patch expends the MPTCP test base to support MPTCP packet scheduler tests. Add the bpf_first scheduler test in it. Use sysctl to set net.mptcp.scheduler to use this sched.
Some code in send_data() is from prog_tests/bpf_tcp_ca.c.
--- .../testing/selftests/bpf/prog_tests/mptcp.c | 112 ++++++++++++++++++ 1 file changed, 112 insertions(+)
tools/testing/selftests/bpf/prog_tests/mptcp.c +112 -0
@@ -4,6 +4,7 @@#include <test_progs.h> #include "cgroup_helpers.h" #include "network_helpers.h" +#include "mptcp_bpf_first.skel.h" #ifndef TCP_CA_NAME_MAX #define TCP_CA_NAME_MAX 16
@@ -19,6 +20,8 @@ struct mptcp_storage { };static char monitor_log_path[64 ]; +static const unsigned int total_bytes =10 *1024 *1024 ; +static int stop, duration;static int verify_tsk (int map_fd,int client_fd) {
@@ -251,8 +254,117 @@ void test_base(void)close (cgroup_fd); } +static void *server (void *arg) +{ +int lfd = (int )(long )arg, err =0 , fd; +ssize_t nr_sent =0 , bytes =0 ; +char batch[1500 ]; + + fd =accept (lfd,NULL ,NULL ); +while (fd ==-1 ) { +if (errno ==EINTR ) +continue ; + err = -errno; + goto done; + } + +if (settimeo (fd,0 )) { + err = -errno; + goto done; + } + +while (bytes < total_bytes && !READ_ONCE (stop)) { + nr_sent =send (fd, &batch, +MIN (total_bytes - bytes,sizeof (batch)),0 ); +if (nr_sent ==-1 && errno ==EINTR ) +continue ; +if (nr_sent ==-1 ) { + err = -errno; +break ; + } + bytes += nr_sent; + } + +CHECK (bytes != total_bytes,"send" ,"%zd != %u nr_sent:%zd errno:%d\n" , + bytes, total_bytes, nr_sent, errno); + +done: +if (fd >=0 ) +close (fd); +if (err) { +WRITE_ONCE (stop,1 ); +return ERR_PTR (err); + } +return NULL ; +} + +static void send_data (int lfd,int fd) +{ +ssize_t nr_recv =0 , bytes =0 ; +pthread_t srv_thread; +void *thread_ret; +char batch[1500 ]; +int err; + +WRITE_ONCE (stop,0 ); + + err =pthread_create (&srv_thread,NULL , server, (void *)(long )lfd); +if (CHECK (err !=0 ,"pthread_create" ,"err:%d errno:%d\n" , err, errno)) +return ; + +/* recv total_bytes */ +while (bytes < total_bytes && !READ_ONCE (stop)) { + nr_recv =recv (fd, &batch, +MIN (total_bytes - bytes,sizeof (batch)),0 ); +if (nr_recv ==-1 && errno ==EINTR ) +continue ; +if (nr_recv ==-1 ) +break ; + bytes += nr_recv; + } + +CHECK (bytes != total_bytes,"recv" ,"%zd != %u nr_recv:%zd errno:%d\n" , + bytes, total_bytes, nr_recv, errno); + +WRITE_ONCE (stop,1 ); + +pthread_join (srv_thread, &thread_ret); +CHECK (IS_ERR (thread_ret),"pthread_join" ,"thread_ret:%ld" , +PTR_ERR (thread_ret)); +} + +static void test_first (void ) +{ +struct mptcp_bpf_first *first_skel; +int server_fd, client_fd; +struct bpf_link *link; + + first_skel =mptcp_bpf_first__open_and_load (); +if (CHECK (!first_skel,"bpf_first__open_and_load" ,"failed\n" )) +return ; + + link =bpf_map__attach_struct_ops (first_skel->maps.first); +if (!ASSERT_OK_PTR (link,"bpf_map__attach_struct_ops" )) { +mptcp_bpf_first__destroy (first_skel); +return ; + } + +system ("sysctl -q net.mptcp.scheduler=bpf_first" ); + server_fd =start_mptcp_server (AF_INET ,NULL ,0 ,0 ); + client_fd =connect_to_mptcp_fd (server_fd,0 ); + +send_data (server_fd, client_fd); + +close (client_fd); +close (server_fd); +bpf_link__destroy (link); +mptcp_bpf_first__destroy (first_skel); +} +void test_mptcp (void ) {if (test__start_subtest ("base" ))test_base (); +if (test__start_subtest ("first" )) +test_first (); }
Signature · 1 line
2.34.1
Re: selftests: bpf: add bpf_first test: Tests Results
Hi Geliang,
Thank you for your modifications, that's great!
Our CI did some validations and here is its report:
- KVM Validation: normal:
- Success! ✅:
- Task: https://cirrus-ci.com/task/5388566946643968
- Summary: https://api.cirrus-ci.com/v1/artifact/task/5388566946643968/summary/summary.txt
- KVM Validation: debug:
- Critical: 1 Call Trace(s) ❌:
- Task: https://cirrus-ci.com/task/6514466853486592
- Summary: https://api.cirrus-ci.com/v1/artifact/task/6514466853486592/summary/summary.txt
Initiator: Patchew Applier
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/4c685a27f844
If there are some issues, you can reproduce them using the same environment as the one used by the CI thanks to a docker image, e.g.:
$ cd [kernel source code]
$ docker run -v "${PWD}:${PWD}:rw" -w "${PWD}" --privileged --rm -it \
--pull always mptcp/mptcp-upstream-virtme-docker:latest \
auto-debugFor more details:
https://github.com/multipath-tcp/mptcp-upstream-virtme-docker
Please note that despite all the efforts that have been already done to have a stable tests suite when executed on a public CI like here, it is possible some reported issues are not due to your modifications. Still, do not hesitate to help us improve that ;-)
Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (Tessares)
Re: [PATCH mptcp-next v17 4/8] mptcp: add get_subflow wrappers
In reply to Geliang TangQuoted 111 lines · Geliang Tang
This patch defines two new wrappers
mptcp_sched_get_send()andmptcp_sched_get_retrans(), invokeget_subflow()ofmsk->schedin them. Use them instead of usingmptcp_subflow_get_send()ormptcp_subflow_get_retrans()directly.Signed-off-by Geliang Tang <[email protected]>---
net/mptcp/protocol.c| 25 ++++++-------------------net/mptcp/protocol.h| 42 ++++++++++++++++++++++++++++++++++++++++++2 files changed, 48 insertions(+), 19 deletions(-)net/mptcp/protocol.c +6
-19@@ -1427,7 +1427,7 @@ bool mptcp_subflow_active(struct mptcp_subflow_context *subflow)* returns the subflow that will transmit the next DSS * additionally updates the rtx timeout */ -static struct sock *mptcp_subflow_get_send (struct mptcp_sock *msk) +struct sock *mptcp_subflow_get_send (struct mptcp_sock *msk) {struct subflow_send_info send_info[SSK_MODE_MAX ];struct mptcp_subflow_context *subflow;@@ -1438,14 +1438,6 @@ static struct sock *mptcp_subflow_get_send(struct mptcp_sock *msk)u64 linger_time;long tout =0 ; -sock_owned_by_me (sk); - -if (__mptcp_check_fallback (msk)) { -if (!msk->first) -return NULL ; -return sk_stream_memory_free (msk->first) ? msk->first :NULL ; - } -/* re-use last subflow, if the burst allow that */ if (msk->last_snd && msk->snd_burst > 0 && sk_stream_memory_free(msk->last_snd) &&@@ -1575,7 +1567,7 @@ void __mptcp_push_pending(struct sock *sk, unsigned int flags)int ret =0 ; prev_ssk = ssk; - ssk =mptcp_subflow_get_send (msk); + ssk =mptcp_sched_get_send (msk);/* First check. If the ssk has changed since * the last round, release prev_ssk @@ -1644,7 +1636,7 @@ static void __mptcp_subflow_push_pending(struct sock *sk, struct sock *ssk)* check for a different subflow usage only after * spooling the first chunk of data */ - xmit_ssk = first ? ssk :mptcp_subflow_get_send (mptcp_sk (sk)); + xmit_ssk = first ? ssk :mptcp_sched_get_send (mptcp_sk (sk));if (!xmit_ssk) goto out;if (xmit_ssk != ssk) {@@ -2218,17 +2210,12 @@ static void mptcp_timeout_timer(struct timer_list *t)* * A backup subflow is returned only if that is the only kind available. */ -static struct sock *mptcp_subflow_get_retrans (struct mptcp_sock *msk) +struct sock *mptcp_subflow_get_retrans (struct mptcp_sock *msk) {struct sock *backup =NULL , *pick =NULL ;struct mptcp_subflow_context *subflow;int min_stale_count =INT_MAX ; -sock_owned_by_me ((const struct sock *)msk); - -if (__mptcp_check_fallback (msk)) -return NULL ; -mptcp_for_each_subflow (msk, subflow) {struct sock *ssk =mptcp_subflow_tcp_sock (subflow);@@ -2481,7 +2468,7 @@ static void __mptcp_retrans(struct sock *sk)mptcp_clean_una_wakeup (sk);/* first check ssk: need to kick "stale" logic */ - ssk =mptcp_subflow_get_retrans (msk); + ssk =mptcp_sched_get_retrans (msk); dfrag =mptcp_rtx_head (sk);if (!dfrag) {if (mptcp_data_fin_enabled (msk)) {@@ -3146,7 +3133,7 @@ void __mptcp_check_push(struct sock *sk, struct sock *ssk)return ;if (!sock_owned_by_user (sk)) { -struct sock *xmit_ssk =mptcp_subflow_get_send (mptcp_sk (sk)); +struct sock *xmit_ssk =mptcp_sched_get_send (mptcp_sk (sk));if (xmit_ssk == ssk)__mptcp_subflow_push_pending (sk, ssk);net/mptcp/protocol.h +9
-0@@ -988,4 +988,46 @@ mptcp_token_join_cookie_init_state(struct mptcp_subflow_request_sock *subflow_restatic inline void mptcp_join_cookie_init (void ) {}#endif +struct sock *mptcp_subflow_get_send (struct mptcp_sock *msk); +struct sock *mptcp_subflow_get_retrans (struct mptcp_sock *msk); + +static inline struct sock *mptcp_sched_get_send (struct mptcp_sock *msk) +{ +struct mptcp_sched_data data; + +sock_owned_by_me ((struct sock *)msk); +
net/mptcp/protocol.h mptcp_token_join_cookie_init_state(struct mptcp_subflow_request_sock *subflow_re · line 1009
Quoted 10 lines
net/mptcp/protocol.h +10
-0+/* the following check is moved out of mptcp_subflow_get_send */ +if (__mptcp_check_fallback (msk)) { +if (!msk->first) +return NULL ; +return sk_stream_memory_free (msk->first) ? msk->first :NULL ; + } + +if (!msk->sched) +return mptcp_subflow_get_send (msk); +
I think it would be good to zero out the mptcp_sched_data here so we aren't passing random data to the BPF code from uninitialized stack memory. Probably good to create a helper function that you can use here and in mptcp_sched_get_retrans(), and then modify the helper when adding struct members in "mptcp: add subflows array in mptcp_sched_data".
- Mat
Quoted 29 lines
net/mptcp/protocol.h +23
-0+ msk->sched->get_subflow (msk, false, &data); + + return data.sock; +} + +static inline struct sock *mptcp_sched_get_retrans (struct mptcp_sock *msk) +{ +struct mptcp_sched_data data; + +sock_owned_by_me ((const struct sock *)msk); + +/* the following check is moved out of mptcp_subflow_get_retrans */ +if (__mptcp_check_fallback (msk)) +return NULL ; + +if (!msk->sched) +return mptcp_subflow_get_retrans (msk); + + msk->sched->get_subflow (msk, true, &data); + +return data.sock; +} +#endif /* __MPTCP_PROTOCOL_H */ Signature · 1 line
2.34.1
--
Mat Martineau
Intel
Re: [PATCH mptcp-next v17 6/8] mptcp: add last_snd write access
In reply to Geliang TangQuoted 3 lines · Geliang Tang
This patch exports the member
last_sndof structmptcp_sockinbpf_mptcp_helpers.h, and adds BPF write access to it.
I still think we should stay away from giving the BPF functions write access to the msk unless it's really necessary.
In this case, both of the example schedulers write identical values to msk->last_snd and data->sock. Will they ever be different? Seems better to keep it simple for the BPF code and have the get_subflow wrappers handle assigning to msk->last_snd if a BPF function was called, and let the built-in scheduler assign last_snd itself.
- Mat
Quoted 68 lines
Signed-off-by Geliang Tang <[email protected]>---
net/mptcp/bpf.c| 15 ++++++++++++---tools/testing/selftests/bpf/bpf_mptcp_helpers.h| 1 +2 files changed, 13 insertions(+), 3 deletions(-)net/mptcp/bpf.c +12
-3@@ -18,8 +18,8 @@#ifdef CONFIG_BPF_JIT extern struct bpf_struct_ops bpf_mptcp_sched_ops;extern struct btf *btf_vmlinux; -static const struct btf_type *mptcp_sched_type__read_mostly ; -static u32 mptcp_sched_id; +static const struct btf_type *mptcp_sock_type, *mptcp_sched_type__read_mostly ; +static u32 mptcp_sock_id, mptcp_sched_id;static u32 optional_ops[] = { offsetof(struct mptcp_sched_ops, init),@@ -47,12 +47,15 @@ static int bpf_mptcp_sched_btf_struct_access(struct bpf_verifier_log *log,return btf_struct_access (log, btf, t, off, size, atype, next_btf_id, flag); -if (t != mptcp_sched_type) { +if (t != mptcp_sock_type && t != mptcp_sched_type) {bpf_log (log,"only access to mptcp_sched is supported\n" );return -EACCES ; }switch (off) { +case offsetof(struct mptcp_sock , last_snd): + end = offsetofend(struct mptcp_sock, last_snd); +break ;case offsetof(struct mptcp_sched_data , sock): end = offsetofend(struct mptcp_sched_data, sock);break ;@@ -145,6 +148,12 @@ static int bpf_mptcp_sched_init(struct btf *btf) {s32 type_id; + type_id =btf_find_by_name_kind (btf,"mptcp_sock" ,BTF_KIND_STRUCT ); +if (type_id <0 ) +return -EINVAL ; + mptcp_sock_id = type_id; + mptcp_sock_type =btf_type_by_id (btf, mptcp_sock_id); + type_id =btf_find_by_name_kind (btf,"mptcp_sched_data" ,BTF_KIND_STRUCT );if (type_id <0 )tools/testing/selftests/bpf/bpf_mptcp_helpers.h +1
-0@@ -27,6 +27,7 @@ struct mptcp_sched_ops {struct mptcp_sock {struct inet_connection_sock sk; +struct sock *last_snd;__u32 token;struct sock *first;struct mptcp_sched_ops *sched;Signature · 1 line
2.34.1
--
Mat Martineau
Intel